No one is invincible!

During a conversation this year at DVCon 2009, Gary Smith classified the Dynamic Play List features of the DV Notebook as belonging to the classic definition of an Intelligent Test Bench.  Specifically, the ability of a test bench to prune unproductive work and avoid wasteful re-run of tests that are not going to find bugs.

Brian Bailey has written a compatible definition of an intelligent test bench in the following quote:

An intelligent testbench can either replace or enhance existing simulation based or formal verification methodologies. Constrained random generation techniques manage to create huge quantities of stimulus, but at the end of the day they have difficulties both with closure (achieving the desired verification goals) and secondly with efficiency (huge server farms required). An intelligent testbench can help either by determining efficient stimulus sets or by finding ways to reach difficult to reach coverage points.

In spite of this, one still needs to be clear about many aspects of intelligent test benches.  Most EDA products that belong in this category are focused on optimizing the outcome of a single test.  The vast majority of those are concerned with a single execution image.  This is in contrast with the idea of looking at a test population to selectively run those tests that are most likely to yield results.  (also see Do tests depreciate?)

There are several different scales to consider:

• Optimizing a single execution to target a desired outcome
• Optimizing runs of a single test to meet particular coverage goals
• Optimally selecting the members (tests) within one or more regression lists
• Optimizing test populations over the life span of a project or on subsequent projects

In part, the problem may be nomenclature.  The last two bullets above probably relate more to wisdom than to intelligence.  Wisdom is well rounded knowledge accumulated through time and experience.

The goal of the DV Notebook products is to enhance existing simulation based verification by determining efficient stimulus sets.  This entails looking more broadly at all available data (e.g. bugs, bug fixes, code check ins, lint results,…) to determine how the code is evolving, how the project is progressing.  This gives a picture of status and overall project health.  It is from that basis that cost reductions in verification can be realized.

This entry was posted on Friday, February 27th, 2009 at 3:42 pm and is filed under DVCon09, Verification. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.