The notion of signoff has many layers to it, both in terms of complexity but also in terms of meaning. In my last blog post, I talked about some of the imprecise attributes of functional verification, like how much functional coverage you should use on a particular design. I promised to next talk about signoff, so here it is.
There are two fundamental steps most users apply to sign-off a new design, a functional milestone and an RTL complete milestone. The difference between the two is that one is really almost entirely focused on functional attributes, whereas the other adds all of the other less subjective RTL tests needed to ensure design correctness. I can generally characterize the first as something that can take a great deal of human engineering effort by dedicated verification teams armed with powerful built-for-purpose verification languages designed for deep verification cycles. I characterize the second as a little more automated and mainly tool driven. You read in your RTL into the tool, and the tool is smart enough to know if it passes based on user inputs. Examples are things like clock-domain crossing checks, synthesis checks, gate-level power-up checks, configuration mode checks, and many more. For the most part, these either work or they don’t; there is no gray matter in between success and failure. Either it passes or it does not, and if it does not it is usually something the RTL designer forgot.
Functional signoff adds an element of ambiguity, especially for a complex SOC. For a given IP verification project, even if you happen to get the best verification engineer on the planet and they code up the most precise set of functional coverage needed for the design and according to a comprehensive and well vetted verification plan, you still have enough variables and ambiguity left such that it is never 100% precise. There are factors such as coverage uniformity, hard-to-reach corner cases, degrees of completeness, depth of verification, diligence of stress test, imprecise use case scenarios, design ambiguity – the list goes on and on. Individually they are not that bad, combine them together and you’ve got challenges. Just like in mechanical design where designs have safety factors, in functional verification you need safety factors as well. Over achievement, limit testing, and applying independent and alternate test methods become the tools of the trade and help to ensure a high-quality design. Just like you use an independent verification engineer to ensure integrity of the design, you also can and should use independent verification techniques for mission-critical designs.
This first milestone, functional signoff, takes the most amount of time because of the nature of functional verification described above. So we tend to focus on it the most. At a minimum, the typical functional signoff list for an IP design would include things like:
- RTL code coverage – check to make sure all aspects of the design have been tested, and code not used or dead code gets waivers
- Functional coverage – check to make sure all aspects of the design are functionally correct using functional covergroups and assertions
- Use case tests – tests the design according to its intended use
- Bug rate – no new bugs have been identified, rate is no longer increasing
Sure, it's nice to add more stress tests, requirements coverage, plan coverage, feature coverage, perhaps stability tests, etc - there is a long list. The Incisive vManager tool can easily display any and all of these functional signoff metrics in an easy-to-view list with a unified ‘overall coverage’ metric which is rolled up and merged with all the other coverage metrics to form a single unified results metric. You combine metrics from multiple engines as well, ensuring data consistency.
This second part of signoff, which I will call RTL signoff, is normally a second and independent milestone, which can only be completed after functional signoff. This second milestone focuses on your RTL structural testing, and aggregates your additional sequence, power-up, and timing-based checks with much of the testing done at gate level post synthesis. For example, the gate-level tests would most likely be:
- Power up
- Shut down
- Configuration modes
- Reset
- Low power
RTL signoff requires additional tests for clocks, synthesis, timing, FSM, and more. Again, depending on the quality goals, your design may have more or less of these RTL-specific signoff goals. The key is, you need both, and both need to be easily displayed and managed as independent milestones. The Incisive vManager tool is primarily focused on functional signoff and provides the infrastructure and interfaces to be able to automatically collect, manage, and analyze functional signoff within one single cockpit, with dedicated views for engineers and dedicated views for managers. Functional signoff is based on your signoff metrics and are different at different levels of integration. For IP designs, the metrics are coverage based, for the SoC, it is much more around the actual use cases. This methodology, metric-driven verification signoff, understands and comprehends these different criteria at different levels. Collecting and integrating the RTL signoff data can be added to the Incisive vManager tool via data imports or API, enabling a single cockpit to view all of the results.
If you would like to learn more, please contact us, we would love to show you.
John Brennan