Functional Verification of
HDL Models
Introduction to Verification
2007
Tehran University
1
Reference Book
• Janick Bergeron [WTB]
Writing Testbenches:
Functional Verification Of HDL Models.
• First Edition, Kluwer, 2000, ISBN: 0-7923-77664
• Second Edition, Kluwer, 2003, ISBN: 1-40207401-8
2007
Tehran University
2
Other references
• Comprehensive Functional Verification: The complete
industry cycle
– B. Wile, J. Goss, W. Roesner [IBM]
• Principles of Functional Verification
– Andreas Meyer
• Functional Verification Coverage & Measurement and
Analysis
– Andrew Piziali
• Advanced Verification Techniques: A SystemC Based
Approach for Successful Tapeout
– Leena Singh, L. Drucker, Neyaz Khan
2007
Tehran University
3
Course Goals
Aim: To familiarise you with the routine tasks in verification
management, and to give you the technical background plus
some of the practical skills expected from a Design
Verification Engineer.
• Prerequisite: Hardware Language programming
On successful completion of this unit, you will be able to:
• understand the complexities and limits of verification;
• carry out functional verification and determine its effectiveness;
• set appropriate verification goals, select suitable verification
methods and assess the associated risks;
• compile a verification plan that fits into the flow of a design
project
2007
Tehran University
4
1974
Brian W. Kernighan
• Debugging is twice as hard as writing the code
in the first place. Therefore, if you write the
code as cleverly as possible, you are, by
definition, not smart enough to debug it.
2007
Tehran University
5
Bugs: What is the Cost?
• Time to Market is affected
• Bugs found early in design have little cost!
• Finding a bug at chip/system level has moderate cost:
– Requires more debug time and isolation time.
– Could require new algorithm, which could effect schedule
and cause board rework.
• Finding a bug in System Test (test floor) requires respin of a chip.
• Finding a bug after customer delivery cost millons
2007
Tehran University
6
Cost of Bugs Over Time
Number of bugs found
Cost of finding bugs
Time
module
2007
chip
Tehran University
system
customer
7
Importance of Verification
• 70% of ASIC / IP / SoC design effort goes to
verification
– Is on the critical path
– Has effects on Schedule, Cost, Quality
– Fewer revisions through fabrication process means lower cost
and faster time-to-market
• Twice more Verification engineers than RTL designer
• Re-spinning cost time and money
• Getting the design right first time is an art
– Due to chip complexity it is more and more difficult
2007
Tehran University
8
Silicon Validation
• Silicon Validation times
increase as features size
decreases
• The difficulty is expected
to increase at 65nm and
below because existing
ad-hoc methodologies do
not scale with the
unprecedented levels of
SoC device complexity.
2007
Tehran University
9
SoC Verification
• Complete system-level verification of a complex
SoC at 90 nm or below is not feasible pre-silicon
– Even the most sophisticated SoC design
methodology cannot fully account for all the
parameters that impact silicon behavior, or for all
logic "corner cases" that occur in the real life of a
chip working at speed and in system [Miron
Abramovici, Sep. 2007 ]
• post-silicon validation has become an essential
step in the design implementation methodology
2007
Tehran University
10
Importance of Verification (Cont.)
• International Technology Roadmap for
Semiconductors identified Verification as a crisis
area (http://public.itrs.net/)
• Shortfalls in qualified engineers and softwaretrained designers
• Verification can be reduced through parallelism,
abstraction and automation
2007
Tehran University
11
Course Outline
Lecture Topics:
• What is Verification? What is a Testbench?
• Verification Tools (including Verilog basics)
• Verilog HDL coding and Verification Plan
• Assertion-based Verification
• Advanced Testbench Design Methodology
• Functional Formal Verification and Property Checking
• Labs (Verilog, ModelSim, C/C++, SystemC)
2007
Tehran University
12
Introduction to IC Design Process
1.
2.
Specification
Behavioral Description
Architectural Synthesis
3. RTL Description
Logical Synthesis
4. Gate Level Description
Physical Synthesis
5. Layout and tape-out
In practice all steps almost start at the same time and run
in parallel
2007
Tehran University
13
Steps in IC Design Process
Architectural Synthesis
Input: behavioral description
• generates architecture for design at RTL
– data path (interconnection of adders, multipliers, etc)
– control unit (signals to control data path)
• scheduling and binding
Logic Synthesis
Input: RTL description
• converts into network of logic primitives (depending on design
• style)
• Two groups:
– sequential/combinatorial
Physical Design
Input: gate level description
• generates geometric patterns that define physical layout of chip
2007
Tehran University
14
Verification Role in IC Design
IC design is a complex process:
• Conflict: design time (decreased) versus complexity
(increased)
• Aim: correct by construction, right-first-time design
More and more time-consuming to obtain an acceptable
level of confidence in correctness of design
• Design time << Verification time
• Needs its own trained verification engineers
• 80% of written code is for the verification environment
2007
Tehran University
15
How does a designer know a circuit
is correct?
concept
Verify: What you specified is what you envisioned
Specification
Verify: What you designed is what you specified
HDL design (RTL)
Verify: What you taped out is what the RTL described
Tape out
Verify: What you manufactured is what you taped out
Silicon
There is no single methodology that covers all!
2007
Tehran University
16
Some Definitions
• What is Verification?
– A process used to demonstrate the functional correctness of
a design in its implementation
 Hence the term Functional Verification
• What is a Testbench?
– Code used to generate predetermined input sequence to a
design
– Implemented using Verilog, VHDL, e or Open Vera, also
include C routines
– Always refers to test case/scenario
2007
Tehran University
17
Testbench and DUV
• Completely closed systems
• No inputs or outputs go in or out
• Generic Structure of a testbench and DUV:
Testbench
Stimulus
Generator
Design
Under
Verification
Checker
Supply stimulus to checker for comparison with the specification
2007
Tehran University
18
Verification is on critical path
Need to minimise verification time e.g. by using:
• Parallelism: Add more resources
• Abstraction: Higher level of abstraction (i.e. C vs
Assembly, TLM)
– This often means a reduction of control!
• Automation: Tools to automate standard processes
– Requires standard processes/methodology.
– Usually a variety of functions, interfaces, protocols, and
transformations must be verified.
– Not all (verification) processes can be automated.
– Randomization can be used as an automation tool
2007
Tehran University
19
Reconvergence Model
• Conceptual representation of the verification
process: Illustrate what is exactly being verified
Transformation
Verification
Purpose of verification is to ensure that the result of some transformation
is as intended or as expected.
2007
Tehran University
20
Source of ambiguity/error
• Human Factor:
– How a designer read and interpret the specification
– Specification can be ambiguous
– Designer can introduce errors by (mis-)interpretation
(re-convergence model)
DANGER: When a designer verifies her/his own
design - then she/he is verifying her/his own
interpretation of the design - not the specification!
2007
Tehran University
21
How to Reduce Human Errors
Reduce human errors by:
• Automation
– Take the human out but not really feasible
– Process not well defined enough, not formal enough
• Make mistake-proof the human intervention by
reducing it to simple steps (Poka-Yoka)
• Redundancy
– Two individuals check each other’s work
– Costly mechanism but still cheaper than re-design and chip
recall
But never let a designer verify his own design
2007
Tehran University
22
What is being verified?
• Choosing a common origin and the reconvergence
points determines what is being verified.
– Determined by the tool used to perform the verification
• The following approaches have different origin and
reconvergence points:
– Formal Verification
– Functional Verification
– Testbench Generators
2007
Tehran University
23
Formal Verification
This is not a tool that mathematically determines the
correctness of a design without having to write
testbenches!
Build a formal mathematical model of some aspect of the
system and calculate whether or not it has some desired
property
• 2 types of Formal Verification
– Equivalence Checking
– Model Checking
2007
Tehran University
24
Equivalence Checking
• Compares two models to check for equivalence
– Mathematically proves that both are logically equivalent
• Commonly used on lower level of design
Synthesis
RTL
Gate
Equivalence Checking
• Make sure that scan chain, clock tree insertion did not
change the functionality
• It is only interested in comparing boolean and
sequential logic functions – no tech mapping
• Why can’t we entirely trust the synthesis tool?
2007
Tehran University
25
Model Checking
• Assertions or characteristics of a design are formally
proven or disproved
• Use to check for generic problems or violations of
user-defined properties of the behavior of the design
• Usually employed at higher levels of design process
• Properties are derived from specification
• Properties are expressed in some (temporal) logic
• Checking often involves a (finite) state machine model
2007
Tehran University
26
Model Checking
Problems:
• Knowing which properties to check and how to
express them needs expertise.
– Very few designers are also good mathematicians.
– (Verification engineers should be closing this gap.)
• Very few user-friendly tools for model checking.
• Tools that do exist can only check small designs.
• There is a lot of research to make Model Checking
more practically usable. For an introduction see
”Model Checking” by Clarke, Grumberg and Peled
2007
Tehran University
27
Functional Verification
• Ensure that a design implements intended
functionality
• Without it, one must trust that the
transformation of a specification into RTL code
was performed correctly
• We can only prove the presence of bugs, but we
cannot prove their absence!
2007
Tehran University
28
Testbench Generators
• Tools to generate stimuli to exercise code or expose bugs
• Designer/Verification engineer input is still required.
• RTL code is the origin and there is no reconvergence point
Code coverage/proof
Metrics
RTL
Testbench
Testbench Generation
• Verification engineer has responsibility to determine if the testbench applies
valid stimuli
• If used with parameters, one can control the generator in order to focus the
testbench on more specific scenarios
• 2 input ”and” gate - ... combinations of inputs (exhaustive)
• Instruction stream generators to verify processor cores.
Is exhaustive verification (always) feasible?
2007
Tehran University
29
Functional Verification Approaches
There are 3 complementary approaches:
• Black-Box Verification
• White-Box Verification
• Grey-Box Verification
2007
Tehran University
30
Black Box Verification
• The black box has inputs, outputs, and performs some function.
• The function may be well documented (or not ...).
• To verify a black box, you need to understand the function and
be able to predict the outputs based on the inputs.
• The black box can be a full system, a chip, a unit of a chip, a
module, etc.
• No knowledge of the actual implementation is required
• Independent of implementation.
• Lacks controllability and observability.
• Difficult to locate source of problem, only exposes effects
• Example: round-robin arbiter
2007
Tehran University
31
White Box Verification
• Opposite of Black Box
• Full visibility and controllability of the internal
structure and implementation of DUV
• Easy to set up interesting conditions
• But testbench is tied to a specific implementation –
needs to be changed if design modified
• Best for module or system level verification
• Example: counter roll-over
2007
Tehran University
32
Grey Box Verification
• Compromise between BB and WB
• Like BB, it observes and controls top level interface
signaling
• But increases coverage metrics by adding some nonfunctional modification to increase visibility
– Include additional software-accessible registers to observe
internal states
• In practice for SoC most environments are GB because
prediction of results are sometimes impossible without
looking at internal signals
2007
Tehran University
33
Perfect Verification
• To fully verify a black box, you must show that
the logic works correctly for all combinations
of inputs.
This means:
• Driving all permutations on the input lines.
• Checking for proper results in all cases.
• Full verification is not practical on large
designs, but the principles are valid across all
verification approaches
2007
Tehran University
34
Verification versus Testing
• Often Confused:
– Test: show the design was manufactured correctly
(post-silicon) – verifies internal nodes can be toggled
by applying test vectors
– Verification: Ensure the design meets its functional
intent (pre-silicon)
HW Design
Spec
Manufacturing
Gates
HDL
Testbench
2007
Synthesis
Equivalence Checking
Tehran University
Silicon
Testing
35
Verification versus Testing (Cont.)
• One test method is scanning: Design For Test
– Links all registers into a long shift register (chain)
– Chain accessible from the I/O pins, observe and control
internal states
• Scan-based testing restricts the design (sync, no gated
clock, single clock) but keeps cost low (reference book:
Digital System Testing and Testable Design,
Abramovici)
Research topic: “Design for Verification”!
Consider what the design is supposed to do? How will it
be verified?
2007
Tehran University
36
Design and Verification Reuse
• You won’t use what you don’t trust.
• How to trust it? (Verify it)
• For reuse, designs must be verified with more strict
requirements
– All claims, possible combinations and uses must be verified.
– Not just how the design is (intended to be) used in one
environment
• Verification/testbenches re-usable to some degrees
– BFMs have different requirements at block or system level
– From project to project
2007
Tehran University
37
Cost of Verification
• It is always too long and costs too much but is
absolutely necessary
– It does not generate revenue!
– As number of bugs decreases, the cost and time for finding
the remaining ones increases: When is the verification done?
• Yet indispensable:
– To create revenue, design must be functionally correct and
provide benefits to customer.
– Proper functional verification demonstrates trustworthiness
of the design.
– Right-first-time designs demonstrate professionalism and
”increase” reputation of design team
2007
Tehran University
38
When Is Verification Done?
Never truly done on complex designs.
• Remember: Verification can only show presence of errors, not
their absence.
• Skill and experience are needed to:
– Determine WHAT needs to be verified.
• Be selective/exhaustive.
• Identify corner cases.
•
•
Given enough time, errors will be uncovered.
Critical question:
– Is the error likely to be severe enough to warrant the effort spent to
find/correct it?
Verification is similar to statistical hypothesis testing.
• Hypothesis ”under test” is: Is the design functionally correct?
2007
Tehran University
39
Hypothesis Matrix
Fail
Pass
Type II
(False Positive)
Bad Design
Good design
Type I
(False Negative)
• Type I mistakes: Easy to identify - found error where none
exists.
• Type II mistakes: Most serious - verification failed to identify
an error!
– Can result in a bad design being shipped unknowingly!
• Knowing where you are in the verification process is much easier
to estimate than how long it will take to complete the job.
2007
Tehran University
40
Summary
• Seen what a testbench is
• Understood how important verification is and why it is so
important.
• Seen that parallelism, automation and abstraction are strategies
to reduce the time necessary to implement testbenches.
• Used reconvergence models to help us identify what is
verified.
• Been introduced to the various verification approaches that
exist.
• Investigated the importance of verification for design reuse.
• Discussed the cost of verification
• Next: Verification tools, especially simulators, waveform viewers
and simulation languages (Verilog)
2007
Tehran University
41