Integrating Ethernet CMS with Internal
Verification Environments
Cadence Verification Challenge, Bristol,
Mike Bartley, TVS
Test and Verification Solutions
21st April 2010
1
Overview
• Background
– The DUT
– The verification environments
• Experiences
– Of integrating verification environments and results
– Of OVM scoreboard
• Results
Test and Verification Solutions
21st April 2010
13 April 2015
2
The DUT: Blocks from Gnodal Ethernet chip
MAC
PCS
Media
Access
Control
Sublayer
Physical
Coding
Sublayer
Data Link
Layer
(Layer 2)
PHY
Layer
(Layer 1)
Blocks from an Ethernet chip from Gnodal
Test and Verification Solutions
21st April 2010
13 April 2015
3
Internal block level verification flow
• Use of PSL + IFV
• Test bench
–
–
–
–
SystemC (sequencers and transactors)
PSL assertions in the design
SystemC coverage and assertion checks
SV covergroups and assertions
• There are numerous simulator tools/languages
– It makes sense to pick the right tool for each job
– For example
• PSL/SVA and SV covergroups are great for checking internal
stuff in the guts of the design, in an unobtrusive way
• Whilst e and SC are good for the higher-level test control…
– Need to consider re-use too (Gnodal had SystemC chip test bench)
Test and Verification Solutions
21st April 2010
13 April 2015
4
Description of the internal test bench
stimulus
KEY
irun
transactor
SystemC
Scoreboard
Verilog
CPU
Coverage
System
Verilog
PSL
Coverage
PCS
Assertions
Assertion
PSL
checks
Loopback
- Injects clock and lane skew
Test and Verification Solutions
21st April 2010
13 April 2015
IFV
5
Description of the internal test bench
irun
KEY
stimulus
transactor
SystemC
Coverage
Coverage
Verilog
Scoreboard CPU
System
Verilog
MAC
Assertion
checks
PSL
Assertions
IFV
PSL
stimulus
Test and Verification Solutions
transactor
21st April 2010
13 April 2015
6
Description of the CMS test bench
KEY
eVC
incl coverage
& assertions
SystemC
Verilog
irun
OVM e
Scoreboard
PCS
CPU
System
Verilog
Assertions
Coverage
e
Test and Verification Solutions
Coverage
Assertion
checks
MAC
Loopback
- Flow control
21st April 2010
13 April 2015
7
Experiences of using the Ethernet eVC/CMS
• Connecting the eVC
– Quick and easy
• Extensive assertions related to IEEE specification
• Extensive coverage targets related to IEEE spec
– Comprehensive and easy to adapt to our needs (using VManager)
• Tests
– Obtained reasonably good coverage out of the box
– Easy to add additional tests
• Injecting errors where we wanted
– Layered approach – but we could only see how to inject errors
after packet construction
– This made it hard to cover all the state machines which required
error injection at specific times/ in specific sequences
Test and Verification Solutions
21st April 2010
13 April 2015
8
OVM e scoreboard : Instance and connect
1. Define new scoreboard type and declare its ports
unit vr_enet_scbd like ovm_scoreboard {
scbd_port in_packet: add vr_enet_packet;
scbd_port out_packet: match vr_enet_packet;
};
2. Connect it to the environment monitor/s
extend vr_enet_env {
scbd : ACTIVE vr_enet_scbd is instance;
connect_ports() is also {
do_bind …..
}
};
Test and Verification Solutions
21st April 2010
13 April 2015
9
OVM scoreboard : Add and match
3. Customize the packet add and match criteria
unit vr_enet_scbd like ovm_scoreboard {
in_packet_predict (pkt : vr_enet_packet) is only {
-- Add as UNCERTAIN;
};
out_packet_reconstruct (pkt : vr_enet_packet) is only {
-- Perform checks, match in scoreboard
};
extend vr_enet_env {
// add tcm to monitor inputs and update packet status
// from UNCERTAIN to PENDING if appropriate
};
Test and Verification Solutions
21st April 2010
13 April 2015
10
Experiences of using the OVM e scoreboard
• Easy to import and adapt
• Predicting which packets get dropped is hard
– Ethernet can silently drop packets!
– Difficult to align with packet boundaries from eVC
– The DUT has windows of time for errors
• We may or may not lose the link
• We therefore slackened off the scoreboard
– With the risk of missing errors
Test and Verification Solutions
21st April 2010
13 April 2015
11
Integrating verification environments in VManager
CMS
vsif file
Internal
vsif file
Apply
internal
perspective
Merge
vsif
files
Target
vsif file
Test and Verification Solutions
21st April 2010
13 April 2015
12
Integrating verification results using VManager
CMS
vsof files
Internal
vsof files
Merge
Single
vsof file
Test and Verification Solutions
21st April 2010
13 April 2015
13
Results
Setup
Run tests
Add
checks
Review coverage
and add tests
Signoff
coverage
holes
Effort
25%
5%
35%
25%
10%
Time
10%
40%
15%
30%
5%
Cvge
State
ment
Expre
ssion
Toggle CMS
Gnodal
functional functional
CMS
88%
73%
40%
92%
28%
Gnodal
100%
95%
91%
NA
76%
Combined
100%
95%
91%
92%
76%
• Mutation
Coverage low
because MAC
in static cfg
Coverage low
because did
not merge in
MAC cvge
– The designer inserted a number of bugs in the design
– The test bench found them all
Test and Verification Solutions
21st April 2010
13 April 2015
14
Conclusions
• The eVC was easy to install and connect
• Comprehensive assertions related to IEEE spec
• The CMS functional coverage targets
– Comprehensive and easy to adapt for our perspective
– Easy to integrate with our coverage targets (VManager)
• The CMS tests
– good coverage and were easy to extend
• Ethernet scoreboarding is hard to predict
• Easy to merge CMS and internal coverage
– Merge functional and code coverage
Test and Verification Solutions
21st April 2010
13 April 2015
15