Integration Verification: Re-Create or Re-Use?
Nick Gatherer
Trident Digital Systems
1
Introduction
Nick Gatherer
SoC Design Manager
Trident Digital Systems
(Formerly BU TV, NXP Semiconductors)
In former NXP role …
• Chair of NXP’s Advanced Functional Verification Working
Group
- NXP’s cross-business expert team on verification
• NXP Business Renewal project manager for System Level
Design & Verification
2
Agenda
Introduction & Verification Landscape
Focus on Integration Verification
Challenges
What does it cost ?
The IP  Integrator Interface
Example verification re-use methodology
Conclusions
Q&A
3
Trident – SoCs for DTV & STB (high complexity !)
• PNX85500
• Industry’s first digital TV processor
in 45nm CMOS
• Most complex SoC ever developed
by NXP/Trident
• Required aggressive TtM & Right First Time
• From packaged parts received to System
bring up in less than 10 days
• Very high level of functionality and performance
• PNX847x/8x/9x
4
• World’s first fully integrated 45nm
Set Top Box SoC platform
• Architectural & IP re-use from PNX85500
(internal & 3rd party IP)
Functional Verification – A Holistic Approach
System validation
HW/SW integration &
co-verification
SW development
HW Integration
Hardware IP
Virtual
Prototype
RTL
Simulation
FPGA
Prototype
Emulation
Verification Technology Landscape
• Select appropriate platform per project phase
• Availability vs Accuracy vs Performance vs Debuggability
• Allocate coverage goals to specific platforms
5
Silicon
Functional Verification – Re-Use Opportunities
System validation
Re-use across verification tasks/abstractions
HW/SW integration &
co-verification
Re-use across projects
SW development
HW Integration
Re-use across verification platforms
Hardware IP
Virtual
Prototype
RTL
Simulation
FPGA
Prototype
Emulation
Silicon
Verification Technology Landscape
• Verification re-use depends on stakeholder alignment
• IP suppliers, IP integrators, verification teams, technology/standards
6
Agenda
Introduction & Verification Landscape
Focus on Integration Verification
Challenges
What does it cost ?
The IP  Integrator Interface
Example verification re-use methodology
Conclusions
Q&A
7
Integration Verification – Challenges
• HW integration verification
• Focus on (structural) integration integrity
• Interconnect & interoperability
• Typical Challenges
• Integrator has limited knowledge of IP implementation
• IP supplier has limited knowledge of target system
• Use of multiple IP suppliers results in inconsistent IP verification views
• Methodology, testbenches, coverage data, etc. Industry standards late.
• Mechanisms for accommodating IP configuration
• Usually need to port IP tests to SoC level / environment in order to re-use
• Huge effort ! What sub-set is needed for integration verification ?
• Awareness and impact of IP implementation changes & known problems
• Concurrent IP dev and SoC integration demands incremental maturity
• Difficult to debug complex interactions between IPs (embedded checks help)
8
• How much coverage is sufficient for integration confidence ?
Agenda
Introduction & Verification Landscape
Focus on Integration Verification
Challenges
What does it cost ?
The IP  Integrator Interface
Example verification re-use methodology
Conclusions
Q&A
9
Integration Verification – What does it cost per IP ?
• Example project (from subsequent data mining) …
Integration Test
IP with no external I/O, IP with external I/O,
no DMA, no master IF DMA, but no master IF
[weeks]
[weeks]
IP with external I/O,
DMA and bus master IF
[weeks]
Register Access
Clock & Reset
Connectivity
Interrupt Connectivity
Stub Integration
I/O Connectivity
DMA Connectivity
Communication
Stub/CPU
Bus Master Interface
Total
0.5 – 1.0
0.5 – 1.0
0.5 – 1.0
0.5 – 1.0
0.5 – 1.0
0.5 – 1.0
0.5 – 1.5
n.a.
n.a.
n.a.
n.a.
0.5 – 1.5
1.0 – 4.0
0.5 – 1.5
1.0 – 2.0
n.a.
0.5 – 1.5
1.0 – 4.0
0.5 – 1.5
1.0 – 2.0
1.0 – 4.0
n.a.
1.5 – 3.5
n.a.
4.0 – 11.0
1.0 – 3.0
6.0 – 18.0
Effort for integration verification is 50%-75% of the total
SoC front end work  Re-use is a must!!!
10
Integration Verification – What does it cost per SoC ?
 For a medium complexity SoC with around 50 IPs …
Integration verification effort is 100 – 250 man weeks
− Effort can vary a lot depending on how much integration verification can be
re-used from previous projects
− Effort can vary a lot depending on what is delivered by the IP provider
(verification components)
Example project plan (based on 50% IP verification re-use)
Total effort reduced by 103 man weeks (approx 2 man years !)
 Significant improvement potential on the IP provider side
 Design for integration verification
 Significant improvement potential on the SoC integrator side
 Align on a common methodology
11
Agenda
Introduction & Verification Landscape
Focus on Integration Verification
Challenges
What does it cost ?
The IP  Integrator Interface
Example verification re-use methodology
Conclusions
Q&A
12
IP  Integrator Interface for verification
HW Integration
(views delivered by
IP suppliers that
support integration)
IP Datasheet (functionality, interfaces, etc) & Maturity index
Verification report …
IP verification strategy/plan & results (RTL, N/L, FPGA)
Code & functional coverage
Checklist/maturity, limitations, waivers, and open issues
Simulation vectors
Full & reduced set
for integration
Emulation models as needed
to support mapping
Simulation models as
needed to support testbench
XML register description to
support test creation
BFMs, eVC’s, OVC’s
Interface & protocol
assertion checkers
IP specification & integration reviews (ideally face-2-face)
Joint silicon bring-up & validation
RTL Simulation
13
Emulation
Agenda
Introduction & Verification Landscape
Focus on Integration Verification
Challenges
What does it cost ?
The IP  Integrator Interface
Example verification re-use methodology
Conclusions
Q&A
14
Verification SW – Methodology Overview
• Objective:
• Enable IP suppliers to
deliver IP integration tests
SoC
CPU
INTC
DMAC MEMC
mem
• Implementation
• Software-centric verification approach
IP
• ‘C’ tests run on any embedded processor
(or transactor)
• Non-invasive and runs on all implementation levels
1
IP2
IPn
IPx
• SystemC, RTL, netlist, FPGA, emulation, silicon
• API defined to ensure standard methodology
• Allows IP supplier to define:
• In-context connectivity verification test
• Register tests can be generated automatically from IP-XACT metadata
• Inter-operability test with generic platform functions
• interrupt, DMA, clock, reset, power
• IP context information is applied at the SoC integration phase
• Allows automatic test program generation from suitable integration environment
15
Verification SW – Re-Use Example
IO-Stub
IO-Stub
IOs
SoC
IOs
CPU
Verification
SW
IP Supplier
IP
DMA
INTC
Bus Interface
IP
Bus Interface
Logic
CGU
CPU
Verification
SW
SoC Integrator
(Chip context applied)
 IP supplier delivers STANDARDIZED verification components
 Plug and Play Integration Verification
16
Agenda
Introduction & Verification Landscape
Focus on Integration Verification
Challenges
What does it cost ?
The IP  Integrator Interface
Example verification re-use methodology
Conclusions
Q&A
17
Conclusions
• Verification continues to consume significant time & effort for
complex SoCs
• Design re-use is now well established
• Verification re-use needs to catch up !
• IP is usually delivered with support for integration verification, but
lacks consistency between suppliers
• There’s a big difference between ‘re-usable’ and ‘almost re-usable’ !
• Proprietary approaches have emerged to overcome lack of industry
standards, but these are not robust against industry dynamics
• Overall ….
• Design IP is a MUST
• IP support for integration verification is a MUST
• Need more standardisation of approach
18
Agenda
Introduction & Verification Landscape
Focus on Integration Verification
Challenges
What does it cost ?
The IP  Integrator Interface
Example verification re-use methodology
Conclusions
Q&A
19