At the core of the user experience.®
On Chip Instrumentation
as a Verification Tool
FMCAD - Pre and Post Silicon Verification
in the Industry Workshop
Nov. 2006
First Silicon Solutions Division
of MIPS Technologies Inc.
Rick Leatherman rickl@fs2.com
Neal Stollon neals@fs2.com
MIPS Technologies copyright 2006 – All rights reserved
1
At the core of the user experience.®

MIPS Technologies is Leader in High Performance
Embedded RISC Processors
TM
®
 MIPS32 34K Multi-threaded RISC Processor Core
 Full range of 32 and 64 bit processor IP based on
common MIPS ISA

FS2 is Instrumentation IP and Tools Division of MIPS
®
 On-Chip Instrumentation (OCI ) Leadership
 Integrated Trace and Analysis Solutions for
Embedded Processor, On- Chip Bus and Logic

Processor and Bus OCI and tools for SoC systems




Eclipse Based Tools Support
Extended performance analysis advanced features
High performance Data trace port options
Tighter integration to EDA flows
MIPS Technologies copyright 2006 – All rights reserved
2
At the core of the user experience.®
The System Analysis Issue
• Methodology migrates from hardware to software bugs
• Need to comprehend the software needs at the hardware stages
• Customers and venders starting to realize
the job does not stop at the dotted line
Focus on Hardware Bugs
RTOS
Integration
System
Analysis
Focus
System
Initialization
RTL
Diagnostics
Instruction
Level/
Bus
Functional
Copyright © 2003 Novas Software, Inc.
Focus on Software Bugs
ESL
Application
Software
Multi-core
Integration
issues
Point were Hardware is
“assumed working”
Hardware
Hardware
Software Debugger
Simulation Prototype/Emulation System platform
Verification Abstraction
MIPS Technologies copyright 2006 – All rights reserved
3
At the core of the user experience.®
Highly Integrated Designs
Require System Level Debug Tools

Test based solutions (Scan, BIST) are not interactive, do not
provide cause and effect information

Processor centric debugging (i.e. Instruction Trace) only solves
part of the problem

Frequently chip bring-up problems are bus related

Saturation, latch-up, etc.

Performance optimization is interaction of cores, buses, IP

Processor debug tools are of limited help

On-chip Bus Analyzers and Multicore Embedded Debug Tools
Needed to Provide System Level Debug Features
MIPS Technologies copyright 2006 – All rights reserved
4
At the core of the user experience.®
Emerging SoC Debug and Analysis Issues

Debug and trace is crucial to embedded design success
 Difficult to fix what you can not see
 Unexpected on-chip Inter-relationships not always intuitive

System on Silicon Debug Requirements
 Visibility into non-observable sub-system interfaces
 On-chip debug analysis complements EDA verification
 Interoperability with processor debug capabilities
 Cross-triggering, synchronization for full view of problems

Performance Analysis on buses, caches, execution cores, coprocessors, interrupts, peripherals, . .

Debug IO requirements
 Leverage existing interfaces ie. JTAG
 Additional options to provide more analysis bandwidth
MIPS Technologies copyright 2006 – All rights reserved
5
At the core of the user experience.®
Multicore Debug Is Becoming Prevalent
Range of On-Chip Instrumentation IP and tools



Customizable for specific customer requirements


Multi-core cross triggering and synchronous control and
actions (go, halt and breaks)
Multi-core trace with wide ranges of features
measure activity on buses, caches, execution cores, coprocessors, interrupts, peripheral device events, . . .
Probes designed for trace and system debug




Merging debug for several cores over single probe
Probe Hardware and tool API’s to support multi-core trace
Simple Control/data transfer via a single JTAG TAP
Support for multiple instantiations of source level debuggers
MIPS Technologies copyright 2006 – All rights reserved
6
At the core of the user experience.®
IP Complaint Solutions to SoC Analysis
IP
Centric
Products
SoC
Synergies
Instrumentation
Centric
Analysis
IP
Synergies
On Chip
IP
Processor
Cores
Enablement
Bus
IP
Prototyping
HW
Customer
Specific
IP
SW Dev.
tools
Bus
Level
Trace
Multi
Core
Debug
Probes
Infrastructure Synergies
MIPS Technologies copyright 2006 – All rights reserved
SoC
Performance
Analysis
System
Debug
tools
7
At the core of the user experience.®
Plug and Play System Debug
Debug Support for Systems with Multiple Processors, Buses and Other IP Provides Complete Visibility Into Highly Integrated Designs
Typical solutions include:

Processor core + on chip bus analyzer
 Support for industry bus standards (AMBA, OCP)
 Custom solutions for proprietary buses
 Supports cross triggering between the bus analyzer and core

Cores from different IP venders
 Support for proprietary cores frequently required
 Support synchronous go/halt/step & cross triggering between cores
 On-chip Logic Analyzers for “other” peripherals, accelerators, etc.

Multiple cores + on chip bus analyzer
 Cross triggering, timestamping, synchronization
 System level event monitoring and debug control
 Trigger between cores and bus analyzer
MIPS Technologies copyright 2006 – All rights reserved
8
At the core of the user experience.®
MultiCore vs. Processor Debug
Multicore Debug addresses different issues for processor debug
Both are aspects of the same problem System Analysis and Optimization
 Communications vs. Computation
 Architecture Optimization vs. Algorithm Optimization
 Complementary- Both are required for full system view
 Equally important – priorities at different stages of debug.
Multicore
Debug
MANY
FACTORS
TO
CONSIDER
Processor
Debug
MIPS Technologies copyright 2006 – All rights reserved
9
At the core of the user experience.®
Platform Based System Design
Typical Features
 Diverse bus fabric options



multiple processors
other blocks of IP
Differing clock domains
IO constrained


(security, video,
imaging . . . )
OCP/AMBA
More than one core




lots of data transfers
Differing bus interfaces
Other IP blocks
MIPS32®
34KTM
System Bus Fabric
OCP/AMBA
Other cores
Embedded operations of interest
not accessible by outside world
Problems show up in real time

Mem
Cntl
RAM
hard to emulate or reproduce
SO HOW CAN WE ADDRESS SoC VERIFICATION BETTER ?!?
MIPS Technologies copyright 2006 – All rights reserved
10
At the core of the user experience.®
MIPS Analyzer GUI
Integrated System Level Debug
System Debug Support for Multiple Processors,Buses
and Other IP - Visibility Into Highly Integrated Designs
 Single link (System Navigator probe) to target SoC for
system debug control, monitoring trace, cross triggers, …
 Access to all EJTAG and PDTrace debug facilities
 Bus Navigator, Logic Navigator, HyperDebug Instruments
JTAG Chain
System
Navigator
Probe
MIPS
EJTAG
PDTrace
24K/34KTMCore
Logic (IP)
Analyzer
(security,
HyperDebug
Cross Trigger
Debug Control
OCP
OCP / AHB
Custom
Bus Analyzer
Bus Navigator GUI
Other IP
video,
imaging . . . )
MIPS32TM
System Bus Fabric
Other MIPS
Other cores
EJTAG
Other FS2 Analyzers
Memory
Controller
HyperDebug
Cross trigger Chain
MIPS Technologies copyright 2006 – All rights reserved
11
At the core of the user experience.®
Starting Point is Processor Debug
MIPS EJTAG - is a leading example
 run control for each core - start, stop, breakpoint, single-step
 Hardware and software breakpoints
PDTrace adds processor trace capability

Real time Instruction and data Trace - up to 64 bits per cycle

Trigger on execution, RAM operations, address or opcode values

Execution trace compression (Branch Trace Messages)

Trace funneling for support of multiple MIPS core instances

GDB and commercial Debugger support
MIPS Technologies copyright 2006 – All rights reserved
12
At the core of the user experience.®
Debug Software for Multi-Core Platforms
• Standards based debug IDE - Plug and play with 3rd party tools
• User transparent concurrent debug access (many cores/many debuggers)
• Standard based Eclipse, Tcl/TK, MDI, XML, text based scripting
MIPS Debugger
GUI
Tcl/TK
Command Line
Interface
Trace/Trigger
GUI
MDI
MIPS Debugger
GUI
MDI
EJTAG API
Nav. API
EJTAG API
Other
Debug GUI
MDI
Other API
Multicore ABI - Probe Drivers
USB, Ethernet
FS2 System
Navigator Probe
Off-Chip
TRACE PORT
OR
JTAG IO
On-Chip
34K
Other MIPS
EJTAG/
PDTrace
EJTAG/ PDTrace
Bus
Trace
OCI
Cross
Trigger
OCI
Other
OCI
Bus Fabric
MIPS Technologies copyright 2006 – All rights reserved
13
At the core of the user experience.®
On Chip Vs. Off Chip Instrumentation Solutions
On Chip (JTAG) trace
• Buffers signal trace on chip
• Requires on chip RAM
• Uses JTAG port for IO
•Simpler, cheaper Sys Nav probe
• All logic analyzer functions on chip
- several 10K gates on-chip
• Trace is function of on chip RAM
For more options with no extra debug
IO, On Chip approach is right choice.
Off Chip (Streaming) trace
• Streams data through parallel port
• Limitation is IO bandwidth
• Simpler OCI - Less logic on chip
• Logic analyzer is in probe
- Trace RAM & trigger logic is in probe
For very deep trace or reduced on-chip
debug gates, Off Chip is right choice.
OCI 1
Bus
Trace
RAM
OCI 2
Perf
Anal.
RAM
JTAG
JTAG
Probe
4 pins
OCI 3
PDTrace
OCI 1
Bus
Trace
OCI 2
Perf
Anal.
RAM
DEBUG
FUNNEL
1-64
pins
Streaming
Trace
Probe
OCI 3
PDTrace
MIPS Technologies copyright 2006 – All rights reserved
14
At the core of the user experience.®
Bus Navigator OCI IP Overview
PDTrace
----------EJTAG
Other IP blocks
MIPS32®
24KTM
(security, video,
BUS NAVIGATORTM OCI
Performance Time
stamp
counters
imaging . . . )
Trigger/trace
control
OCP/AMBA
System Backplane
JTAG IF
Trace Port
Trace
RAM
MIPS32®
34KTM
PDTrace
----------EJTAG
Mem
Cntl
Cross triggers





Wide trace capability (256 signals)
to support for >1 bus channel
Programmable event monitoring
Configurable triggers, counters
Trace filtering and alignment
Trace limited by RAM size or Port Bandwidth
MIPS Technologies copyright 2006 – All rights reserved
15
At the core of the user experience.®
Bus Navigator Trace GUI
Complete Visibility into Your SOC Design
Waveform
Display
State
Display
Timestamps and Complex Event Triggers
MIPS Technologies copyright 2006 – All rights reserved
16
At the core of the user experience.®
Bus Navigator Integrated with Processor Debug
• Processor operations can drive system debug operations and vice versa
• Example: Cross-Triggering Bus Trace with MIPS Source Code
MIPS Technologies copyright 2006 – All rights reserved
17
At the core of the user experience.®
Case Study : Mobileye EyeQ2 Architecture

Complex automotive image processor IC





Debug Performance and bottlenecks
 Bus latency and throughput analysis






10 cores (two MIPS 34K, DSPs, DMA)
Complex OCP/AMBA system network
Multi-layer Sonics SMX
Complex memory subsystems
Trace concurrent bus transactions
Trace full address for any transaction
Correlate Bus transactions to processor
Instructions, load/store operations
Track active threads to bus operations
Trace at least one image frame (>1 Gbyte)
Low overhead debug solution (minimal buffering)
MIPS Technologies copyright 2006 – All rights reserved
18
At the core of the user experience.®
Instrumentation Solutions for Mobileye (RRT)


Request-Response Trace – Real time Bus Transaction Analysis
Solution built around FS2 System Navigator Pro Probe

64 pin target interface


Deep (2 Gbyte) Probe Trace Buffer




Complex triggers, timestamps, . . .
Minimized on chip Logic
Complex trace resources in probe
Multicore PDtrace Funnel IP
PDtrace/RRT Correlation in SW



Request-Response delay analysis
PDtrace/RRT Bus Correlation
Inferred thread to bus analysis
RRT Agent
(capture,
filter,
format)
Multicore Mictor 1
PDtrace
Funnel/
Port
JTAG
port
FS2
System
Navigator
Pro
Probe
RRT Agent
Upgraded PDtrace for multicore trace


MIPS 1
(34K 2x
OCP)
Multi-frame trace
RRT agents and funnel IP


MIPS 0
(34K 2x
OCP)
Allows processor and bus trace
VCE
(XB1
OCP)
DMA
(AHB)
RRT Trace Funnel
(combiner/
Scheduler)
RRT
Trace
Port
Mictor 2
RRT Agent
RRT Agent
MIPS Technologies copyright 2006 – All rights reserved
19
At the core of the user experience.®
System Level Performance Analysis

Types of analysis that Navigator supports






Measure activity - bus utilization, caches efficiency,
coprocessors activity, interrupts, peripheral device events
Measure latencies - interrupts, bus access, DMA transfers
Loop times of processing network packets, DSP Blocks
Monitoring bus bandwidth utilization, efficiency
Caches hit/miss ratios, DRAM pages, processor stalls
FS2 Navigator enables Analysis of System operations

Free running timestamps allow interval measurements
relative to real time

Counter and trigger resources allow real time event rates or
duration measures
MIPS Technologies copyright 2006 – All rights reserved
20
At the core of the user experience.®
So. What are open issues

Standardization

Most current solutions are proprietary.

There are standardization efforts ongoing - next slide

More end user and academic support will be needed

Better Methods

Biggest bottleneck is trace bandwidth and size

New, better trace compression is needed for bigger SoC

Design for Debug Awareness

Too often, problems are not understood until post silicon

More formal methodology for addressing instrumentation needs

EDA integration

Post silicon analysis and pre-silicon analysis are handled differently

Open areas in addressing common tools and interoperability
MIPS Technologies copyright 2006 – All rights reserved
21
At the core of the user experience.®
Some Post Silicon Verification Related
Standards efforts

Nexus Forum – IEEE 5001

Nexus Addresses higher performance processor debug interfaces

Affiliate membership is available to Academic groups

OCP-IP Debug Working Group

OCP is open bus protocol for complex SoC designs

Debug group is working on standardized methods for SoC
Instrumentation

SPIRIT Debug Working Group

SPIRIT is vender neutral format for passing IP data to EDA tools

Debug working group is working on standardized extensions for
Instrumentation and debug SW#

MultiCore Association Debug Working Group

Addressing API level interfaces for Multicore debug
MIPS Technologies copyright 2006 – All rights reserved
22
At the core of the user experience.®
FS2 Solutions for MIPS Systems

FS2 offers sophisticated debug tools for SoC systems.

System Navigator probe concurrently supports range of System OCI
options for SoC debug and analysis

Trace Support for all MIPS cores
 True Plug-and-Play integration with leading MIPS Debuggers

Bus (OCP/AMBA/Custom) Navigator debug/trace solutions
 Customizable for specific customer requirements

Supported by FS2 Cross triggering, and analysis software
www.fs2.com gives you the details

System Navigator Pro (deep trace) probe expends options

Allows smaller on-chip component (more debug pins)

System On Chip Instrumentation for is available now
MIPS Technologies copyright 2006 – All rights reserved
23
At the core of the user experience.®
FS2 Comprehensive
Debug Solutions
Software Tools and Interfaces
• Integrated with GDB/Insight
• New Eclipse IDE
• Supports 3rd party debuggers via MDI
• RTOS awareness
Embedded Instrumentation IP
• EJTAG, PDTrace, iFLowtrace
• Complex Triggers
• Integrated Bus Analyzers
• Multi-Core debug
• Performance Analysis
• Application specific debug blocks
System Navigator Probes
• USB 2.0 and Ethernet host
PC connections
• 14-pin (EJTAG & on-chip PDTrace)
or 38-pin Mictor (EJTAG & off-chip
• PDTrace) target connections
MIPS Technologies copyright 2006 – All rights reserved
24
At the core of the user experience.®
Eclipse Gaining Wide Acceptance

Eclipse IDE and CDT Debugger
 Eclipse is an modern IDE gaining wide acceptance in
the EDA and tools marketplace
 CDT (or C/C++ Debug Tools) is the native Eclipse
debugger for C/C++ code
 Eclipse “Plug-Ins” are built on well defined API’s
allowing third parties to enhance the debug
experience with their specific tools
 Create “views” for application specific solutions
Eclipse is an integrated development environment (IDE) that
combines a code editor, compiler, debugger, text editor, graphical user
interface (GUI) builder, and other components into a single, user-friendly application
MIPS Technologies copyright 2006 – All rights reserved
25
At the core of the user experience.®
View all thread
information after hitting
breakpoint, including
name, running state,
priority, etc
If thread suspended,
shows responsible
kernel component(s)
causing suspension
“cpu context” view
added to denote
execution in nonthread or ISR context
View of ThreadX
objects – TIMER,
QUEUE,
SEMAPHORE,
MUTEX,
BYTE_POOL
MIPS Technologies copyright 2006 – All rights reserved
26