Massachusetts Institute of Technology
Kavli Institute for Astrophysics and Space
Research (MKI)
FPE/DHU FPGA Firmware Test Plan
Dwg. No. 37-32003
Revision A
Date 03/23/2015
37-32003
Page 1 of 17
Revision A
Table of Contents
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
2
INTRODUCTION
4
PURPOSE
SCOPE
ROLES AND RESPONSIBILITIES
TOOLS
DESIGN/SIMULATION WORKFLOW
TEST LEVELS SUMMARY
TBD/TBR
4
4
4
4
4
5
5
DOCUMENTS AND DEFINITIONS
6
2.1 APPLICABLE DOCUMENTS
2.2 ACRONYMS
3
6
6
TEST LEVELS
7
3.1 FPE FIRMWARE SIMULATION
3.1.1 FPE TEST BENCH
3.1.2 FPE TEST CASES
3.2 DHU FIRMWARE SIMULATION
3.2.1 DHU TEST BENCH
3.2.2 DHU TEST CASES
3.3 FPE STAND-ALONE LAB TESTING
3.3.1 FPE LAB TEST CASES
3.3.2 FPE LAB TEST FLOW
3.4 DHU LAB TESTING
3.4.1 TEST COMPONENTS
3.4.2 DHU LAB TEST CASES
3.4.3 DHU LAB TEST FLOW
7
7
9
10
10
11
13
14
14
14
14
15
15
4
SIMULATION SCRIPT FORMATS
16
5
BUG REPORTING
17
6
ACCEPTANCE AND REGRESSION TESTS
17
Tables in Document
Table 1. TESS Firmware Test Levels..................................................................................................................... 5
Table 2. TBD/TBR List. .............................................................................................................................................. 5
Table 3. Applicable Documents. ............................................................................................................................. 6
Table 4. Acronyms. ...................................................................................................................................................... 6
Table 5. FPE Test Cases.............................................................................................................................................. 9
Table 6. DHU Test Cases. ........................................................................................................................................ 12
Table 7. FPE Lab Test Cases. ................................................................................................................................. 14
Table 8. DHU Lab Test Cases................................................................................................................................. 15
Table 9. Script Formats. .......................................................................................................................................... 16
37-32003
Page 2 of 17
Revision A
Table 10. Bug Report Form. .................................................................................................................................. 17
Figures in Document
Figure 1. FPE FPGA Block Diagram. ...................................................................................................................... 7
Figure 2. FPE FPGA Test Bench. ............................................................................................................................. 8
Figure 3. DHU Interfaces. ....................................................................................................................................... 10
Figure 4. DHU FPGA Test Bench. ......................................................................................................................... 11
Figure 5: FPE Test Environment. ........................................................................................................................ 13
Figure 6: DHU Test Environment........................................................................................................................ 15
37-32003
Page 3 of 17
Revision A
1 Introduction
1.1 Purpose
This document outlines the firmware simulation and test procedures for the Transiting
Exoplanet Survey Satellite (TESS) Focal Plane Electronics (FPE) FPGA and the Data Handling
Unit (DHU) FPGA. It details the methods to be used to verify that each FPGA design in the
system meets the specifications described in the FPE FPGA Design Specifications and the DHU
FPGA Design Specifications documents.
1.2 Scope
This document covers VHDL test bench simulations to be run on each FPGA design separately,
as well as simulations to be run on the DHU and FPE designs interconnected in the test bench
as they will be in the physical system. This will not include mixed signal simulations (printed
circuit board impedances, etc.). A procedure for running a subset of test scripts on the physical
system will also be covered.
1.3 Roles and Responsibilities
For the purpose of this document, the Design Engineer is the individual responsible for the
design and development of the FPGA designs, while the Test Engineer is responsible for
running simulations to ensure that the designs perform as expected. The Design Engineer
creates the source code for all design entities, as well as constraints files, implementation
scripts, etc., and takes the design through synthesis, place and route and bit file generation.
While it is normal for the Design Engineer to run simulations on any and all modules in the
design as the work progresses, those simulations are not required by this test plan.
Separating the design and simulation tasks between two individuals provides an extra layer of
confidence due to the independent interpretation of requirements. A single engineer
performing both functions can conceivably miss a problem by designing the same error into
both the design entity and the test bench.
1.4 Tools
Simulations will be done using the Xilinx Vivado Simulation Tool. Tests will be run in
waveform mode, to confirm correct behavior and for complete visibility to all internal signals,
and self-checking, which will automatically generate test vectors and check them against
predicted results.
1.5 Design/Simulation Workflow
A common project repository will be set up with design source files and test bench and script
files in separate directories. Only the Design Engineer will have Write/Modify access to design
37-32003
Page 4 of 17
Revision A
source files, and the Test Engineer will have Write/Modify access to test bench and script
source files. The Test Engineer will report bugs or suspect behavior to the Design Engineer,
with instructions on which script to run and at what simulation time to observe the noted
behavior. The case will be resolved in one of three ways:

a fix in the design followed by a re-check using the original script that found the bug or
suspect behavior
a fix to the test bench or script after confirming that the requirements were unclear or
misinterpreted
clarification of the relevant requirements and no change to either the design or test
bench


1.6 Test Levels Summary
Table 1. TESS Firmware Test Levels.
Level
1
2
3
4
Description
Purpose
Relevant Requirements
FPE Firmware
Simulation
DHU Firmware
Simulation
FPE Stand-alone Lab
Testing
DHU Lab Testing
Check all functions of the
FPE Firmware
Check all functions of the
DHU Firmware
Check FPE Firmware
functions in hardware
Check DHU Firmware
functions in hardware
L7_FPE_FW_1 - L7_FPE_FW_8
L5_DHU_FW_1 - L5_DHU_FW_18
L7_FPE_FW_1 - L7_FPE_FW_16
L5_DHU_FW_1 - L5_DHU_FW_23
1.7 TBD/TBR
Table 2. TBD/TBR List.
TBD/TBR
TBR
Page
8
TBR
11
TBR
17
37-32003
Description
The configuration files are manually created or
generated via a script from a master file (FPE VHDL
simulation scripts)
The configuration files are manually created or
generated via a script from a master file (DHU VHDL
simulation scripts)
0003 0000ffff # address and data widths TBR (referring
to script commands)
Page 5 of 17
Revision A
2 Documents and Definitions
2.1 Applicable Documents
Table 3. Applicable Documents.
Item
1
2
3
4
5
6
Document ID
37-14010, Revision B,
01/05/2015
37-14011, Revision B,
01/07/2015
37-12050, Revision Aq,
12/04/2014
ds180_7Series_Overview.pdf
DS 180 (v1.16.1),
12/17/2014
Level 5 FPE Firmware
Requirements_Rev03q.xlsx
Level 5 DHU Firmware
Requirements_Rev03.xlsx
Title
DHU Firmware Design Specification
FPE Firmware Design Specification
Camera-DHU Interface Document
Xilinx FPGA 7-series Data Overview
(contains links to all relevant FPGA
documentation)
Level 5 FPE Firmware Requirements
Rev 03q
Level 5 DHU Firmware Requirements
Rev 03
2.2 Acronyms
Table 4. Acronyms.
ADC
CCD
CMD
DAC
DDL
DDR
DHU
FPE
FPGA
FW
GSE
HK
I/F
JTAG
PCIe
PPS
RST, Rst
RSTn
37-32003
Analog to Digital Converter
Charged Coupled Device – type of image sensor
Command
Digital to Analog Converter
Data DownLink
Double Data Rate
Data Handling Unit
Focal Place Electronics
Field Programmable Gate Array
Firmware
Ground Support Equipment
Housekeeping (analog) data
Interface
Joint Test Action Group – defines configuration and
boundary scan port
Peripheral Component Interconnect Express
Pulse Per Second
Reset
Reset not (inverted)
Page 6 of 17
Revision A
SBC
S/C
SEU
SGMII
TAP
UUT
VHDL
Single Board Computer
Spacecraft
Single Event Upset
Serial Gigabit Media II – type of high speed serial
channel
Test Access Port
Unit Under Test
Very High Speed Integrated Circuit (VHSIC) Hardware
Description Language
3 Test Levels
3.1 FPE Firmware Simulation
3.1.1 FPE Test Bench
The FPE Test Bench is a VHDL file that instantiates the FPE design file and its hierarchy as the
Unit Under Test (UUT). It provides stimulation for all of the UUT’s inputs, and provides viewing
and/or data comparison access to the UUT’s outputs. The FPE FPGA has the input and output
ports shown in the block diagram below.
Figure 1. FPE FPGA Block Diagram.
37-32003
Page 7 of 17
Revision A
The Test Bench writes and reads registers and memory via the SPI port labeled “From DHU” in
the diagram. It also sources the CCD Data, which consists of 16 discrete ADC ports, and the
Housekeeping (HK) ADC port. The relationship between the Test bench and the UUT is shown
in the diagram below.
TEST BENCH
CONTROL
FILE
REGISTER
CONFIG
FILE
MEMORY
CONFIG
FILE
MAIN
DRIVER
PROCESS
CCD
ADC
FILE
HK
ADC
FILE
RST & CLK
Logic
RST & Clock
JTAG
TAP
Model
JTAG
DHU
Model
DHU CMD/DATA
CCD
ADC
Model
CCD ADC Pixels
HK
ADC
Model
HK ADC Input
UUT
(FPE FPGA)
ADC Convert Line
SELF
CHECKER
FILE
Data
Compare
CCD Clocks
DAC Control & Data
Figure 2. FPE FPGA Test Bench.
For the JTAG, DHU, CCD ADC and HK ADC ports, the Test bench contains a model to emulate
each interface. A Main Process reads configuration files and manages the distribution of data to
the UUT and to the simulation models. The configuration files are manually created or
generated via a script from a master file (TBR). The Register and Memory configuration files
specify data to be loaded into registers and memory respectively, and this occurs at the
beginning of the test run. The CCD ADC file specifies ADC values for each of the 16 CCD nodes,
in a deterministic sequence. The CCD Model, which uses this data, responds to the CCD clocks
37-32003
Page 8 of 17
Revision A
output by the UUT. The Data Compare block compares CCD Clock waveforms with waveforms
stored in the Self Checker file, and also compares DAC data values with stored values.
3.1.2 FPE Test Cases
For each test case, waveforms are displayed by the simulation tool and inspected for
correctness. In addition, the test bench performs checks of stored expected data with UUTgenerated data and logs results, errors, etc. into a log file.
Below is a table of FPE test case categories. Except for the Initial Conditions category, each one
will have a number of variations as required to fully check the UUT.
Table 5. FPE Test Cases.
Test
Case
Description
Stimulus
Check
1
Initial conditions
DHU Cmd port
access
All output ports to
specified initial states
Write and read values
match
L7_FPE_FW_12
2
3
DAC voltage setting
Issue Power-On
Reset
Write to registers
and memory via
DHU Cmd port.
Send sump
command to read
values back via
DDL.
Write to DAC
channels via Cmd
port
L7_FPE_FW_6
4
DDL Housekeeping
standalone test
DAC Model compares
data in DAC commands
from UUT to script
stored data
Check HK Packet;
compare received HK
data with script stored
HK data
5
DDL Frame
generation
Check Pixel Packet;
compare pixel data with
script stored data
L7_FPE_FW_1
L7_FPE_FW_2
L7_FPE_FW_4
L7_FPE_FW_7
6
Full DDL channel
test
Check all pixel and HK
data
L7_FPE_FW_4
7
DHU Cmd port error
recovery
8
SEU Recovery
37-32003
Enable HK
channels to
transmit; disable
Pixel Packet
generation
Enable frame
generation, with
parameters set for
small frame;
disable HK packet
generation
Enable frame
generation and HK
channels
Inject errors into
DHU command
stream
Force single bit
errors into SEU
protected
registers/memory
Check proper handling
of Cmd errors (ignore
message, etc.)
Bit error cleared
Page 9 of 17
Relevant
Requirement(s)
L7_FPE_FW_8
L7_FPE_FW_3
L7_FPE_FW_5
L7_FPE_FW_11
Revision A
3.2 DHU Firmware Simulation
3.2.1 DHU Test Bench
The DHU Test Bench instantiates the DHU design file and its hierarchy as the UUT. It provides
stimulation for all of the UUT’s inputs, and provides viewing and/or data comparison access to
the UUT’s outputs. The DHU FPGA has the input and output ports shown in the diagram below.
Figure 3. DHU Interfaces.
The Test Bench provides stimulus for the JTAG and SBC PCIe ports, and the S/C 1PPS, Hard
RSTn, Interrupts, FPGA ID and clocks. Models are instantiated for the two FPE modules and the
DDR2 memories. With the FPE FPGA design previously simulated, sections of that design may
be used in place of models in the DHU simulation.
The relationship between the Test bench, UUT and models is shown in the diagram below.
37-32003
Page 10 of 17
Revision A
TEST BENCH
Reset
& Clocks
Logic
CONTROL
FILE
PCIe
Model
MAIN
DRIVER
PROCESS
JTAG
PCIe (fast data)
UUT
(DHU FPGA)
FPE1 Cmd
FPE
Model 1
FPE1 DDL
FPE2 Cmd
FPE 2
CONFIG
FILE
FPE
Model 2
DDR2
CONFIG
FILE
SELF
CHECKER
FILE
1PPS, FPGA ID
JTAG
TAP
Model
REGISTER
CONFIG
FILE
FPE 1
CONFIG
FILE
RST, Clocks, Ints,
FPE2 DDL
DDR2
Model 1
DDR2_1 I/F
DDR2
Model 2
DDR2_2 I/F
Data
Compare
Figure 4. DHU FPGA Test Bench.
Vendor models will be used for the PCIe and DDR2 interfaces. Models or sections of the FPE
design will be used for FPE1 and FPE2. A Main Process reads configuration files and manages
the distribution of data to the UUT and also the simulation models. The configuration files are
manually created or generated via a script from a master file (TBR). The Register and Memory
configuration files specify data to be loaded into registers and memory respectively, and this
occurs at the beginning of the test run. The Data Compare block compares actual UUT outputs
with expected results.
3.2.2 DHU Test Cases
For each test case, waveforms are displayed by the simulation tool and inspected for
correctness. In addition, the test bench performs checks of stored expected data with UUTgenerated data and logs results, errors, etc. into a log file.
37-32003
Page 11 of 17
Revision A
Below is a table of DHU test case categories. Except for the Initial Conditions category, each
one will have a number of variations as required to fully simulate the UUT.
Table 6. DHU Test Cases.
Test
Case
Description
Stimulus
Check
1
Initial conditions
2
PCIe port access
All output ports to
specified initial states
Write and read values
match
3
PCIe port
sequential
reads/writes
PCIe port
interleaved
reads/writes
PCIe Soft Reset
(FPE1 and FPE2)
Issue Power-On Reset
(hard reset)
Write and read
registers and memory
via PCIe port
Write and read
sequentially via PCIe
port
Write and read
interleaved via PCIe
port
Separately issue soft
reset command to
FPE1 and FPE2
4
5
6
7
FPE Control test
(FPE 1 and 2)
DDL input stream
symbols tests
Write and read FPE
registers and memory
Via FPE model,
generate proper
symbols and all error
types (unknown
control value, missing
symbol, etc.)
Set up DHU and
targeted FPE model
for small frame
capture test both FPE
inputs)
8
Single Camera,
Small Frame tests
9
Dual Camera,
Small Frame tests
Set up DHU and both
FPE models for small
frame capture
10
Dual Camera,
Small Frame tests
with variable PCIe
bus traffic
Set up DHU and both
FPE models for small
frame capture
Perform PCIe
reads/writes to
emulate bus traffic
37-32003
Relevant
Requirement(s)
L5_DHU_FW_20
Write and read values
match
Write and read values
match
Target FPE output
ports to specified
initial states (other
FPE unaffected)
Check FPE Command
interface operation
Check for detection of
errors
L5_DHU_FW_20
L5_DHU_FW_22
Check DDL Interface,
DDR2 arbitration,
data throughput,
saturation, separation
of camera logic,
output of all DPMs
(full frame, guide
stars, etc.)
Check DDL Interface,
DDR2 arbitration,
data throughput,
saturation, separation
of camera logic,
output of all DPMs
(full frame, guide
stars, etc.)
Check DDL Interface,
DDR2 arbitration,
data throughput,
saturation, separation
of camera logic,
output of all DPMs
L5_DHU_FW_6
L5_DHU_FW_7
L5_DHU_FW_17
L5_DHU_FW_18
Page 12 of 17
L5_DHU_FW_16
L5_DHU_FW_17
L5_DHU_FW_6
L5_DHU_FW_7
L5_DHU_FW_17
L5_DHU_FW_5
L5_DHU_FW_6
L5_DHU_FW_7
L5_DHU_FW_17
Revision A
11
Target Pixel
Processing
Receive pixel data,
store target pixel data
12
Guide Star
processing
Select and store Guide
Star data
13
Raw Image
Integration
Single Exposure
test
Integrate 15 to 60
raw images
Collect and store
single exposure pixel
data
Issue 1PPS pulse
14
15
16
17
18
1PPS timestamp
generation
DHU
Housekeeping test
FPE
Housekeeping test
SEU Recovery
Collect DHU HK data
Receive HK data
packets from FPE1
and 2
Force single bit errors
into SEU protected
registers/memory
(full frame, guide
stars, etc.)
Adequate memory
allocated and target
pixel data stores
Memory allocated for
Guide Star, and data
correct
Image sum correct
Memory allocated and
data properly stores
Check counter and
register values.
Check HK data
L5_DHU_FW_3
L5_DHU_FW_4
L5_DHU_FW_17
L5_DHU_FW_9
L5_DHU_FW_10
L5_DHU_FW_1
L5_DHU_FW_17
L5_DHU_FW_14
L5_DHU_FW_15
L5_DHU_FW_17
L5_DHU_FW_2
L5_DHU_FW_11
HK data properly
received
L5_DHU_FW_12
L5_DHU_FW_13
Bit error cleared
L5_DHU_FW_19
L5_DHU_FW_21
3.3 FPE Stand-alone Lab Testing
Before being tested against the DHU, lab testing of the FPE FW will be conducted with a DHU
simulator, as shown in the following diagram.
Figure 5: FPE Test Environment.
37-32003
Page 13 of 17
Revision A
3.3.1 FPE Lab Test Cases
Table 7. FPE Lab Test Cases.
Test Case
Typical raw-to-processed
frame data flow, with
multiple full frames and
typical integrations.
Should be run
continuously for several
hours to ensure each
integration is valid.
Housekeeping and
memory dumps via the
DDL interface. Various
housekeeping
configurations.
Checks
DDL interface
CMD Interface
Memory writes
Resets
Clock Level Voltage Interface
CCD Interface
Notes
Housekeeping Interface
Status Registers
Memory reads
3.3.2 FPE Lab Test Flow
Tests will be run with full frame sizes and typical integrations.
1. The FPGA bit file can be loaded either via JTAG or the Serial Synchronous Interface from
the Zynq board. Testing JTAG is not a requirement, as flight will use the Serial
Synchronous Interface.
2. Frame images are loaded into the Zynq board via an Ethernet SGMII port. The Zynq has
an FPGA that can be programmed to transmit this to the FPE. Alternatively, the FPE can
be set into test mode, thus generating the pixel data without being loaded.
3. The Zynq board can be configured to send configuration information to initialize the
FPE registers and memory, as well as send a Frame Start and process incoming data.
4. Data from the FPE can be read from the Zynq board via an Ethernet cable.
3.4 DHU Lab Testing
3.4.1 Test Components
Prior to hooking it up to the Focal Plane Electronics, lab testing of the DHU FW will be
conducted with an FPE simulator, as shown in the diagram below.
37-32003
Page 14 of 17
Revision A
Figure 6: DHU Test Environment.
3.4.2 DHU Lab Test Cases
Table 8. DHU Lab Test Cases.
Test Case
Typical raw-to-processed
frame data flow, with
multiple full frames and
typical integrations.
Should be run
continuously for several
hours to ensure
Checks
DDL Interface
CMD Interface
PCIe interface
DDR2 interface
DPM processing
Timestamps
Notes
While SBC
software/board is in
bringup, the USB to JTAG
bus can be used as an
alternative in configuring
the DHU Virtex FPGAs and
checking the DDR2
memory.
3.4.3 DHU Lab Test Flow
Tests will be run with full frame sizes and typical integrations.
1. The FPGA bit file can be loaded either via the SBC, through an Ethernet write into Flash
and SBC code to initiate the boot. The USB to JTAG is an alternative method, but testing
the JTAG interface is not a requirement.
2. Frame images are loaded into the Zynq board via an Ethernet SGMII port. The Zynq has
an FPGA that can be programmed with FPE FPGA logic. The FPE can be set into test
mode, thus accepting test data from memory instead of from ADCs, as on the FPE board.
37-32003
Page 15 of 17
Revision A
3. The GSE FPE can be configured in two ways—via the Zynq processor or the SBC
processor. Only the SBC processor interface (through the DHU FPGA) is required to be
tested, but the alternate method might be useful during bringup.
4. The DHU can be configured in two ways—over the JTAG interface, or via the SBC
processor. The SBC processor is the only method that needs to be tested.
5. The FPE transmits the housekeeping and image data over the Data Downlink bus to the
DHU, which will accept and process image frames.
6. DHU completes data processing.
7. Data from the DDR2 can be checked in two ways. The JTAG can poll the interrupt
register and read data over the JTAG to USB. The SBC can take interrupts and transfer
data to the SBC DDR2 memory, which can be read via SGMII Ethernet and checked for
accuracy.
4 Simulation Script Formats
For a given simulation, the test bench parses a master script file which handles any necessary
setup and guides the flow of the simulation. Some commands specify a configuration to be used
for writes to registers or memory, or for comparison in the case of reads. The script commands
for the FPE and DHU simulations are almost identical.
Table 9. Script Formats.
Command
Write
Syntax
wr <port> <type> <file>
Read/Compare rd <port> <type> <file>
Wait For
Wait Until
Wait Event
Soft Reset
Comment
wt <time>
wu <time>
we <type>
rs
#
Definitions
port: j = JTAG, d = DHU (for FTE), p = PCIe (for
DHU), 1 = FTE1 model, 2 = FTE2
model
type: r = Register, m = Memory,
d = DAC,
c = CCD model, k = HK model
port: d = DHU, j = JTAG
type: r = Register, m = Memory,
d = DAC, p = Pixel,
c = CCD model, k = HK model
time in uS
time in uS
type: 1 = packet received
(DHU only)
An example of a Master Script for the FTE follows:
37-32003
Page 16 of 17
Revision A
wr
wr
rd
we
rd
d
d
d
1
d
r
m
r
#
p
reg_load_01.txt # Load registers as defined in the file reg_load_01.txt
mem_load_01.txt # Load memories as defined in the file mem_load_01.txt
reg_load.txt # Read registers specified in the file reg_load_01.txt and compare
wait until a packet is received from the FPE
pixel_file.txt # Compare the received packet against the expected packet
Each line in a register load file specifies a register address and a value:
0003 0000ffff # address and data widths TBR
5 Bug Reporting
A log in spreadsheet form will be maintained in the project repository, and will have the
following headings:
Table 10. Bug Report Form.
ID
1
2
3
Severity
feature
mild
major
Status
open
fixed
tb fixed
Build
Script
Description
Resolution
The Design Engineer can use the script listed in the bug report to confirm or deny the bug
itself, and then to check the bug fix after any modifications to the design. An Acceptance Test
script, consisting of a composite of key scripts to date, can also be used by the Design Engineer
to confirm that no other problems have been introduced. Once the bug log has been updated
by the Design Engineer, the Test Engineer will re-run the listed script and the Acceptance Test
script as a further confirmation. Regression testing will also be performed (see Section 6)
6 Acceptance and Regression Tests
While there will be any number of short, specific test scripts used to focus on functions within
the overall designs, there will also be two special test scripts: Acceptance Test and Regression
Test.
Both types of test scripts include tests for all major functions in a given design (FPE or DHU),
and will grow into full tests by the end of the test bench design cycle. The difference is that the
Acceptance Test is designed to have a minimal run time, while the Regression Test is more
detailed and will necessarily take longer to run. The Acceptance test will be run more often,
and the Regression test will be run periodically, but less often, by the Test Engineer.
37-32003
Page 17 of 17
Revision A