Implementing Video Systems
Using the Altera Video Framework
© 2011 Altera Corporation—Internal
AGENDA
What is the Video Framework?

 Overview

VIP Core Details

Boards/Reference Designs
© 2011 Altera Corporation—Internal
2
Altera Video Framework
Building block IP cores to speed up
development (VIP suite)
Open, low-overhead interface standard to
mix-and-match custom or off-the shelf IP
blocks
Format conversion reference designs
that showcase silicon capability, provide a
starting point for design
System Level Tools and design
methodology
Variety of development boards/kits for
rapid design prototyping





© 2011 Altera Corporation—Internal
3
The Video IP framework – The IP
Building block IP to speed up development



Interfaces (SDI, DVI, etc), High Performance Memory Controllers, Nios
Video IP suite: library of interoperable Video IP blocks using common standards
and protocols
Open interface standard to mix-and-match custom or offthe shelf IP



Avalon Memory-Mapped and Avalon-Streaming Video protocol
Open Source HDL Template to facilitate integration of custom IP into framework
Avalon-ST Video (over Avalon-ST)
Input
interface
Clocked
Video
Input
Altera
VIP 1
Nios
Video in
© 2011 Altera Corporation—Internal
4
Avalon-MM
Custom
IP
Altera
VIP 2
Clocked
Video
Input
Output
interface
Memory controller
Video out
The Video IP framework – Design Tools
SOPC Builder
Quartus II
1080p reference designs: out-ofthe-box ASSP-level capabilities
and/or great starting points
V-series, M-series, partner designs
Pin mapping,
Interface IP,
Top-level HDL
Abstract design capture
Switch fabric generation
IP Catalogue tool
Nios II IDE
Nios II IDE / Quartus II
Software library to control
the cores of the VIP Suite
and for interrupt servicing
Development boards/kits for
rapid design prototyping
Debug (Nios II & SignalTap)
© 2011 Altera Corporation—Internal
5
Stratix II GX, Cyclone III
Stratix IV GX PCIe, Arria II GX
Software control and
configuration
AGENDA
What is the Video Framework?

 Overview

VIP Core Details

Boards/Reference Designs
© 2011 Altera Corporation—Internal
6
Video Image Processing (VIP) Suite
BT 656

Avalon ST video
Scaler
v10.1
Chroma
resampler
Deinterlacer
v11.0
Deinterlacer II
Switch
2D FIR filter
Control Sync
Frame buffer
2D median filter
Frame Reader
Image clipper
Color space
converter
Avalon ST video

BT 656
Gamma
correction
Test pattern
generator
Color plane
sequencer
Scaler II
Alpha
blending mixer
Half a dozen hardware verified video processing
reference designs using these functions
© 2011 Altera Corporation - Public
7
Available TODAY!
New Cores in VIP Suite (2011)
Scaler II

 Reduces area and improves performance
 Compared to first-generation scaler
 Supports 4:2:2 chroma data sampling rate
 Further reduces area used
Deinterlacer II

 Supports 3:2 cadence detection
 Supports low-angle edge detection
 Delivers lower latency
© 2011 Altera Corporation - Public
8
VIP Function Details
Chroma Resampler

This function allows you to
change between 4:4:4, 4:2:2
and 4:2:0 sampling rates

For 4:2:2  4:4:4, the filtered
algorithm uses a 4-tap filter
with fixed Lanczos-2
coefficients

For 4:4:4  4:2:2, the filtered
algorithm uses a 7-tap filter
with fixed Lanczos-2
coefficients
© 2011 Altera Corporation—Internal
9
VIP Function Details
Color Space Converter

This function provides a
flexible and efficient
means to convert image
data from one color space
to another

You can select from a list
of common color space
conversion

Or choose a custom color
space
© 2011 Altera Corporation—Internal
10
VIP Function Details
Deinterlacer / Deinterlacer II (motion adaptive)
Select the input image height and width
Choose the deinterlacing algorithm –
bob, weave, motion adaptive
The weave and motion adaptive algorithms
require external frame buffering
© 2011 Altera Corporation—Internal
11
Understanding Algorithmic Constraints
Deinterlacer (motion adaptive)
Motion adaptive deinterlacer requires five
(master) accesses to the DDR memory





4:4:4 Mode, Motion Bleed is ON
1 field write
2 field reads
1 motion vector write
1 motion vector read
Deinterlacer (MA)
Calculating DDR memory bandwidth


Input format: 1080i, 60 fields/sec, 10-bit color
 1920 × 1080 × 30bits × 60/2 = 1.866Gbit/s

Output format: 1080p, 60 frames/sec, 10-bit color
 1920 × 1080 × 30bits × 60 = 3.732Gbit/s

Motion format: Only use 10bits for the motion values
 1920 × 1080 × 10bits × 60/2 = 0.622Gbit/s





1 × write at input rate: 1.866Gbit/s
1 × write at motion rate: 0.622Gbit/s
1 × read at motion rate: 0.622Gbit/s
2 × read at output rate: 7.464Gbit/s
Total: 10.574Gbit/s

© 2011 Altera Corporation—Internal
12
DDR2
Memory access:
4:4:4 Mode
Understanding Algorithmic Constraints
Deinterlacer (motion adaptive)
Motion adaptive deinterlacer requires five
(master) accesses to the DDR memory





1 field write
2 field reads
1 motion vector write
1 motion vector read
4:2:2 Mode, Motion Bleed is ON
Deinterlacer (MA)
Calculating DDR memory bandwidth


Input format: 1080i, 60 fields/sec, 10-bit color
 1920 × 1080 × 20bits × 60/2 = 1.24Gbit/s

Output format: 1080p, 60 frames/sec, 10-bit color
 1920 × 1080 × 20bits × 60 = 2.48Gbit/s

Motion format: Only use 10bits for the motion values
 1920 × 1080 × 10bits × 60/2 = 0.622Gbit/s

Memory access:




1 × write at input rate: 1.24Gbit/s
1 × write at motion rate: 0.622Gbit/s
1 × read at motion rate: 0.622Gbit/s
2 × read at output rate: 4.96Gbit/s
Total: 7.44Gbit/s

© 2011 Altera Corporation—Internal
13
DDR2
4:2:2 Mode, 30% reduction in DDR
bandwidth
VIP Function Details
Requires progressive 4:4:4 input
Scaler
Select the input and output image sizes
Choose the scaling algorithm – number
of taps/phases
Choose from a selection of pre-built filter
functions or choose a custom function
© 2011 Altera Corporation—Internal
14
VIP Function Details
Scaler II – Utilizes Fewer Resources
Reduces area and improves performance

 Compared to first-generation scaler
Supports 4:2:2 chroma data sampling rate

 Further reduces area used

Algorithm supports
 Linear and polyphase with
run-time coefficient load

New core in HDL
 More efficient in using resources
© 2011 Altera Corporation - Public
15
VIP Function Details
Alpha Blending Mixer
Select the size of the image
Select the number of
layers to be mixed
The level of blending is controlled by
the number of alpha bits (up to 8)
© 2011 Altera Corporation—Internal
16
VIP Function Details
Generate a color bar test pattern
© 2011 Altera Corporation—Internal
17
BT656  Avalon ST Video

The Clocked Video Input
MegaCore function converts
from clocked video formats
(such as BT656 and DVI) to
Avalon-ST Video

It strips the incoming clocked
video of horizontal and
vertical blanking, leaving only
active picture data

No conversion is done to the
active picture data

The color plane information
remains the same as in the
clocked video format.
© 2011 Altera Corporation—Internal
18
BT656  Avalon ST Video

The Clocked Video Output
MegaCore function converts
from Avalon-ST Video to
clocked video formats (such
as BT656 and DVI)

It formats Avalon-ST Video
into clocked video by
inserting horizontal and
vertical blanking and
generating horizontal and
vertical sync information

No conversion is done to the
active picture data

The color plane information
remains the same as in the
Avalon-ST Video format.
© 2011 Altera Corporation—Internal
19
Frame Buffer
© 2011 Altera Corporation—Internal
20
Run-time Control of the Video Chain
Build video signal chains that can be updated onthe-fly without changing the HDL or bit-stream

 Using the streaming interface protocol: Avalon ST Video
 Using an embedded processor
© 2011 Altera Corporation—Internal
21
Run-time Control of the Video Chain
Video function 1
Video function 1
Avalon ST
READY
DATA
VALID
SOP
EOP
Examples of Avalon-ST Video Control Packets
© 2011 Altera Corporation—Internal
22

Avalon-ST Video protocol
is a packet-oriented way to
send video and control data
 First packet contains a frame of
video data
 Second packet contains control
information that applies to the
subsequent video packet/frame
Run-time Control of the Video Chain
Video function 1
Avalon MM control plane
Avalon MM
Master port
Processor



Most video functions permit run-time control of some
aspects of their behavior, using a common type of AvalonMM slave interface
Each slave interface provides access to a set of control
registers which must be set by an embedded
processor/logic
Generally, a function can be updated on a frame boundary
© 2011 Altera Corporation—Internal
23
NIOS Control: Cyclone III Example Design
Initialize daughter card
Initialize daughter card
Start the IP cores
Control the mixer and
Scaler parameters at runtime
© 2011 Altera Corporation—Internal
24
AGENDA
What is the Video Framework?

 Overview

VIP Core Details

Boards/Reference Designs
© 2011 Altera Corporation—Internal
25
Video Development Kits
Cyclone III FPGA, $1,895
Cyclone IV FPGA, $1,295
Stratix IV FPGA, $4,995
Arria II GX FPGA, $2,995
$495
$795
Cyclone III FPGA, $2,995
© 2011 Altera Corporation—Internal
26
New Demo in 2011 (VEEK)


Demo # 1 – Simple Video Format Conversion
Input
Processing
composite
de-interlaced video /w txt overlay
(NTSC to 800x600 with real-time
resizing, zoom and panning)
Output
VEEK display
VGA out
Demo # 2 – Camera Input Scaling
Input
Processing
Camera on VEEK
Camera Input/w txt overlay
(Camera input scaling with real-time
resizing, zoom and panning)
Output
VEEK display
VGA out
Download Design Files on Altera Video Wiki
www.alterawiki.com/wiki/Videoframework
© 2011 Altera Corporation—Internal
27
New Demo in 2011 (VEEK)


Demo # 3 – Picture in Picture (PiP)
Input
Processing
Output
composite
camera on VEEK
de-interlaced video /w txt overlay
PIP - (camera source)
VEEK display
VGA out
Demo # 4 - Multi-view
Input
Processing
Output
composite
camera (GPIO)
DVI (HSMC)
CH 1 / 3: de-interlaced video
CH 2 : camera video
CH 4: DVI video (800x600 down scaling)
DVI (HSMC) @ 800x600
Watch Demo of VIP Suite
© 2011 Altera Corporation—Internal
28
New Low Cost UDX Reference Design
C4GX150 board

 Develop in Quartus II software v11.0 using Qsys
 Available at end of July
1 channel UDX design

 SDI / DVI input switch
 SDI / DVI output
 Resolution supported
 1920 x 1080P 50/59.94/60Hz
 1280 x 720P 50/59.94/60Hz
 Text overlay and testing pattern
© 2011 Altera Corporation—Internal
29
Video Reference Designs
Design
Device
Features
I/Os
Video and
image
processing
(VIP) demo
Cyclone III,
Cyclone IV
National Television System Committee
(NTSC)-to-480p scaling, 720p camera input
scaling, mixing for PiP, multiview ,
1-channel UDX design
Analog, digital video
interface (DVI)
Cyclone® III
FPGA design
Cyclone III
NTSC-to-720p scaling, MA deinterlacer,
scaler, mixing for PiP
Analog, DVI
UDX2
Arria® II GX
Same as UDX4
Serial digital interface
(SDI), DVI
UDX3/4
Stratix® IV GX
V1, on-screen display (OSD),
switch, active format description (AFD),
mixing, multiport front-end (MPFE), 1080p,
720p, 1080i, ntsc/pal, 4:2:2 scaler, MA
deinterlacer with low-angle edge
SDI, DVI, high-definition
multimedia interface
(HDMI)
4K scaling
Stratix IV GX
1080p  4K scaling, component design
methodology
SDI
4x SDI for output
Broad Portfolio of Reference Designs
to Help You Get Started
© 2011 Altera Corporation—Internal
30
Summary

Altera video framework enables rapid
development
 Mix and match existing IP – Leverage Altera’s open interface
standard
 Automatically integrate embedded processors and arbitration
logic
 Leverage building block IP provided by Altera
 Use existing reference designs as starting points

And rapid prototyping
 Implement design using the appropriate development boards
 Test the design with actual video signals
© 2011 Altera Corporation - Public
31
Next Step

Coming Video Framework Workshop, please contact Altera Sales orDistributors
for more information

Provide an introduction to Video and Image Processing suite including the cores,
reference designs, roadmap, video framework, etc.

For information on:






DDR-based Memory controllers


http://www.altera.com/education/training/courses/IMEM210
http://www.altera.com/technology/memory/mem-index.jsp
SOPC Builder


http://www.altera.com/education/training/courses/OEMB1115
http://www.altera.com/literature/lit-sop.jsp
System Console


http://www.altera.com/education/training/courses/OEMB1117
http://www.altera.com/literature/ug/ug_system_console.pdf
NIOS-II Flow


http://www.altera.com/education/training/courses/IEMB115
http://www.altera.com/literature/lit-nio2.jsp
Video and Image Processing Suite



http://www.altera.com/products/ip/dsp/image_video_processing/m-alt-vipsuite.html
http://www.altera.com/end-markets/broadcast/1080p/bro-1080p.html
http://www.altera.com/support/refdesigns/sys-sol/broadcast/ref-format-conversion.html
On-chip Debugging, including SignalTap-II


http://www.altera.com/education/training/courses/ODSW1164
http://www.altera.com/support/software/debugging/sof-qts-debugging.html
© 2011 Altera Corporation—Internal
32
Backup
© 2011 Altera Corporation—Internal
33