ECE 354
Lab 2: Capturing and Displaying
Digital Image
ECE Department: University of Massachusetts, Amherst
Big Picture Introduction
 Understand the existing SOPC builder setup with new
modification
 Capturing image using camera
 Storing the captured image in the frame buffer
 Display the image on a CRT monitor with the use of a VGA
controller
 Perform simple image processing
 Primarily written in C code
ECE 354
2
Additional Hardware
 TRDB_D5M Digital Camera
 Serial Connector and
Cable
 CRT Monitor
ECE 354
3
Skills to learn
 Integrate code with preexisting code
 Understanding previously written code
 Connect to a device that you did not design
ECE 354
4
Steps taken to complete project
 Begin by looking over camera documentation
 Understand SOPC builder setup given
 Write a C program to convert the color image from camera
to black and white
 C program to transfer data from FIFO to flash memory
 Gain knowledge of how the DE2_NIOS_HOST_MOUSE_VGA
project displays an image to the CRT monitor
 Use that knowledge to display your image
 Finally have fun performing image processing
ECE 354
5
Lab2 Setup
[1]
[1] TRDB_D5M_Userguide
http://www.terasic.com.tw/attachment/archive/281/TRDB_D5M_UserGuide.pdf
ECE 354
6
Overview of DE2_NIOS_HOST_MOUSE_VGA project
 Implements a monochrome display, with a preloaded image,
where the user can draw on it with a mouse
 USB mouse should be connected USB HOST port
 CRT monitor should be connected to the VGA port
 You will integrate your code so that the project displays the
image received from digital camera
ECE 354
7
SOPC builder setup:








Based on DE2_NIOS_HOST_MOUSE_VGA
The program code is stored in SRAM memory
Parallel Input/output interface(PIO) for LED, switches
Camera Avalon Interface(Camera_IF) helps in transferring
image data
The dedicated SDRAM is used as a frame buffer that holds
a single frame of video at the time.
The SDRAM is controlled and accessed by using the fourport controller hardware
 Controller can be used to read or write to the SDRAM
through its four FIFO buffers.
Flash controller is used to control the data transfer to flash
memory
VGA controller displaying data on to VGA
ECE 354
8
Block diagram of Terasic camera system

TRDB_D5M_Userguide
http://www.terasic.com.tw/attachment/archive/281/TRDB_D5M_UserGui
de.pdf
ECE 354
9
Hardware Modules for this Lab
 SDRAM-Multiport Controller:
This module is a generic 4 port SDRAM controller.
It uses two FIFO buffers for inputs (writing) and
two FIFO buffers for outputs. The FIFO buffers
are made using Altera MegaFunctions and are 16
bit wide and can hold up to 512 words of this
size. In this system the SDRAM controller is used
as a frame buffer to hold a single picture frame
 Flash Controller
 VGA controller
 Camera Avalon Interface module
ECE 354
10
To get started with the lab
 Run the DE2_NOIS_MOUSE_VGA_project present in the
project folder given to you
 Download the project content files put up online under lab2
 Run SOPC builder and understand each of the components
added and configuration present
 Hardware component files are present under IP folder
•
•
It contains SDRAM_4 port controller components
Camera Avalon Interface
 Software components are present under the software folder
• hello_led_0 has all the required software component files
• Some part of code is given in this project which is present in
file hello_led.c
ECE 354
11
Information for C program
 C code to capture camera data and store it in frame buffer
 Picture taken at 640x480 has 3 data per pixel times 8 bits
(at least) each, this would mean a total of 7.37 Mbit or
921.6 Kbyte
 So efficient storage design required
 Recommended C code function
• Camera Capture
• Copying RGB(pixels) data from FIFO to Flash memory
• Using flash memory to store the pixel data before
transferring to VGA output
 Program space memory is less (SRAM).Avoid unnecessary
buffers. Write programs as efficient as possible
ECE 354
12
Displaying image through VGA controller
 Look through VGA.h for
VGA functions
 Write C code similar to
that shown below that
calls functions from VGA.h
to display each pixel
ECE 354
13
Reading pixels from Flash Memory(hello_led.c)
 The methods
called inside this
file are NIOS II
APIs.
 Use this skeleton
to develop your
code.
ECE 354
14
Function.c
 This file needs to
be updated with
your code.
 Various skeleton
functons are
defined. You need
to modify/extend
these functions.
ECE 354
15
Additional information
 The image should be 1-bit per pixel with a resolution of
640x480
 Uncompressed
ECE 354
16
Image Processing
 It is required that you implement two forms of
image processing (out of 5)
 Each bullet below correspond to one image
processing technique you need to show on the
VGA
 Recommendations include:
•
•
•
•
•
ECE 354
Add timestamp onto image
Counter to keep track of number of pictures taken
Rotate, mirror, invert image
Simple edge detection (challenging)
Detect changes in images (challenging)
17
References for software and hardware design in Lab2
 I recommend looking over section II of the NIOS II Software
Developer’s Handbook
 Go through project titled- Embedded Demonstrator for Video
Presentation and Manipulation by Cato Marwell Jonasse (Google
it) to get clear insight on hardware description for this project
(You can go through chapter 9 specifically)
 DE2_NIOS_HOST_MOUSE_VGA project
 NIOS II Software Developer’s Handbook
• http://www.altera.com/literature/lit-nio2.jsp
ECE 354
18
Where does this project lead?
 Next lab will be on the topic sending data over a network
• Sending image between DE2 boards
• Don’t worry if you have not taken computer networking
ECE 354
19
Questions and Comments
ECE 354
20
Back up
ECE 354
21
Readout modes: D5M camera
 The Terasic D5M camera supports a function
called :
1. binning -This function reduces the resolution
of the image by averaging pixels together
2. Skipping - reduces the output resolution
without affecting the field-of-view. It does this
by not sampling entire rows and columns of
pixels. A skip 2X mode skips one of pixels for
every pair of output.
ECE 354
22