Cerify – File Based Quality Control
Yannick LE DREAU
Outline

The Business Case

Cerify Workflows and Applications
Wednesday, 08 April 2015
The Business Case
Wednesday, 08 April 2015
Business types

Segmentation
– Business areas
–
–
–
–
–
Content Services
Traditional Broadcasters
Advertiser Services
VoD & IPTV
Post Production Houses
– Applications
– Ingest
– Repurposing
– Delivery
Wednesday, 08 April 2015
Business area examples
 Segments include
– Content Services
– Ingest once, re-purpose many
– iTunes encoding and delivery
– archive/asset management at high bitrate encoded video
– Traditional Broadcasters
– Ingest once, re-purpose many
– Ingest and playout of high bitrate encoded video
– Advertiser Services
– Advertisement/commercial encoding and delivery
– VoD & IPTV
– Encoding for web, VoD, mobile
– MPEG 2 Transport streams
– 3GPP files
– Windows Media
– Post Production Houses Content services
– File based workflow and delivery to broadcaster
– Mainly commercials
Wednesday, 08 April 2015
Typical Applications of different “SD” codecs
Video Codec
6
Typical applications (new)
MPEG-2
Broadcast, cable, VOD, DVD
H.264 / AVC / MPEG-4 Pt
10
Broadcast, cable, VOD, HD, Internet, HD
video disks*
WM9/VC-1
Broadcast, HD, Internet TV, HD video discs –
Blu-Ray
DV/DVC-Pro 25/50
Broadcast, Acquisition, Editing
MPEG-4 Pt 2
Mobile
Wednesday, 08 April 2015
Typical Applications of different “HD” codecs
Video Codec
7
Typical applications (new)
DNxHD/VC-3
Editing, Broadcast Playout
AVC-Intra
Acquisition, Editing
Pro-Res
Editing, iTunes
JPEG 2000
D-Cinema, broadcast internal use/archive
DVC-Pro100
Acquisition, Editing, Broadcast Playout
Wednesday, 08 April 2015
MPEG-2: Typical Configurations
 SDTV, DVD
– MP@ML, 720 × 576, 4:2:0 colour format, 25 fps
 HDTV
– MP@HL, 1920 × 1080i or 1280 × 720p, 4:2:0 colour format, 25 fps
 Ingest SD (Sony IMX MPEG 2 I-Frame Only)

422P@ML, 720 × 608, 4:2:2 colour format, 25 fps
 Ingest HD (Sony XD-Cam MPEG 2 long GOP)
– 422P@HL, 1920x1080 4:2:2 colour format, 25fps
8
Wednesday, 08 April 2015
AVC/H.264/MPEG4 part 10
 Is an “open standard”, compared to closed formats, like Microsoft's
Windows Media
 Performs significantly better than any prior standard can, under a
wide variety of circumstances in a wide variety of application
environments
 H.264 can often perform radically better than MPEG-2 video—
typically obtaining the same quality at half of the bit rate or less
 Can be carried by MPEG-2 Transport Streams making it increasingly
used in HD VoD applications.
 AVC-Intra is a Panasonic P2 implementation of Intra frame coded
AVC. Gaining popularity for acquisition and editing.
9
Wednesday, 08 April 2015
DNxHD/VC-3
 Avid’s DNxHD is a HD codec technology engineered to create
mastering-quality HD media at reduced bit rates to allow HD
editing and production using SD infrastructure.
 In January 08 it was approved to be compliant with SMPTE
2019 VC-3
 There is support from EVS and Omneon amongst others for
playout.
Image © Avid.com
10
Wednesday, 08 April 2015
DNxHD
 Key benefits
11
–
Optimal mastering picture quality
–
Reduced storage requirements
–
Enables real-time HD sharing and collaboration
–
Improved multi-stream performance
–
Specifically engineered for multi-generation compositing and post
production
–
Full frame raster sampling
–
Choice of 8- or 10-bit sampling as well as three user-selectable bit
rates
–
Open standard: HD Codec source code is licensable free of charge
Wednesday, 08 April 2015
Apple Pro Res 422

Unlike DNxHD, ProRes 422 provides full functionality at advanced
resolutions (2K and 4K cinema) and SD.

It is DCT based, and is therefore simpler to decode than distribution
oriented formats like H.264.

Full-width 1920x1080 and 1280x720

4:2:2 chroma sampling

8 and 10-bit sample depth

I frame-only encoding

Variable bit-rate (VBR) encoding

Normal 145 Mbit/s and High-Quality 220 Mbit/s for HD resolution

Normal 42 Mbit/s and High-Quality 63 Mbit/s for SD resolution

Fast encoding and decoding (both at full size and half size)
12
Wednesday, 08 April 2015
MXF – Wrapper

Media eXchange Format

Developed as a global wrapper format for all video, audio and metadata

Specification is layered. “OP1A” configuration is most commonly used today
(simplest implementation of MXF)
 File interchange for networked AV file distribution
 Format independent
–
–
–
Picture Size
Interlaced or Progressive
Audio Sampling Rates
 Wraps all formats
–
–
–
13
Current and future formats
Metadata only files allowed
Compressed and uncompressed
Wednesday, 08 April 2015
Broadcast Operational challenges

Ingest and Playout are in transition
–

Methods of storing video are changing
–

Analogue to Compressed Digital
Tape to File based stored on video Servers
Many different new formats
–
–
–
Terrestrial, Satellite, Cable, VoD, IPTV
Sub SD, SD, HD,
MPEG-2, MPEG-4, H.264/AVC, VC-1 etc

Content is being re-purposed at different
resolutions and frame rates, using new
codecs and at different bit-rates, with
different technical requirements

Explosion of content from many sources for
different channels and in different formats
Wednesday, 08 April 2015
The file based QC challenge
Visible
Errors
Invisible
Errors
15
Wednesday, 08 April 2015
Broadcasting Human Quality Control Challenges
 Human QC/play-out testing is time consuming and expensive
 Visual/audio errors are easily missed with human play-out
–
–
just by blinking or losing concentration for a second
different skill level/experience/training of testers
 Simple errors – human may not get constraints correct
 Consistency – cannot repeat exactly the same test on content, day
after day, week after week
 Person cannot easily look inside the file for example:
–
–
–
–
–
–
correctness of encoding used for compression for video, audio
bit-rates of video, audio and overall
syntax errors in the file which may prevent it playing correctly
correct playtime of the video and audio
audio on correct channels
visual quality ...
Wednesday, 08 April 2015
File Based video Quality Control
 File Based Video QC Application Challenges
– Traditional video T&M only checks baseband (luma/chroma/signal) levels
in analog and digital video streams
– File-based video is different. It must be checked for
– Correct Encoding Syntax – at digital level, audio/video must be
encoded without errors, so it plays out correctly at the Customer’s
STB / playout device
– Correct Parameters – audio/video bitrates, GOP structure, Colorspace, Color depth, Frame size, Frame rate, Aspect ratio,
Quantization levels
– Correct Baseband and Quality levels - analog parameters for
Signal levels, Luma, Chroma, Gamut and Quality levels of Black
frames, Blockiness, Loss of audio, Audio clipping, Video/Audio
playtime
Wednesday, 08 April 2015
Video Quality Control

Video QC is a delivery requirement by many
broadcasters. QC Operators would be
looking at:
–
–
–
–
–
–
–
–

Black and white level
RGB Gamut and possibly composite gamut
Number of audio channels, level and loudness
peaks, phase
Picture action and graphic safe areas in all
deliverable aspect ratios
Creative quality and attached metadata such as
program duration, commercial breaks and clock
details
Video blanking
Timecode presence and continuity
General picture quality, standards conversion and
compression artefacts, tape drop out
Some of the above can be automated
Wednesday, 08 April 2015
Why Automate Quality Control?
 We can automate our classic QC tools such as WFM and WVR
– What do our customers do today?
– They use logging
– They use SNMP traps
 Broadcasters are moving to a Service Orientated Workflow
– Using more file based technology
– File base acquisition
– Panasonic P2 Cards
– Sony XDCam SxS cards
– File based editing
– Apple Final Cut Pro
– Avid Media Composer
– File based archive
– Isilon
– Omneon MediaGrid
– File based Playout
– Omneon Spectrum
– Grass Valley K2
Wednesday, 08 April 2015
Why Automate Quality Control
 Save time
– Non-linear workflow
– QC can happen at the same time as proxy creation/transcoding
 Save resources
– Skilled Quality Control operators are few and expensive
– Automated QC is a tool for the department
 Give Consistency
– Repeatable results
 QC Operators looking at a broadcast monitor cannot
easily “look inside” the file
– Are the correct parameters of encoding used for compression for video,
audio?
– Are the bit-rates of video, audio and overall correct?
– Syntax errors in the encoding may prevent it playing correctly further on
in the workflow
– Is the metadata correct?
Wednesday, 08 April 2015
Content Verification
Service Level Agreement (SLA) Audit Points
Broadcaster
Ingest
Edit /
Transcode
Playout
Cable
Archive
Post
Encode/Reformat
Ingest
Edit /
Transcode
Playout
Library
Content On Demand
Cerify File based Audit point
Wednesday, 08 April 2015
On Line Service Provider
What does Cerify test?
 Systems level tests
–
–
–
–
Correct Standard and integrity
File size, Bitrate, Playtime, Number of video/audio streams, CableLabs VOD
compliance,
Video and audio PIDs, Signalling of Closed captions
Playtime of video vs playtime of audio
 Video Tests
–
–
–
–
–
–
–
–
Correct encoding Standard, Profile, Level and Syntax checks for encoding errors
GOP structure, Frame rate, Bitrate, Frame size, Interlaced/ progressive, Aspect
ratio
Baseband tests including Gamut levels, Luma, Chroma, Signal levels
Letterbox/ Pillarbox, Playtime
Freeze frame
Color depth, Color format (4:2:0, 4:2:2), Copyright
Field order, Telecine pulldown
Quality - Black frames (lead in, lead out and during the video), blockiness,
Quantization
Wednesday, 08 April 2015
What does Cerify test? - cont
 Audio Tests
–
–
–
–
Correct encoding Standard, Profile, and Syntax checks for encoder errors
Sample rate, Bitrate, Playtime
Number of Channels, Peak and Minimum Signal levels on each channel
Audio Silence (lead in, lead out, during video), clipping, mute, test tones
 Reporting
–
–
–
–
Web based on-screen job reports and detailed drill down to error details
Copy or move file on success or error
Email alerts with test reports
CeriTalk API for fully automated working and integration with Automation /
Asset Management systems
Wednesday, 08 April 2015
Benefits of Automated QC
 Quality Control Strategy
–
–
–
Saves time and money over human playout testing
Works 24/7
Enables all of your file-based content to be checked all of the time
 Provides a Content Filter to catch errors that people can easily miss,
and perform repeatable objective testing
–
–
Performs automatic, consistent and thorough checks of content files against userdefined templates
Allows QC staff to concentrate on 5% of rejected files not watching the 95% of good
files
 Ingest / Transcode / Archiving
–
–
–
Check encoded content at ingest to resolve problems early – prevent problems from
entering your workflow
Monitor your Service Level Agreements and the quality of your supplier’s content
Check before and after transcode, edit and archiving
 Playout
–
–
Check your files are ready to playout
Improves broadcast QoS by minimizing risk of transmission and reception defects
Wednesday, 08 April 2015
Cerify Workflows and
Applications
Wednesday, 08 April 2015
Identifying the workflow
 There are 3 main types of workflow available with Cerify
– Manual job creation and results viewing
– Semi-automatic using Watchfolder/dropbox
–
–
–
–
Manual viewing of results in web GUI
Automatic email reports generated and sent
Automatic XML reports generated and saved on network
Custom Action Scripts
– CeriTalk API
– Uses SOAP
– Large list of members of the Cerify Developer Community – new Partners
added monthly
– Best way to control Cerify
– May require development effort from Automation/Asset Management Partner
 Workflow chosen depends on application/size of facility
Wednesday, 08 April 2015
Cerify Workflow – Network Setup
Wednesday, 08 April 2015
Manual Workflow – monitoring results
 Files are added to MediaSets manually
 Jobs are created manually
 Results are reviewed by an operator
Wednesday, 08 April 2015
“See and Solve” Test Results
Test summary for
each file in “Job”
Details for each file
within the “Job”
Alert traffic
lights

Details of Cerify results are
displayed on the user GUI in
levels

The amount of detail
increases as you “drill down”
to lower levels
Video thumbnails and audio
waveform shown in frames
surrounding the alert
Thumbnail of
erroneous frame
Alerts for each error
within the file
Details of a specific
error alert
Wednesday, 08 April 2015
Semi-Automatic Workflow – Cerify Email Reports

Files are added to MediaSets automatically using “watch folders” or “drop
boxes”

Jobs are created manually using these mediasets

Results are sent by email and reviewed by an operator

The operator can “click through” from the email to the Cerify web GUI for
thumbnail viewing
Wednesday, 08 April 2015
Semi-Automatic Workflow – Cerify XML Reports
 Files are added to MediaSets automatically using “watch folders” or
“drop boxes”
 Jobs are created manually using these mediasets
 Results are stored on a server in XML
 They can be reviewed by an operator or picked up by an asset
management system
 The operator can “click through” from the XML to the Cerify web GUI
for thumbnail viewing or an asset management system can use these
URLs to gather the thumbnails
Wednesday, 08 April 2015
Automated Workflow – using the CeriTalk API

Tektronix has a Partner program to aid Cerify Integration
– This is the Cerify Developer Community or CDC

3rd Party Applications (typically Asset Management) control Cerify
– File is ingested
– Job is created on Cerify by automation system
– Results are fed back to automation system
Wednesday, 08 April 2015
Cerify Developer Community - CDC
Working together for integrated solutions with Cerify automated QC
Note: The community is growing rapidly. Please check with Tektronix for latest list or visit http://www.tek.com/products/video_test/cerify/cdc.html
Wednesday, 08 April 2015
Case Study - VoD
 Overview
– The customer encodes a master file in HD
– SD and HD low bitrate versions are made from the master file
– The master file is archived for later re-use
 Workflow
–
–
–
–
High resolution/high bitrate master file ingested manually
Encoded file is moved to a high bitrate watch folder
Transcoder picks up the master file and creates 2 low bitrate versions
Auto Quality Check tests the files in these watch folders for encoding
syntax, parameters and various baseband values and sends emails on
error to alert QC Operator
– Dolby DP600 used for Dolby AC-3 encoding, correction of audio
metadata and levels
 Cost benefits of auto QC
– As volume of media goes up costs can remain low
– The complete file is tested not just 3 or 5 point spot check
Wednesday, 08 April 2015
Workflow Example - VoD
Wednesday, 08 April 2015
Case Study – Broadcast Playout Centre

Overview
– Tape and file ingest controlled by Media Asset Management and automation
system
– Important decisions on the format taken e.g. AVC-Intra, IMX, MXF, XDCam
– Can be complex and highly automated
– Uses Cerify’s web services SOAP API
– Low number of transcode/un-wrap/re-wrap steps

Workflow
–
–
–
–
–

Asset Management requests media to be ingested
After ingest job is created on Cerify for testing
Cerify is polled for the results/progress
Results are provided on completion as XML over SOAP
Results are displayed in GUI
Cost benefits of auto QC
– Clear penalties for playout centre failing to schedule a show or commercial –
Service Level Agreements
– Quality of the channel brand upheld
Wednesday, 08 April 2015
Workflow Example - Broadcast Playout Centre
Wednesday, 08 April 2015
How fast is Cerify?
Wednesday, 08 April 2015
Cerify Benchmarks
 Cerify Lite is difficult to bench mark because the hardware is customer
supplied – it will process one file at a time
– Typically on a 3GHz processor with 2Gb RAM
– 50Mb/s SD MPEG2 is realtime
– 12Mb/s SD MPEG2 is 0.5 x realtime
– 8MB/s SD H.264 is 2x realtime
 CYC200 & CYM200 will process 4 files at the same time
– 50Mb/s SD MPEG2 is realtime
– 12Mb/s SD MPEG2 is 0.5 x realtime
– 8MB/s SD H.264 is 2x realtime
Wednesday, 08 April 2015
Cerify Processing
 To define the speed
– Firstly the codec
– MPEG2 is faster than H.264
– The syntax testing takes the most time
– Secondly the bitrate and complexity
– I frame only is simpler than long GOP
– Low bitrate is quicker than high
– Thirdly the resolution
– HD takes longer to process than SD
– Fourthly – the baseband tests
– Colour Gamut and luminance tests take more processing power than black
frame detection
 The most important test is the syntax
– This takes most of the processing
 Limit the number of tests switched on
– Limit the baseband tests to those most appropriate
Wednesday, 08 April 2015
Questions?
Wednesday, 08 April 2015