Topics in Signal Processing
Course ID: EE5359
Project Proposal: HEVC
Lossless Coding and
Improvements
SUBMITTED BY: SUJATHA GOPALAKRISHNAN
STUDENT ID: 1001024145
Table of Contents
ACRONYMS: ................................................................................................................................. 3
HEVC .............................................................................................................................................. 5
Block Diagram HEVC .................................................................................................................... 5
HEVC Lossless Coding .................................................................................................................. 6
Block Diagram HEVC Lossless Coding ......................................................................................... 7
Basic Definitions ............................................................................................................................. 7
H.264[1][2] ................................................................................................................................. 7
Inter Frame .................................................................................................................................. 7
Intra Frame .................................................................................................................................. 7
Loop Filters ................................................................................................................................. 7
De-blocking Filter ................................................................................................................... 8
Sample Adaptive Offset .......................................................................................................... 8
Block-Based Angular Intra Prediction ........................................................................................ 8
Sample-Based Angular Intra Prediction ..................................................................................... 8
Coding Tools ............................................................................................................................... 8
LCU/CTU ............................................................................................................................... 9
Parallel Processing .................................................................................................................. 9
Entropy Coding ....................................................................................................................... 9
Motion Estimation .................................................................................................................. 9
Motion Compensation........................................................................................................... 10
Algorithm of Sample Based Angular Intra Prediction .................................................................. 10
Proposed Project ........................................................................................................................... 12
Projected Enhancement in the Project .......................................................................................... 12
References ..................................................................................................................................... 12
ACRONYMS:
Two dimension
3D
Three dimension
ACM MoVid Association for Computer Machinery Mobile Video
AHG
Ad Hoc Groups
AIF
Adaptive Interpolation Filter
ALF
Adaptive Loop Filter
Advanced Motion Vector Prediction
AMVP
APIF
Adaptive Pre-Interpolation Filter
ASIC
Application-Specific Integrated Circuit
Advanced Video Coding
AVC
AVS
Audio Video Standard
BBC
British Broadcasting Corporation
BD
Bjontegaard Distortion
BL
Base Layer
bpp
Bits per pixel
BS
Boundary Strength
CU
Coding Unit
CI
Confidence Interval
CABAC
Context Adaptive Binary Arithmetic Coding
Central Processing Unit
CPU
CRA
Clean Random Access
CSVT
Circuits and Systems for Video Technology
CU
Coding Unit
Discrete Cosine Transform
DCT
DCTIF
Discrete Cosine Transform Interpolation Filters
DDCT
Directional Discrete Cosine Transform
DSP
Digital Signal Processing
DST
Digital Sine Transform
EC
Error Concealment
Finite Impulse Response
FIR
FPGA
Field Programmable Gate Array
fps
Frames per second
GPU
Graphics Processing Unit
HDR
High Definition Range
High efficiency video coding
HEVC
HEVStream High Efficiency Video Stream
HTTP
Hyper Text Transfer Protocol
IEEE Conference on Industrial Electronics and Applications
ICIEA
Institute of Electrical and Electronics Engineers
IEEE
INTDCT
Integer Discrete Cosine Transform
intra HE
Intra high efficiency
IPTV
Internet Protocol Television
IS & T
Information Systems and Technology
2D
ISO
ITU-T
IVMSP
JCTVC
JM
JPEG
JPEG-XR
JSVM
JTC
LR
Mbit/s
MC
MDDCT
MDDT
ME
MJPEG
MMSP
MPEG
Mpixel
Mpm
MV
NAB
NAL
PCM
PSNR
PU
QP
RD
RDOQ
RDPCM
ROT
RTP
SAO
SHVC
SVC
SELC
SIP
SSVC
TB
TU
International Organization for Standardization
Telecommunication Standardization Sector of the International
Telecommunications Union
Image, Video, and Multidimensional Signal Processing
Joint Collaborative Team on Video Coding
Joint Model
Joint Photographic Experts Group
JPEG extended range
Joint Scalable Video Model
Joint Technical Committee
Low Resolution
Megabit per second
Motion Compensation
Modified Directional Discrete Cosine Transform
Mode-Dependent Directional Transform
Motion Estimation
Motion JPEG
Multimedia Signal Processing
Moving Picture Experts Group
Megapixel
Most Probable Modes
Motion Vector
National Association of Broadcasters
Network Abstraction Layer
Pulse Code Modulation
Peak-to-peak signal to noise ratio
Prediction Unit
Quantizer parameter
Rate Distortion
Rate-distortion optimized quantization
Residual Differential Pulse Code Modulation
Rotational Transform
Real-time Transport Protocol
Sample adaptive offset
Scalable High Efficiency Video Coding
Scalable Video Coding
Sample based weighted prediction for Enhancement Layer Coding
Signal and Image Processing
Spatially Scalable Video Coding
Transform Block
Transform Unit
HEVC





High Efficiency Video Coding (HEVC) [1] [2] is a video compression standard, a
successor to H.264/MPEG-4 AVC [22]. HEVC is said to double the data
compression ratio compared to H.264/MPEG-4 AVC [1] at the same level of video quality
[2].
The design of most video coding standards is primarily aimed at having the highest coding
efficiency
HEVC benefits from the use of larger Coding Tree Unit (CTU) sizes.
The HEVC video coding layer uses the same "hybrid" approach used in all modern video
standards, starting from H.261 [1], in that it uses inter-/intra-picture prediction and 2D
transform coding.
The main goal of the HEVC standardization effort is to enable significantly improved
compression performance relative to existing standards, in the range of 50% bit rate
reduction for equal perceptual video quality [10] [11].
Block Diagram HEVC
Figure 1: HEVC Encoder [2]
Figure 2: HEVC Decoder Block Diagram [3]
Some differences in HEVC [1][2] are coding tree units instead of macro blocks, single entropy
coding methods-Context Adaptive Binary Arithmetic Coding (CABAC) [15] method and features
like tiles , wave front parallel processing and dependent slices to enhance parallel processing.
HEVC Lossless Coding





The lossless coding mode of HEVC main profile bypasses transform quantization and
in-loop filters as shown in the fig.2 [4][19].
Comparing it with non-lossless coding mode, it has smallest quantization parameter value.
Lossless coding mode provides perfect fidelity and average bit rate reduction.
Outperforms the existing lossless compression solution such as JPEG-2000 [22] and
JPEG-LS [22].
It employs Sample Angular-based Intra-Prediction (SAP) [4].
 Same prediction mode signaling method.
 Same interpolation method of HEVC.
 Uses adjacent neighbors as reference shown in fig.7.
Block Diagram HEVC Lossless Coding
Figure 3: HEVC lossless Algorithm Block Diagram [4]
The blocks that are marked bypass are not being used when implementing a HEVC [1][2] lossless
algorithm, thereby providing average bit rate reduction.
Basic Definitions
H.264 [1][2]
H.264/MPEG-4 AVC [1] [2] [22] is a block-oriented, motion-compensation based video
compression standard.
Inter Frame
An inter frame is a frame in a video compression stream which is expressed in terms of one or
more neighboring frames. The "inter" part of the term refers to the use of Inter frame prediction.
Intra Frame
The term intra-frame refers to the various lossless and lossy compression techniques that happens
relative to information which is contained only within the current frame and not relative to any
other frame in the video sequence.
Loop Filters
HEVC [1] specifies two loop filters that are applied sequentially; the de-blocking filter (DBF) [4]
applied first and the sample adaptive offset (SAO) filter applied afterwards. Both loop filters are
applied in the inter-picture prediction loop, i.e. the filtered image is stored in the decoded picture
buffer (DPB) as a reference for inter-picture prediction.
De-blocking Filter
 The DBF is similar to the one used by H.264/MPEG-4 AVC [1] [2], but with a simpler
design and better support for parallel processing.
 DBF first apply horizontal filtering for vertical edges to the picture and only after that does,
it apply vertical filtering for horizontal edges to the picture. This allows for multiple
parallel threads to be used for the DBF [1].
Sample Adaptive Offset
The SAO filter is applied after the DBF and is designed to allow for better reconstruction of the
original signal amplitudes by applying offsets stored in a lookup table in the bit stream.
Block-Based Angular Intra Prediction
It is a method of computing predicted samples produced by PU when lossless coding is not
enabled. It is defined to exploit spatial sample redundancy in intra coded CUs. As shown
in the fig.4, a total of 33 angles are defined for the angular prediction, which can be
categorized into two classes: vertical and horizontal angular predictions as illustrated [14].
Figure 4: Block Based Angular Intra Prediction in HEVC [4]
Sample-Based Angular Intra Prediction
It is a method of computing predicted samples produced by PU when lossless coding is enabled. It
is explained detail in the following.
Coding Tools
Coding efficiency is the ability to encode video at the lowest possible bit rate while maintaining a
certain level of video quality. This could be achieved with the following coding tools.
LCU/CTU
Coding tree unit (CTU) is the basic processing unit of the HEVC video standard and conceptually
corresponds in structure to macroblock units, which were used in several previous video
standards. CTU is also referred to as largest coding unit (LCU) [1] [2]. In the HEVC, one frame is
divided into a series of non-overlapped Coding Tree Unit (CTU) [9] [12].
Figure 5: Division of a CTB into CBs and transform blocks TB [2]
Parallel Processing
A picture is divided into tiles. Main purpose of these tiles is that, they can be decoded /encoded
individually in a simultaneous way called parallel processing. Parallel computing is basically a
technique in which multiple computation tasks are assigned to multiple processes and process the
job simultaneously. The basic approach for parallel processing is to break the task into multiple
smaller tasks and further assign each task to each of the thread which performs required operations
in parallel. Parallelization can sometimes get complicated due to race conditions, data dependency,
synchronization and communication among different threads [13].
Entropy Coding
HEVC uses a context-adaptive binary arithmetic coding (CABAC) algorithm that is
fundamentally similar to CABAC in H.264/MPEG-4 AVC. CABAC is the only entropy encoder
method that is allowed in HEVC while there are two entropy encoder methods allowed by
H.264/MPEG-4 AVC. CABAC and the entropy coding of transform coefficients in HEVC are
designed for a higher throughput than H.264 while maintaining higher compression efficiency for
larger transform block sizes relative to simple extensions [15][16]. These techniques include
reducing context coded bins, grouping bypass bins, grouping bins with the same context, reducing
context selection dependencies, reducing total bins, and reducing parsing dependencies. It also
describes reductions to memory requirements that benefit both throughput and implementation
costs.
Motion Estimation
Motion estimation [5] is an essential process in many video coding standards like MPEG-2,
H.264/AVC and HEVC [1] [2]. Motion estimation has been used at the encoder. Motion
Estimation itself consumes more than 50% coding complexity or time to encode. To reduce the
computation time, many fast motion estimation Algorithms were proposed and implemented
[5]. HEVC allows for two MV modes which are Advanced Motion Vector Prediction (AMVP) and
merge mode. AMVP uses data from the reference picture and can also use data from adjacent
prediction blocks. The merge mode allows for the MVs to be inherited from neighboring
prediction blocks. Merge mode in HEVC is similar to "skipped" and "direct" motion inference
modes in H.264.
Motion estimation process in HEVC consumes more than 50% coding complexity or time to
encode with equal perceptual quality [6] [7]. Many block based motion estimation algorithms [8]
[9] are proposed and also implemented to reduce the computation time.
Figure 6: Illustration of Motion Estimation process [5]
Motion Compensation
The interpolation of fractional luma sample positions HEVC uses separable application of
one-dimensional half-sample interpolation, with an 8-tap filter or quarter-sample interpolation
with a 7-tap filter [5]. While H.264/MPEG-4 AVC[1][2] uses a two-stage process that first derives
values at half-sample positions, using separable one-dimensional 6-tap interpolation followed by
integer rounding; then applies linear interpolation between values at nearby half-sample positions
to generate values at quarter-sample positions. HEVC has improved precision due to the longer
interpolation filter and the elimination of the intermediate rounding error. As in H.264/MPEG-4
AVC [1] [2], a scaling and offset operation may be applied to the prediction signal(s) in a manner
known as weighted prediction [1].
Algorithm of Sample Based Angular Intra Prediction
The SAP [4] is designed to better exploit the spatial redundancy in the lossless coding mode
by generating intra prediction samples from adjacent neighbors. The design principle here is
very similar to the sample-based DPCM in [21][4] H.264/MPEG-4 AVC [20][4] lossless
coding, but SAP[4] is fully harmonized with the HEVC block-based angular intra prediction,
and can be applied to all the angular intra prediction modes specified in HEVC [4].
As shown in the fig.7 SAP is performed sample by sample. The adjacent neighboring samples
, of the current sample in the current PU are used for prediction. That is, the reference samples
used for prediction are not limited to those boundary reference samples from the left and upper
neighboring PUs. The SAP has to be processed in a predefined order to ensure the availability of
these adjacent neighbors for prediction.
Figure 7: Algorithm of SAP [4]
Proposed Project
The aim of this proposal is to analyze HEVC lossless coding. Hence if a scheme is
designed that will reduce the computation time using HEVC lossless coding it will further increase
its performance and reduce encoding time without much increase in complexity [21].
The simulation will be conducted using HM Software 16.0 [18] [26], with different video
sequences [29], search range, block sizes and number of frames using GPU multicore computing.
Bit rate differences, Compression ratio, Distribution of intra-coded ratio (ratio between
intra-coded and coded blocks), and results based on it, will be calculated for different algorithms
using various search patterns.
Projected Enhancement in the Project
 Performance analysis of HEVC Lossless Mode using HM16.0 [18] software.
 Performance analysis of SAP [4] using HM16.0 [18] software.
 Complete Study of HEVC Lossless Coding [4].
References
[1] G.J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE
Trans. CSVT, vol. 22, pp.1649-1668, Dec 2012.
[2] G.J. Sullivan et al, “Standardized Extensions of High Efficiency Video Coding (HEVC)”,
IEEE Journal of selected topics in Signal Processing, vol.7, pp.1001-1016, Dec 2013.
[3] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC Joint Collaborative
Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July 2012.
[4] M. Zhou et al, “HEVC lossless coding and improvements”, IEEE Trans. CSVT, vol.22,
pp.1839-1843, Dec 2013.
[5] N. Purnachand et al, "Fast Motion Estimation Algorithm for HEVC", IEEE Second
International Conference on Consumer Electronics-Berlin (ICCE-Berlin), vol.11, pp.34-37, Sep
2012.
[6] P. Hanhart et al, “ Subjective quality evaluation of the upcoming HEVC video compression
standard”, SPIE Optical Engineering+ Applications. International Society for Optics and
Photonics, vol. 8499, pp.84990v-84990v, Aug 2012.
[7] M. Horowitz et al, “Informal subjective quality comparison of video compression performance
of the HEVC and H.264/MPEG - 4 AVC standards for low delay applications”, SPIE Optical
Engineering+ Applications. International Society for Optics and Photonics, vol.84990,
pp.84990w-84990w, Aug 2012.
[8] A. Abdelazim, W. Masri and B. Noaman., "Motion estimation optimization tools for the
emerging high efficiency video coding (HEVC)", IS&T/SPIE Electronic Imaging. International
Society for Optics and Photonics, vol. 9029, pp. 902905-902905, Feb 17 2014.
[9] M. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms-a survey”,
Journal of Opto-Electronics Review, vol. 21, pp.86-102, Mar 2013.
[10] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”,
Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC),
Jan. 2013, available on
http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=7243
[11] J. Ascenso et al, "Improving Frame Interpolation with Spatial Motion Smoothing for Pixel
Domain Distributed Video Coding", 5th EURASIP Conference on Speech and Image Processing,
Multimedia Communications and Services, pp.1-6, Smolenice, Slovak Republic, July 2005.
[12] X. Wang et al, “Paralleling Variable Block Size Motion Estimation of HEVC on Multicore
CPU plus GPU platform”, IEEE International Conference on Image Processing (ICIP), vol.22, pp.
1836-1839, Sep. 2013.
[13] Introduction to parallel computing
https://computing.llnl.gov/tutorials/parallel_comp/#Whatis
[14] L. Zhao et al, “Group-Based Fast mode decision algorithm for intra prediction in HEVC”,
IEEE Eighth international Conference on Signal Image Technology and Internet based Systems.
Article no.6115979, pp. 225-229, Nov 2011.
[15] V. Sze and M. Budagavi, "High Throughput CABAC Entropy Coding in HEVC", IEEE
Transactions on Circuits and Systems for Video Technology, vol.22, no.12, pp.1778-1791, Dec
2012.
[16] T.Nguyen et al, "Transform Coding Techniques in HEVC", IEEE Journal of Selected Topics
in Signal Processing, vol.7, pp.978–989, Dec 2013.
[17] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html
[18] HEVC Reference Software HM16.0.
https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.0rc1/
[19] B. Bross et al,“High Efficiency Video Coding (HEVC)Text Specification Draft 8”, JCT-VC
document, JCTVC-J1003, Stockholm, Sweden, Jul. 2012.
http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=6465
[20] Joint Video Team, “Advanced Video Coding for Generic Audiovisual Services”, ITU-T Rec.
H.264 and ISO/IEC, 14496-10 (MPEG-4) AVC, pp.H.100-H.869, Feb 2014.
[21] Y.L. Lee et al, "Improved lossless intra coding for H.264/MPEG-4 AVC", IEEE Trans. Image
Process., vol.15, no.9, pp.2610-2615, Sep 2006.
[22]K.R. Rao, D.N. Kim and J.J Hwang, “High Efficiency Video Coding (HEVC)
Revised/Updated Chapter from the book Video Coding Standards”–Springer 2014.
[23] ITU-T website: http://www.itu.int/ITU-T/index.html
[24] JCT-VC documents are publicly available at http://ftp3.itu.ch/av-arch/jctvc-site and
http://phenix.it-sudparis.eu/jct/
[25] V.Sze, M.Budagavi, and G.J. Sullivan “High Efficiency Video Coding (HEVC)” Springer,
2014.
[26] Software reference manual for HM:
https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-9.2-dev/doc/software-man
ual.pdf
[27] M. Wien, “High efficiency video coding: Tools and specification”, Springer, 2015.
[28] I.E. Richardson, “Coding video: A practical guide to HEVC and beyond”, Wiley, 11 May
2015
[29] Video Sequences:
http://ultravideo.cs.tut.fi/