2018 IEEE International Conference on Consumer Electronics (ICCE)
Optimization of HEVC λ-domain Rate Control
Algorithm for HDR Video
Junaid Mir, Dumidu S. Talagala, and Anil Fernando
Center for Vision Speech and Signal Processing, University of Surrey, United Kingdom
Email: {j.mir, d.talagala, w.fernando}@surrey.ac.uk
Abstract—Rate Control (RC) will play an important role in
conceiving high-fidelity HDR video distribution through transmission and broadcast in imminent future. However, as video coding standards, like HEVC, are generally designed and optimized
for the LDR content, this can result in an inefficient HDR video
compression in the Rate-Distortion (RD) sense. In this paper,
the state-of-the-art HEVC λ-domain RC algorithm is optimized
for HDR video coding by proposing a new λ−QP relation
after investigating the suitable RD model for HDR content.
The updated RC algorithm with proposed relation outperforms
the default HEVC RC algorithm achieving averaged PU-PSNR
improvements of up to 1.36 dB for HDR video test sequences.
Further, performance improvement is also evident from the HDRVDP-2.2 Q and HDR-VQM based RD performance curves.
I. I NTRODUCTION
High Dynamic Range (HDR) video is considered to be
the next frontier in digital multimedia that seeks to provide
the high-fidelity representation of the real-world scene. As
HDR video stores the light levels and colours inherent in
the original capture of content, it offers the quality and
sensation closer to true-to-life visual experience. Foreseeing
this, support for HDR video has been shown by all major
Hollywood film studios from Universal to Warner Bros, and
video streaming services like Netflix, Vudu, etc. Although
there is sufficient Over-the-Top (OTT) HDR content available,
to make HDR content consumption by masses a reality, HDR
video distribution through broadcast or transmission is certain
in the near future. This is also vital for the wide adoption
of HDR display technology among end-consumers which is
already becoming mainstream in the consumer electronic (CE)
market.
Rate Control (RC) plays a pivotal role in high-fidelity video
distribution via video communication channel as it allows
the efficient utilization of the available channel bandwidth
while maintaining the optimal video quality. Considering the
essential importance of RC to a video encoder, RC algorithm is
generally included in all video coding standards including the
state-of-the-art High Efficiency Video Coding (HEVC) [1]. An
HEVC compliant encoder utilizes λ-domain model [2] in RC
algorithm which is particularly adopted for the HEVC due to
its suitability and increased Rate-Distortion (RD) performance
in comparison to the conventional Q-domain model [3] and
ρ-domain model [4] adopted in the previous video coding
standards.
Work is supported by the EU H2020 project CONTENT4ALL funded
under H2020-INDUSTRIAL LEADERSHIP program. (Project ID: 762021).
Although HEVC λ-domain RC algorithm can be utilized
for HDR video distribution, it is designed and its performance
is optimized for Low Dynamic Range (LDR) content. For
example, the RD relation in RC algorithm is characterized
by the Hyperbolic function [2] which is known to result
in an acceptable approximation error relative to the actual
measurements for LDR video coding. As Hyperbolic function
is only validated to fit the RD curve in which distortion D is
expressed in terms of Mean Square error (MSE), it might not
be optimal for HDR video coding as LDR measure like MSE
is well known for its inability to measure HDR distortions
[5]. Further, the λ-QP relation [6] utilized in the RC algorithm
plays a critical role in RC coding performance as the resulting
value of QP derived from the λ would set the quantization
step and thereby the resulting distortion. The relationship is
presently modelled for the LDR video coding and needs to
be updated for HDR video coding. In this paper, we address
these problems to optimize the HEVC λ-domain RC algorithm
for HDR video coding. To the authors best of knowledge, this
work is the first attempt towards investigating and optimizing
the RC algorithm in HEVC for the HDR video. We first
investigate an appropriate model able to characterize the RD
relation for HDR content. Then, the relationship between λ
and QP is evaluated for HDR video coding to optimize the
RC algorithm for HDR content.
The rest of the paper is organized as follows. Section II
presents the RD model and λ−QP relation to be utilized
in the HEVC λ-domain RC algorithm for efficient HDR
video coding. Section III presents the experimental results and
discussion, and is followed by the conclusion in Section IV.
II. HEVC λ- DOMAIN RC A LGORITHM FOR HDR VIDEO
To optimize the RC Algorithm for HDR video coding, we
first find the relationship between bitrate R and HDR distortion D to investigate whether currently employed Hyperbolic
function in HEVC λ-domain RC algorithm can model the RD
relationship for HDR video coding. Then, we find new λ−QP
relation for HDR video coding.
A. RD Analysis for HDR video
To analyse the RD relationship for HDR video, we encoded
different HDR sequences with multiple QP values using IBBB
and IPPP coding structures with only one reference picture.
The resulting HDR distortion D expressed in the terms of PUMSE [7] and bitrate R expressed in the terms of bpp (bits per
978-1-5386-3025-9/18/$31.00 ©2018 IEEE
0
PU-MSE
PU-MSE
RD Curve
Fitted RD Curve
30
25
20
15
10
5
0
0.002 0.004 0.006 0.008 0.01
Bitrate (bpp)
RD Curve
Fitted RD Curve
0.01
0.02
0.03
Bitrate (bpp)
Balloon
y = 1.294x-0.467
R² = 0.9802
RD Curve
Fitted RD Curve
0
FireEater2
y = 0.124x-0.849
R² = 0.9957
0
70
60
50
40
30
20
10
0
0.005
300
250
200
150
100
50
0
0.01 0.015
Bitrate (bpp)
QP
SUN
y = 0.1037x-0.859
R² = 0.9909
PU-MSE
PU-MSE
70
60
50
40
30
20
10
0
0.02
Market3
RD Curve
Fitted RD Curve
0
0.02
0.04
0.06
Bitrate (bpp)
80
SUN
y = 0.1243x-0.868
R² = 0.9944
40
PU-MSE
PU-MSE
80
RD Curve
Fitted RD Curve
20
0
0
0
40
0.002 0.004 0.006 0.008 0.01
Bitrate (bpp)
0
FireEater2
y = 0.1203x-0.881
R² = 0.9971
30
20
PU-MSE
PU-MSE
RD Curve
Fitted RD Curve
20
RD Curve
Fitted RD Curve
10
0
0
0.01
0.02
0.03
Bitrate (bpp)
0.04
0.01
300
250
200
150
100
50
0
0.02
0.03
Bitrate (bpp)
0.04
Market3
y = 3.7291x-0.749
R² = 0.9982
RD Curve
Fitted RD Curve
0
0.03
0.06
0.09
Bitrate (bpp)
0.12
(b)
Fig. 1. RD curve fitting according to the Hyperbolic function for HDR video
encoded in (a) IBBB coding structure, and (b) IPPP coding structure.
pixel) are calculated by using Eq. 1 and Eq. 2, respectively,
given as:
R
bpp =
(f × W × H)
D=
2
1 PU(Li ) − PU(Li )
N i
6
8
ln (ߣ)
10
12
Relationship between λ and QP through curve fitting.
RD relationship for HDR video. Therefore, HEVC λ-domain
RC algorithm, whose foundation is laid on the existence of
robust correspondence between λ and R derived from the
Hyperbolic function based RD relation, can be utilized for
HDR video coding.
Balloon
40
4
0.08
y = 1.8782x-0.44
R² = 0.9774
60
λ-QP Points
Fitted Curve
Fig. 2.
(a)
60
y = 2.9638x + 10.52
2
y = 2.3711x-0.793
R² = 0.9968
0.04
50
45
40
35
30
25
20
15
B. λ−QP model for HDR video
To find the relationship between the λ and QP, we encoded
the HDR sequences with flat QP values of 22, 27, 32, 37 and
41. The flat QP assures that all frames of the tested HDR
sequences are encoded with the same QP irrespective of their
hierarchical position in the GOP. This would result in a single
λ value being used for all the coding levels for a particular
QP used for the encoding. Then, the Hyperbolic function is
fitted on the consumed bitrate R (as a result of the tested QP
value) and the resulting distortion D in PU-MSE, as shown in
the Fig. 1(a) and Fig. 1(b). The resulting Hyperbolic function
defining the relationship between bitrate R and HDR distortion
D is given as:
(3)
D(R) = CR−K
where C and K are the model parameters related to the
characteristics of the source and R is expressed in bpp. As
λ is the slope of RD curve, it can be computed as:
(1)
∂D
(4)
∂R
substituting Eq. 3 in Eq. 4 and calculating partial derivative
will yield λ value as:
(2)
λ = CKR−K−1
λ=−
where R is the consumed bitrate at fixed QP, f is the frame
rate, W and H are
the width and height of the picture respec
tively. Li and Li are the original and reconstructed HDR pixel
values representing linear luminance values of the encoded
picture. PU (L) is the luminance value in the Perceptually
Uniform (PU) domain after which pixel difference metrics like
MSE can be used to measure HDR distortions.
Fig. 1(a) and Fig. 1(b) presents the original RD curve
obtained from the experiments and fitted RD curve according
to the Hyperbolic function for HDR sequences encoded in
the IBBB and IPPP coding structures, respectively. It can be
seen from the fitted curves and the corresponding correlation
coefficient values that Hyperbolic function can characterize the
(5)
Eq. 5 is used to calculate the λ value for each of the
tested QP value for different HDR sequences encoded in the
IBBB and IPPP coding structures. Resulting λ−QP points
for all the tested HDR sequences are shown in the Fig. 2
along with the curve fitted to the data to find the optimized
relationship between λ and QP for HDR video content. The
new relationship to be used to calculate QP value from the λ
value in HEVC RC algorithm for HDR video coding is
QP = 2.9638 × ln(λ) + 10.52
(6)
The resulting relationship has different coefficients in comparison to the relationship in the default RC algorithm defined as:
QPdef ault = 4.2005 × ln(λ) + 13.7122
(7)
56
HDR-VDP-2.2 Q
0
66
100
200 300
Bitrate (kbps)
61
Default RC
Updated RC
51
0
500 1000 1500 2000
Bitrate (kbps)
61
0
63
2000 4000 6000
Bitrate (kbps)
0.2
Default RC
0.1
Updated RC
0
0
8000
(d) Market3
HDR-VQM
Default RC
Updated RC
62
400
(c) FireEater2
56
63
(a) SUN
0.3
0.09
100 200 300
Bitrate (kbps)
400
Default RC
Updated RC
53
48
0
2500 5000 7500 10000
Bitrate (kbps)
Fig. 3. HDR-VDP-2.2 Quality (Q) based RD curves for the default RC and
the updated RC algorithm in the reference HEVC encoder.
The new λ−QP relationship given by Eq. 6 for HDR video
coding would result generally in smaller QP in comparison
to the default λ−QP relation given by Eq. 7. As the use
of smaller QP would result in more bitrate utilization than
initially assigned at the beginning of rate-controlled encoding,
this would result in value of λ being adjusted thereby opting
different coding structure than the default RC algorithm. This
is indicated in the next section where we present results for the
default and updated RC algorithm to assess the effect of the
proposed change in the λ−QP relation for HDR video coding.
III. R ESULTS AND D ISCUSSIONS
To evaluate performance of the RC algorithm with proposed
λ−QP relation for HDR video coding, HEVC reference encoder software HM 16.2 is utilized. 10-bit Random Access
(RA) Main profile with GOP size of 8 and Low Delay (LD)
Main profile with GOP size of 4 are utilized for encoding
nine HDR video test sequences at 4 different rate points.
The 10-bit YUV 420 representation of the HDR video test
sequences is generated through Perceptual Quantizer (PQ)
based HDR video broadcast chain as detailed in [8]. For
an objective evaluation of the updated RC algorithm (with
proposed λ−QP relation integrated in HEVC) with default
HEVC RC algorithm, the HDR perceptual metrics HDR-VDP2.2 [9], HDR-VQM [10] and PU-PSNR [7] are used.
Fig. 3 shows HDR-VDP-2.2 Q factor based and Fig. 4
presents HDR-VQM based (lower value is better) RD performance curves for the updated and default HEVC RC algorithms for few HDR video sequences. RD curves reveal, that at
the same bitrate, HDR quality is improved in comparison to the
default RC algorithm. Further, Table I shows the improvement
in the HDR video quality in terms of the PU-PSNR computed
through BD metric [11]. An average quality gain of 1.24 dB
and 1.50 dB is achieved for the HDR videos encoded in the
RA and LD coding structures, respectively.
Results suggest that, with the new relationship between λ
and QP, the coding efficiency of the HEVC RC algorithm is
improved for the HDR video sequences. To understand this
improvement, we studied the resulting λ values used inside
Default RC
Updated RC
0.07
0.06
0
500 1000 1500 2000
Bitrate (kbps)
(b) Balloon
Default RC
Updated RC
0
(c) FireEater2
0.08
58
0.25
0.2
0.15
0.1
0.05
0
HDR-VQM
Updated RC
0.4
(b) Balloon
64
HDR-VQM
Default RC
60
65
HDR-VQM
64
HDR-VDP-2.2 Q
(a) SUN
HDR-VDP-2.2 Q
HDR-VDP-2.2 Q
68
2000 4000 6000
Bitrate (kbps)
8000
(d) Market3
0.4
0.3
0.2
Default RC
Updated RC
0.1
0
0
2500 5000 7500 10000
Bitrate (kbps)
Fig. 4. HDR-VQM quality (lower value is better) based RD curves for the
default RC and the updated RC algorithm in the reference HEVC encoder.
the HEVC to interpret the implication of the new relationship.
Fig. 5 shows some comparison of the λ values used per picture
of the HDR sequences encoded in the LD coding structure for
the default and the updated RC algorithm. It can be seen from
the Fig. 5 that for the proposed λ−QP relation, the λ value
used inside the updated RC algorithm is generally larger than
the default RC algorithm. This is due to the new relationship
which will result in a lower QP value in comparison to the
QP value used in the default RC algorithm for the same λ.
The use of lower QP value will result in more bitrate being
utilized than initially assigned for that picture in the updated
RC algorithm. This will make λ value for next pictures larger
to adjust the bitrate for the remaining pictures in the GOP.
When the λ value is large, RDO process may tend to select
a mode costing smaller number of bits as the best mode which
means that coding structure and bitrate utilization per picture
will be different in the updated RC algorithm from the default
RC algorithm. This is shown in Fig. 6 which presents the
bits consumed per picture in RC algorithms. It can be seen
that bits per picture utilized for the default and the updated
RC algorithms are different. This could be either due to the
different pre-allocation of the bitrate in the default and the
updated RC algorithms or simply due to the inefficiency of RC
TABLE I
AVERAGE BD GAIN IN PU-PSNR HDR QUALITY METRIC
HDR Sequences
SUN
Tunnel
Hallway
ExArea
Students
Balloon
FireEater2
Tibul2
Market3
Average
BD-PUPSNR (dB)
Random Access
Low Delay
1.0191
1.1466
1.1260
1.5905
1.1251
1.3183
0.9198
1.4385
1.5469
1.9686
0.8360
0.9162
2.4951
2.4574
1.1509
1.230
0.9123
1.3939
1.2368
1.4960
100
Fig. 5.
14
20
λ per picture for Sun and FireEater2 HDR video sequences.
80
SUN
13
12
Default RC
Updated RC
11
100
Fig. 6.
FireEater2
76
72
20
algorithm in achieving the pre-allocated bitrate. To check this,
we have computed the Normalized Root Mean Square Error
(NRMSE) (defined in [12]) which is the measure of bit estimation accuracy. NRMSE values of the SUN and FireEater2 HDR
video sequences for the default and the updated RC algorithms
are summarized in the Table II where a smaller value means
that better bit estimation result is achieved. From the table,
it can be observed that both RC algorithms have generally
similar bit-budget estimation accuracy which indicates that
different bit per picture utilization in the default and the
updated RC algorithm is due to the different pre-allocation
of the assigned bitrate. This observation is consistent for all
the tested HDR video sequences encoded at different bitrates.
With different bit per picture utilization as shown in Fig. 6,
it is interesting to see that bit cost for certain pictures e.g., in
FireEater2, is less in the updated RC algorithm in comparison
to the default RC algorithm. However this does not imply that
the quality is degraded for these particular pictures. This can
be seen from Fig. 7 which presents the quality per picture
measured in terms of the PU-PSNR (dB) and Q factor of
the HDR-VDP-2.2 metric. It can be seen from the curves
that improvement in the HDR quality is consistent for all
the pictures. These observations which are also seen in the
other HDR video sequences indicates that bitrate allocation
for different coding levels is done intelligently in the updated
RC algorithm. This is due to the proposed λ−QP relation
which is based on the HDR quality and thereby optimizes the
RC algorithm for the HDR video coding and results in an
improved compression efficiency for HDR content.
TABLE II
NRMSE VALUES FOR THE DEFAULT AND THE PROPOSED RC ALGORITHM .
Target Bitrate
400 kbps
400 kbps
1922 kbps
1922 kbps
60
Default RC
Updated RC
58
100
66
64
Default RC
Updated RC
62
Fig. 7.
55
50
45
120
140
160
180
Picture Order Count (POC)
Deafult RC
Updated RC
40
20
120
140
160
180
Picture Order Count (POC)
SUN
68
FireEater2
60
40
60
80
100
Picture Order Count (POC)
FireEater2
68
64
60
Default RC
Updated RC
56
20
40
60
80
100
Picture Order Count (POC)
Quality per picture for Sun and FireEater2 HDR video sequences.
40
60
80
100
Picture Order Count (POC)
Bits per picture for Sun and FireEater2 HDR video sequences.
HDR Sequences
SUN (LD)
SUN (RA)
FireEater2 (LD)
FireEater2 (RA)
62
100
Deafult RC
Updated RC
68
64
120
140
160
180
Picture Order Count (POC)
64
100
40
60
80
Picture Order Count (POC)
HDR-VDP-2.2 Q
120
140
160
180
Picture Order Count (POC)
SUN
PU-PSNR (dB)
200
66
HDR-VDP-2.2 Q
Lambda (ߣ)
Deafult RC
Updated RC
0
100
Bit Cost (Kb)
FireEater2
300
PU-PSNR (dB)
SUN
Default RC
Updated RC
Bit Cost (Kb)
Lambda (ߣ)
50
40
30
20
10
0
Default RC
0.5662
1.051
0.7034
0.7726
Updated RC
0.5645
0.9661
0.7070
0.7423
IV. C ONCLUSION
HEVC λ−domain RC algorithm is updated and optimized
for HDR video coding. A new λ−QP relation is proposed
which better estimates the relationship between the HDR
distortions and the bitrate utilized. As a result, the updated
RC algorithm with the proposed relation results in an improved
HDR quality at the same bitrate in comparison to the default
RC algorithm. Subjective tests will be done in the future work
to further validate the results.
R EFERENCES
[1] G. J. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand, “Overview of the
high efficiency video coding (HEVC) standard,” IEEE Transactions on
Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649–
1668, 2012.
[2] B. Li, H. Li, L. Li, and J. Zhang, “λ− Domain Rate Control Algorithm
for High Efficiency Video Coding,” IEEE Transactions on Image Processing, vol. 23, no. 9, pp. 3841–3854, 2014.
[3] T. Chiang and Y.-Q. Zhang, “A new rate control scheme using quadratic
rate distortion model,” IEEE Transactions on Circuits and Systems for
Video Technology, vol. 7, no. 1, pp. 246–250, 1997.
[4] Z. He, Y. K. Kim, and S. K. Mitra, “Low-delay rate control for DCT
video coding via ρ−domain source modeling,” IEEE Transactions on
Circuits and Systems for Video Technology, vol. 11, no. 8, pp. 928–940,
2001.
[5] J. Mir, G. Kulupana, D. S. Talagala, H. K. Arachchi, and A. Fernando,
“Efficient rate-distortion optimization for hdr video compression,” in
IEEE International Conference on Consumer Electronics (ICCE), 2017,
pp. 256–257.
[6] B. Li, J. Xu, D. Zhang, and H. Li, “QP refinement according to Lagrange
multiplier for High Efficiency Video Coding,” in IEEE International
Symposium on Circuits and Systems (ISCAS), 2013, pp. 477–480.
[7] T. O. Aydın, R. Mantiuk, and H.-P. Seidel, “Extending quality metrics
to full luminance range images,” in SPIE Journal of Electronic Imaging,
2008, pp. 68 060B–68 060B.
[8] High dynamic range television for production and international programme exchange, ITU-R BT.2390-1, 2016.
[9] R. Mantiuk, K. J. Kim, A. G. Rempel, and W. Heidrich, “HDR-VDP2: a calibrated visual metric for visibility and quality predictions in all
luminance conditions,” vol. 30, no. 4, p. 40, 2011.
[10] M. Narwaria, M. P. Da Silva, and P. Le Callet, “HDR-VQM: An
objective quality measure for high dynamic range video,” Elsevier
Journal of Signal Processing: Image Communication, vol. 35, pp. 46–60,
2015.
[11] G. Bjontegaard, “Calculation of average PSNR differences between RDcurves,” ITU SG16 Doc. VCEG-M33, 2001.
[12] M. Wang, K. N. Ngan, and H. Li, “An efficient frame-content based
intra frame rate control for high efficiency video coding,” IEEE Signal
Processing Letters, vol. 22, no. 7, pp. 896–900, 2015.