Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG
(ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6)
2nd Meeting: Geneva, CH, Jan. 29 - Feb. 1, 2002
Title:
Short tap filter for high resolution sequences
Status:
Input Document to JVT
Purpose:
Proposal
Author(s) or
Contact(s):
Kei-ichi Chono and Yoshihiro Miyamoto
NEC Multimedia Laboratories,
4-1-1 Miyazaki Miyamae Kawasaki,
216-8555 JAPAN
Source:
Tel:
Fax:
Email:
Document: JVT-B077
Filename: JVT-B077.doc
Generated: 2002-01-23
+81 44 856 8141
+81 44 856 2232
chono@dsp.cl.nec.co.jp
miyamoto@dsp.cl.nec.co.jp
NEC Corporation
Kawasaki, 216-8555 JAPAN
_____________________________
1. Background
In VCEG-O16 of the Pattaya meeting adaptive interpolation scheme, estimating and transmitting
6-tap filter coefficients per picture, was proposed [1] To consider non-stationary statistical
properties of video signals, this adaptive interpolation scheme seems to be beneficial [1][2].
However, there are some issues: complexity and so on. AHG was formed to study these issues
for the in-corporation of this feature into TML.
We agree to the adaptive interpolation scheme, and have compared coding quality gains of
some interpolation filters, for a given set selection of filter coefficients. Experiment results show
us two facts. First fact is that TML9 6-tap filter coding gain to Telenor 4-tap filter depends on
sequences. Second fact is that 4-tap filter implementation reduces the amount of fractional-pel
interpolation calculation to 50% at decoder. We propose that encoder/decoder may use shorter
tap filter for high-resolution sequences, in order to reduce the complexity.
2. Experiments
2.1 Interpolation
In order to produce 2:1 resolution, Telenor 4-tap filter, with impulse response (-4, 20, 20, -4)/32
[3], is used. For 4:1 resolution, Bilinear filter is used as second filter. Hence, the difference
between 4-tap filter interpolation and TML9 is first filter coefficients.
2.2 Test sequences and coding conditions
Test sequences are shown in Table 1.
File:231199546
Page: 1
Date Printed: 9/29/2021
Table 1: Test sequence
Test sequcence
Resolution Frame rate (fps)Frames
Coastguard
QCIF
10
300
Container
QCIF
10
300
Foreman
QCIF
10
400
News
QCIF
10
300
Silent
QCIF
15
300
Coastguard
CIF
10
300
Container
CIF
10
300
Foreman(Simens version)
CIF
10
300
News
CIF
10
300
Silent
CIF
15
300
Paris
CIF
15
300
Mobile
CIF
30
300
Tempete
CIF
30
260
Flower
SIF
30
150
Football
SIF
30
150
Flower
CCIR
30
150
Football
CCIR
30
150
Mobile
CCIR
30
150
The coding conditions are: Quantization parameter for I-picture(QI) set to 24, 16 pixel search
range(32 pixel is only used for Football CCIR), RestrictSearchRange set to two, UseHadamard
set to on, quarter-pel motion, one reference frame, UVLC in entropy coding, RD optimization set
to on, no B-picture.
CCIR sequences are interlace-formant, not supported by TML9 yet. The encoder treated 2 fields
as 1 frame described Fig 1.
frame 2
frame 1
frame 0
Top field
Top field
Top field
Bottom field
Bottom field
Bottom field
time
0
Fig 1
Input YUV format at CCIR encoding
2.3 Results
TML9 filter coding gain compared to Telenor 4-tap filters is the following:
QCIF results:
TML9 filter leads to average gain 0.19[dB] with avsnr4, up to 0.48[dB
File:231199546
Page: 2
Date Printed: 9/29/2021
CIF and SIF results:
TML9 filter leads to average gain 0.17[dB] with avsnr4, up to 0.60[dB]
CCIR results:
TML9 filter leads to average gain 0.24[dB] with avsnr4, up to 0.54[dB]
Avsnr4 is used to calculate objective differences for common measurement method [4]. The
avsnr4-results are shown in Table 1-3, and rate-distortion curves are attached.
3. Proposal : adaptive filter taps for reducing coding calculation
Experiment results show that TML9’s coding gain to Telenor 4-tap filter depends on sequences.
The total amount of fractional-pel calculation increases in proportion to the spatial resolution. The
current TML9 uses direct-interpolation for reducing multiplications and memory requirement.
Telenor 4-tap direct-interpolation can reduce multiplication to half of 6-tap’s [5]. We have
evaluated the decoding time improvement by the multiplication reduction. Evaluation condition is:
Pentium4 1.7GHz, 512MB memory, Microsoft Windows2000, and Intel VTune5 with 1msec
sampling rate. The multiplication reduction cuts half of interpolation calculation, and 10 percent of
total decoding calculation (See Chart 1-3).
From view of complexity reduction, we think that encoder/decoder may use shorter tap filter for
high-resolution sequences, because TML9’s coding gain depends on the sequence.
4. Conclusion
In this report, two facts are shown:
[1] TML 6-tap filter coding gain to Telenor 4-tap filter depends on sequences
[2] 4-tap direct interpolation cuts half of interpolation calculation, and 10 percent of total
decoding calculation
We propose that encoder/decoder may use shorter tap filter for high-resolution sequences, in
order to reduce the complexity. The additional study on higher resolution sequences, as HDTV,
may be valuable for confirming this interpolation filter issue.
Reference
[1] Thomas Wedi, “H.26L Interpolation Filtering Ad Hoc Report”, ITU-T SG16, doc. VCEG-O11,
Pattaya, Thailand, 4-6 Dec., 2001
[2] Thomas Wedi, “Adaptive Interpolation Filter for H.26L”, ITU-T SG16, doc. VCEG-N28, Santa
Barbara, CA, USA, Sep. 2001
[3] Thomas Wedi, “Interpolation Filter for motion compensated prediction with 1/4 and 1/8-pel
accuracy”, ITU-T SG16, doc.Q15-J-14, Osaka, Japan, May. 2000.
[4] Gary Sullivan and Gisle Bjontegaard, “Recommended Simulation Conditions for H.26L
Coding efficiency Experiment on Low-Resolution Progressive-Scan Source Material”, ITU-T
SG16, doc. VCEG-N81, Santa Barbara, CA, USA, Sep. 2001
[5] Thomas Wedi,” Results on complexity and coding performance investigations: Displacement
vector resolution and interpolation filter tap size”, ITU-T SG16, doc. VCEG-M46, Austin,
Texas, USA, April, 2001.
File:231199546
Page: 3
Date Printed: 9/29/2021
Avsnr4-results
Table1: QCIF results
TML9 average [dB]
TML9 average %
Sequence(QCIF)PSNR gain
bitsavings (avsnr4)
(avsnr4)
Coastguard
0.11
3.11
Container
0.48
11.20
Foreman
0.26
5.43
News
0.09
2.01
Silent
0.00
0.26
Average
0.19
4.40
Max
0.48
11.20
Min
0.00
0.26
Table2: CIF and SIF results
TML9 average [dB]
TML9 average %
Sequence
PSNR gain
bitsavings (avsnr4)
Coastguard(CIF (avsnr4)
0.06
1.65
)
Container (CIF)
0.27
8.04
Foreman (CIF)
0.13
2.95
News (CIF)
0.03
0.79
Silent (CIF)
0.00
0.12
Paris (CIF)
0.09
2.06
Mobile (CIF)
0.60
12.17
Tempete (CIF)
0.33
7.84
Flower (SIF)
0.16
3.15
Football (SIF)
0.02
0.44
Average
0.17
3.92
Max
0.60
12.17
Min
0.00
0.12
Table3: CCIR results
Sequence
Flower
Football
Mobile
Average
TML9 average [dB]
TML9 average %
PSNR gain
bitsavings (avsnr4)
(avsnr4)
0.13
2.49
0.06
0.55
0.54
10.15
0.24
4.40
Max
0.54
10.15
Min
0.06
0.55
File:231199546
Page: 4
Date Printed: 9/29/2021
Decoding samples
TM L9(77kbps)
P roposed(80kbps)
0
500
get_quarterpel_pixel
1000
S am ples
decode_one_m acroblock
1500
in-loop filter
itrans
2000
U V LC
else
Chart 1 Decoding samples (Sequence: Foreman with QCIF, 10fps, QI=24, QP=20, and 133 frames)
TM L9(251kbps)
P roposed(256kbps)
0
500
get_quarterpel_pixel
1000
1500
2000
decode_one_m acroblock
2500 3000
S am ples
3500
in-loop filter
4000
itrans
4500
U V LC
5000
else
Chart 2 Decoding samples (Sequence: Paris with CIF, 15fps, QI=24, QP=20, and 150 frames)
TM L9(4.1M bps)
4-tap filter(4.2M bps)
0
5000
10000
15000
20000
25000
S am ples
get_quarterpel_pixel
decode_one_m acroblock
in-loop filter
itrans
U V LC
else
Chart 3 Decoding samples (Sequence: Flower with CCIR, 30fps, QI=24, QP=20, and 150 frames)
File:231199546
Page: 5
Date Printed: 9/29/2021
Rate-distortion curves
PSNR [dB]
Coastguard QCIF 10fps
34.4
33.9
33.4
32.9
32.4
31.9
31.4
30.9
30.4
29.9
29.4
28.9
28.4
27.9
27.4
26.9
26.4
TM L
4-tap Telenor filter
0
20
40
60
80
100
120
140
160
Bitrate kbps
PSNR [dB]
C ontainer Q C IF 10fps
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
TM L
4-tap Telenor filter
0
5
10
15
20
25
30
35
B itrate kbps
PSNR [dB]
Forem an Q C IF 10fps
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
TM L
4-tap Telenor filter
0
10
20
30
40
50
60
70
80
90
100
B itrate kbps
File:231199546
Page: 6
Date Printed: 9/29/2021
PSNR [dB]
N ew s Q C IF 10fps
36.5
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
TM L
4-tap Telenor filter
0
10
20
30
40
50
60
B itrete kbps
PSNR [dB]
S ilent Q C IF 15fps
36.5
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
TM L
4-tap Telenor filter
0
10
20
30
40
50
60
70
B itrete kbps
PSNR [dB]
C oastguard C IF 10fps
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
27.5
27.0
TM L
4-tap Telenor filter
0
100
200
300
400
500
600
700
B itrate kbps
File:231199546
Page: 7
Date Printed: 9/29/2021
PSNR [dB]
C ontainer C IF 10fps
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
TM L
4-tap Telenor filter
0
20
40
60
80
100
120
140
160
B itrate kbps
PSNR [dB]
Forem an C IF 10fps
37.0
36.5
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
TM L
4-tap Telenor filter
0
50
100
150
200
250
300
B itrate kbps
PSNR [dB]
N ew s C IF 10fps
38.0
37.5
37.0
36.5
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
TM L
4-tap Telenor filter
0
20
40
60
80
100
120
140
160
B itrate kbps
File:231199546
Page: 8
Date Printed: 9/29/2021
PSNR [dB]
S ilent C IF 15fps
37.0
36.5
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
TM L
4-tap Telenor filter
0
20
40
60
80
100
120
140
160
180
200
B itrate kbps
PSNR [dB]
Paris C IF 15fps
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
27.5
TM L
4-tap Telenor filter
0
50
100
150
200
250
300
350
400
B itrate kbps
PSNR [dB]
M obile C IF 30fps
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
27.5
27.0
26.5
26.0
25.5
25.0
24.5
24.0
TM L
4-tap Telenor filter
0
500
1,000
1,500
2,000
2,500
B itrate kbps
File:231199546
Page: 9
Date Printed: 9/29/2021
PSNR [dB]
Tem pete C IF 30fps
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
27.5
27.0
26.5
26.0
25.5
25.0
TM L
4-tap Telenor filter
0
200
400
600
800
1,000
1,200
1,400
1,600
B itrate kbps
PSNR [dB]
Flow er SIF 30fps
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
27.5
27.0
26.5
26.0
25.5
25.0
24.5
24.0
23.5
23.0
TM L9
4-tap Telenor filter
0
500
1000
1500
2000
2500
B itrate kbps
PSNR [dB]
FootballS IF 30fps
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
TM L9
4-tap Telenor filter
0
500
1000
1500
B itrate kbps
File:231199546
Page: 10
Date Printed: 9/29/2021
PSNR [dB]
Flow er C C IR 30fps
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
27.5
27.0
26.5
26.0
25.5
25.0
24.5
TM L9
4-tap Telenor filter
0
1000
2000
3000
4000
5000
6000
7000
8000
B itrate kbps
PSNR [dB]
FootballC C IR 30fps
36.5
36.0
35.5
35.0
34.5
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
TM L9
4-tap Telenor filter
0
1000
2000
3000
4000
5000
B itrate kbps
PSNR [dB]
M obile C C IR 30fps
34.0
33.5
33.0
32.5
32.0
31.5
31.0
30.5
30.0
29.5
29.0
28.5
28.0
27.5
27.0
26.5
26.0
25.5
25.0
24.5
24.0
23.5
TM L9
4-tap Telenor filter
0
1000
2000
3000
4000
5000
6000
7000
8000
B itrate kbps
File:231199546
Page: 11
Date Printed: 9/29/2021
JVT Patent Disclosure Form
International Telecommunication Union
Telecommunication Standardization Sector
International Organization
for Standardization
International Electrotechnical Commission
Joint Video Coding Experts Group - Patent Disclosure Form
(Typically one per contribution and one per Standard | Recommendation)
Please send to:
JVT Rapporteur Gary Sullivan, Microsoft Corp., One Microsoft Way, Bldg. 9, Redmond WA
98052-6399, USA
Email (preferred): Gary.Sullivan@itu.int Fax: +1 425 706 7329 (+1 425 70MSFAX)
This form provides the ITU-T | ISO/IEC Joint Video Coding Experts Group (JVT) with
information about the patent status of techniques used in or proposed for incorporation in a
Recommendation | Standard. JVT requires that all technical contributions be accompanied
with this form. Anyone with knowledge of any patent affecting the use of JVT work, of their own
or of any other entity (“third parties”), is strongly encouraged to submit this form as well.
This information will be maintained in a “living list” by JVT during the progress of their work, on
a best effort basis. If a given technical proposal is not incorporated in a Recommendation |
Standard, the relevant patent information will be removed from the “living list”. The intent is
that the JVT experts should know in advance of any patent issues with particular proposals or
techniques, so that these may be addressed well before final approval.
This is not a binding legal document; it is provided to JVT for information only, on a best effort,
good faith basis. Please submit corrected or updated forms if your knowledge or situation
changes.
This form is not a substitute for the ITU ISO IEC Patent Statement and Licensing Declaration, which should be
submitted by Patent Holders to the ITU TSB Director and ISO Secretary General before final approval.
Submitting Organization or Person:
Organization name
NEC CORPORATION
Mailing address
4-1-1 Miyazaki, Miyamae-Ku, Kawasaki 216-8555, Japan
Country
Japan
Contact person
Yoshihiro Miyamoto
Telephone
+81-44-856-8141
Fax
+81-44-856-2232
Email
miyamoto@dsp.cl.nec.co.jp
Place and date of
submission
2nd JVT Meeting Jan.29-Feb.1, Geneva
Relevant Recommendation | Standard and, if applicable, Contribution:
Name (ex: “JVT”)
JVT
Title
Short tap filter for high res sequences
Contribution number
JVT-B077
(Form continues on next page)
File:231199546
Page: 12
Date Printed: 9/29/2021
Disclosure information – Submitting Organization/Person
X
2.0
(choose one box)
The submitter is not aware of having any granted, pending, or planned patents
associated with the technical content of the Recommendation | Standard or Contribution.
or,
The submitter (Patent Holder) has granted, pending, or planned patents associated with the technical
content of the Recommendation | Standard or Contribution. In which case,
2.1
The Patent Holder is prepared to grant – on the basis of reciprocity for the above
Recommendation | Standard – a free license to an unrestricted number of applicants
on a worldwide, non-discriminatory basis to manufacture, use and/or sell
implementations of the above Recommendation | Standard.
2.2
The Patent Holder is prepared to grant – on the basis of reciprocity for the above
Recommendation | Standard – a license to an unrestricted number of applicants on a
worldwide, non-discriminatory basis and on reasonable terms and conditions to
manufacture, use and/ or sell implementations of the above Recommendation |
Standard.
Such negotiations are left to the parties concerned and are performed outside the ITU |
ISO/IEC.
2.2.1 The same as box 2.2 above, but in addition the Patent Holder is prepared to grant a
“royalty-free” license to anyone on condition that all other patent holders do the same.
2.3
The Patent Holder is unwilling to grant licenses according to the provisions of either
2.1, 2.2, or 2.2.1 above. In this case, the following information must be provided as
part of this declaration:

patent registration/application number;

an indication of which portions of the Recommendation | Standard are affected.

a description of the patent claims covering the Recommendation | Standard;
In the case of any box other than 2.0 above, please provide the following:
Patent
number(s)/status
Inventor(s)/Assigne
e(s)
Relevance to JVT
Any other remarks:
(please provide attachments if more space is needed)
(form continues on next page)
File:231199546
Page: 13
Date Printed: 9/29/2021
Third party patent information – fill in based on your best knowledge of relevant patents granted,
pending, or planned by other people or by organizations other than your own.
Disclosure information – Third Party Patents (choose one box)
X
3.1
The submitter is not aware of any granted, pending, or planned patents held by third
parties associated with the technical content of the Recommendation | Standard or
Contribution.
3.2
The submitter believes third parties may have granted, pending, or planned patents
associated with the technical content of the Recommendation | Standard or Contribution.
For box 3.2, please provide as much information as is known (provide attachments if more space
needed) - JVT will attempt to contact third parties to obtain more information:
3rd party name(s)
Mailing address
Country
Contact person
Telephone
Fax
Email
Patent
number/status
Inventor/Assignee
Relevance to JVT
Any other comments or remarks:
File:231199546
Page: 14
Date Printed: 9/29/2021