© ISO/IEC 2006 – All rights reserved
INTERNATIONAL ORGANIZATION FOR STANDARDIZATION
ORGANISATION INTERNATIONALE DE NORMALISATION
ISO/IEC JTC 1/SC 29/WG 11
CODING OF MOVING PICTURES AND AUDIO
ISO/IEC JTC 1/SC 29/WG 11
N8634
October 2006, Hangzhou, China
Source:
Audio Subgroup
Title:
ISO/IEC 23003-1:2006/PDAM 1, MPEG Surround conformance testing
Status:
Approved
This document constitutes the PDAM text of the conformance testing specification for MPEG Surround.
Document type: International Standard
Document subtype: Amendment
Document stage: (30) Committee
Document language: E
D:\106738811.doc STD Version 2.1c2
ISO/IEC JTC 1/SC 29 N
Date: 2006-12-15
ISO/IEC 23003-1:2006/PDAM 1
ISO/IEC JTC 1/SC 29/WG 11
Secretariat:
Information technology — MPEG audio technologies — Part 1: MPEG
Surround, AMENDMENT 1: Conformance testing
Élément introductif — Élément central — Partie 1: Titre de la partie
Warning
This document is not an ISO International Standard. It is distributed for review and comment. It is subject to
change without notice and may not be referred to as an International Standard.
Recipients of this draft are invited to submit, with their comments, notification of any relevant patent rights of
which they are aware and to provide supporting documentation.
ISO/IEC 23003-1:2006/PDAM 1
Copyright notice
This ISO document is a working draft or committee draft and is copyright-protected by ISO. While the
reproduction of working drafts or committee drafts in any form for use by participants in the ISO standards
development process is permitted without prior permission from ISO, neither this document nor any extract
from it may be reproduced, stored or transmitted in any form for any other purpose without prior written
permission from ISO.
Requests for permission to reproduce this document for the purpose of selling it should be addressed as
shown below or to ISO's member body in the country of the requester:
[Indicate the full address, telephone number, fax number, telex number, and electronic mail address, as
appropriate, of the Copyright Manger of the ISO member body responsible for the secretariat of the TC or
SC within the framework of which the working document has been prepared.]
Reproduction for sales purposes may be subject to royalty payments or a licensing agreement.
Violators may be prosecuted.
© ISO/IEC 2006 – All rights reserved
iii
ISO/IEC 23003-1:2006/PDAM 1
Contents
Page
Foreword ..............................................................................................................................................................v
8
8.1
8.2
8.3
8.4
8.4.1
8.5
8.5.1
8.5.2
8.5.3
iv
Conformance testing .............................................................................................................................1
Introduction ............................................................................................................................................1
Terms and definitions ...........................................................................................................................1
MPEG surround conformance testing .................................................................................................1
Compressed data ...................................................................................................................................1
Test procedure .......................................................................................................................................1
Decoders .................................................................................................................................................2
Characteristics .......................................................................................................................................2
Test procedure .......................................................................................................................................2
Test sequences ......................................................................................................................................4
© ISO/IEC 2006 – All rights reserved
ISO/IEC 23003-1:2006/PDAM 1
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
Amendment 1 to ISO/IEC 23003-1:2006 was prepared by Joint Technical Committee ISO/IEC JTC 1,
Information Technology, Subcommittee SC 29, Coding of Audio, Picture, Multimedia and Hypermedia
Information.

Part 1: MPEG Surround
© ISO/IEC 2006 – All rights reserved
v
ISO/IEC 23003-1:2006/PDAM 1
Information technology — MPEG audio technologies — Part 1:
MPEG Surround, AMENDMENT 1: Conformance testing
Add a new clause 8, Conformance testing, as given below:
8
Conformance testing
8.1 Introduction
This clause of ISO/IEC 23003-1 specifies conformance criteria for both compressed data and decoders of the
technologies standardized in the preceding clauses. This is done to assist implementers and to ensure
interoperability.
8.2 Terms and definitions
Relevant terms and definitions for this part of ISO/IEC 23003 can be found in ISO/IEC 23003-1. Furthermore,
the following MPEG surround related terms and definitions will be used throughout this clause:
conformance data – Conformance test sequences and conformance tools.
conformance tools – tools which are provided within an electronic annex of ISO/IEC 23003-2 to check
certain conformance criteria.
conformance test sequences – The superset of compressed data and its reference waveforms provided as
examples within an electronic annex of this document.
compressed data – encoded data according to ISO/IEC 23003-1.
reference waveforms – Decoded counterparts of the compressed data.
8.3 MPEG surround conformance testing
For Audio conformance points related to ISO/IEC14496-4, please refer to ISO/IEC14496-4, subclause 6.3
“Audio Conformance Points”.
ISO/IEC 23003-1 defines the Baseline MPEG Surround Profile consisting of 5 levels. Some conformance
requirements apply to the MPEG Surround technology in general, while some are specific to the Baseline
MPEG Surround profile. Conformance is always tested against a certain level within that profile.
8.4 Compressed data
8.4.1
Test procedure
Each compressed data shall meet the syntactic and semantic requirements specified in ISO/IEC 23003-1.
This subclause describes a set of semantic tests to be performed on decoder relevant data. The procedure to
verify whether the syntax is correct is straightforward and therefore not defined in this subclause. In the
description of the semantic tests it is assumed that the tested compressed data contains no errors due to
© ISO/IEC 2006 – All rights reserved
1
ISO/IEC 23003-1:2006/PDAM 1
transmission or other causes. For each test the condition or conditions that must be satisfied are given, as
well as the prerequisites or conditions in which the test can be applied.
8.4.1.1
Configuration header
8.4.1.1.1
SpatialSpecificConfig
TODO: List elements here.
8.4.1.2
Bitstream payload
8.4.1.2.1
Spatial_bitstream() ?
TODO: List elements here.
8.5 Decoders
8.5.1
8.5.1.1
Characteristics
General
The MPEG Surround decoder can be implemented in two different versions:

High quality MPS

Low Power MPS

Other configurations
8.5.1.2
Baseline MPEG Surround profile

The ability to skip over residual data embedded in the MPS bitstream is mandatory for decoders of
level 1 and 2. Decoders of level 3 and higher shall be able to decode and apply a residual bitstream.

Add further requirements here
8.5.2
8.5.2.1
Test procedure
Downmix decoders
For the sake of simplicity the conformance of the underlying downmix decoder shall be satisfied separately.
The MPS conformance test procedure contains a tool to remove the influence of a potentially inaccurate
downmix decoder to the maximum extend possible.
8.5.2.2
MPS decoders
The conformance test procedure for MPS decoders internally creates a reference for comparison, given the
conformance test sequence and the output from the decoder under test. In order to accomplish this, every
MPS conformance sequence is divided into two parts as outlined in Figure AMD1-1. The downmix bitstream
for the two parts is identical but the MPS payload of the first part signals bsXXXdataMode = 0, causing the
MPEG surround decoder to pass the downmix signal through the QMF filterbank without applying any of the
MPS tools, while the second part contains the actual MPS payload that is intended for use for conformance
testing.
2
© ISO/IEC 2006 – All rights reserved
ISO/IEC 23003-1:2006/PDAM 1
TODO: insert figure outlining the above
Figure AMD1-1 — The disposition of the MPEG surround conformance test sequences
The conformance test procedure will:

read the conformance test sequence;

store the output signal of the decoder under test during the first half of the bitstream
Since this signal is identical to the output-signal from the downmix decoder after QMF analysis and QMF
synthesis, it can by means of a polyphase correction filter, be approximated to be the same signal as was
used by the MPS decoder in the decoder under test.
For cases where the downmix decoder is connected directly in the QMF domain like the combination of AAC
and SBR, the MPEG surround conformance test procedure can only approximate the output of the decoder
that was used prior to the QMF analysis step. The MPEG surround conformance test sequences for such
decoder combinations are designed to keep the influence of the QMF domain based downmix decoder
minimal.
In parallel to storing the signal it shall also be fed to the reference MPS decoder along with the first half of the
compressed MPS conformance sequence. This way, the reference MPS decoder will perform the exact same
processing steps as the MPS decoder in the device under test, thereby ensuring that all internal states of both
decoders match. The output signal of this shall be compared to the input signal, i.e. the output signal of the
decoder under test. This serves as a QMF test of the first half of the conformance file.
For the second half of the bitstream, an MPS processed reference signal shall be created based on the
previously derived signal. This makes it possible to test the accuracy of the MPS decoder alone without
influence from the underlying downmix decoder. Furthermore, the accuracy of the QMF implementation of the
decoder under test is tested separately for every conformance sequence.
If HE AAC is used as downmix coder, and a complex QMF filter bank with a modified internal phase angle
(hereinafter referred to as twiddles) is used in the decoder under test, the al_sbr_twi_* sequences shall be
tested first. For further information regarding these sequences, please refer to ISO/IEC 14496-4 clause
6.6.17.2.2 (SBR conformance test procedure).
In order to ensure that the QMF is implemented correctly, the output from the QMF test specific sequences
*_qmf_* are compared to the internal reference applying the polyphase correction filter and storing the first
half of the test sequence. This is done to detect errors in the QMF implementation, which would otherwise
appear to be errors in the downmix decoder. By, for the QMF specific sequences, omitting the parts of the tool
that are designed to neglect differences in the downmix decoder implementation it is ensured that the QMF is
implemented correctly.
If the decoder under test passes the conformance criteria for the dedicated QMF test sequences, this is a
good indication that the QMF implementation is accurate. However, it is no definite guarantee, and hence it
could happen that a QMF implementation that barely passes the conformance for the QMF test, does not pass
conformance for other parts of the system due to the QMF implementation. Therefore, it is useful to observe
the result from the QMF test for the first half for any of the conformance sequences. This can give a good
indication of the origin of a potential error.
Figure 2 outlines a flowchart of the MPEG surround conformance test procedure.
TODO: insert flowchart.
Figure 2 — Flowchart of the SBR conformance test procedure
The test sequences specified in Table 2 have to be applied. The testing shall be done using the test
procedure defined above. The conformance of the underlying AAC decoder shall be tested before
© ISO/IEC 2006 – All rights reserved
3
ISO/IEC 23003-1:2006/PDAM 1
conformance testing is done for the SBR Tool. The SBR Tool is based on a pseudo QMF filterbank. This is the
most critical part of the SBR Tool in terms of precision. Hence, in order to simplify conformance testing, it is
recommended to first check the accuracy of the QMF implementation, after proper identification of the
filterbank twiddling in the decoder under test.
The Low Power SBR Tool has an aliasing detection algorithm that includes threshold binary decisions which
can give rise to large output differences due to small rounding errors. In the aliasing detection algorithm a
decision is made on how to modify gain calculation in the envelope adjuster. If potentially strong aliasing is
detected, a clear indication on how to modify the gain values will be obtained, and no rounding problems will
occur. However, if no strong aliasing is detected, the gain values will be modified anyway, albeit to a much
smaller extent. Since no strong aliasing is detected, the modification decision could vary due to rounding
effects in the aliasing detection algorithm. A small rounding error will then cause the gain calculation algorithm
to modify different gain values, and hence an inaudible, but clearly measurable difference is observed in the
output. The conformance sequences are designed to avoid the above problem, but differences can appear for
other sequences.
The inverse filtering in the HF generator is another module that is prone to give output differences due to
rounding effects. Similarly to the differences due to rounding in the aliasing detection, these are inaudible
differences.
Reference waveform output files from decoding the compressed MPEG-4 data conformance sequences with
the MPEG-4 reference software are also available. These files are available for reference only, and are not
part of the official conformance testing procedure.
Notes: The reference software implementation of the conformance test procedure will behave differently for
the test sequences named al_sbr_twi_*, since for these test sequences the QMF implementation is identified
with respect to the twiddle factors used on the analysis and synthesis side of the QMF.
The reference software implementation of the conformance test procedure will behave differently for the test
sequences named al_sbr_qmf_*, since for these test sequences only the QMF implementation is tested, as
outlined above in the tool description. For these sequences only the output from the QMF test is displayed.
This QMF test is not informative as for the other sequences, it is normative and shall be passed in order for
the device under test to be conformant.
8.5.3
Test sequences
To test MPEG Surround decoders, ISO/IEC JTC 1/SC 29/WG 11 supplies a number of test sequences. For all
conformance test sequences, the file name convention given in Table 1 is used. The test sequences are
defined in Table 2.
File basename
Compressed data properties
_qmf_
Tests the QMF implementation
Table AMD 1 — Naming conventions for MPEG surround conformance test sequences
Table 2 — List of MPEG surround conformance test sequences
4
© ISO/IEC 2006 – All rights reserved