Proposal for New Work Item for WG13 on MPEG-4 AVC / H.264 Video Coding Introduction It is proposed that the DICOM Committee approve a Work Item for the development of a new supplement on the addition of MPEG-4 AVC/H.264 as a transfer Syntax for encoded Video sequences. Certain medical video workflows, notably diagnostics and video streaming, have turned to the widely adopted MPEG-4 AVC / H.264 video coding standard to satisfy their particular requirements. These requirements range from the need to deliver the best possible video quality at low fixed bitrates (video streaming applications), to the provision of video streams with very high levels of detail, whether in the form of larger picture sizes (above 1920x1080), higher bit depths, and/or improved color sampling, and/or display aspect ratio. The main intent of this work item proposal is to ensure that the MPEG-4 AVC / H.264 video streams produced by these applications are accommodated within DICOM, allowing them to be archived and shared by relevant actors within the healthcare enterprise. Limitations of Current Standard and Proposal In order to provide the possibility to store HD video, a new supplement is currently under development, which applies MPEG-2 to cover the compression of HD video in DICOM. Considering the trade off between complexity and coding efficiency, this MPEG-2 HD supplement under development is a good choice for many applications. MPEG-2 provides reasonable compression efficiency and picture quality at moderate computational complexity. However, the following functional improvements targeting specific video applications such as diagnostics and video streaming cannot be fulfilled by either the current DICOM video or the MPEG-2 HD supplement. 1. 2. 3. 4. Higher coding efficiency Bit depths above 8-bit, up to 14-bit Color sampling of 4:2:2 and 4:4:4 Picture sizes larger than 1920x1080 5. Arbitrary display aspect ratios In the following more details on each functional improvement and a proposal for a solution are given. 1. Coding Efficiency The compression efficiency of MPEG-2 is limited. Small file size of videos is extremely important, first because videos are primarily acquired in the operating room (OR) and are transmitted or streamed over the hospital network from the OR to the PACS system or from the PACS server to a DICOM viewer after the surgery, and second because video files are currently the largest data files stored in PACS systems. The OR is one of the most expensive rooms in the hospital and therefore the transmission has to be done as efficiently and quickly as possible. With MPEG-4 AVC / H.264 we see the opportunity to reduce file size and network traffic by 50-60 % compared to MPEG-2 for video streams with the same image quality. It is expected that revisions to the DICOM standard and supplements for HD video compression adopted in the near term are likely to govern HD video compression for the next 10 years. Therefore it is important to support those widely adopted standards, which are well suited for technological reasons for use with particular applications. On the Internet, compression formats with a compression performance similar to MPEG-4 AVC / H.264 are available and have gained considerable traction in the marketplace. If solely MPEG-2 would be standardized, there is a possibility that various other compression formats will be used as de-facto standards and slow down the adoption of DICOM. If MPEG-4 AVC / H.264 would be standardized, such kind of confusion could be avoided, because there is no other compression format that is superior from a compression ratio as well as from practical point of view. 2. Bit-depth Due to difficult lighting conditions, the current limitation to 8-bit in DICOM video, limits the ability to provide good resolutions for darker and brighter parts of a video signal at the same time. The new professional profiles of MPEG-4 AVC / H.264 target bit depths between 8 and 14 bits per sample and color component. The use of these bit depths would significantly increase the dynamic range of the video. 3. Color sampling Due to the limitation to 4:2:0 color sampling in DICOM video and the MPEG-2 HD supplement, the spatial resolution of each chrominance component is 4 times lower than the luminance component. But, since color is an important factor for diagnostics, the 4:2:0 limitation affects the quality of diagnostics. The professional profiles of MPEG-4 AVC / H.264 provide color samplings of 4:2:2 and 4:4:4 at excellent compression efficiency and good parallel processing capabilities. 4. Picture size The MPEG-2 standard limits the maximum video resolution to 1920x1080. However, for future applications even higher resolutions are possible, which would significantly improve the possibilities for diagnostics. MPEG-4 AVC / H.264 provides excellent profiles supporting picture sizes beyond HD, such as 4Kx2K. 5. Arbitrary display aspect ratios The MPEG-2 HD supplement limits the display aspect ratio to 16:9. But various applications such as endoscopy have originally different aspect ratios. MPEG-4 AVC / H.264 is able to handle arbitrary aspect ratios. Details of Proposal In order to address the above-mentioned functional improvements, it is proposed to standardize MPEG-4 AVC / H.264 in DICOM. First of all, MPEG-4 AVC / H.264 provides significantly improved compression efficiency compared to MPEG-2. Furthermore, MPEG-4 AVC / H.264 is the state of the art standard, which provides professional profiles to handle higher resolution video, higher-bit depth, and higher color samplings. Since it would be too much to standardize all features and profiles in one supplement, it is proposed to use an incremental approach. The proposal addresses only coding efficiency improvements in the first step: 1. Standardize MPEG-4 AVC / H.264 for 4:2:0, 8-bit 2. Add the support of more functionality and profiles of MPEG-4 AVC / H.264 in future steps o Better color samplings: 4:2:2, 4:4:4 o Higher bit depth: 10-14 bit per sample and color component o Higher spatial resolution such as 4k Ideally, each step or profile would result in its own transfer Syntax UID. This facilitates the statement of compliance for DICOM devices that support only a subset of the proposed MPEG-4 AVC / H.264 profiles in DICOM. Parts of Standard Affected This supplement adds a new Transfer Syntax. It does not introduce any new SOP Classes or IODs and may be used with existing multi-frame SOP Classes that satisfy the constraints of the Transfer Syntax, such as the Video Endoscopic Image Storage SOP Class, the Video Microscopic Image Storage SOP Class, the Video Photographic Image Storage SOP Class or the Multi-frame RGB Secondary Capture Image Storage SOP Class. This supplement makes changes to PS 3.5 Data Structures and Encoding and PS 3.6 Data Dictionary parts of the DICOM standard. Workload. It is intended to finish the first MPEG-4 AVC / H.264 supplement (4:2:0, 8-bit) at the end of FY2009. Several organizations such as Panasonic, Karl Storz, ETIAM, JAHIS, Stryker have declared to provide the necessary amount of work to fulfill this deadline. Since these companies are not directly involved in the development of the MPEG-2 HD supplement, which is hosted in the same working group, it is expected that the progress of this supplement will not be adversely affected. It is expected that six hours of WG 6 meeting time on two occasions during 2009 will be spent for the review and approval of public comment and letter ballot text of the first MPEG-4 AVC / H.264 supplement. In further steps the following functionalities (or parts thereof) will be added in additional MPEG-4 AVC / H.264 supplements. Better color samplings: 4:2:2, 4:4:4 Higher bit depth: 10-14 bit per sample and color component Higher spatial resolution such as 4k Appendix Some more facts about MPEG-4 AVC / H.264: MPEG-4 AVC / H.264 is a widely used standard for video compression, and there are many products using this standard. More than one billion H.264 decoders are expected to be sold by the end of this year Since it has been adopted into the HD broadcast and HD optical disk standards, it can be expected that it will be the main standard especially for HD devices. Year of standardization: MPEG-4 AVC / H.264 Main Profile in 2004, High Profile in 2005 The compression ratio is approx. 2 times higher than in MPEG-2 HD decoders that are compliant with MPEG-4 AVC / H.264 can easily also support MPEG-2, which is simpler to decode. Trade-off between complexity and compression ratio gain is scalable E.g. depending on encoder settings, similar (or even lower) complexity than MPEG-2 can be achieved Appropriate parameters can be pre-set by the encoder engine and can also be recommended in an informative section of the DICOM supplement Standards in which it is applied: Optical disk (BD-ROM, AVCHD, AVCRec), HD broadcast (DVB/ATSC/ARIB.. France, UK, Northern Europe etc.）, 3GPP, broadcast for mobile devices (1-Seg, Korean DMB, DVB-H） HD encoder and decoder applications in hardware or software are available from various major manufactures. Products： BD recorder / player, TV, STB, Camcorder, video game console, … Licenses: Main license by MPEG-LA (same than for MPEG-2 case), which is providing fees at a non-discriminatory level. Since there are already many consumer and professional devices that are MPEG-4 AVC / H.264 compliant, there will be many existing devices, which can be used for visible light application in medicine.