Multimedia Technology Research
in HKUST
Oscar Au (PhD, Princeton University)
Dept. of ECE, HKUST
Clearwater Bay, Hong Kong
Tel: +852 2358-7053
Email: eeau@ust.hk
www.ee.ust.hk/~eeau
Oscar Au’s background
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
BA Sc - Toronto (82-86), MA/PhD/Postdoc - Princeton (86-92)
Professor, HKUST (1992 to now).
Director, Multimedia Technology Center, HKUST.
Associate Editor of 6 journals: IEEE TCSVT, IEEE TIP, IEEE
TCAS-1, JSPS, JMM, JFI
Chair of IEEE CAS MSATC, Vice Chair/Chair of IEEE SP
MMSP TC, member of IEEE CAS VSPC/DSP TC, IEEE SP
IVMSP TC, IEEE ComSoC MMTC
Steering Committee of IEEE TMM/ICME. IEEE Fellow/DLP.
General Chair, PCM 2007/PV 2010/ICME 2010.
Host MPEG in 1/2005.
320 technical papers. 80 patents filed. 9 granted. 3 book
chapters.
67 MPEG/HEVC/AVS contributions. Several accepted into
MPEG4, AVS and HEVC.
Forensic expert. 7 press conferences. Talks/media interviews.
My multimedia expertise
• Video: coding (MPEG-1/2/4, H.261/3/4, AVS), transcoding,
enhancement, error resilience/concealment, denoising,
deinterlacing, HDR, SVC, MVC, 3DTV, DVC, GPU video
processing, depth estimation, view interpolation, subpixel
rendering, white balancing, color demosaicking,
compressive sensing, light weight encryption, watermarking,
motion estimation, rate control, perceptual quantization
• Image: coding (JPEG, JPEG2000), watermarking,
restoration, enhancement, processing
• Audio/speech: coding, watermarking, processing
• Special achievement: Our fast motion estimation technology
was accepted into MPEG-4. Up to 8000 times faster with
same quality. First ever in Hong Kong and China.
Video Processing
•
Standard-based image/video compression for SDTV/ HDTV:
MPEG-1/2/4, H.261/3/4, AVS, HEVC, etc.
•
Patented technologies accepted into standard: 1 for MPEG4, 2 for AVS, 2 for HEVC
•
Video transcoding, ITF, HKD 4,000,000, contract research
•
Watermarking, light-weight encryption, forensics: ~10 patents,
ITF, HKD 7,000,000.
•
Fast motion estimation, ~5+ patent, HKTIIT, HKD 10,000,000
•
3DTV, free viewpoint TV, distributed/scalable Video coding
•
Denoising, deinterlacing, error resilience, enhancement
•
Hardware-friendly algorithms for GPU and ASIC, Vision chip
•
~80 patents, IEEE Fellow, Associate Editors
Image Processing
•
•
•
•
•
•
•
•
•
High dynamic range (HDR) imaging
Subpixel rendering
Automatic white balancing (AWB)
Color interpolation (demosaicing)
Denoising, enhancement, restoration
Watermarking, data hiding, forensic
Halftone image
Texture synthesis
Segmentation,
Capture / Display - subpixel
processing (contract + GRF,
HKD 1,000,000)
Video coding (encoder optimization)
•
•
•
•
•
•
•
•
Fast motion estimation, PMVFAST
Fast subblock ME
Fast multi-frame ME
Subpixel ME
Fast intra-prediction
Luma/chroma coding
Rate control (block level), R-D modeling
Rate allocation (frame level)
Video coding related issues
•
•
•
•
•
•
Denoising – temporal, spatial, inter-color
Deinterlacing
Deblocking
Color processing
Transcoding
Error resilience
GPU/Hardware
• GPU-based processing
– ME, intra prediction
• Hardware architecture (SW/HW co-design)
– ME, deblocking, demosaicking,
– Filtering, subpixel rendering
– Data dependency, data loading, data re-use, low
power, low latency, reconfigurability
Beyond simple video coding
• Scalable video coding
• Distributed video coding
• Multiview video coding
– Intra-prediction
– Subblock motion estimation
– Multiple reference frame motion estimation
– Subpixel motion estimation
– Rate control
• JPEG2000 optimization
Imaging
•
•
•
•
•
•
•
•
•
Demosaicking
Automatic white balancing
Color correction
Subpixel rendering
High dynamic range imaging
Compressive sensing
Audio: variable speed playback
Audio: coding, MP3, AAC HE, SBR
Speech: recognitions
Security/forensics/others
•
•
•
•
•
•
Image/halftone watermarking
Attacks and steganography
Capacity estimation
Image forensics
Light-weight encryption
Encrypted domain processing