EE 6886: Topics in Signal Processing
-- Multimedia Security System
Lecture I: Introduction
Ching-Yung Lin
Columbia University
New York, NY 10027, USA
1/18/2006 | Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
© 2006 Columbia University
E 6886 Topics in Signal Processing: Multimedia Security Systems
Course Outline
Multimedia Security :
Multimedia Standards – Ubiquitous MM
Encryption and Key Management – Confidential MM
Watermarking – Uninfringible MM
Authentication – Trustworthy MM
Security Applications of Multimedia:
Audio-Visual Person Identification – Access Control, Identifying Suspects
Surveillance Applications – Abnormality Detection
Media Sensor Networks – Event Understanding, Information Aggregation
2
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1
E 6886 Topics in Signal Processing: Multimedia Security Systems
About this course
Instructor: Ching-Yung Lin
Email: cylin@ee.columbia.edu
Office Hour: Wednesday 6:40 – 7:10 pm, Mudd 1312, or by appointment
Course Webpage: http://www.ee.columbia.edu/~cylin/course/mss/
Course Time: Wednesday 4:10 – 6:40 pm
Course Format: Lecture 100 – 120 mins + Presentation/Discussion 20 – 40 mins
Location: Mudd 535
TA: TBD
TA Office Hour: TBD
Grading: 3 homeworks: 50%, final project: 50%
3
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Homeworks
HW #1: Image compression and encryption experiments.
HW #2: Video watermarking experiments
HW #3: Audio speaker authentication experiments
Software Requirements: C, C++, Java, Matlab, or others
Hardware Requirements: Windows, Unix/Linux or Mac
4
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
2
E 6886 Topics in Signal Processing: Multimedia Security Systems
Final Project
Team work is encouraged (1 – 3 students)
Implement components of multimedia security systems or surveys of
emerging technologies
Oral presentations at the mid-term project proposal and the final
presentation.
Final project report due at the end of semester.
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
5
E 6886 Topics in Signal Processing: Multimedia Security Systems
Examples of Final Project Topic
Digital Rights Management in Mobile Environment
Steganography and steganoanalysis
Multimedia Forensics
Human Vision Systems – implementations and experiments
Art authentication
Types of paintings: modern, abstract, impression, etc.
Tampering detection, Natural / CG detection
Face recognition in videos
Fingerprint recognition
Human behavior authentication:
Keyboard
Email records
Event detection from camera(s)
Audio/Visual Sensor Network
6
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
3
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
Multimedia Security :
Multimedia Standards – Ubiquitous MM
Encryption and Key Management – Confidential MM
Watermarking – Uninfringible MM
Authentication – Trustworthy MM
Security Applications of Multimedia:
Audio-Visual Person Identification – Access Control, Identifying Suspects
Surveillance Applications – Abnormality Detection
Media Sensor Networks – Event Understanding, Information Aggregation
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
7
E 6886 Topics in Signal Processing: Multimedia Security Systems
Multimedia Standards: Towards Knowledge Management and Transaction
Enrichment for Digital Media
Signals
Signals
Features
Features
Semantics
Semantics
Knowledge
Knowledge
Near
future
Recent
past
Applications
MPEG-1,-2,-4
Video storage
Broadband
Streaming video
delivery
MPEG-4,-7
Content-based retrieval
Multimedia filtering
Content adaptation
Compression
Coding
Communications
Similarity searching
Object- and featurebased coding
MPEG-7
Semantic-based retrieval
and filtering
Enterprise content mgmt.
MPEG-21
E-commerce of
Electronic content
Digital items
Problems and Innovations
MPEG
Modeling and classification
Personalization and
summarization
Media mining
Decision support
IPMP (rights)
- 1, - 2
MPEG
-4
MPEG
- 7
MPEG
8
1/18/06: Lecture 1 -- Introduction
- 21
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
4
E 6886 Topics in Signal Processing: Multimedia Security Systems
MPEG-1,2 Overview
I
B
B
P
B
B
P
B
B
P B
B
I
Intraframe: I frames
Interframe: P and B frames
MPEG-1: 352x240 or 352x264 – for VCD
MPEG-2: (1) multiple resolutions, e.g., 1024x768 – for compatibility
with TV. (2) field-based compression
MPEG-1 Audio Layer 3 – MP3
9
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
MPEG-4 Overview
Object-based compression --- x
Low-bit rate coding for mobile applications
Natural-Synthetic hybrid compression --- x
The latest MPEG-4 standard: H.264/AVC
10
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
5
E 6886 Topics in Signal Processing: Multimedia Security Systems
MPEG-7 Overview
XML Metadata for Multimedia Content Description
Compression
Coding
MPEG-1,-2,-4
Management
Filtering
Transmission
Retrieval
Streaming
MPEG-7 Normative elements:
Acquisition
Authoring
Editing
Descriptors and Description Schemes
DDL for defining Description Schemes
Extensible for application domains
MPEG-7
Standard
DDL
DS
DS
MPEG-7
Searching
Indexing
D
Browsing
Navigation
DS
D
Time
Axis
Segment Tree
Shot1
Shot2
D
D
Application
domain
D
Event Tree
i.e., Medical
imaging
Remotesensing
images
Surveillance
video
Computer
animations
and graphics
Shot3
Rich, highly granular descriptions:
Segment 1
• Introduction
Sub-segment 1
• Summary
Sub-segment 2
• Program logo
• Studio
Sub-segment 3
Video segments, moving regions, shots, frames, …
Audio-visual features: color, texture, shape, …
Semantics: people, events, objects, scenes, …
• Overview
Sub-segment 4
• News Presenter
segment 2
• News Items
• International
Segment 3
• Clinton Case
• Pope in Cuba
Segment 4
• National
Segment 5
• Twins
• Sports
Segment 6
• Closing
Reference Region
Reference Region
Reference Region
Motion
Motion
Motion
Segment 7
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
11
E 6886 Topics in Signal Processing: Multimedia Security Systems
MPEG-21 Multimedia Framework
“Transactions of Digital Items”
Users and participants in the content value chain
seamlessly exchange content in form of “digital
items” across networks and devices
Framework supporting all forms of electronic
content/intellectual property (video, music, learning
objects, on-line reports, etc.)
Digital Item = bundling of:
Essence (i.e., media resources)
Metadata
Rights expressions
Identifiers
Example: Digital music package
Descriptor
text/xml
MPEG-7
Creation
Information
Item
Descriptor
text/plain
Music package
(one song plus
two photos)
Item
music.mp3
Item
photo1.jpg
Item
photo2.jpg
12
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
6
E 6886 Topics in Signal Processing: Multimedia Security Systems
Types of Security
Computer Security
Protect data on a computer
Network Security
Multimedia Security
Protect data during transmission
Content Security
Protect intellectual property
Provide Trustworthiness
COMPUTER SECURITY
+
+
NETWORK SECURITY
CONTENT SECURITY
Courtesy of Prof. Kundur, Texas A&M
1/18/06: Lecture 1 -- Introduction
13
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Multimedia Security
Content
Provider
N
N
N
14
???
Content
Receiver
Data Authentication:
-- assure the credibility of multimedia content.
Confidentiality:
-- secure content transmission privacy.
Copy Control:
-- protect multimedia data from illegal distribution and theft.
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
7
E 6886 Topics in Signal Processing: Multimedia Security Systems
Digital Rights Management (DRM) System
Definition (from Iannella, 2001)
Digital Rights Management (DRM) involves the description,
identification, trading, protection, monitoring, and tracking of all forms
of rights usages over both tangible and intangible assets – both in
physical and digital form – including management of Rights Holders
relationships.
Digital management of use rights to content
Links specific user rights to media to control access, viewing,
duplication, and sharing.
Ideally, balances information protection, usability, and cost to provide a
beneficial environment for all parties involved.
15
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
An Example of Digital Rights Management System
Creator
Creation
Unique
Number
Creation
Description
Unique
Unique?
Number
Issuer
Creation
Provider
Value
As
Va
si g
lu
nm
e
en
to
fR
ig
ht
s
Rights
Holder
IMPRIMATUR DRM Model
16
1/18/06: Lecture 1 -- Introduction
Certification
Authority
ck
he
C
ID
of
Creation
D
M
Check P
identity
Media
Distributor
ce
lue cen
e
Va or li
nc
f
i ce
l.
L
p
l
Ap
na
itio
nd
o
C
IPR data
C
au he
th ck
or
isa MD
tio
n
Value
T & Cs of use
Creation
R
IP
Purchaser
Lo
g
fo
In
Monitoring
Service
Provider
IPR
Database
Log
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
8
E 6886 Topics in Signal Processing: Multimedia Security Systems
Multiple Aspects of DRM
Technical
Enforcement by engineering mechanisms/systems
Business
Commercially viable products/services
Social
User privacy, limits on user behavior, etc.
Legal
Enforcement by legislation
17
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
Multimedia Security :
Multimedia Standards – Ubiquitous MM
Encryption and Key Management – Confidential MM
Watermarking – Uninfringible MM
Authentication – Trustworthy MM
Security Applications of Multimedia:
Audio-Visual Person Identification – Access Control, Identifying Suspects
Surveillance Applications – Abnormality Detection
Media Sensor Networks – Event Understanding, Information Aggregation
18
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
9
E 6886 Topics in Signal Processing: Multimedia Security Systems
Security Services (X.800)
Person Authentication
Assurance that communicating entity is the one claimed
Access Control
Prevention of unauthorized use of a resource
Data Confidentiality
Protection of data from unauthorized disclosure
Data Integrity
Assurance that data received is as sent
Non-Repudiation
Protection against denial by the parties in a communication
19
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Confidentiality
Encryption
Real-time requirements
Integration with compression
EAVESDROPPING
Courtesy of Prof. Kundur, Texas A&M
20
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
10
E 6886 Topics in Signal Processing: Multimedia Security Systems
Access Control
Encryption
Authentication
Query???
Courtesy of Prof. Kundur, Texas A&M
21
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Revocation
Early detection and reaction
Courtesy of Prof. Kundur, Texas A&M
22
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
11
E 6886 Topics in Signal Processing: Multimedia Security Systems
Conventional Encryption Algorithms
Data Encryption Standard (DES)
The most widely used encryption scheme
DES is a block cipher – the plaintext is processed in 64-bit blocks
The key is 56-bits in length
Based on Feistel Cipher Structure
Triple DES
Effective key length of 112/168 bits
Advanced Encryption Standard (AES)
128-bit data, 128/192/256-bit keys
Stronger & faster than Triple-DES
Public key encryption: asymmetric key
key
key
plaintext
23
plaintext
ciphertext
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
Multimedia Security :
Multimedia Standards – Ubiquitous MM
Encryption and Key Management – Confidential MM
Watermarking – Uninfringible MM
Authentication – Trustworthy MM
Security Applications of Multimedia:
Audio-Visual Person Identification – Access Control, Identifying Suspects
Surveillance Applications – Abnormality Detection
Media Sensor Networks – Event Understanding, Information Aggregation
24
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
12
E 6886 Topics in Signal Processing: Multimedia Security Systems
Piracy Protection
Compliant Players
Courtesy of Prof. Kundur, Texas A&M
25
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Copyright Protection and Copy Control
Scramble
Tx
Rx
Descramble
VCR
Because of content-preserving transcoding:
Ownership Identification, Copy Control have to survive multi-stage
transcoding
Use robust watermarking
26
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
13
E 6886 Topics in Signal Processing: Multimedia Security Systems
Watermarking
• Embedding
Visible/Invisible Codes
in Multimedia Data for
(or not for) Security
Purpose
PIL or content- based
feature codes
Tx
Verify the
watermark
Watermark
27
1/18/06: Lecture 1 -- Introduction
Rx
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Visible Watermark
Purpose:
Claim the ownership and prevent content piracy.
Properties:
Robust: Watermarks must be very difficult, if not impossible, to be
removed.
Non-obtrusive: Watermarks must not affect the audiovisual contents
too much.
28
Visible: It must be visible, but it had better to be insensible.
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
14
E 6886 Topics in Signal Processing: Multimedia Security Systems
Visible Watermark Example
Description:
♦ Robustness:
Pixel brightness were altered pixel by pixel.
Depending on the brightness of each mask
pixel, if it is larger than a threshold, then we
add a value to the corresponding pixel of the
original image to make it brighter. Otherwise, if
it is smaller than a threshold, then we subtract
a value to make the corresponding pixel darker.
–
–
–
Randomly set the adjustment factor.
Randomly locate the mask.
The mask pixels may depend on the image
contents.
Alternation Function:
Watermark mask
29
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Invisible Watermark
Purpose:
Protect ownership and trace illegal use.
Properties -- Transmit a bitstream through a very noisy channel, i.e. the
original picture.
Robust: The watermark must be very difficult, if not impossible, to
remove. It must be able to survive manipulations to the images, such
as: lossy compression, format transformation, shifting, scaling,
cropping, quantization, filtering, xeroxing, printing, and scanning.
Invisible: The watermark should not visually affect the image/video
content.
30
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
15
E 6886 Topics in Signal Processing: Multimedia Security Systems
What is Watermarking ? –
Multimedia as a Communication Channel
Basic communication system:
Information
W
Decoder
W
Analog Communication -
Channel
Encoder
Encoder/ Decoder:
Amplitude Modulation (AM),
Frequency Modulation (FM).
Multiplexing: use different carrier frequencies.
Channel: air, wire, water, space, …
Watermarking:
Information
W
Encoder
Decoder
W
Image/Video/Audio
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
31
E 6886 Topics in Signal Processing: Multimedia Security Systems
Watermarking -- Multimedia as Communication Channel
X ≤P
Sw
Sw
Encoder
W
Decoder
W
Information
S
Perceptual
Model
Distortion
Model
Z ≤N
Private/ Public
S: Source Image (Side Information)
W: Embedded Information
X: Watermark (Power Constraint: P)
Z: Noise (Power Constraint: N )
Source Image
Encoder may include two stages: Coding and Modulation.
Coding: Error Correction Codes, Scrambling (use cryptographic keys).
Modulation:
Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Code
Division Multiple Access (CDMA).
Spread Spectrum is a CDMA technique, which needs modulation keys for Frequency Hopping
or other specific codes.
32
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
16
E 6886 Topics in Signal Processing: Multimedia Security Systems
Example: Watermark surviving Print-and-Scan
Original Image [384x256]
Watermarked Image, PSNR 43.8dB, ρ=0.84, Z=7.02
After PS, Crop to 360x240 &
JPEG CR: 95:1 => ρ=0.64, Z=4.30
After Print & Scan, Crop to
402x266 => ρ=0.80, Z=6.46
33
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Example: IBM Digital Music Content Platform Project
•Automatic generation of cue sheets using audio watermarking
•Secure and easy distribution of music content
•Japanese government funding the project
•Cooperation by popular FM radio stations and major Japanese labels
FM Radio Broadcast
Host
Audio
Stream
ThinkPad
Audio-in
Content Authoring
(Watermark Embedding)
ＡＤ
Conversion
Audio-out
Real-time
Watermark
Embedder
Watermarked
Audio
Stream
Broadcast Monitoring
(Watermark Detection)
ＤＡ
Conversion
Monitoring Center
34
1/18/06: Lecture 1 -- Introduction
Content ID © 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
17
E 6886 Topics in Signal Processing: Multimedia Security Systems
DMCP Audio Data Hiding Performance
Data Payload
Standard Version : 72bit/30second (for STEP and DMCP)
Short Window Detection : 27bit/2-5second
Benchmarking: STEP – JASRAC (Japan), BIEM, CISAC (France)
Robustness:
Analogue Conversion
Down-mixing (2ch->1ch)
Time scaling (10%)
Pitch shifting (10%)
Down-sampling (16kHz)
ATRAC3 compression
MP3 compression (96kbps)
AAC compression 128kbps
ATRAC compression
RealAudio compression (64kbps)
Windows Media Audio compression
(64kbps)
Dynamic range compression
FM broadcast
AM broadcast
PCM broadcast
Noise addition (-30dB)
Acoustic Quality
DVD-Audio quality Psycho-acoustic model
Joint evaluation test at a Audio device maker’s DVD-Audio studio in 2000
Reliability
Controllable false alarm rate (e.g. 10-5-10-12)
Low bit error rate achieved through built-in Error Correcting Code
35
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
Multimedia Security :
Multimedia Standards – Ubiquitous MM
Encryption and Key Management – Confidential MM
Watermarking – Uninfringible MM
Authentication – Trustworthy MM
Security Applications of Multimedia:
Audio-Visual Person Identification – Access Control, Identifying Suspects
Surveillance Applications – Abnormality Detection
Media Sensor Networks – Event Understanding, Information Aggregation
36
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
18
E 6886 Topics in Signal Processing: Multimedia Security Systems
A Photographer’s Shot
President Clinton and First Lady strolled in the White House
37
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Somebody Manipulates It….
Another proof of their relationship ???
38
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
19
E 6886 Topics in Signal Processing: Multimedia Security Systems
Hillary’s Revenge???
Hillary and Ching-Yung walked together???
39
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Integrity
Hash Functions
Traditional approaches sensitive to format conversion and minor bit
changes
Existing software tools enable seamless tampering
TAMPERING
Courtesy of Prof. Kundur, Texas A&M
40
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
20
E 6886 Topics in Signal Processing: Multimedia Security Systems
Person Authentication
Digital signatures
Biometrics
IMPERSONATION
Courtesy of Prof. Kundur, Texas A&M
41
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Nonrepudiation
Digital certificates
Biometrics
Courtesy of Prof. Kundur, Texas A&M
42
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
21
E 6886 Topics in Signal Processing: Multimedia Security Systems
(Robust) Digital Signature
Digital Signature, Diffie and Hellman (1976).
Verify the data integrity which is endorsed by the signer.
Trustworthy Digital Camera, Friedman (1993).
Non-Repudiation Signature to prove reality
Content-Based Digital Signature, Schneider, Lin, Chang (1996, 1997)
Using content-related feature codes instead of the hash values.
Image
Private Key Encrypted
Hash
Feature
Codes
Extraction
Digital Camera
Digital
Signature
Digital Image
Public Key Decrypted
Digital
Signature
Digital Image
Comparator
Feature
Hash
Codes
Extraction
Result
Image Authenticator
43
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Self Authentication-and-Recovery Images (SARI)
add
watermark
watermarked SARI
image
original image
manipulation
authentication
authentication
& recovery
image after crop-and-replacement
and JPEG lossy compression
44
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
22
E 6886 Topics in Signal Processing: Multimedia Security Systems
Other Applications
Semantic Authentication
Image/Document Forensics
Who is the actual author of a picture or a novel?
Is the media data manipulated?
45
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
Multimedia Security :
Multimedia Standards – Ubiquitous MM
Encryption and Key Management – Confidential MM
Watermarking – Uninfringible MM
Authentication – Trustworthy MM
Security Applications of Multimedia:
Audio-Visual Person Identification – Access Control, Identifying Suspects
Surveillance Applications – Abnormality Detection
Media Sensor Networks – Event Understanding, Information Aggregation
46
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
23
E 6886 Topics in Signal Processing: Multimedia Security Systems
Biometric Features for Person Authentication
47
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Example: Fingerprint-based Authentication
Fingerprint Match
Fingerprint minutiae
48
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
24
E 6886 Topics in Signal Processing: Multimedia Security Systems
Speaker Recognition
Speaker Verification and Speaker Identification:
IBM system
Basic Model Unit
Speech
Signal
Feature
Extraction
Clustering
GMM
GMM + T
Population
Counts
Gradient
Descent
T
Transformation
Enhanced
Model Unit
Features: (TREC2002) 38 dimensional MFCC, (Journal 1/03) 19
GMM: (TREC 2002) 1536 or 2048 mixtures, (Journal 1/03) 64 mixtures
Universal Background model (UBM): [TREC2002] trained by 60
speakers (two minutes for training, one minute for testing),
49
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Outline -- Introduction
Multimedia Security :
Multimedia Standards – Ubiquitous MM
Encryption and Key Management – Confidential MM
Watermarking – Uninfringible MM
Authentication – Trustworthy MM
Security Applications of Multimedia:
Audio-Visual Person Identification – Access Control, Identifying Suspects
Surveillance Applications – Abnormality Detection
Media Sensor Networks – Event Understanding, Information Aggregation
50
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
25
E 6886 Topics in Signal Processing: Multimedia Security Systems
Event Understanding
Another View
Event Understanding:
Objects:
Visual Objects:, Tree, Person, Hands, …
Audio Objects: Music, Speech, Sound, …
Scenes:
Background: Building, Outdoors, Sky
Relationships:
The (time, spatial) relationships between objects & scenes
Activities:
Holding Hand in Hand, Looking for Stars
1/18/06: Lecture 1 -- Introduction
51
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Example: Low-cost Multimodality Sensors for Sleep Situation
Monitoring
Understand human night-time activity –
Sleep
What we have done:
What we are going to do:
52
Using visual, audio, heartbeat, infrared
sensors to monitor a person’s sleep patterns
Measurement of sleep quality
Logging and retrieval of sleep situation
Early detection and long term monitoring of
sleep related diseases
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
26
IBMUnit
Research
Business
or Product Name
Distributed Video Signal Understanding
TV broadcast,
VCR,
DVD discs,
Video
File Database,
Webcam
(Distributed Smart Sensors) Block diagram of the
smart sensors
(Server) Concept Detection
Processing Elements
PE1: 9.2.63.66: 1220
PE4: 9.2.63.66:1235
Face
MPEGCDS
PE2: 9.2.63.67
Encod- 1/2
GOP
Feature
Event Features
ing
Extraction
Extraction 22.4 Extraction 2.8 Outdoors
1.5
320
Mbps
Kbps
Kbps
Kbps
Female
PE5: 9.2.63.66: 1240 Meta-
PE3: 9.2.63.68
Indoors
data
Male
PE6: 9.2.63.66
Airplane
Sensor 1
PE7: 9.2.63.66
Sensor 2
Sensor N
Smart Cam
600
bps
Chair
Sensor 3
PE100: 9.2.63.66
Clock
Display and
Information
Aggregation
Modules
User
Interests
Control
Modules
Resource
Constraints
Learning and Mining from Multimodal Sensing Signals | Ching-Yung Lin | IBM
© 2003
2005 IBM Corporation
E 6886 Topics in Signal Processing: Multimedia Security Systems
References
• K. Hill, “Imprimatur Business Model,” MCPS 2nd Consensus Forum, Stockholm, May 1997.
• ISO/IEC JTC 1/SC 29/WG 11 N4269, “Information Technology – Multimedia Framework
(MPEG-21) – Part 4: Intellectual Property Management and Protection,” July 2001.
G. Kesden, “Introduction on Content Scrambling System,” Lecture on Operating Systems:
Design and Implementation, CMU, Dec. 2000.
R. Tachibana, “Audio Watermarking for Live Performance,” SPIE Security and
Watermarking of Multimedia Contents V, San Jose, January 2003.
J. A. Bloom, I. J. Cox, T. Kalker, J.-P. Linnartz, M. L. Miller and B. Traw, “Copy
Protection for DVD Video,” Proceedings of the IEEE, July 1999.
J. A. Bloom, M. L. Miller and I. J. Cox, “Digital Watermarking”, Morgan Kaufmann
Publishers, 2001.
C.-Y. Lin and S.-F. Chang, “Semi-Fragile Watermarking for Authenticating JPEG Visual
Content,” SPIE Security and Watermarking in Multimedia Contents II, Vol. 3971, Jan. 2000.
U. Uludag, S. Pankanti, S. Prabhakar and A. K. Jain, “Biometric Cryptosystems: Issues and
Challenges,” Prof. of the IEEE, June 2004.
54
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
27
E 6886 Topics in Signal Processing: Multimedia Security Systems
M.S. Research Projects Available (Spring 2006, 3Pts)
Developing Distributed Smart Video Cameras (Phase II):
Large-Scale Data Mining, Community Modeling, Human Behavior
Modeling
55
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
1/18/06: Lecture 1 -- Introduction
E 6886 Topics in Signal Processing: Multimedia Security Systems
Other Related Research Issues
Issues in multimedia security
copyright protection, authentication, fingerprinting: system, theory and
techniques
public watermarking techniques, watermarking attacks, quality
evaluations and benchmarks
perceptual models, noise models, information theoretical models
conditional access
Traitor tracing
legal aspects
watermarking protocols
security in JPEG2000, MPEG-4, MPEG-7 or MPEG21
biometrics and multimedia security
watermarking/ information hiding applications
http://www.research.ibm.com/people/c/cylin
Acknowledgment: Thanks for Prof. Deepa Kundur’s (Texas A&M U.) assistance on the
graphical examples of MM security objectives.
56
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
28
E 6886 Topics in Signal Processing: Multimedia Security Systems
Columbia and IBM
T. J. Watson Research Center
The first
supercomputer
(1954)
1953-1970
(612 W115th St.)
1945-1953
The first computer science course
(1947)
The founding of ACM (1947)
The first “personal” computer (1948)
1970 –
(Yorktown Heights)
(612 W116th St.)
57
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
E 6886 Topics in Signal Processing: Multimedia Security Systems
Open Discussion
58
1/18/06: Lecture 1 -- Introduction
© 2006 Ching-Yung Lin, Dept. of Electrical Engineering, Columbia Univ.
29