Multimedia Databases
Eran Stiller
Omer Gertel
In this presentation…
Introduction


What is a Multimedia Database?
Why do we need it?
Modeling Multimedia Databases
Design Methodology
Modeling the WWW as a Multimedia
Database
WebSQL
 Using the Multimedia Design Methodology
Graphical extension to SQL: SQL+D

Eran Stiller
Omer Gertel
Multimedia Databases
2
What is a Multimedia
Database?
Multimedia Data-types

Movies, Pictures, Sound Clips, Documents
etc…
Advanced Querying Possibilities
Eran Stiller
Omer Gertel
Multimedia Databases
3
Why Do We Need It?
Tremendous Amounts of
Digitized Multimedia Content

Video servers, Ifilms, IMDB, AllMusic, etc…
Querying for Digitized Information

“A Picture is Worth a Thousand Words”
Eran Stiller
Omer Gertel
Multimedia Databases
4
The Leading Concept
Extracting Information from Multimedia
Objects “On the Fly”
Creating a Design Methodology
Simple, Intuitive, Upward Compatible
 Unbound by Data-type, Application, Usage

Eran Stiller
Omer Gertel
Multimedia Databases
5
Modeling Multimedia
Databases
Using Object Relational Data Models

Data Description Language
Eran Stiller
Omer Gertel
Multimedia Databases
6
Data Description Language
Example:

Movie Database:


MOVIE(Title, Year, Director, Length)
What About the Movie?
Add It to the Database
 Create Additional Functions

Eran Stiller
Omer Gertel
Multimedia Databases
7
Data Description Language
First Approach:

Complex Type:
Create Type Movie_t(
Title varchar(20),
Year int,
Director varchar(20),
Length int,
MV blob);
 Create Table Movie of type Movie_t;

Eran Stiller
Omer Gertel
Multimedia Databases
8
Data Description Language
Second Approach:

Extended Type:

Eran Stiller
Omer Gertel
Create Table Movies(
Title varchar(20),
Year int,
Director varchar(20),
Length int,
MV movie_blob);
Multimedia Databases
9
Data Description Language
Concerns

Distinguishing the Digitized Information
Conclusion
Naming Convention
 The ‘Core’ Special Attribute

Eran Stiller
Omer Gertel
Multimedia Databases
10
Data Description Language
Example – Continued:

A New Relation:


MOVIE(Title, Year, Director, Length, Core)
What About Querying?

Eran Stiller
Omer Gertel
“Show the opening sequence of all movies
directed by Spielberg in 1993.”
Multimedia Databases
11
Data Description Language
SQL-like Language:

Select Movie.opening_seq
From Movie
Where Year=1993 AND
Director=“Spielberg”;
Eran Stiller
Omer Gertel
Multimedia Databases
12
Data Description Language
SQL Solution:

Declare External opening_seq(movie_blob)
Returns movie_blob
Language C;
Smart Interpreter
Eran Stiller
Omer Gertel
Multimedia Databases
13
A Design Methodology
Core-ER Diagram
Core-ORDB schema
Eran Stiller
Omer Gertel
Multimedia Databases
14
Core-ER Diagram
Entities
Abstract Entities
 Core Entities

Eran Stiller
Omer Gertel
Multimedia Databases
15
Core-ER Diagram
Attributes
Conventional Attributes
 Unconventional Attributes

Eran Stiller
Omer Gertel
Multimedia Databases
16
Core-ER Diagram
Methods
Core as Input
 Entity as Input

Eran Stiller
Omer Gertel
Multimedia Databases
17
Core-ER Diagram
Keys and Relationships

Exactly as in ER Diagrams
Eran Stiller
Omer Gertel
Multimedia Databases
18
Core-ER Diagram
Example
Eran Stiller
Omer Gertel
Multimedia Databases
19
CER to Core-ORDB Schema
We Need Rules For:




Abstract Entities
Core Entities
Methods
Relationships


Embedded ER Diagram
Keys

Core Can Be a Key
Example
Eran Stiller
Omer Gertel
Multimedia Databases
20
Employee(SSN, name, picture, position, salary, Dept)
Eran Stiller
Omer Gertel
Multimedia Databases
21
EMail(EMailID, From, To, Date, Date R, Size,
RespTo, Core)
Eran Stiller
Omer Gertel
Multimedia Databases
22
Declare External Keyword(Email, string) Returns BOOLEAN
<Function Definition>;
Eran Stiller
Omer Gertel
Multimedia Databases
23
Has(SSN, EMailID)
Eran Stiller
Omer Gertel
Multimedia Databases
24
Note: Core Can Be A Key!
Eran Stiller
Omer Gertel
Multimedia Databases
25
A Design Methodology
Conclusion
Full Upward Compatibility
 An Easy to Use Solution

Eran Stiller
Omer Gertel
Multimedia Databases
26
Modeling the WWW
WebSQL
Using the Multimedia Design
Methodology
Eran Stiller
Omer Gertel
Multimedia Databases
27
WebSQL
SQL-Like Language
Minimal Relational Approach
Queries Based Both On The Content
And On The Hypertext Structure Of The
Web
Eran Stiller
Omer Gertel
Multimedia Databases
28
WebSQL
Content Based Queries

Document(url, title, text, type, length,
lastModify)
Eran Stiller
Omer Gertel
Multimedia Databases
29
WebSQL
Example:

Find All HTML Documents About
“Hypertext”

Eran Stiller
Omer Gertel
Select D.url, D.title, D.length, D.lastModify
From Document D such that D mentions
“Hypertext”
Where D.type = “text/html”;
Multimedia Databases
30
WebSQL
Hypertext Structure Based Queries
Document(url, title, text, type, length,
lastModify)
 Anchor(base, href, label)

Eran Stiller
Omer Gertel
Multimedia Databases
31
WebSQL
Example:

Find All Links to Applet from Documents
About “Java”

Eran Stiller
Omer Gertel
Select Y.label, Y.href
From Document X such that X mentions “Java”
Anchor Y such that base = X
Where Y.label contains “Applet”;
Multimedia Databases
32
WebSQL
Hypertext Link Classification



Interior
Local
Global
()
()
()
Empty Path
(=)
Path Regular Expressions



Alternation
(|)
Repetition
(*)
Concatenation ( . )
Eran Stiller
Omer Gertel
Multimedia Databases
33
WebSQL
Example:

Select D.url, D.title
From Document D such that
“http://www.cs.huji.ac.il” = |  | . D
Where D.title contains “database”
Eran Stiller
Omer Gertel
Multimedia Databases
34
WebSQL
Example:

Select X.url, X.title, Y.url, Y.title
From
Document X such that X mentions
“Computer Science”,
Document Y such that X = |  | .  Y;
Eran Stiller
Omer Gertel
Multimedia Databases
35
Modeling the WWW
WebSQL
Already Implemented in Java
 Specific Solution
 Refers Only To Documents

Using the Multimedia Design
Methodology
Eran Stiller
Omer Gertel
Multimedia Databases
36
Multimedia Databases
We’ll Be Back, Right After The Break…
Modeling the WWW
WebSQL
Already Implemented in Java
 Specific Solution
 Refers Only To Documents

Using the Multimedia Design
Methodology
Eran Stiller
Omer Gertel
Multimedia Databases
38
Multimedia Design
Methodology
Entities
HTMLdoc(h_url, title, type, length,
lastModify, core)
 Links(l_url, label)
 Include(h_url, l_url)

Eran Stiller
Omer Gertel
Multimedia Databases
39
Multimedia Design
Methodology
Methods
Contains(HTMLdoc.title, string)
 Reach_by(HTMLdoc.url, url_to, by_n,
l_type)
 Mentions(HTMLdoc, string)
 Linktype(HTMLdoc, url)

Eran Stiller
Omer Gertel
Multimedia Databases
40
Multimedia Design
Methodology
Example:

Find All HTML Documents About
“Hypertext”

Eran Stiller
Omer Gertel
Select HTMLdoc.h_url
From HTMLdoc
Where Contains(HTMLdoc.title, “Hypertext”);
Multimedia Databases
41
Multimedia Design
Methodology
Example:

Find All Links to Applet from Documents
About “Java”

Eran Stiller
Omer Gertel
Select Links.l_url
From HTMLdoc, Links, Include
Where mentions(HTMLdoc, “java”) AND
HTMLdoc.h_url = Include.h_url AND
Links.l_url = Include.l_url AND
substring(“applet”, Links.label);
Multimedia Databases
42
Multimedia Design
Methodology
Example:

Select Links.l_url
From HTMLdoc, Links, Include
Where substring(“database”, HTMLdoc.title) AND
HTMLdoc.h_url = Include.h_url AND
Links.l_url = Include.l_url AND
reach_by(“http://www.cs.huji.ac.il”,
Links.l_url, 2, local);
Hint: We’ve seen it before…
Eran Stiller
Omer Gertel
Multimedia Databases
43
Multimedia Design
Methodology
Example:

Select HTMLdoc.h_url, Links.l_url
From HTMLdoc, Links
Where mentions(HTMLdoc, “Computer Science”)
AND
reach_by(HTMLdoc.h_url, Links.l_url, 2, local);
Eran Stiller
Omer Gertel
Multimedia Databases
44
Multimedia Design
Methodology
Simplified Model, But…
Complicated Real World
Taking Another Step
Eran Stiller
Omer Gertel
Multimedia Databases
45
Multimedia Design
Methodology
Entities
HTMLdoc(h_url, title, type, length,
lastModify, core)
 WebObject(w_url, label, objectType,
interactive, core)
 Properties(position, size, props)
 Include(h_url, w_url, position, size, props)

Eran Stiller
Omer Gertel
Multimedia Databases
46
Multimedia Design
Methodology
Methods




Contains(HTMLdoc.title, string)
Reach_by(HTMLdoc.url, url_to, by_n, l_type)
Mentions(HTMLdoc, string)
Linktype(HTMLdoc, url)
And Additional Methods


DisplayDoc(HTMLdoc)
DisplayObj(WebObject, Properties.position,
Properties.size, Properties.props)
Eran Stiller
Omer Gertel
Multimedia Databases
47
Multimedia Design
Methodology
Example:

List All Documents That Have a Video-Clip Or a
Picture Labeled “Sharon”

Eran Stiller
Omer Gertel
Select HTMLdoc.h_url
From HTMLdoc, WebObject, Include
Where HTMLdoc.h_url = Include.h_url AND
WebObject.w_url = Include.w_url AND
(WebObject.objectType = “Image” OR
WebObject.objectType = “Video”) AND
WebObject.label = “Sharon”;
Multimedia Databases
48
Multimedia Design
Methodology
Still Not Complicated Enough…
The Last Step
Eran Stiller
Omer Gertel
Multimedia Databases
49
Modeling the WWW
WebSQL



Already Implemented in Java
Specific Solution
Refers Only To Documents
Using the Multimedia Design Methodology



General Methodology Used
Doesn’t Require a Special Web SQL-Like
Language
Easily Expandable
Eran Stiller
Omer Gertel
Multimedia Databases
50
SQL+D
Graphical Extension to SQL
Answering a Multimedia Database
Query
Easier than Creating Specific Display
Solutions
Display Specifications Are Included in
the Query
Eran Stiller
Omer Gertel
Multimedia Databases
51
SQL+D
Video Store Database
Movie(available, title, director, producer,
date, classification, rating, core, poster)
 Movie_Actors(title, name, role)
 Actors(name, age, biography, picture)

Eran Stiller
Omer Gertel
Multimedia Databases
52
SQL+D
List All Actors in “Gone With the Wind” with
Their Pictures and Biographies

Select MA.name, A.biography, A.picture
From Movie_Actors MA, Actors A
Where MA.title = “Gone With the Wind” AND
A.name = MA.name
Display Panel main, Panel info On main(east),
With MA.name As list On main(west),
A.picture As image On info(north),
A.biography As text On info(south);
Eran Stiller
Omer Gertel
Multimedia Databases
53
Eran Stiller
Omer Gertel
Multimedia Databases
54
SQL+D
Display All Action Movies and Their Posters,
Allowing to Play the movie If Desired

Select title, poster, core
From Movie
Where classification = “Action”
Display Panel main, Panel b On main(east)
With poster As image On main(west),
title As list On b(north),
“Play” As button On b(south)
Trigger ‘mpegplay(‘+ core +’)’
Eran Stiller
Omer Gertel
Multimedia Databases
55
Eran Stiller
Omer Gertel
Multimedia Databases
56
SQL+D
Campus Map Database
Campus(campus_area,area_map)
 Building(building_name, building_picture,
campus_area, coordinates)
 Floor(building_name, floor, floor_map)
 Room(building_name, floor, room_name,
description, homepage, info, coordinates)

Eran Stiller
Omer Gertel
Multimedia Databases
57
SQL+D
Show a Map of Main Campus With a
List of the Buildings in it

Select C.area_map, B.building_name
From Campus C, Building B
Where C.campus_area = “Main Campus”
Display Panel main
With B.building_name As list On main(east),
C.area_map As image On main(west);
Eran Stiller
Omer Gertel
Multimedia Databases
58
Eran Stiller
Omer Gertel
Multimedia Databases
59
SQL+D
Show a Map of Main Campus With a
List of the Buildings in it

Select C.area_map, B.building_name,
B.coordinates
From Campus C, Building B
Where C.campus_area = “Main Campus”
Display Panel main
With B.building_name As list On main(east),
C.area_map As image On main(west)
All B.coordinates As polygon On main(west);
Eran Stiller
Omer Gertel
Multimedia Databases
60
Eran Stiller
Omer Gertel
Multimedia Databases
61
SQL+D
Show a Map of Main Campus With a
List of the Buildings in it

Select C.area_map, B.building_name
From Campus C, Building B
Where C.campus_area = “Main Campus”
Display Panel main
With B.building_name As list On main(east)
Trigger ‘building_maps(‘+ building_name +’)’
C.area_map As image On main(west);
Eran Stiller
Omer Gertel
Multimedia Databases
62
SQL+D
Where building_maps(input) is

Select B.building_name, B.building_picture, F.floor
From Building B, Floor F
Where B.building_name = + input + AND
F.building_name = B.building_name
Display Panel main, Panel b On main(south)
With building_picture As image On main(north),
building_name As title On main(north),
All floor As button On b(east)
Trigger ‘floor_buildings( ‘+ building_name +’,
‘+ floor +’)’
“Select Floor To Visit” On b(west);
Eran Stiller
Omer Gertel
Multimedia Databases
63
Eran Stiller
Omer Gertel
Multimedia Databases
64
SQL+D
And floor_buildings(input1, input2) is

Select F.floor_map, R.floor, R.room_name,
R.info, R.homepage, R.coordinates
From Floor F, Room R
Where F.building_name = +input1+ AND
F.floor = +input2+ AND
R.floor = F.floor
Eran Stiller
Omer Gertel
Multimedia Databases
65
SQL+D
And floor_buildings(input1, input2) is

Display Panel main,
Panel b On main(west),
Panel c On main(east),
With floor_map As image On c(north),
All coordinates As poly On c(north),
room_name As list On b(north),
info As text On b(center),
All Distinct floor As button On c(south)
Trigger floor_buildings(‘+building_name+’,‘+floor+’)’
homepage As button On b(south);
Eran Stiller
Omer Gertel
Multimedia Databases
66
Eran Stiller
Omer Gertel
Multimedia Databases
67
SQL+D
SQL-Like Queries
Application-Like Display Controller
Makes Most of Both Worlds
Simple to Use
 User Friendly Output

Eran Stiller
Omer Gertel
Multimedia Databases
68
Conclusion
Multimedia Content Becomes More and
More Popular
Correct Generic Models Bring Easy to
Use Results
Clean, Simple Solutions Come From
Extending the Existing Format
Eran Stiller
Omer Gertel
Multimedia Databases
69
References
Chitta Baral, Graciela Gonzalez and Tran Son, “Conceptual Modeling
and Quering in Multi-media Databases”, Department of CS, UTEP
Alberto O. Mendelzon, George A. Mihaila and Tova Milo, “Quering the
World Wide Web”, University of Toronto and Tel-Aviv University, 1996
Chitta Baral, Graciela Gonzalez and Tran Son, “A Multimedia Display
Extension to SQL: Language Design and Architecture”, Department of
CS, UTEP, 1997
Chitta Baral, Graciela Gonzalez and Tran Son, “Issues in Querying
Multimedia Databases”, Department of CS, UTEP
Isabel F. Cruz, Michael Averbuch, Wendy T. Lucas, Mellisa
Radzyminski, Kirby Zhang, “Delaunay: a Database Visualization
System”, Database Visualization Reaserch Group, Tufts University
Eran Stiller
Omer Gertel
Multimedia Databases
70
Th…Th…That’s All Folks!
Eran Stiller
Omer Gertel
Multimedia Databases
71