Logical and physical data
structures for very small
databases (VSDB)
Fabio A. Schreiber
Cristiana Bolchini
Letizia Tanca
Fabio Salice
Politecnico di Milano
DATABASES AND SYSTEMS
LARGE
VLDB
VSDB
?
SYSTEM
DB
SMALL
SMALL
DATABASE
LARGE
Logical/Physical Data Structures in
VSDB
1
1
PRESENTATION OUTLINE
• MOTIVATIONS
• VSDB DESIGN METHODOLOGY
• LOGICAL AND PHYSICAL DATA STRUCTURES
• ACCESS MODES
• COMPLEXITY AND POWER CONSUMPTION ANALYSIS
• WORK IN PROGRESS ...
Logical/Physical Data Structures in
VSDB
2
MOTIVATIONS
• FROM PC DOWNWARDS
PALM AND HANDHELD PC
SMART CARDS
CELLULAR PHONES (SIM CARDS)
• PROFESSIONAL EMBEDDED SYSTEMS
DEVICE CONTROL AND TELEMETRY
SYSTEMS AND INDUSTRIAL PROCESS
CONTROL
DISTRIBUTED NETWORK MANAGEMENT
Logical/Physical Data Structures in
VSDB
3
2
PHYSICAL STORAGE MEDIUM
Flash EEPROM
TECHNOLOGY ISSUES
• CONSTRAINTS ON MODIFY (ERASE/WRITE) OPERATIONS
– WRITE = PROGRAMMING
– WRITE ONLY ON VIRGIN OR ERASED LOCATIONS
– ERASE ONLY AT BLOCK LEVEL
• TIMING ISSUES
• BLOCK ALLOCATION/MANAGEMENT
• ENDURANCE
PERSPECTIVE FROM
• PERFORMANCE VIEWPOINT
• POWER CONSUMPTION VIEWPOINT
Logical/Physical Data Structures in
VSDB
4
PHYSICAL STORAGE MEDIUM
EEPROM Flash-EEPROM
Read (nsec)
Program
( µ sec/byte)
80 @ 5V
150 @ 3V
150
157 (64 B)
625 (16 B)
10 @ 5V
17 @ 3V
Erase (sec/block)
N.A.
0.45 @ 5V
0.5 @ 3V
Cell Size (µ
µm2 @ 0.4µ
µm
technology)
4.2
2
Cost per bit
Medium
Endurance (program,
erase operations)
10K to 100K
write cycle
/byte
Low
10K to 100K
erase cycle
/block
Logical/Physical Data Structures in
VSDB
5
3
PHYSICAL STORAGE MEDIUM:
THE SMART CARD EXAMPLE
ROM
RAM
Flash-EEPROM
Time Base &
Watchdog
CPU
Random
Number
Generator
Mod Arithmetic
Processor
Control logic
Interrupt Controller
Port
Ck
Rst Gnd Vcc
Logical/Physical Data Structures in
VSDB
6
FLASH MEMORY ORGANIZATIONS
64 KB Main Block
16 KB Boot Block
8 KB parameter Block
64 KB Main Block
8 KB parameter Block
96 KB Main Block
64 KB Main Block
128 KB Main Block
128 KB Main Block
64 KB Main Block
Symmetrically blocked component
128 KB Main Block
Asymmetrically blocked component
Logical/Physical Data Structures in
VSDB
7
4
VSDB DESIGN METHODOLOGY
IDENTIFY
IDENTIFYINFORMATION
INFORMATIONAREAS
AREASAND
ANDDEFINE
DEFINEVIEWS
VIEWS
INTEGRATE VIEWS INTO THE CONCEPTUAL SCHEMA AND
INTEGRATE VIEWS INTO THE CONCEPTUAL SCHEMA AND
TRANSLATE THEM INTO THE LOGICAL DATA MODEL
TRANSLATE THEM INTO THE LOGICAL DATA MODEL
DEFINE
DEFINELOGICAL
LOGICALVIEWS
VIEWSAND
ANDALLOCATE
ALLOCATETHEM
THEM
INTO INFORMATION AREAS
INTO INFORMATION AREAS
ESTIMATE LOGICAL VIEW SIZE
ESTIMATE LOGICAL VIEW SIZE
ALLOCATE
ALLOCATEEACH
EACHLOGICAL
LOGICALVIEW
VIEW
“ON
“ONCARD”
CARD”OR
ORTO
TOAN
ANOUTER
OUTERSYSTEM
SYSTEM
FOR EACH FRAGMENT
ALLOCATED “ON CARD”
DEFINE
DEFINEPRIVACY
PRIVACYPROFILE
PROFILE
AND
ANDREFINE
REFINEVIEWS
VIEWS
ESTIMATE ACCESS TYPE
ESTIMATE ACCESS TYPE
AND
ANDDATA
DATAVOLATILITY
VOLATILITY
SELECT
SELECTTHE
THELOGICAL
LOGICAL
DATA
DATASTRUCTURE
STRUCTURE
DESIGN
DESIGNMEMORY
MEMORYPROTECTION
PROTECTION
MECHANISM
MECHANISM
SELECT
SELECTTHE
THEPHYSICAL
PHYSICAL
DATA
DATASTRUCTURE
STRUCTURE
CHOOSE
CHOOSEENCRYPTION
ENCRYPTION
ALGORITHMS
ALGORITHMS
Logical/Physical Data Structures in
VSDB
8
CASE STUDY:
PORTABLE INTERNET ACCESS DATABASE
url
description
web_site
t,o
id
category
web_login
1:n
1:1
has
history
web_access
bookmark
1:n
login
password
visited_on
amount
last_visited
date
mvmnt
time
description
1:n
id
web_purse
balance
Logical/Physical Data Structures in
VSDB
9
5
RELATIONAL DB MODEL
Logical/Physical Data Structures in 10
VSDB
LOGICAL/PHYSICAL DATA STRUCTURES
• Heap RELATION
LIMITED CARDINALITY (<= 10 records)
• Sorted RELATION
MEDIUM CARDINALITY (~100 - ~1000 records)
SORTED W.R.T. A FIELD
• Circular list RELATION
MEDIUM CARDINALITY
STORED AND MANAGED AS A CIRCULAR LIST
TYPICALLY SORTED BY date/time
• Generic RELATION
MULTI-INDEX STRUCTURE
Logical/Physical Data Structures in 11
VSDB
6
DB ANNOTATION
Logical/Physical Data Structures in 12
VSDB
PHYSICAL DESIGN: DATA ORGANIZATION
CIRCULAR LIST
SORTED
DATA STRUCTURES
ARE ASSIGNED ON
THE BASIS OF THE
ANNOTATIONS ON
THE RELATIONS
HEAP
HEAP
SORTED
CIRCULAR
LIST
Logical/Physical Data Structures in 13
VSDB
7
ACCESS MODES: QUERY
• SCAN: FETCH ALL RECORDS IN THE TABLE
• SEARCH WITH EQUALITY SELECTION
FETCH ALL RECORDS THAT SATISFY AN
EQUALITY SELECTION
"find the VisitedURL for day = #05/01/2001#”
• SEARCH WITH RANGE SELECTION
FETCH ALL RECORDS
RANGE SELECTION
THAT
SATISFY
A
"find the movements with amount between €10
and €100"
Logical/Physical Data Structures in 14
VSDB
ACCESS MODES: UPDATE
• INSERT: REQUIRES RECORD SHIFTING WHEN
SORTED
FETCH BLOCK
INCLUDE THE NEW RECORD
WRITE THE BLOCK BACK
• DELETE: REQUIRES RECORD SHIFTING WHEN
SORTED
FETCH BLOCK
DELETE RECORD
WRITE THE BLOCK BACK
• MODIFY:
FETCH BLOCK
MODIFY RECORD
WRITE THE BLOCK BACK
Logical/Physical Data Structures in 15
VSDB
8
PHYSICAL DESIGN: BLOCK ALLOCATION
GOAL:
• REDUCE THE NUMBER OF ERASE/WRITE
OPERATIONS
ACHIEVE GOOD PERFORMANCE
ACHIEVE LOW POWER CONSUMPTION
PRESERVE THE LIFE OF THE FLASH DEVICE
SOLUTIONS:
• ADD A validity_bit PER RECORD
• ADD A deleted_bit PER RECORD
• INSERT DUMMY RECORDS IN BLOCKS
• PERFORM GARBAGE COLLECTION DURING
ERASE/PROGRAM CYCLES
Logical/Physical Data Structures in 16
VSDB
PHYSICAL DESIGN: BLOCK ALLOCATION
CIRCULAR LIST
Sliding window
ERASE
Validity Bit
Deleted Bit
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
(a)
(b)
●●
●●
●●
●●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
(c)
●
●●
●
●
●
●
●
●
●
(d)
Logical/Physical Data Structures in 17
VSDB
9
PHYSICAL DESIGN: BLOCK ALLOCATION
SORTED DATA STRUCTURE
• HIGH PROBABILITY OF CASCADED BLOCK ERASURE /
REWRITE
• LEAVE SOME EMPTY (DUMMY) RECORDS IN EACH
BLOCK
• ADJACENT (EFFICIENT SEARCH)
• DISTRIBUTED (EFFICIENT UPDATE)
Logical/Physical Data Structures in 18
VSDB
PHYSICAL DESIGN: BLOCK ALLOCATION
SORTED DATA STRUCTURE
• CONCENTRATED
LOWER SEARCH TIME
• DISTRIBUTED
FEWER INSERTION CONFLICTS
Block 1
n
dummy
used records
d
empty records
n
Block 2
new
dummy
barber
barber
barber
d
new
new
host
mom
mom
new
Logical/Physical Data Structures in 19
VSDB
10
COMPLEXITY AND
POWER CONSUMPTION ANALYSIS
DIFFERENT MEMORY ALLOCATION SOLUTIONS
Adj. Dummy / No Dummy
Distributed Dummy
Adj. Dummy / No Dummy
Distributed Dummy
Logical/Physical Data Structures in 20
VSDB
EXPERIMENTAL RESULTS
Data
Block Erasures
Transmitted Bits on Bus
structure Strategy 10%-30% 40%-60% 70%-90% 10%-30% 40%-60% 70%-90%
Heap
Simple
1
Delete bit
0
Simple
1
Sorted Delete bit 0.83
Dummy
conc.
0.83
Dummy
dist.
0.10
Circular Simple
1
List
Delete bit
0
1
0.38
1
0.68
1
0.98
1
0.79
1
0.38
1
0.74
1
0.54
1
0.71
1
1.00
1
0.77
0.51
0.44
0.74
0.57
0.45
0.12
1
0
0.24
1
0.05
0.03
1
0.07
0.06
1
0.07
0.22
1
0.15
occurred block erasures and transmitted bytes on the system bus
w.r.t. “the naive”, no deleted bit, no dummy records solution
Logical/Physical Data Structures in 21
VSDB
11
WORK IN PROGRESS ...
• ACCURATE LOW POWER ANALYSIS
• USE OF “ADAPTIVE” DATA STRUCTURES
• DETAILED DEFINITION OF THE
METHODOLOGY
• DESIGN OF TOOLS SUPPORTING THE
AUTOMATIC AND THE SEMIAUTOMATIC
STEPS OF THE METHODOLOGY
Logical/Physical Data Structures in 22
VSDB
BIBLIOGRAPHY
•
•
•
•
C. Bobineau, L. Bouganim, P. Pucheral, P. Valduriez, "PicoDBMS:
Scaling down Database Techniques for Smart card," Proc. 26th Int.
Conf. on Very Large Databases (VLDB), 2000, pp.11-20.
C. Bolchini, F. Salice, F. Schreiber, L. Tanca, "Logical and Physical
Design issues for Smart Card Databases", Technical Report no.
2001.68, Politecnico di Milano, to be published in ACM-TOIS.
C. Bolchini, F. Salice, F. Schreiber, L. Tanca, "Logical and Physical
Design for Very Small Databases", Proc. SEBD’02, Portoferraio, june
2002
W. Rankl, W. Ewffing, Smart Card Handbook – Second Ed., Wiley,
1999
Logical/Physical Data Structures in 23
VSDB
12
Personal Portable Folder
Information System
CIVIL AVIATION AUTHORITY
TRAFFIC CONTROL AUTHORITY
DRIVING
LICENSE
owns
number
MUNICIPAL REGISTRY
birth
fam. name place birth
SSN first name date
death
citizenship
date
domicile
PERSON
death
residence
place
hairs
degree height
profession eyes colour
possesses
colour
ELECTORAL
CERTIFICATE
code
date
state
birth
place
birth
date
date
MEDICAL
date
start
date
state
time min. Max.
BLOOD
PRESSURE
code
MEDICAL
RECORD
(LOCAL)
fam. name first name
type
date of
choice
FAMILY
DOCTOR
date
issues
manages
state
blood
group
code
suffering from
result
JUDICIAL
REGISTER authority
date
date start
date end
has
license
action
JUDICIAL AUTHORITY
updates
outcome
type
action
date
end
subjected
to
fam. name
first name
takes out
allergiesorgan donation
company
POLICY
number
premium
HEALTH
AUTHORITY
EXAM
SSN
chronic rate
diseases
MEDICAL
RECORD
PRESCRIPTION
name dose
DRIVING/FLYING LICENSES
has
VACCINATION
code
undergoes
date of
issue
SUSPENSION
visit
type
dependent
on
LICENSE
limitations
place of
issue
has
record
signature civil photo
status
type
number
owns
state
VACCINATION
holds civil
rights
DRUG
RENEWAL
SSN fam. name first name
citizenship
OWNER
(PERSON)
residence
has
spouse
place
date
agency
vaccinated
agency
IDENTIFICATION
has
son
LIFE
FAMILY PUBLIC
LIABILITY
CAR PUBLIC
LIABILITY
takes out
POLICY
INSURANCE COMPANY (1 … N)
number
HOSPITALIZATION
INSURANCE COMPANY 1
facility date cause
on smart-card
on external media
virtual
relationship
INSURANCE COMPANY N
Logical/Physical Data Structures in 24
VSDB
13