DMJ
Distributed Media Journaling
http://www.ifi.uio.no/~dmj
• !#"%$'&( )* !#+,-.) ( -!%/ $"0+%( !0(1$%/.* !0 $'( !%/32 / 4 "0* ! ( !05 , +6* -7$%( +.+,
/ 4 ( * 8)( !%/.2 ! ( 9+6* :!%/5-) ( -7!%/ $4 !4 4 +4
• !#2 $')( +%-;$%( "05"0* !'$%( +.+, /! <6!%/$%/3$+'( $%( !%/3*%!%"0+*0/3+,)4 !%*0
/!0, !%/.! =6!0( 41+6"0" )*%* >? 3( @!#4 !%4 4 +
Database
’”“•.–—˜7™”š—›“œ •ž Ÿ
1999 - 2002
’”“•.–—˜7™¢¡ž•ž£ž¤'Ÿ
Address and devise solutions for an extensible framework
for on-line content analysis, indexing and annotation of
(live) networked multimedia sessions.
¥ —7¦F—;£›“˜F§¨˜§F£ž¤ ¤ —›©F¡ª—7¦¢Ÿ
Network
On-line content analysis of networked multimedia sessions
under uncertainty (evidence of content missed or hallucinated).
Event
Sensor
Content analysis
ALARM
Retrieval
«¬šªšž“•ž£;˜›§­Ÿ
Control
A 'BC 'B 6
DE1 'F
6 . G%H IJ K6L0M N KOQP R6S T N U N T L0V N H KW
XQY6P V J L0T VT H KV S K6VZ'IS J [%\
]QS KS J N TR'M L KO0S KS J L0V H'J W
^J L KP M L0V S PI6P S JT H K6V S KVZ%I6S J [#N KV HQL
‡  ‚ x ƒ y u † ~ | ƒ ˆ
T H KU N O0I6J L0V N H K_H%U6`S a'N LQRJ H0T S P P N K6OQU I6KT V N H K6P
‹ yƒ x
L0T T H%J aN K6O1V HQL1bN S J L%J T b6N T L0M%R6M L K6\
'
c
V
J
0
L
V
S O [#O S KS J L0V H%J W
ˆ ‹ y v ƒ  y u  v u y vˆ  „ 
y
ƒ
y

|
†

~
u
ƒ
Ž  ˆ y ƒ y  u v  
c'S M S T V PLQ`S a'N LQRJ H0T S P P N K6OdP V J L0V S O [dU J H0`eV bS
6
b
N S J L%J T bN T L0M'RM L K\0^J L%a'S PH0U UZ%IL0M N V L0V N f%SL K6a
‘6y u x ƒ  u ‚ ˆ  y †  y Œ u † v   „
Z%I6L K6V N V L0V N f%SJ S Z%I6N J S `_S KV PH0UV b6SI6P S JL%O L0N KP V
z
Network
L
%
f
L0N M L0Y6M SJ S P H I6J T S P \
t'u ˆ Šy ƒ ‡ v tu ƒ | xu v ~  „ | x w | ‚x v y { z y v u'{ u v u
gQKH hQM S aO S i Y6L%P S1j g_kl W
c'S VH0Ua%H0`L0N KRJ H0U N M S PiN KU H%J `L0V N H K?L0YH I6V
 w … y † v z y { | u6} ~  €6x tu v u w u x y
T H0`Y'N KN KO1M H hQi M S f%S MZ%IL KV N V L0V N f%S1L K6L0M ['P N P
N K6V H1b6N O b6S J i M S f'S M%T H K6V S K6VJ S T H%O K6N V N H K\
‹ yƒ x
‡  † ‚   ƒ ƒ v u  ~ | ~ƒ ˆ
mL0T V H%J [W
‡ | y ‚ € yƒ  u ~
n K6P V L0KV N L0V S PR6J H0T S P P N KOQP V J L0V S O [dL'PLdT H0M M S T V N H K
‰
tx „ Šx v ‡y
H0UN K6V S J L0T V N KOdP H0U V hQL'J ST H0`?RH K6S KV PN KV bSo1pq
z
oQN P V J N Y%IV S a_L KL0M ['P N PH0Y r S T V W
n K6P V L0KV N L0V S a#R6J H0T S P P N K6O#P V J L0V S O [%\
G%H IJ K6L0M N KO1T H%KV J H%M W
s1IK6i V N `SS f%S K6VJ S R'H%J V N K6O#L Ka#L%a%L%RV L0V N H K6\
øù ú í û ñ üò óì ô ê õ ð ò ý ù û
ö ÷ óô
öò ö
éê ë ñ ò ý óë ì0ô õ í ò î ë ï ð ê
÷ñö
õ ô
÷ö
÷ ñ ö õ ô ÷ ñ ö õ ô
ù ù ð ñ í ý ëõ ù ô ü ð þ0í î ë ï ð êÿ í ê ð ê û0î ê
÷ ö õòó
÷ ô õ ô ñ ô
÷ö õò ó
øî ù ú ý í ûý íüí ì0î ê ÷ ëð í ý ù ô ù õû ûô ñ ô
öò ö
ç_N S J L%J T b6N T L%M%R'M L K_L%PL KH0Y r S T V i H%J N S KV S KadkL [%S P N L K?KS V hQH%J èW
kL [%S P N L K_K6S V hQH%J è_H0Y r S T V
$1IV R'I6Vf%L%J N L0Y6M S
÷ñö
õ ô
$1IV R'I6V6V HdN K6R'I6VJ S M L0V N H K
ñ ò ó6ô õ ò n K6R%IV'fL%J N L0Y6M S
#7L0RRN KO1H0Uf%L%J N L0Y6M S
pj mL0M P SRH%P N V N f'S P l
pj mL0M P SK6S O L%V N f'S P l
pj %QH%aSf%N P N V S a%l
0| † v ‚  y%{ | } } y  y ƒ † yv  y x  ~ {
ƒ
„yx
tƒ  v  y    v z  ‰ | ƒ ˆ y  x  ƒ
! ~  "
 | x  v  |ˆ ƒ  ˆu z  y † ~x u  x ƒ x | } | y {u x
z ‰
ƒ
„yx
t6'ƒ  v  y    v z  ‰ | ƒ ˆ6 y  x  ƒ
‘ y    v z  ‰ | ƒ ˆ6 y  x  ƒ
#7S a'N LQR6J H0T S P P N KOQP V J L0V S O [dL%PL
T M L%P P N U N T L0V N H KDj aS T N P N H K'lV J S S
0ô õ
0ô õ
* @
?
@
Â Ç ¼ ½À
>
?
?
S5T U V W"X
metadata
Y[Z 4 Z=\=Z1] 0
?
@
?
@
( )
÷ ö õ õò ó
>
@
Â Ç ¼ ½À
* ÷ñ ö
õ ô
÷ ô õ ô ñ ô
¹;ºQ»d¼ ½»d¼d¾?¿1½À ÁD »Ã
based on conceptual information through the
use of concept templates which define possible concept relations
and how concepts can be decomposed into media processing
algorithms.
¹;ºQ»dĽ żd¼ ½Æ3Å;Ç Èd¼ ½É
(Bayesian network objects) construction/
customization on the fly through instance-based learning.
Automatic
construction of resource- and reliability-aware media
processing strategies based on classification (decision) trees.
¹;ºQ»d¼ ½Ðd¼ Ë ÑÀ½½›Èd¼ ¼ À Íd¿1¼ ½7Ã_ÀÈQÆeƞÈ#À'É
combined with OOBNs to
achieve a more expressive framework.
Ê ½Ç  ÈdÍ#Â Ç Â ¼ Á?ËÈ#»QÌ¢À½Éºd¿QÀ6Ä½Ë ÈQÎÈdÀ½›Äº1»d¼ ½»d¼#ÈQ»Q»#ºQ¼ Èd¼  º1»
achieved
through dynamic cooperation with a resource manager.
®¯•ª˜7™•¢“£F¤;šª“•–—F˜™e•›°ªÒӜ0Ô3™•¢“eÕÓÖD×ؕ­¤ Úٔœ ž—eŸ
The focus of this thesis is system support for distributed
analysis of distributed media. The following sub areas
will be addressed:
ÛFÜ ½»#¼#ÍQÀ6ºdݽÀ;ÈQÀÄÞ? ¼ ½Ä¼ ¿1À½Ï
>
Â Ç ¼ ½À
mediastream
Mediastream (from camera 1)
Distributable architecture based on event broker service
Partitioning, composition and
deployment of processing units for analysis and filtering.
Ê ½Éºd¿1À6ĽFÆeÈQ»QÈýƢ½»d¼ Ï
Reliability and latency adaptable to
resources currently available. Improve resource usage by
load balancing processing between different nodes.
mediastream
?
-/.1032541687:9<;=037
@
^ ÷ ÷ ôñ õ ö ô õ ô ñ ô
÷ ö õ õò ó
( )
This thesis will focus on how object-oriented Bayesian networks
(OOBNs), an effective hierarchical and modular framework for
knowledge representation and reasoning under uncertainty, can
support generic hierarchical representation of multimedia content,
representation and use of domain knowledge, and annotation
under uncertainty. The following sub topics will be examined:
ß ½ Å;Ç ºÁ#Æ¢½»d¼_º ÑÅ7ÀºQĽÉÉ »Ã_Ï
ACB"D EF G'H:I JKLEMI B:N/H"O JI
I JPMH:F I JNQJ D:R O
Mediastream (from camera 2)
÷
®¯•ª˜7™•¢“£F¤;šª“•–—F˜™e•›°­±¬¤ —F²6³´§ª“œ ¦;™•F°°—ž“¢µ¶“£F©ž·¸•”Ÿ
An event broker is the glue
which connects the DMJ components, as illustrated in
figure “Distributed analysis object”. Important issues
are scalability and performance.
' 'F æ0 ?'&0
B'
+ , Combination of probabilistic, knowledge-based content
analysis and QoS/resource awareness, packaged into a
generic extensible framework.
Ê ½Éºd¿1À6ĽËÈQ»dÌ¢À½Ç  ÈdÍ#Â Ç Â ¼ Á?Ë ÈQÎÈdÀ½FÆ¢½Ì_ È.Å7ÀºQĽÉ6É »Ã_Ï
ä; BC B6å ´Få'B6
' 0 7 'æ
éê ëñ ò ì óí î ô ë õ ï ò ð ê
ö ÷ óô ÷ ñ ö õ ô
Computational complexity of feature extraction and object
recognition and massive amount of data to be analyzed under
real-time requirements.
PD: primitive event detector
CD: composite event detector
P: event producer
C: event consumer
઺á­ÈdÎÈQÀ½Qâº#¿À»#ÈdÇ Â »Ã_Ï
The quality of the journal may
depend on events detected by the analysis system. For
example, the video sequences containing moving persons
may be stored in high quality video while other sequences
are stored in low quality.
¥ —7¦ªãª¤ ™¦¢Ÿ
2 papers submitted to conferences
6 presentations/invited talks
First prototype under development (office journaling)