Portlets and Web Services for
Collaboration and Videoconferencing
NESC 16 July 2003
PTLIU Laboratory for Community Grids
Geoffrey Fox, Marlon Pierce, Wenjun Wu
Ahmet Uyar, Hasan Bulut
Indiana University, Bloomington IN 47404
gcf@indiana.edu
2016/5/31
1
Abstract




We define interoperable framework called XGSP (XML
based General Session Protocol) based on Web services
technology for creating and controlling sessions
We developed a common dynamic messaging environment
(NaradaBrokering) for Grid and collaboration applications
Based on the web-services framework and
NaradaBrokering messaging environment, we are
developing Global Multimedia Collaboration System
(Global-MMCS) with Beihang University China
We use MVC (Model View Control) paradigm to develop
Web Service model/data – Messaging Control -- Jetspeed
style aggregation – portlet style view
2016/5/31
2
Collaboration and Web Services

Collaboration has
a) Mechanism to set up members (people, devices) of a
“collaborative sessions”
b) Shared generic tools such as text chat, white boards, audiovideo conferencing
c) Shared applications such as Web Pages, PowerPoint,
Visualization, maps, (medical) instruments ….

b) and c) are “just shared objects” where objects
could be Web Services but rarely are at moment
•

We can port objects to Web Services and build a general
approach for making Web services collaborative
a) is a “Service” which is set up in many different
ways (H323 SIP JXTA are standards supported by
multiple implementations) – we should make it a WS
2016/5/31
3
Shared Event Collaboration

All collaboration is about sharing events defining state changes
• Audio/Video conferencing shares events specifying in compressed form
audio or video
• Shared display shares events corresponding to change in pixels of a frame
buffer
• Instant Messengers share updates to text message streams
• Microsoft events for shared PowerPoint (file replicated between clients) as
in Access Grid


Finite State Change NOT Finite State Machine architecture
Using Web services allows one to expose updates of all kinds as
messages
• “Event service” for collaboration is similar to Grid notification service
and we effectively define SDE’s (service data elements) in OGSI

Group (Session) communication service is needed for the
delivery of the update events
• Using Event Messaging middleware makes messaging universal
2016/5/31
4
Portals and Web Services




Web Services allow us to build a component model for resources.
Each resource naturally has a user interface (which might be
customized for user)
Web Service <--> Portlet
Natural to use a component model for portal building displayed
web page from collection of portlets
• So can customize each portlet and customize which portlets you want
• Need to redirect portlets to cell-phones etc.



What is form of MVC (Model-View-Controller) paradigm
What happens if “real applications” like PowerPoint are
formulated as Web Services
Jetspeed uses HTML to build a portal – as well as Java, SVG
(Scalable Vector Graphics) is interesting
2016/5/31
5
XGSP MCU Architecture
Use Multiple Media servers to scale to many codecs and many
versions of audio/video mixing
Session Server
XGSP-based Control
NB Scales as
distributed
Admire
Web
Services
NaradaBrokering
All Messaging
SIP
H323
Media Servers
Filters
High Performance (RTP)
and XML/SOAP and ..
Access Grid
Native XGSP
Gateways convert to uniform XGSP Messaging
2016/5/31
6
Global-MMCS 2.0 (1) XGSP MCU



We are building an open source protocol independent Web
Service “MCU” which will scale to an arbitrary number of users
and provide integrated thousands of simultaneous users
collaboration services.
We will deploy it globally and hope to test with later this year.
The function of A/V media server will be distributed using
NaradaBrokering architecture.
• Media Servers mix and convert A/V streams

Open XGSP MCU based on the following open source projects
•
•
•
•
2016/5/31
openh323 is basis of H323 Gateway
NIST SIP stack is basis of SIP Gateway
NaradaBrokering is open source messaging from Indiana
Java Media Framework basis of Media Servers
7
Shared Output Port Collaboration
Collaboration as a WS
Set up Session with XGSP
Web Service Message
Interceptor
F
I
WSDL
R
O
U
Application or
Content source
Web Service
Text Chat
Whiteboard
Multiple
masters
2016/5/31
Master
O
F
I
Event
(Message)
Service
WS
Viewer
WS
Display
WS
Viewer
WS
Display
Other
Participants
WS
Viewer
WS
Display
8
Shared Input Port (Replicated WS) Collaboration
Collaboration as a WS
Set up Session with XGSP
R
U
Web
F
Servic
I
I
e
O
O
F
WS
Viewer
WS
Display
Master
U
Web
F
Servic
I
I
e
O
O
F
Event
(Message)
Service
R
R
U
Web
F
Servic
I
I
e
O
O
F
2016/5/31
WS
Viewer
WS
Display
Other
Participants
WS
Viewer
WS
Display
9
XGSP Conference Control
Framework Components

User session management
• User session management supports user sign-in, user
create/terminate/join/leave/invite-into XGSP sessions.

Application Session Management
• XGSP application session management provides the services
to A/V and data application endpoints and communities,
controlling multipoint A/V RTP and data channels.

Floor Control
• Floor control manages the access to shared collaboration
resources.
2016/5/31
10
XGSP Application Session Management

XGSP signaling protocol for
• H.323 signaling protocols (H.225, H.245)
• SIP signaling protocol ( Invite, Bye Message )
• Access Grid ( Unicast VIC & RAT )
Join XGSP Session, Leave XGSP Session,
Invite into XGSP Session, Expel from XGSP Session

Activate the XGSP session
• the XGSP session server will link all the “rooms” in the session
together by connecting multipoint A/V and data channels from
different communities to the XGSP A/V Media and Data Channel
Services.
Link/Disconnect XGSP SubSession
2016/5/31
11
XGSP Floor Control

XGSP should provide:
• Floor control primitives, including: request floor, release floor,
grant floor, cancel floor, remove floor request
• mediator-controlled floor control: to support the mediator
control policy
• Collaboration applications have to define their own roles in
the XGSP registration so that the mediator could assign the
role of the application to each user.
for example, a shared PowerPoint application should define
master/slave role.
2016/5/31
12
NaradaBrokering

Based on a network of cooperating broker nodes
• Cluster based architecture allows system to scale to arbitrary
size


Originally designed to provide uniform software
multicast to support real-time collaboration linked to
publish-subscribe for asynchronous systems.
Now has five major core functions
• Message transport (based on performance measurement) in
heterogeneous multi-link fashion
• General publish-subscribe including JMS & JXTA and
support for RTP-based audio/video conferencing
• Distributed XML data-base using P/S XPATH metaphor
• Filtering for heterogeneous clients
• Federation of multiple instances of Grid services as illustrated
by JXTA peer-group linkage
2016/5/31
13
vic and RealVideo views of multiple streams
2016/5/31
14
Polycom view of multiple video streams
2016/5/31
15
Performance Test : GlobalMMCS1.0

We conducted extensive performance tests on audio
and video servers.

Video:
• The test shows that our video server is capable of supporting 300
clients if there is only one video sender.
• Video Server Machine : 1.2GHz Intel Pentium III dual CPU,
1GB MEM, RedHat Linux 7.3
Audio:
• Our tests show that audio server can support 5 concurrent
sessions (250 participants in total) without any packet droppings.

• Audio Server Machine: 2.5GHz Pentium 4 CPU, 512MB
Windows XP machine

memory,
Scale with logarithmic Broker network
2016/5/31
16
Transit delay for message samples in NaradaBrokering
Different communication hops - Internal Machines
9
8
7
hop-2
hop-3
hop-5
hop-7
Sender/receiver/broker - (Pentium-3, 1
GHz, 256 MB RAM). 100 Mbps LAN.
JDK-1.3, Red Hat Linux 7.3
6
5
4
3
2
1
1000
2016/5/31
1500
2000
2500
3000
3500
Message Payload Size
(Bytes)
4000
4500
5000
17
Standard Deviation for message samples in NaradaBrokering
Different communication hops - Internal Machines
0.8
hop-2
hop-3
hop-5
hop-7
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1000
2016/5/31
1500
2000
2500
3000
3500
Message Payload Size
(Bytes)
4000
4500
5000
18
Comparison between the performance of
NaradaBrokering and JMF
Average delays/packet for 12 (of the 400 total) video-clients.
NaradaBrokering Avg=80.76 ms, JMF Avg=229.23 ms
450
NaradaBrokering-RTP
JMF-RTP
400
350
300
250
200
150
100
50
0
2016/5/31
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
19
Comparison between the performance of
NaradaBrokering and JMF
Average jitter/packet for 12 (of the 400 total) video clients.
NaradaBrokering Avg=13.38 ms, JMF Avg=15.55 ms
25
NaradaBrokering-RTP
JMF-RTP
20
15
10
5
0
2016/5/31
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
20
Global-MMCS 2.0 (2) Portlets


Collaboration clients will be built into portlets by
creating Java Applet or ActiveX controls for the nonHTML clients and adding them into HTML pages.
A collaboration portlet opens local services for XGSP
application session management and floor control.
• Node Manager portlet invoke the service to control local portlets


Apache Jetspeed seems good open source technology
supporting this model
Portlets such as Access Grid portlet (really a VIC portlet)
can be reused by Grid Portal Developers
2016/5/31
21
Unicast AG Portlet
2016/5/31
22
Multicast Multistream AG Portlet


2016/5/31
Java applet supports
multicast AG with
multiple streams
In Jetspeed, easiest to
have fixed size but this
doesn’t fit well natural
range of 1-20 separate
streams
23
Collaborative SVG Web Service

SVG is W3C 2D Vector Graphics standard and is interesting for
visualization and as a simple PowerPoint like application
• Further SVG is built on W3C DOM and one can generalize results
to all W3C DOM-based applications (“all” in future?)

Apache Batik SVG is Java and open source and so it is practical
to modify it to explore
• Real Applications as a Web Service
• Collaboration as a Web Service
• MVC model and web services with implications for portlets

We intend to use NaradaBrokering and XGSP to control
collaboration; support PDA Cell-phone and desktop clients;
restructure Batik as MVC Web Service
• Good progress in all areas see
• http://www.svgarena.org for SVG Games
• http://grids.ucs.indiana.edu/ptliupages/projects/carousel/ for PDA
2016/5/31
24
Collaborative SVG Chess
Game in Batik Browser
Players
5/31/2016
Observers
25
Data to
other clients
R
Master client
SVG browser 1
Session control
Server
Control to/from
all SVG browsers
in the collaborative
session
Control to/from
XGSP
I
Data from
master client
R
Other client
SVG browser 2
F
Control to/from
XGSP
I
Data from
master client
R
O
Other client
SVG browser n
F
Control to/from
XGSP
5/31/2016
O
•••
Event (Message) Service Infrastructure
XGSP
NaradaBrokering
F
I
O
Figure 3 Architecture of collaborative SVG browser on PC
26
Architecture of collaborative SVG browser on PC
Classic MVC Paradigm
Model View Controller
Model
Controller
Mouse event
Keyboard events
5/31/2016
Figure
View
Display
MVC Model
27
Model View Controller
Decomposition of SVG Browser
Semantic
Model
Events as
messages
Rendering as
messages
Controller
High Level UI
Input port
Output port
View
Events as
messages
Rendering as
messages
Raw UI
Display
Display
a. MVC Model
Messages contain control information
b. Three-stage pipeline
Figure 1 Reformulation
SVG to to
message
based MVC in based
a Web Service
Model in a
Reformulation
of ofSVG
message
MVC
5/31/2016
28
Web Service Model
XGSP
Architecture of
multiplayer game
with SVG
Control to/from
Session control
Server
Rendering to
SVG display 2
SVG WS 2
SVG WS n
Control to/from
Internet
Game
SVG display 1
Rendering from
SVG WS 2
SVG display 2
Control to/from
SVG display 2
•••
•••
Control to/from XGSP,
SVG display 2
NaradaBrokering
SVG WS 1
Event (Message) Service Infrastructure
SVG WS1,2, …, n
SVG display n
SVG WS1,2, …, n
5/31/2016
Web
Services
Messaging
Displays
Figure 4 Architecture of collaborative Web Services drawn for
29
Collaborative
SVG As
Aclients
Web Service
Control flow for collaborative
SVG
From
Collaboration
As a WS
Application as a Web service
Application as a Web service
Events
Rendering
From Master
W3C DOM Events
User Interface
Participating Client
From
Collaboration
As a WS
Application as a Web service
Application as a Web service
Events
Rendering
To Collaborative Clients
W3C DOM Events
User Interface
5/31/2016
Master
Client
30
Shared SVG Browser on PC
Shared SVG Browser on PDA
Event (Message) Service
Collaborative
Web Service
Events and
messages
Event (Message) Service
Collaborative
Web Service
Events and
messages
Internet Game
Event (Message) Service
Collaborative
Web Service
RFIO
RFIO
Web Service
Web Service
RFIO
Semantic
Semantic
High Level UI
SVG Browser
Input port
Events as
messages
UFIO
Input port
UFIO
Semantic
High Level UI
Raw UI
Display
Output port
Rendering as
messages
Events as
messages
Messages contain control information
Output port
Rendering as
messages
High Level UI
Raw UI
Display
Raw UI
Display
Messages contain control information
Events and
messages
Messages contain control information
a. Non-decomposed collaborative SVG
b. Decomposed WS optimized for thin
c. Decomposed WS optimized for
requiring minimal changes to the original
source code
clients
performance
Figure 2 Three among the different ways of decomposing SVG between client and Web Service component
Three among the different ways of decomposing
5/31/2016between client and Web Service component
31
SVG
Real-time Community Collaboration
5/31/2016
32
Scalable Vector Graphics (SVG) via “Shared Web Service”