Project details
Title of the project:
Development of a multimedia application
Student name:
Carlos García Porcel
Degree:
Enginyeria Tècnica en Informàtica de Sistemes
Credits:
22,5
Director/Speaker:
Carlos Villavieja Prados
Department:
AC
Board members (name and signature)
President:
Jose Navarro Mas
Member:
Josep Anton Sánchez Espiguares
Secretary:
Xavier Martorell Bofill
Qualification
Numeric qualification:
Descriptive qualification:
Date:
Development of a multimedia application
2
Development of a multimedia application
Abstract
This project will consist of developing a software application focused on
joining video conference and video reproduction. The diversity of target users
is as big as its possibilities since the final application is intended to be a common
basis or a prototype of these kinds of services.
For this reason, the main objectives will be reproducing video streams (no
matter what the source is) via the Internet and the chance of watching several
streams simultaneously.
3
Development of a multimedia application
Summary
Figure 1-1 Perla Logo
Perla is a project that involves the development of a multimedia
application, which permits the local video files, web cameras and remote streams
in an attractive interface, designed following the guidelines of several software for
the GNU/Linux platform, actually to an Ubuntu distribution. It also contains a server
application that will be used as a client indexer and video repository. It includes a
preliminary study of the available tools in order to develop good media support
which is user friendly.
An initial brainstorm has been carried out in order to determine which
features the client and the server application ought to have to offer a different
service from what is currently given.
As we had all the freedom to choose under
which requirements and conditions the tool was to be designed and created, we
came to the conclusion that Perla would be built under the multireproductor
premise, permitting the final user to play several streams simultaneously, so
the application would be hybrid between a classic reproduction program and a video
conference solution.
Then, a server was needed in order to serve as a common meeting point
between every Perla user. This application was in charge of storing a data base with
the clients' information as well as being used as a common media broadcasting
environment to let the users exchange streams.
4
Development of a multimedia application
In order to succeed on developing both applications, different technologies
and tools have been used:
 The C programming language, under the GNU/Linux environment.
 GTK+ library for the interface design.
 Gstreamer library as the core for the multimedia services.
 Glib library, in order to give some extra support to the C.
 Glade, the application that uses GTK+ to design the graphical user interface.
 Icecast as the stream publishing server
5
Development of a multimedia application
Index
2. Introduction ...................................................................... 12
2.1 Current situation .......................................................... 12
2.2 Introduction to the objectives ........................................ 13
3. Objectives ........................................................................ 15
3.1 Desktop application ...................................................... 15
3.2 Media reproduction software .......................................... 15
3.3 User: unaware of the current technologies ...................... 15
3.4 Possibility of watching streams from other users .............. 15
3.5 Supporting features ...................................................... 15
4. Analysis of requirements ................................................... 16
4.1 Functional requirements ............................................... 16
4.2 Non functional requirements .......................................... 19
4.2.1 Perla must be open source .................................. 19
4.2.2 Perla must be free ............................................. 19
4.2.3 Perla must be easy to use ................................... 19
4.2.4 Perla must be efficient ........................................ 19
5. State of the art .................................................................. 20
5.1 Target Markets ............................................................ 20
5.1.1 Entertainment ................................................... 20
5.1.2 Education ......................................................... 21
5.1.3 Video surveillance .............................................. 21
5.2 Reproduction and videoconferencing tools ....................... 22
5.2.1 VLC .................................................................. 22
5.2.2 Totem Media Player............................................ 23
5.2.3 Skype............................................................... 24
5.2.4 Cheese ............................................................. 24
6. APIs and applications ........................................................ 25
6.1 C, the programming language ....................................... 25
6.2 GTK+ (The Gimp Toolkit) .............................................. 26
6.3 Glib
........................................................................ 26
6.4 Glade ........................................................................ 27
6.5 Gstreamer ................................................................... 27
6
Development of a multimedia application
6.6 Icecast .......................................................................
7. Users’ guide ....................................................................... 29
7.1 Main Window ............................................................... 29
7.1.1 The tool bar ...................................................... 29
7.1.2 Central part ...................................................... 30
7.1.3 Right column ..................................................... 30
7.1.4 Lower bar ......................................................... 31
7.2 Audio ........................................................................ 32
7.2.1 On mode player ................................................. 32
7.2.2 On mode video conference .................................. 33
7.3 Controls ...................................................................... 34
7.3.1 Open/Eject ........................................................ 34
7.3.2 Play/Pause ........................................................ 34
7.3.3 Take screen shot ............................................... 35
7.3.4 Go/Leave full screen .......................................... 35
7.3.5 Navigation ........................................................ 36
7.4 Information ................................................................. 37
7.4.1 Connection status .............................................. 37
7.4.2 Source type ...................................................... 37
7.4.3 Source destination ............................................. 38
7.4.4 Time line .......................................................... 38
7.4.5 Current screen .................................................. 38
7.5 Connection .................................................................. 39
7.6 Opening a source ......................................................... 40
7.6.1 Local source ...................................................... 40
7.6.2 Remote source .................................................. 42
7.6.3 Special source ................................................... 43
7.7 About Window ............................................................. 44
8. Design
........................................................................ 45
9. Use case
........................................................................ 47
9.1 Use case: Client registration .......................................... 48
9.2 Use case: Client connection ........................................... 49
9.3 Use case: Client disconnection ....................................... 50
9.4 Use case: Client update ................................................ 51
9.5 Use case: Source publication ......................................... 52
7
28
Development of a multimedia application
9.6 Use case: Source conceal .............................................. 53
9.7 Use case: Source browse .............................................. 54
9.8 Use case: Client check .................................................. 55
9.9 Use case: Opening a web cam source ............................. 56
10. Implementation: Gstreamer ............................................ 57
10.1 Bins and pipelines ...................................................... 57
10.2 Pads and capabilities .................................................. 57
10.3 Threading .................................................................. 58
10.4 An overview of Gstreamer operation ............................. 59
11. Reference manual ............................................................ 65
11.1 Perla-gui ................................................................... 65
11.2 Perla-gst ................................................................... 68
11.3 Perla-client ................................................................ 74
11.4 Perla ........................................................................ 76
11.5 Perla-server ............................................................... 77
11.6 How to run Perla applications ....................................... 81
12. Economical study ............................................................. 84
13. Conclusions...................................................................... 87
13.1 Achieved objectives .................................................... 87
13.2 Future lines of work .................................................... 88
13.2.1 Critical ............................................................ 88
13.2.2 Desired ........................................................... 89
13.2.3 Optional .......................................................... 90
14. Bibliography .................................................................... 91
8
Development of a multimedia application
Index of figures
Figure 5-1 VLC Media Player logo ................................................ 22
Figure 5-1 Totem Media Player main window ................................ 23
Figure 5-3 Skype logo ............................................................... 24
Figure 5-4 Cheese logo .............................................................. 24
Figure 6-1 APIs and applications in Perla architecture .................... 25
Figure 6-2 GTK+ logo ................................................................ 26
Figure 6-3 Glade logo ................................................................ 27
Figure 6-4 Gstreamer logo ......................................................... 27
Figure 6-5 Icecast logo .............................................................. 28
Figure 7-1 Main window ............................................................. 29
Figure 7-2 Tool bar ................................................................... 29
Figure 7-3 Group of screens ....................................................... 30
Figure 7-4 Right column ............................................................ 30
Figure 7-5 lower bar ................................................................. 31
Figure 7-6 Different audio modes ................................................ 32
Figure 7-7 Audio enabled ........................................................... 33
Figure 7-8 Audio disabled .......................................................... 33
Figure 7-9 Open ....................................................................... 34
Figure 7-10 Eject ...................................................................... 34
Figure 7-11 Play ....................................................................... 34
Figure 7-12 Pause ..................................................................... 34
Figure 7-13 Take a screenshot ................................................... 35
Figure 7-14 go full screen .......................................................... 35
Figure 7-15 Leave full screen ..................................................... 35
Figure 7-16 View modes ............................................................ 35
Figure 7-17 Navigation arrows .................................................... 36
Figure 7-18 Connected .............................................................. 37
Figure 7-19 Disconnected .......................................................... 37
Figure 7-20 nothing is playing .................................................... 37
Figure 7-21 Video file playing ..................................................... 37
Figure 7-22 Web cam device playing ........................................... 37
Figure 7-23 Special stream playing ............................................. 37
Figure 7-24 Information window ................................................. 38
Figure 7-25 Private or remote .................................................... 38
9
Development of a multimedia application
Figure 7-26 Broadcasting ........................................................... 38
Figure 7-27 Time line ................................................................ 38
Figure 7-28 Current screen information ....................................... 38
Figure 7-29 Connection window .................................................. 39
Figure 7-30 Open/Cancel ........................................................... 40
Figure 7-31 Open local source .................................................... 40
Figure 7-32 Select source .......................................................... 41
Figure 7-33 Select web cam device ............................................. 41
Figure 7-34 Select preview + preview ......................................... 41
Figure 7-35 Broadcasting option ................................................. 41
Figure 7-36 Open remote source ................................................ 42
Figure 7-37 Search engine ......................................................... 42
Figure 7-38 Open special source ................................................. 43
Figure 7-39 YouTube logo .......................................................... 43
Figure 7-40 about window ......................................................... 44
Figure 9-1 Client/server architecture in Perla ............................... 47
Figure 10-1 Perla pipeline .......................................................... 59
Figure 11-1 Debian package ....................................................... 81
Figure 11-2 Perla-server running ................................................ 83
Figure 12-1 Percentatge of dedication in hours ............................. 85
10
Development of a multimedia application
PERLA
11
Development of a multimedia application
2. Introduction
2.1 Current Situation
Today new media technologies are very important to the evolution of society,
and they are used in many fields: research, education, entertainment, etc. Its
development has become a critical part of any study or job as they are the most
reliable tools to achieve good results and to satisfy the widest range of clients.
In order to simplify the usage of these technologies, industry has pursued the
synergy of its multimedia services. Little by little, mobile phones offer other
services apart from communication between people, adding a camera gadget,
Internet wireless connection or GPS, among others; the PDA is mixture of computer
and agenda; media centers have invaded the majority of living rooms fusing video
and music players.
12
Development of a multimedia application
2.2 Introduction to the objectives
An attractive and intuitive service is going to be created where the user is
able to watch and listen to his multimedia elements in a comfortable way.
Perla intends to be a basis for more specialized multimedia software, or
a simple application that merges the entire possible multimedia source into a
unique interface.
For that very reason, Perla needs be open to the local system as well as to the
Internet, where a huge amount of web cams, videoclips, movies or TV series are
stored.
In
addition,
it
may
give
multiple
visualization
of
several
streams
simultaneously, so a set of displays must be built, considering a rational quantity to
not to overload the system
Within the main interface, the user must have independent control of
every running screen, making possible some orders to be given: pause and
resume, control the volume – defining heights and let the user navigate between
them, and other special features, that are going to be defined during the
development of the project.
It is also desirable that it had a little information about the playing streams in
order to identify them and treat them providing what its source is.
Another important task is giving the user the chance to publish local
streams to the Internet, and retrieve other ones from other the rest of the
clients as easily as opening a local file. So it has to have a background environment
in order to share information among all the Perla users. So a server application is
necessary to be implemented in order to save the said information as well as serve
as a common access point, where the users (or clients) could publish its streams
and pick them up from other clients. All these operations had to be possible no
matter where the client was, so certain rules had to be applied.
A further explanation of the capabilities and requirements of the tool will be
found in the following chapters.
As a result, the following elements will be created:
 A Debian package containing the Perla Client application (or Perla)
 A .tar.gz with the Perla Server source code and the corresponding files to
compile it.
13
Development of a multimedia application
In general terms, the achievement of the entire project are performed
through:
 A preliminary study of the development tools
 A complete specification of the features that Perla must have
 Information about similar applications and its way of working
14
Development of a multimedia application
3. Objectives
The following chapter outlines the main objectives and premises for Perla
development:
3.1 Desktop application
 Under GNU/Linux, actually to Debian-based Ubuntu distribution
 The C programming language as basis
 Standard protocols
3.2 Media reproduction software
 Local video files
 Local web cams
 Remote streams
 Integration of existent platforms
 Chance to watch different streams simultaneously
3.3 User: unaware of the current technologies
 User friendly environment
 Easy and quick mechanisms of stream control
3.4 Possibility of watching streams from other users
 Availability under local networks
 Stream publication
 Stream tagging
3.5 Supporting features
 Different audio modes
 Hierarchy with reference to the visualization
 Screen shots
These ideas might be considered when a more detailed analysis has been
carried out
15
Development of a multimedia application
4. Analysis of requirements
4.1 Functional requirements
In the initial interface must be reflected, if not the whole set of services, the
majority of them (at least the essential ones to make a reproduction work). It will
display several screens, where the user will be able to watch the files or web cams
he uses.
The user will have at his disposal the connect/disconnect options. If
he wants to connect to a server in order to send and receive remote streams, an
input window will appear. At this stage, the user must enter his user name,
password and the name of the server that he wants to connect to. If it is his first
time, the user must tick the register option. After some exchanges between the
application and the server, the client will be notified if he has successfully
connected to the selected server or not.
Another option will be to open a new source. Once the user has selected
the screen where he wants to place the reproduction, an upcoming window will
show up. It ought to have different options: select whether he wants to play a
source from his local system or connecting to the network of the application or
reproduce it from a location outside the framework.
If the user wants to play from local, Perla will show several input options:
name, description, image which will represent the source, etc.
With reference to the source of the stream, it is necessary that Perla allows
him to choose between a video file and a web cam.
When the video option is selected, the user will be able to choose the desired
file through a graphical way. On the other hand, if he chooses to play from a web
cam, he will have the chance to select what device he wants to play.
Once all the desired optional attributes are set, as long as the user is
connected, there will be the option to publish his source to the connected server.
In contrast, if the user wants to play a remote source, he will need to be
previously connected. There will be a text input, where the user will write down the
name of the user whose published source he wants to play. When the search
finishes, the system will inform him if the searched client has any published stream,
and if it does, provide some general information of each found source. Then, the
user will select one of the sources, and accept.
16
Development of a multimedia application
Finally, if the user wants to play any supported multimedia file from outside
Perla framework (meaning the clients' publications and the local sources), he will
choose among different options, each one with different modes of obtaining the
source. The possible sources vary from a simple URL to more complex structures
and sites.
Once the user is playing some source/s, more options become available. Perla
will give the opportunity of retrieving some information about the playing
streams, as the previously completed information (name, description, representing
image, etc.) or the duration of the each one.
Moreover, a button with the proper indication will be responsible for taking
pictures of the current view, indicated in some visible part of the interface.
The user will be able to give more importance to a chosen stream than the
others, displaying it on a bigger screen, then go back and reallocate the stream on
its original screen.
With reference to audio, as we want to permit the user to use it as a player
or as a video conference application indistinctly, Perla will offer an easy protocol
to switch between audio modes. If the user is using it as a player, a volume
scrollbar will be shown. With this option, only one playing stream will be heard
(with the selected volume) while the others are mute. On the other hand, if he is
using it as a video conference application, several volume controls will appear,
making the unique volume scroll bar disappear. So the user will be able to mute or
“unmute” the playing streams.
As the user will only be allowed, in video player mode, to hear a screen, this
screen will be pointed in some place on the interface. Therefore, and with the idea
of taking a screen shot or making full screen, Perla will have the element of the
current view or current screen. This screen will have the total control of every
option shown that is for only one screen (it excludes the volume control when it is
on video conference mode). When this screen is selected, it will be able to be
controlled with the previous options as well as with a play/pause button and an
eject button.
If the user presses the play/pause button, the state will switch between these
two modes; if the eject button is pressed, the stream on the current screen will be
stopped, its resources released and the options to this screen unset. This case will
17
Development of a multimedia application
occur when the stream ends, too (for instance, a remote stream is stopped or a
video file reaches the end).
Finally, in order to switch among screens, there will be some navigation
options: a pair of buttons to iterate them or a click with the mouse that will
change automatically the current screen to new pressed one.
18
Development of a multimedia application
4.2 Non functional requirements
4.2.1 Perla must be open source
This is very important, since if we want others to use our application as a
basis of another specialized application, we need to let different programmers study
the code and adapt/change it (combined with the following requirement).
4.2.2 Perla must be free
As we want to release our application under General Public License (from now
on GPL), every used external application, library or component must be released
under this license or any compatible one.
4.2.3 Perla must be easy to use
The target client of the application is not a computer science expert, not even
a frequent user of media programs. For this reason, if it is wanted that the
application is assimilated by the maximum number of people, it must have a clear
interface and an understandable set of options.
If the application is loaded with too many options it is possible that the user
feels uncomfortable and exceeded by the software itself. If the options are hardly
accessible, the final user will not use it or will waste too much time, breaking the
efficiency of the system.
4.2.4 Perla must be efficient
As it is dealing with multimedia and it usually requires watching the source in
time, we need to not to load the supporting system too much.
In addition, communication between entities must be as fast as possible, and
the errors or exceptions must be handled in a way that does not interrupt the usual
flow of the user, or at least that this can be taken up again when the unusual
behavior is fixed.
19
Development of a multimedia application
5. State of the art
The following chapter talks about different factors and tools which have
served as a reference to develop Perla.
First of all, I will briefly go over the markets where the application is meant to
be used, describing how I understand these sectors and why Perla could give a new
sort of service to them.
Finally, I will talk about some existing applications giving A solution in a
similar way to Perla, so it will be reflected how the project is guided towards these
tool proposals.
5.1 Target markets
5.1.1 Entertainment
In the recent past, there has been a dramatic increase of the entertainment
supply, and the channels to acquire it have totally been changed from the
traditional TV broadcasting to the video-on-demand practiced via the Internet.
People from different countries are able to watch new material with very few hours
of difference or even simultaneously.
So the audience demands new mechanisms to have complete access to these
sources, and the industry must look for new formulas to attract people to its
products and make profits. The demand is there and some changes from TV
management have been made, but a total revision of the rules of this game and its
strategies is necessary.
Perla already has a publishing service, where people can exchange their web
camera streams, but further versions are being implemented in order to permit the
broadcast of video files. With a specialization of the Perla framework, we can turn
the application into a new paradigm of entertainment: cinema at home.
With a proper protocol of identification and the possible integration of online
payment methods, people can watch the
are.
20
premiere of films no matter where they
Development of a multimedia application
5.1.2 Education
World-wide access to an adequate
education is one of the principles of the
Universal Declaration of Human Rights. Catastrophes, wars and lack of investment
in welfare restrict the number of people who have access to this basic right. In
order to solve these problems, there have been lots of initiatives, such as OLPC,
which has seen the creation of very cheap computers with free software addressed
to poor countries.
Perla could provide a software solution for people with access difficulties;
provide remote communication with teachers, pedagogy studies or private tuition.
If it is integrated with a sort of online library or a repository of didactic material, it
could be a good solution for these problems as well as a new way of giving classes.
5.1.3 Video surveillance
Insecurity is one of the major problems in the current society. Day by day,
there are more companies involved in personal security and surveillance. For
instance, a Closed Circuit Television (CCTV) network has been set up in London in
order to help the police detect and control crime.
But these kinds of services are not cheap, and the average citizen is unable to
pay for these services.
Perla could be a cheap (meaning totally free) application in order to control
several video cameras in only one interface. So no more than one screen will be
needed and a group of cameras could be watched non-stop, as others applications
do not allow switching between views, hence the sight of one of them is lost during
a certain period. It could be also specialized so a movement tracking will be created
or a stop-motion capture service (with its screen shot feature) could be possible.
21
Development of a multimedia application
5.2 Reproduction and videoconferencing
tools
5.2.1 VLC
Figure 5-1 VLC Media Player Logo [1]
Web site: http://www.videolan.org/vlc/
This cross-platform project is one of the most used applications IN
reproducing, thanks to its ability to cope with almost every media format without
the support of any codec else. As WinAmp, it has a configurable interface and it is
supported by the main operating systems. It also supports streaming protocols.
One of its most interesting features is libVLC, a library containing an API to
integrate VLC in other media programs1
1
There is a plausible option to change Gstreamer core of Perla to libVLC, although a preliminary study is
in progress
22
Development of a multimedia application
5.2.2 Totem Media Player
Figure 5-2 Totem Media Player main window [2]
Web site: http://www.gnome.org/projects/totem/
Totem Media Player (or Mplayer) is the official GNOME desktop media player.
It is capable to play video as well as audio and it has lots of interesting features,
such as subtitle integration, telestrator (drawing on the image as sports journalists
do) and a Mozilla plug-in, which allows this web browser to reproduce online videos.
Its backend is based on Gstreamer and it works based on the installed plugins the system has.
23
Development of a multimedia application
5.2.3 Skype
Figure 5-3 Skype Logo [3]
Web site: http://www.skype.com
Skype is proprietary software of instant messaging, videoconferencing and
call. It uses several standard protocols - voIP, P2P – to allow users communicate
with each other via chat, voice and web cam. It also provides a phone call service,
where you can use your computer as a telephone (even getting its own phone
number).
Recently, some mobile phones have been released with compatible features
with Skype and can be used as a good alternative to analogical phone calls.
5.2.4 Cheese
Figure 5-4 Cheese Logo [4]
Web site: http://live.gnome.org/Cheese
Cheese is a GNOME desktop application for camera recording. Among its
features we can find the possibility of taking pictures, recording video and applying
effects (such as vertical flip or noir/blanc).
It is based on the Photobooth application by Apple, and it uses Gstreamer as
its core framework.
24
Development of a multimedia application
6. APIs and applications
Figure 6-1 APIs and applications in Perla architecture
6.1 C, the programming language
C is a programming language developed in 1972 for use with the UNIX
operating system.
Nowadays, along with Python, it is one of the most used languages in order to
develop applications for the GNU/Linux platform because of its low-level profile,
which makes it very suitable in order to give access to the core of the system.
Despite its natural profile, the capabilities of C can be extended using standard
libraries, such as the ones which handle network sockets or I/O operations. It also
has system-dependent libraries; in order to take advantage of every system is
running on.
The major disadvantage of C is that it is not object-oriented, but it is possible
to simulate an OO environment encapsulating it in classes or using several created
classes, such as GObject and its derivations.
25
Development of a multimedia application
6.2 GTK+ (The GIMP Toolkit)
Figure 6-2 Gtk + Logo [5]
GTK consists on a set of cross-platform libraries (written in C), in order to
design
and
create
user
interfaces.
It
is widely
spread
across
GNU/Linux
environments, such as the GNOME desktop or OLPC.
It is based on certain elements, called widgets, representing from simple
components to the dialog windows or text render. Each widget is packed in its
parent, so they can be treated as a unity.
The present version of this library is GTK2+, that improves the accessibility
and further versions are going to be released including more features related to the
gnome desktop.
6.3 Glib
Glib firstly was a part of the GTK+ project, but derived into an independent
entity. It deals with portable implementations of the majority of standard libraries
and adds some new features, such as string types or safe thread handling. Its
major advantage is its C based API, which makes it totally adaptable and
understandable to C programmers, in addition to its similarities to Gstreamer and
GTK+ APIs.
26
Development of a multimedia application
6.4 Glade
Figure 6-3 Glade Logo [6]
Glade is a Gnome application for developing user interfaces with GTK+ on
run-time.
Because of its main feature, it is very recommended to develop user
interfaces using the said library, as it does not produce code but an XML file, that is
queried (using some additional libraries) so the GUI is built. In addition, its clear
interface and its binding with Devhelp, the debian helping tool to developers, makes
it a good candidate to let the new developer enter the interface design.
6.5 Gstreamer
Figure 6-4 Gstreamer Logo [7]
Gstreamer is an open, free multimedia-framework based on GObject. It is
written in the C programming language and designed to operate cross-platform.
Its core and strength lays (as well as GTK+) on separate elements called
plug-ins. Every plug-in implements a function, such as reading from a file or parse
from a video encoding to another, and several of them linked and added to a bin
make up a pipeline. These plug-ins are included in three different official categories,
depending on the reliability and stability of it.
Although Gstreamer framework is new (the first project was founded in 1999)
it is said to be included in the Linux Standard Base, which is the standard internal
structure of Linux.
27
Development of a multimedia application
6.6 Icecast
Figure 6-5 Icecast Logo
Icecast is a server project for streaming media. It supports the publication of
Vorbis audio and Theora video formats (free formats as well), and MP3 files.
The administration of the server and the streaming are controlled via HTTP, so
it can be installed behind any local subnet and it is very suitable to set up a radio
station.
Icecast was added to the project since it was very easy to use and it is
supported by Gstreamer framework.
28
Development of a multimedia application
7. Users' guide
7.1 Main window
Figure 7-1 Main Window
When Perla starts, the main window is shown. As it can be seen, we can
distinguish several components in the initial interface.
7.1.1 The tool bar
Figure 7-2 Toolbar
This part consists of three drop-down menus, where different options are
placed:
File
It contains the connection and disconnection options, in order to start
communication with a server and a quit menu item that will end the application
when pressed (the X button in the upper right side of the window).
Mode
Inside this menu the user will be able to switch between the player and the
video conference modes
29
Development of a multimedia application
Help
An about option is nested in this menu.
7.1.2 Central Part
Figure 7-3 Group of screens
In this element are placed the main screens used to display the streams
played. The user will be able to navigate through them (see 1.2 controls) and the
rest of the controls are set up to interact with the six screens.
7.1.3 Right column
Figure 7-4 Right column
On the right of the interface are situated several controls, such as the volume
scrollbar, the navigation arrows, the screen shot button and the full screen button.
30
Development of a multimedia application
7.1.4 Lower bar
Figure 7-5 Lower bar
The lower part of the application window displays the current screen
information as well as the play/pause and open/eject buttons.
It is important to say that some options will not be available unless the
selected screen (from now current screen) is loaded with a stream. When it is not
doing so, the option will be disabled and it will not be possible to use it.
31
Development of a multimedia application
7.2 Audio
Figure 7-6 Different audio modes
7.2.1 on mode player
If we choose only listening to one playing stream, we will be able to control
the volume level.
100%
The audio will reach the higher level
0% or muted
There will be no audio
Scrollbar
The user will have the option of regulating the desired volume level
32
Development of a multimedia application
7.2.2 on mode video conference
If the option mode->videoconference is active, a set of buttons will appear.
Each button controls a screen audio, so if a screen is not playing anything, its
corresponding button will be disabled.
Enabled
Figure 7-7 Audio enabled
It is heard at the level it has when it was on player mode (50% by default)
Disabled
Figure 7-8 Audio disabled
It will not be heard at all.
33
Development of a multimedia application
7.3 Controls
7.3.1 Open/Eject
This button is placed down the screens, on the left side, and it has the
following behavior:
Figure 7-9 Open
If the current screen does not have anything playing, this button will give the
option to open a stream.
Figure 7-10 Eject
On the other hand, if the current screen is occupied,
the stream will be
stopped and the screen released when the eject button is pressed.
7.3.2 Play/Pause
It is placed in the central part of the lower bar, and it has the usual behavior:
Figure 7-11 Play
If the current screen has been stopped, it can be taken up again in the
previous position.
Figure 7-12 Pause
If the current screen is running, it could be stopped.
34
Development of a multimedia application
7.3.3 Take screen shot
Figure 7-13 Take a Screenshot
When this button is pressed, the view of the current screen will be captured
and a picture will be created. If, for any reason, it cannot be taken, the user will be
informed with an error message.
7.3.4 Go/ leave full screen
Figure 7-14 Go full screen
If the user wants to give more importance to one stream than the others by
reproducing it on a bigger screen, he will have to press the full screen button. Then,
the six screens will be changed to five smaller ones, and a bigger one. The five
small screens, placed on the left side of the interface, will show the resting streams,
and the bigger screen will display the current screen.
Figure 7-15 Leave full screen
Once the user wants to back down and go to the usual view, he will only have
to press the leave full screen button.
Note: If the current stream ends or it is ejected, the application will come
back to the initial interface automatically.
Figure 7-16 View modes
35
Development of a multimedia application
7.3.5 Navigation
There are two different ways of changing the current screen.
Figure 7-17 Navigation arrows
The direction arrows will make the current screen iterate across the screens.
The other way to select the current screen is by clicking on it. But, if the
screen is free, this will cause the appearance of the open
36
Development of a multimedia application
7.4 Information
7.4.1 Connection status
This image will show whether we are connected to a server or not. It is very
useful to know if we can broadcast our sources or look for remote sources. It is not
necessary if we want to play special sources. This sign is placed on the lower left
corner of the application.
Figure 7-18 Connected
Figure 7-19 Disconnected
7.4.2 Source type
This button will indicate the kind of source we are playing on the current
screen, looking at the overprinted image.
Figure 7-20 Nothing is playing
Figure 7-21 Video file playing
Figure 7-22 Web cam device playing
Figure 7-23 Special stream playing
37
Development of a multimedia application
In addition, if we click on it, it will make a window appear, retrieving all the
information from the current screen.
Figure 7-24 Information window
7.4.3 Source destination
This image will be displayed if the current stream is published or not.
Figure 7-25 Private or remote
Figure 7-26 Broadcasting
7.4.4 Time line
These clocks show the current position of the time line and the duration of the
stream (or N/A if it is web cam).
Figure 7-27 Time line
7.4.5 Current screen
The number of the current screen and the name of the source it is being
played on it are displayed on the lower bar, in its central part.
Figure 7-28 Current screen information
38
Development of a multimedia application
7.5 Connection
Figure 7-29 Connection window
If the user attends File->Connect, a window like this will be shown.
The connecting process is the following:
 Fill in the form with the user username, password and the IP of the server
he wants to connect.
 If it is the first time he is connecting with the given username, he will have
to tick the “Are you registering?” option
 Once the “connect” button is pressed, a client / server exchange will start.
 If “cancel” is pressed, all the data will be cleared and the connection window
will disappear
The procedure ends in two different ways:
 The client has successfully connected to the server and the connection
status will change to connected
 The client is not able to connect to the server for several reasons. The
system will inform the user with an explanatory message (user/password incorrect,
the user already exists and another name must be chosen in order to register,
server unavailable, etc.)
In order to disconnect, it is only necessary to press File->Disconnect.
39
Development of a multimedia application
7.6 Opening a source
Figure 7-30 Open/Cancel
The common buttons of each page have the following behavior:
Open
All the opening process will be triggered. If something is incorrect, the system
will inform the user.
Cancel
This button cancels the entire operation.
There are three different main options in order to open a source.
7.6.1 Local source
Figure 7-31 Open local source
In order to open a local source, the following options will be handled:
 Give it a name and a description: These will identify the source to the user.
They are optional, and if they are left unfilled, the system will assign them a text by
default.
 Choose between opening a video file or the web cam device.
40
Development of a multimedia application
- If the user wants to play a video file he will have to browse its directory
system and look for a supported video file. Note: The supported file only depends
on the installed gstreamer plug-in.
Figure 7-32 Select source
 If the web cam option is chosen, Perla will let the user choose which device
he wants to connect to, and the type of driver it runs under (disabled in this
version)
Figure 7-33 Select web cam device
 Select a preview picture: A picture representing the source will be asked for.
As the name/description entries, it is totally optional, and it is left clear, Perla will
assign one picture by default.
Figure 7-34 Select preview + preview
 Tick the broadcasting option: As long as we are properly connected to a
server and (this version) the chosen source is a web cam, there will be the option
of publishing the source on the server, so other clients will be able to access and
play in their application.
Figure 7-35 Broadcasting option
It is possible that an error occurs, such as selecting a wrong file source or a
busy device. An error message will be reported to the user in these cases.
41
Development of a multimedia application
7.6.2 Remote source
Figure 7-36 Open remote source
If it is a stream from a known client the one which the user wants to play, he
must click this flap and follow these steps:
- Type the user name of the known client of the user connected server, and
press the search button.
Figure 7-37 Search engine
 The system will ask the server for the published sources by the written
client.
 If the user is disconnected or has not got any published stream, the system
will inform the user
 On the other hand, if there is a stream that can be selected, the server will
give the information to the application and it will show it to the client in a table.
 Then, the user will select from the table the source which he wants to play,
and he will press “open”.
42
Development of a multimedia application
7.6.3 Special source
Figure 7-38 Open special source
If the user wants to play from one of the special locations Perla offers, he will
have to follow different protocols for each one.
It is important to underline the warning this window displays. As it is the most
basic implementation of getting these kind of sources, it is possible that Perla is not
suitable if these required
sources are too big, especially when getting videos from
YouTube.
The Web
If a media file has its own URL, Perla can reproduce the said file. An instance
of a possible source available to be played by the application is:
www.lostzilla.net/multimedia/do_no_harm.wav
YouTube
Figure 7-39 YouTube logo
Perla has a protocol to play videos from the famous website.
 Access to the desired video via the web site:
http://es.youtube.com/watch?v=CMNry4PE93Y
 Write down the video id
CMNry4PE93Y
 Wait for the complete downloading of the video and play.
43
Development of a multimedia application
7.7 About window
Figure 7-40 About Window
This window is only informative. It contains the name and the version of the
application, and some legal features, as Credits or a copy of the GPL License.
44
Development of a multimedia application
8. Design
In the following section, I am listing what decisions I took in order to develop
Perla and why I decided on them.
It will be developed under GNU/Linux, actually under
Ubuntu distribution
As one of the requirements imposed in the previous analysis, Perla had to be
open source software, since it is desired to be specialized afterwards.
So, in order to be consistent with its philosophy, it must be running under an
open source environment, hence a Linux distribution was selected. The reason why
Ubuntu was the chosen one was because I was more used to debian based
distributions, it has a clean package system and it integrates GNOME desktop,
which it is defined by its elegance and ease.
C will be the programming language to use
The C is the programming language that I am more familiar with, so that fact
was an obvious reason why I chose it as the development API. Furthermore, it is a
very flexible language because almost all frameworks and libraries are based on C,
so it is very easy to find an element that implements a new feature, such as GTK+
and Gstreamer.
GTK+ will be the graphics library
The Gimp tool kit is deeply-rooted to Ubuntu and C; therefore it was a rational
decision, as well as the fact that it is licensed under LGPL and it has stable releases
which means that not a lot of significant changes would have to be made to
procedure.
Gstreamer will be the multimedia framework
After long research, I found out that Gstreamer was the best solution to
implement the multimedia back end of Perla, rather than trying to handle that sort
45
Development of a multimedia application
of data by myself (it would have entailed lots of extra work, even a new project
could have been worked out).
The main reasons I chose Gstreamer as the core library was because of its
access to documentation, the applications where it is used (Totem, Cheese, N800
media environment...) and its light-weight API, which I got used to in a reasonable
short time.
The network architecture will be based on a client /
server role
Initially, several P2P solutions were sized up in order to implement the
network component of Perla, but the lack of clear implementations in C about it and
a more accurate acknowledge of how servers and clients exchange information
were the final reasons why I opted for the current solution. Besides, Perla needed
to centralize its activities in order to have more control over the publishing streams
(in order to implement restrictive policies, for instance).
Perla will make the most of standard protocols and access
points
For a major acceptance of any application, it must exploit the major range of
standard protocols before implementing new ones. That is because the final user is
reluctant to adapt his system to unknown applications. That was why HTTP ports
(8000 and 8080) are used to communicate with the server, or GNOME desktop
related libraries are used, too.
If your application begins to depend on certain non-standard processes, the
user will eventually stop using it, as it will cause more problems than solutions.
Icecast will be the multimedia server to support Perla
server
Gstreamer implements a plug-in (shout2send) that is able to connect to an
Icecast server and Icecast is licensed under GPL, which makes it totally compatible
with Perla philosophy.
Moreover, it was very easy to set up and control and its usage of standard
protocols made Icecast a perfect candidate for the multimedia service provider.
46
Development of a multimedia application
9. Use case
As it was not a complicated but important part of Perla, I found it interesting
to explain how the server works when it comes to exchange information with the
client. In the following use cases, the client will act behind the system.
The last use case will handle the opening of a web cam source by the user.
Figure 9-1 Client/Server architecture in Perla
47
Development of a multimedia application
9.1 Use case: Client registration
Description
A client asks for registration
Sequence of events
Event
Response
1.- The server asks for user
name and pass word
2.- The system retrieves
user name and pass word
3.- The server checks the
availability of the user
name and sends an ID
number
4.- The system receive the
information
5.- The server closes the
connection line
Alternative course
Line 3: The user name is occupied. The server will send an error message to
the system
48
Development of a multimedia application
9.2 Use case: Client connection
Description
A client asks for connection
Sequence of events
Event
Response
1.- The server asks for user
name and pass word
2.- The system retrieves
user name and pass word
3.- The server checks the
matching
between
user
name and pass word, and
sends an ID number
4.- The system receive the
information
5.- The server closes the
connection line
Alternative course
Line 3: The user name does not exist. The server will send an error message
to the system
Line 3: The pass word does not match with the user name. The server will
send and error message to the system.
49
Development of a multimedia application
9.3 Use case: Client disconnection
Description
A client asks for disconnection
Sequence of events
Event
Response
1.- The server asks for ID
2.- The system
the ID
3.- The server flags the
user as disconnected and
frees
all
the
related
resources
50
retrieves
Development of a multimedia application
9.4 Use case: Client update
Description
A client updates its online status to the server
Sequence of events
Event
Response
1.- The server asks for an
ID
2.- The system
the ID
3.- The system
communication line
4.- The server updates the
timestamps of the client
related to the given ID
51
retrieves
closes
Development of a multimedia application
9.5 Use case: Source publication
Description
A client wants to broadcast a source
Sequence of events
Event
Response
1.- The server asks for ID
2.- The system
the ID
retrieves
3.- The server asks for
information
about
the
stream to publish
4.- The system retrieves
the requested information
5.- The server assigns an
available ID to this stream
and sends it to the system
6.The
server
communication line
52
closes
Development of a multimedia application
9.6 Use case: Source conceal
Description
A client wants to stop broadcasting a source
Sequence of events
Event
Response
1.- The server asks for ID
2.- The system
the ID
retrieves
4.- The system
the stream ID
retrieves
3.- The server asks for the
stream ID
5.- The server gets the ID
and frees all its resources
6.The
server
closes
communication with the
system
53
Development of a multimedia application
9.7 Use case: Source browse
Description
A client wants to know about other user published sources
Sequence of events
Event
Response
1.- The server asks for an
user name
2.- The system retrieves
the asked user name
3.- The server look for the
ID related to the user name
4.- The server collects all
the information about the
published
streams
and
sends it to the client
5.The
server
closes
communication with the
system.
Alternative course
Line 3: The user name is wrong. The server sends an error message to the
system
Line 3: The user is not connected. The server sends an error message to the
system
Line 4: There is no stream published by the said user. The server sends the
information to the system.
54
Development of a multimedia application
9.8 Use case: Client check
Description
Checking the online status of the registered users
Sequence of events
Event
Response
1.- The server asks for an
user timestamps
2.- The system retrieves
the timestamps
3.- The server compares
the given timestamps with
the current one
4.- The server loops the use
case from 1 to 3 until all
the users are checked
Alternative course
Line 3: The current timestamps and the given ones differ for a considerable
amount of time. The server, then, flags the user as disconnected.
55
Development of a multimedia application
9.9 Use case: Opening a web cam source
Description
A user is accessing to a web cam device in order to play it
Sequence of events
Event
Response
1.- The user selects a screen
2.- The
window
system
shows
up
a
3.- The system asks for name
and description for the source
4.- The user retrieve a name and
a description for the source
5.- The system asks for the
device source
6.- The user selects the device
source
7.- The system asks for a picture
to identify graphically the web
camera
8.- The user selects a picture
9.- The system asks whether the
stream is published
10.- The
system
user
answers
the
11.- The system
confirmation
asks
for
12.- The user confirms
13.- The system triggers the
opening operation and plays the
stream into the selected screen
Alternative course
Line 2: The screen is busy. There is not window to show.
Lines 4, 6, 8, 10, 12: The user cancels the use case
Lines 4, 8: The user skips the answer. A default attribute is set
Line 6: The device is busy. The system sends an error message to the user
56
Development of a multimedia application
10. Implementation:
Gstreamer
In the following lines, I will explain some theory about the selected
multimedia framework, Gstreamer, and then I will put forward the internal
operation of the component concerning the web cam reproduction.
10.1 Bins and pipelines
A bin is the main element of a Gstreamer application. It acts as a container of
other elements and performs an action (changing capabilities, image displaying,
threading...) inside it. So every plug-in is a bin with a certain function. Every
element, in order to be susceptible to be used, must be added to a bin.
A complete bin (meaning that has an independent deal with data) is called “a
pipeline”. Actually, a pipeline is a closed bin that can be played, paused or null.
10.2 Pads and capabilities
A pad can be defined as a gate of a bin. These pads are in charge of linking
bins and they pass data across them in order to mount the pipeline.
Elements can have two types of pads: source pads, where its belonging bins
place data, and sink pads, where bins retrieve data from. Obviously, each bin links
its pads with the adjoining one in a rational way, that is to say, source pads with
sink pads.
Another important function that pads do is filtering. They are responsible of
accept specific kinds of data. For instance, an element that controls the volume has
to deny access to video data, so its sink pad filters that sort of data in order to not
allow it to pass through the element. That supported kind of data that a bin
supports is called the capabilities of an element, and it can be explicitly filtered
when a pipeline implementation is done.
57
Development of a multimedia application
10.3 Threading
Gstreamer has the great advantage of creating and controlling the necessary
threads by its own, so lots of code lines can be avoided and makes developing
multimedia applications easier. But there are some situations where extra threading
is needed, such as when two elements need to synchronize outputs or need to
handle possibility of data loss.
There is a special element, the queue, which forces these actions. As it can
be seen below, lots of specific threading has been implemented.
Further information about Gstreamer core design, event handling or plug-in
categorization can be found at http://gstreamer.freedesktop.org/documentation/
and in the gst-inspect tool for the Linux terminal.
58
Development of a multimedia application
10.4 An overview of Gstreamer operation
An example: Web cam pipeline
Next I will explain how Gstreamer can be used to play a web cam device in
Perla context. The reason why I chose to write this process as an example was
because the video feature has not been improved to show all the Gstreamer
potential and because, in an effort to standardize the process, they share almost all
of the elements' usage.
Note:
It is possible that a certain element supports more sort of data
encoding than it is reflected in the following list, but I did not find relevant to tell
them all.
Figure 10-1 Perla Pipeline
1. Source element
2. Video sink
3. Audio sink
4. Video converter
5. Audio converter
6. Broadcasting sink
7. Photo sink
59
Development of a multimedia application
Source element
This element (actually a bin) contains the web cam and audio emitters.
Plug-in
Sink pad
capability
Source pad
capability
Description
v4lsrc
-
video/x-raw-yuv
It reads from a set video
device (/dev/video0 by
default).It has to be
considered that this element
only accepts devices whose
drivers are implemented with
Video4Linux.
alsasrc
-
audio/x-raw-int
It reads data from a set
audio device (from the ALSA
source attribute from the
system by default
60
Development of a multimedia application
Video sink
This bin contains three branches: one if them is used to display the media to
the local interface; the other one is treated to broadcast to an Icecast server and
the last one handle the screen shot feature (Element 7).
Plug-in
Sink pad
capability
Source pad
capability
Description
tee
ANY
ANY
The tee element function
consists on cloning the input
into several requested outputs.
This element is suitable to
treating the same data
different independent times.
ffmpegcolorspace
video/x-raw-yuv
video/x-raw-rgb
It converts data from a color
space to another
videoscale
video/x-raw-rgb
video/x-raw-rgb
It is possible that some
previous element filters data in
a specific capability. Apart
from data encoding,
capabilities also deal with video
size, bit rate or color depth.
Videoscale allows data to
change its width and height
ximagesink
video/x-raw-rgb
-
This element outputs video
data into an X Window. The
major drawback is that the
responsibility of displaying
video lays in CPU, not it the
graphics card, but it is
necessary in Perla
implementation, as almost all
graphics cards only supports
one output port.
61
Development of a multimedia application
Audio sink
This bin performs two functions: the first one outputs the audio source to the
loudspeakers and the second one transforms the raw audio into an application
interpretation in order to be broadcast.
Plug-in
Sink pad
capability
Source pad
capability
Description
volume
audio/x-rawint
audio/x-raw-int
As its name says, this
element is in charge of
volume control. As a
curiosity, this element is
dynamically changed by
means of a graphic scrollbar
alsasink
audio/x-rawint
-
It outputs the audio data to
the audio driver port (the PC
loudspeaker by default)
Video converter
This element represents the video conversion for broadcasting treatment.
Plug-in
Sink pad
capability
Source pad
capability
Description
theoraenc
video/x-raw-yuv
video/x-theora
It converts a raw video
data into an interpreted
theora video encoding. This
element is necessary
because its next element
(oggmux) expects video/xtheora data
62
Development of a multimedia application
Audio converter
This element represents the audio conversion for broadcasting treatment.
Plug-in
Sink pad
capability
Source pad
capability
Description
audioconvert
audio/x-raw-int
audio/x-raw-float
It makes audio/x-rawfloat data out if audio/xraw-int encoded data. It
is obligatory, as its
following element does
not accept audio/x-rawint
vorbisenc
audio/x-raw-float
audio/x-vorbis
As it was converted
before, audio data is now
converted to x-vorbis
data type. Along with
video/x-theora, are the
data to pass to the
oggmux
Broadcasting sink
This element is in charge of joining audio and video, create a unique file and
finally broadcast it.
Plug-in
oggmux
Sink pad
capability
audio/x-vorbis
Source pad
capability
application/ogg
It is a special element because
it does not have only one pad,
but two. One of them accepts
audio data and the other one
video. Then, inside the element
are assembled and it produces
ogg file, that is a free file
format to house media
-
Element
that
sends
the
incoming data to an Icecast
server
video/x-theora
shout2send
application/ogg
Description
63
Development of a multimedia application
Photo sink
This elements implements the screen shot feature
Plug-in
Sink pad
capability
Source pad
capability
Description
ffmpegcolorspace
video/x-raw-rgb
video/x-rawrgb
Even though it is the same element
as the previous one, in this stage of
Perla it has another job
(ffmpegcolorspace is a good
example of an element with multiple
capabilities). In this cases, it used
to determine a certain bit rate and
color depth for the screen shot
creation
fakesink
ANY
-
It actually accepts any capability, as
it is used as a dead end to data. It
is very often used to cancel some
sort of data, but in this case it is
used as a standard data receptor
that has connected a signal.
This signal will inform if there is
some data ready to be read
(consequently, this data will be the
screen shot image to save into a
file)
64
Development of a multimedia application
11. Reference Manual
11.1 perla-gui
Synopsis
Class to handle graphics in Perla environment
#include “perla-gui.h”
void
pgui_init_interface
();
GtkWidget*
load_widget
(int ngxml,
const char *str);
void
pgui_init_menubar
();
void
pgui_init_screens
();
void
pgui_init_buttons
();
void
pgui_init_remote_tree
();
void
pgui_select_screen_cb
(GtkWidget * widget,
gpointer data);
void
pgui_previous_screen_cb
(GtkWidget * widget,
gpointer data);
void
pgui_next_screen_cb
(GtkWidget * widget,
gpointer data);
void
pgui_fullscreen_cb
(GtkWidget * widget,
gpointer data);
void
pgui_show_info_cb
(GtkWidget * widget,
gpointer data);
void
pgui_eject_cb
(GtkWidget * widget,
gpointer data);
void
pgui_prepare_local_pipe
();
void
pgui_clear_tree
();
GtkTreeModel*
pgui_create_pipeline_model
();
void
pgui_add_to_model
(gchar * name,
gchar * description,
gchar type,
gchar * location);
65
Development of a multimedia application
gboolean
pgui_show_tree_row
(GtkTreeSelection*selection,
GtkTreeModel * model,
GtkTreePath * path,
gboolean
path_currently_selected,
gpointer userdata);
void
pgui_connect_cb
(GtkWidget * widget,
gpointer data);
void
pgui_disconnect_cb
(GtkWidget * widget,
gpointer data);
void
pgui_menu_change_mode_cb
(GtkWidget * widget,
gpointer data);
void
pgui_menu_help_about_cb
(GtkWidget * widget,
gpointer data);
void
pgui_addl_browse_preview_cb
(GtkWidget * widget,
gpointer data);
void
pgui_addl_browse_source_cb
(GtkWidget * widget,
gpointer data);
void
pgui_addl_webcam_cb
(GtkWidget * widget,
gpointer data);
void
pgui_addl_video_cb
(GtkWidget * widget,
gpointer data);
void
pgui_update_gui
(gint new_screen);
void
pgui_update_seek
(gchar * position,
gchar * duration);
void
pgui_set_control
(gboolean is_active);
void
pgui_set_info
(const gchar * icon_name);
void
pgui_set_play
();
void
pgui_set_pause
();
void
pgui_execute
();
void
pgui_executed
();
void
pgui_prepare_local_pipe
();
void
pgui_set_view
(const gint npage);
void
pgui_set_mode
(const gint mode);
void
pgui_set_confvolume
(gboolean is_active);
void
pgui_special_web_cb
(GtkWidget * widget,
gpointer data);
66
Development of a multimedia application
void
pgui_special_youtube_cb
(GtkWidget * widget,
gpointer data);
void
pgui_prepare_special_pipe
();
void
pgui_prepare_youtube_pipe
(gint screen_name,
const gchar * location);
void
pgui_prepare_web_pipe
(gint screen_name,
const gchar * location);
Description
perla-gui is in charge of all the functions related to loading, creation,
modification and querying about interface features.
It has an important function, load_widget (), that returns the GtkWidget
placed in the Glade file stored in the index and named after the second parameter.
Enumerations and structures
PifaceGUI
typedef struct
{
GladeXML *gxml[2];
GtkWidget *window;
} PIfaceGUI;
Structure used to store outstanding graphics related data.
GladeXML* gxml [2];
Element that stores .glade files
GtkWidget* window;
Main window of the application
PguiViewMode
typedef enum {
VIEW6
= 0,
FULLSCREEN
=1
}PguiViewMode;
These flags determine whether the interface is on mode full screen or not.
VIEW6
Normal mode, six size-alike screens are displayed
FULLSCREEN
Full screen mode.
67
Development of a multimedia application
11.2 perla-gst
Synopsis
Class in charge of multimedia access in the Perla environment
#include “perla-gst.h”
gboolean
pgst_expose_cb
(GtkWidget * widget,
GdkEventExpose * event,
gpointer data);
gboolean
pgst_bus_callback
(GstBus * bus,
GstMessage * message,
gpointer data);
gboolean
pgst_create_local_video
(gint pipe_num,
gboolean is_broadcasting);
gboolean
pgst_create_local_webcam
(gint pipe_num,
gboolean is_broadcasting);
gboolean
pgst_create_remote_pipeline
(gint pipe_num);
gboolean
pgst_create_special_pipeline
(gint pipe_num);
GstElement*
pgst_create_broadcasting_atee
(gint pipe_num);
GstElement*
pgst_create_broadcasting_vtee
(gint pipe_num);
GstElement*
pgst_create_broadcasting_audio
();
GstElement*
pgst_create_broadcasting_video
(gint pipe_num);
GstElement*
pgst_create_broadcasting_sink
(gint pipe_num);
GstElement*
pgst_create_photosink
(gint pipe_num);
GstElement*
pgst_create_webcam_source
(gint pipe_num);
GstElement*
pgst_create_audiosink
(gint pipe_num);
GstElement*
pgst_create_videosink
(gint pipe_num);
void
pgst_connect_bus
(GstElement * pipe,
gint pipe_num);
void
pgst_connect_expose_callback
(GstElement * vsink,
gint pipe_num);
gboolean
pgst_webcam_available
(gint pipe_num);
void
pgst_unmute
(PGstPipeline * pipe);
void
pgst_mute
(PGstPipeline * pipe);
void
pgst_volume_cb
(GtkWidget * widget,
gpointer data);
68
Development of a multimedia application
void
pgst_volume_high_cb
(GtkWidget * widget,
gpointer data);
void
pgst_volume_mute_cb
(GtkWidget * widget,
gpointer data);
void
pgst_volume_on_conf_cb
(GtkWidget * widget,
gpointer data);
void
pgst_set_volume
(PGstPipeline * pipe,
gdouble level);
gdouble
pgst_get_volume
(PGstPipeline * pipe);
gboolean
pgst_get_position
(gpointer data);
void
pgst_init_pipelines
();
void
pgst_destroy_pipelines
();
gboolean
pgst_execute_local
(gint screen_num);
gboolean
pgst_execute_remote
(gint screen_num);
void
pgst_play
(GtkWidget * widget,
gpointer data);
void
pgst_eject
(gint pscreen);
void
pgst_go_fullscreen
(PGstPipeline * pipe);
void
pgst_leave_fullscreen
();
void
pgst_disconnect_fullscreen
();
void
pgst_screenshot_cb
(GtkWidget * widget,
gpointer data);
gboolean
pgst_buffer_probe_callback
(GstElement * image_sink,
GstBuffer * buffer,
GstPad * pad,
gpointer data);
gboolean
pgst_create_photo
(unsigned char *data);
void
pgst_load_pipe
(gint pipe_num,
const gchar * name,
const gchar * description,
const gchar * owner,
GdkPixbuf * preview,
const gchar * location,
const gint type);
void
pgst_fill_seek
(gint pipe_num);
void
pgst_free_pipeline
(gint pipe_num);
gboolean
pgst_valid_uri
(gchar * uri);
69
Development of a multimedia application
Description
perla-gst has the control of all functions with reference to Gstreamer access,
such as the load of PgstPipeline data, event handling or pipeline creation.
As it is seen above, it works very closely with Gtk+, so perla-gui and perla-gst
are complementary (and the most important components) of the Perla client
application.
Enumerations and structures
PgstSourceType
typedef enum{
SOURCE_NONE
= 0,
SOURCE_VIDEO
= 1,
SOURCE_WEBCAM
= 2,
SOURCE_SPECIAL
=3
} PgstSourceType;
These flags determine how a pipeline will be treated.
SOURCE_NONE
There is no data in the pipeline
SOURCE_VIDEO
The pipeline is a video pipeline
SOURCE_WEBCAM
The pipeline is a web cam pipeline
SOURCE_SPECIAL
The pipelines is neither video nor web cam pipeline
a PgstSpecialSourceType flag mus be set
70
Development of a multimedia application
PgstSourceInfo
typedef enum
{
COLUMN_NAME
= 0,
COLUMN_DESCRIPTION
= 1,
COLUMN_LOCATION
= 2,
COLUMN_TYPE
= 3,
NUM_COLUMNS
=4
} PgstSourceInfo;
This enumeration is needed to express PgstPipelines as a GtkListStore for
remote information retrieving.
COLUMN_NAME
The first column, where the name of the remote
will be placed
COLUMN_DESCRIPTION
The second column, where the description of the
remote will be placed
COLUMN_LOCATION
The third column, where the location of the
remote will be placed
COLUMN_TYPE
The fourth column, where the type of the remote
will be placed
NUM_COLUMNS
This is not a column but a sentinel to determine
how many columns there are
PgstSpecialSourceType
typedef enum
{
SPECIAL_WEB
= 0,
SPECIAL_YOUTUBE
= 1,
} PgstSpecialSourceType;
When PgstSourceType is set to SOURCE_SPECIAL, it is necessary to choose
what kind of special source is.
SPECIAL_WEB
The special source comes from a URL
SPECIAL_YOUTUBE
The special source comes from a Youtube video
71
Development of a multimedia application
PipeInfo
typedef struct
{
gchar *name;
gchar *description;
gchar *owner;
gchar *location;
gchar *public_location;
gchar *duration;
GdkPixbuf *preview;
gint type;
gboolean status;
} PipeInfo;
A structure designed for the metadata of pipeline storage.
gchar *name;
Name of the source
gchar *description;
Description of the source
gchar *owner;
Name of the user that the source belongs to
gchar *location;
Location of the source
gchar *public_location;
Location which other users see. Only set if the
source is broadcast
gchar *duration;
Duration of the source
GdkPixbuf *preview;
Preview picture that represents the source
gint type;
Type of the source
gboolean status;
Flags that tells whether the source is broadcast
72
Development of a multimedia application
PGstPipeline
typedef struct
{
gint ID;
GstElement *videosink;
GstElement *audiosink;
GstElement *photosink;
GstElement *filesource;
GstElement *player;
PipeInfo source;
gulong expose_id;
} PgstPipeline;
Structure that keeps information about relevant elements of a GstPipeline
gint ID;
Public ID of the pipeline
GstElement *videosink;
Bin that handles video output
GstElement *audiosink;
Bin that handles audio output
GstElement *photosink;
Bin that handles the screen shot feature
GstElement *filesource;
Bin that handles the data input
GstElement *player;
Bin that contains the previous ones
PipeInfo source;
Information about the source in player
gulong expose_id;
ID of the expose callback
73
Development of a multimedia application
11.3 Perla-client
Synopsis
Class that deals with connection features and maintains basic structures about
data exchange
#include “perla-client.h”
void
pcli_search_cb
(GtkWidget * widget,
gpointer data);
void
pcli_init_client
();
gboolean
pcli_connect
(gchar * name,
gchar * passwd,
gchar * server);
gboolean
pcli_register
(gchar * name,
gchar * passwd,
gchar * server);
gboolean
pcli_disconnect
();
gboolean
pcli_start_update
();
gboolean
pcli_stop_update
();
gboolean
pcli_update_trigger
(gpointer data);
gboolean*
pcli_update
(gpointer data);
gboolean
pcli_publish
(gint index);
gboolean
pcli_unpublish
(gint index);
void
pcli_set_server
(gchar * server);
void
pcli_set_client
(gint ID,
gchar * name,
gchar * passwd);
gint
pcli_init_connection
(gchar * ip);
void
pcli_close_connection
(gint sfd);
gboolean
pcli_write
(gint sfd, gchar * buf,
size_t count,
const gchar * error_msg);
gboolean
pcli_read
(gint sfd,
gchar * buf,
size_t count,
const gchar * error_msg);
gboolean*
pcli_download_youtube
74
(gpointer data);
Development of a multimedia application
Description
perla-client, as its name tells, is the client part of the Perla network. It is in
charge of exchange information with some Perla server and keep connection with it
Enumerations and structures
PSourceStatus
typedef enum
{
UNPUBLISHED
= 0,
PUBLISHED
=1
} PSourceStatus;
Flags that determines the status of a PgstPipeline
UNPUBLISHED The source is only watched at local
PUBLISHED
The source has been broadcast
PClient
typedef struct
{
gint ID;
gchar *name;
gchar *password;
} Pclient;
Structure that contains the essential information about the set Perla client
gint ID;
ID of the Client in the connected Perla server
gchar *name;
User name of the client
gchar *password;
Pass word of the client
75
Development of a multimedia application
11.4 perla
Synopsis
Main class of Perla client application
#include “perla.h”
void
perla_on_exit
(GtkWidget * widget,
gpointer data);
void
perla_set_screen
(const gint screen);
gint
perla_get_mode
();
void
perla_set_mode
(gint mode);
Description
perla is the main file of Perla. It triggers the initialitation of every Perla
component and starts the application loop.
It also has some functions about the current screen and audio modes.
Enumerations and structures
PAudioMode
typedef enum{
PLAYER
= 0,
VIDEOCONFERENCE
=1
}PAudioMode;
The flags that determine the audio modes
PLAYER
Perla is in player audio mode
VIDEOCONFERENCE
Perla is in videoconferencing audio mode
76
Development of a multimedia application
11.5 perla-server
Synopsis
The main file of a Perla server.
#include “perla-server.h”
gboolean
psrv_init_server
();
gboolean
psrv_init_client_structure
();
Client
psrv_create_client
(gchar * name,
gchar * passwd);
Source
psrv_create_source
(gint client_id,
gint source_id,
gchar * name,
gchar * description,
guint8 * preview,
gint type);
void
psrv_free_source
(gint client_id, gint source_id);
gboolean
psrv_accept_cb
(GIOChannel * source,
GIOCondition condition,
gpointer data);
gboolean
psrv_check_connections
(gpointer data);
void
psrv_client_set_offline
(gint index);
gboolean
psrv_init_connection
();
gchar*
psrv_get_ip
(gint sfd);
gboolean
psrv_bind
(gint sfd);
gboolean
psrv_listen
(gint sfd);
gboolean
psrv_open_channel
(gint sfd);
gboolean
psrv_success
(gchar * origin,
gchar * message);
gboolean
psrv_failure
(gchar * origin,
gchar * message);
gboolean*
psrv_talk_to_client
(gpointer data);
void
psrv_client_connect
(GIOChannel * reader);
void
psrv_client_register
(GIOChannel * reader);
void
psrv_client_disconnect
(GIOChannel * reader);
void
psrv_client_publish
(GIOChannel * reader);
void
psrv_client_unpublish
(GIOChannel * reader);
77
Development of a multimedia application
void
psrv_client_update
(GIOChannel * reader);
void
psrv_client_search
(GIOChannel * reader);
gint
psrv_client_init
(gchar * name,
gchar * passwd,
gboolean is_registring);
void
psrv_client_update_timestamp
(gint index);
Description
The server file opens a gate where every client connects and exchanges
information, in order to eventually publish their source into the Icecast server
running in the same machine. It is necessary to say that it is not the Icecast server,
but a supporting application to FILTER access and grant the good behavior of the
media server.
Enumerations and structures
ClientStatus
typedef enum
{
OFFLINE
= FALSE,
ONLINE
= TRUE
}ClientStatus;
Flags that determine the status of a registered client
OFFLINE
The client is not connected
ONLINE The client is connected
78
Development of a multimedia application
Source
typedef struct
{
gint ID;
gchar *name;
gchar *description;
gchar *location;
guint8 *preview;
gint type;
} Source;
Structure that stores metadata from the received streams.
gint ID;
ID of the stream
gchar *name;
Name of the stream
gchar *description;
Description of the stream
gchar *location;
Location of the stream in the Icecast server
guint8 *preview;
Picture that represents the source encoded to be sent
gint type;
Type of the source
79
Development of a multimedia application
Client
typedef struct
{
gint ID;
gchar *name;
gchar *passwd;
gboolean status;
Source sources [6];
gulong timestamp;
} Client;
Structure that saves the main information of a registered client
gint ID;
The ID of the client in the server
gchar *name; User name of the client
gchar *passwd;
Password of the client
gboolean status;
ClientStatus determining its availability
Source sources [6];
List of source metedata
gulong timestamp;
The last time to exchange information with the server
80
Development of a multimedia application
11.6 How to run Perla applications
Inside the CD enclosed with this memory, we can find three files that must be
installed in order to create the Perla environment.
Two of them, perla-server.tar.gz and icecast-2.3.1.tar.gz are needed to
set up a Perla server, while perla-0.4-1_i386.deb contains the Perla client
application.
Note that every installation step is meant to be applied under Ubuntu
Client
From .deb package
Figure 11-1 Debian package
Just double-click the package and the installation will start.
It is possible that for some reason, Perla cannot be installed. The most
common reason is because some depending library (such as gstreamer, glib or
gtk+) is missing. The installation manager will show that missing library.
From source code
Perla it is also meant to be installed via autoconf tools.
In your terminal, and after uncompressing it (from perla-0.4.orig.tar.gz),
type:
./configure
make
make install
In order to install it, it is necessary to have root privileges
81
Development of a multimedia application
Server
First of all, we must install icecast-2.3.1.tar.gz (it can also be installed via
aptitude) and carry out the following steps:
 In
order
to
configure
the
environment,
edit
the
archive
/etc/icecast2/icecast.xml. There are lots of flags and attributes that you can
change, but the most important ones are:
<listen-socket>
<port>8000</port>
<bind-address>192.168.1.34</bind-address>
</listen-socket>
Here you have to write the IP of your machine (you can guess it via ifconfig).
Important: Do not change the port, as the Perla Client is set to communicate with
the Icecast server through port 8000
And then:
<source-password>cv1ll4v1</source-password>
This attribute is necessary for the stream publication. In this version, the
password set to try Perla is cv1ll4v1, so write it down in the convenient tag.
 When the server is configured, we need to start the icecast2 daemon:
sudo /etc/init.d/icecast2 start
Note that it is very important to have icecast daemon started when perlaserver is running, as it will not complain (not in this version), but the server
environment will not be completely set.
When icecast is already installed (and it can be started), we will process to
install perla-server, using the Makefile packed in the archive.
As long as the port 8080 of the concerning machine is available and it is
connected to the Internet, the server will show the IP to use in the icecast
configuration and when a connection by a client is wanted.
82
Development of a multimedia application
It is important to advise that Perla-server will acquire its IP by asking the eth1
interface. If we want to change the IP origin, we need to change the code of the
function psrv_get_ip () and re-compile.
Figure 11-2 Perla-server running
83
Development of a multimedia application
12. Economical study
In order to extract the actual cost of the Perla Project I will evaluate the
stages on the basis of an ordinary software life cycle. Therefore, we can observe its
time breakdown:
Task
Dedication
Necessities and specifications
Documentation
15 hours
Meetings with the tutor
15 hours
Study of technologies
40 hours
Preliminary applications
100 hours
GUI
110 hours
Multimedia
265 hours
Network
60 hours
Debugging
20 hours
Tests
40 hours
Analysis and design
Implementation
Tests and validations
Essay writing
55 hours
Total
720 hours
Following up on the previous statistics, we can realize how critical the
implementation stage is, since it requires lots of hours of preparation, typing and
code cleaning, as well as error correcting or redesigning development strategies.
The analysis and design part also took a lot of time, as we needed to get used to
these new technologies, read their reference manuals and create some boilerplate
applications, whose techniques and acquired knowledge can be reused for new
applications.
It is also important to say that, although the project was started a long time
ago, it did not took its eventual way until some months after, because we initially
have another project in mind (developing some applications to a Nokia N800 Tablet
PC), which cannot be taken as wasted time, but it could have been more profitable
if we had focused earlier on the real services we wanted to offer.
Finally, I want to emphasize that this previous reason did not let me study the
actual length of the project, but these statistics are pretty accurate.
84
Development of a multimedia application
Figure 12-1 Percentage of dedication in hours
85
Development of a multimedia application
If we want to talk about Perla in numbers, we might know a few details:
 All the used software was free software, including the operating system,
libraries and supporting applications. So the software entails no economical
penalization.
 We need to know the salaries of each role involved in Perla development:
Role
€/hour
Analyst
45 € per hour
Designer
55 € per hour
Programmer
35 € per hour
Project leader
60 € per hour
Now we have a brief estimation regarding the tasks of the project:
Task
Hours
Role
Cost
Documentation
15
Analyst
675 €
Meetings
15
Project leader
900 €
Study of technologies
40
Analyst
1800 €
Preliminary applications
100
Programmer
3500 €
GUI implementation
110
Designer
6050 €
Multimedia
implementation
265
Programmer
9275 €
Network implementation
60
Programmer
2100 €
Debugging
20
Programmer
700 €
Tests
40
Programmer
1400 €
Essay
55
Programmer
1925 €
Total
--
--
28325 €
86
Development of a multimedia application
13. Conclusions
First of all, let’s read over the initial objectives.
Perla will have consisted on a debian package, written in C and as many
standard protocols as possible.
The main objective was reproduce video streams from local files, web devices
and remote stream, all on the same interface and simultaneously. In addition, it
had to permit user publish their streams.
Moreover, Perla had to implement some extra features, such as different
audio modes, full screen or screen shots.
All this work ought to be focused in a user friendly way.
13.1 Achieved objectives
Almost all the main objectives are largely completed.
We already have a reproducing application. If we want to play a media
source, we have a graphical and easy way to do it. Following just two steps leads to
the basic reproduction of web cam or video file.
The local web cam publication is now available, just adding few steps to the
process. If we tick the broadcasting option and if we are connected, it can be
published.
Moreover, the connection tools are ready to use and easy to deal with as well.
An input window with some text fields to be completed is shown and the rest of the
client/server exchange is being executed in background.
With reference to playing several streams simultaneously, Perla is one of the
first applications to add this feature with success. The easiness of switching among
screens and the different audio modes lays the foundations of more specialized
services and they are a good beginning.
Some feedback from the system is also available to the client, trying to solve
problems by the application itself and communicating to the client some critical
aspects of the flow.
87
Development of a multimedia application
Finally, I want to explain what I think of the development of the applications
and the project itself.
I think this project has been carried out with effort and
dedication. Getting a complete application out of technologies that were not
previously learned or studied means applying a special effort in front of decisive
aspects and implementation, but someone with little experience can avoid them.
Consequently, I learned the importance of good planning before starting any
project and how useful it is to achieve little objectives day by day and try to
accomplish them in a rational time, so a fluent work flow is set. Only thus can the
success of any project be ensured (and I am not only talking about software
project, but every challenge in life). It has encouraged me to go on with Perla
beyond the delivery of this final degree paper.
13.2 Future lines of work
I divided the possible improvements of the current version in several
categories: a critical category, that includes the obligatory tasks in order to release
an official version (a 1.0 version); another list that says what changes or inclusions
are desired to be included therefore the application is able to offer a complete
range of services on the multimedia aspect; and a third category, that includes
optional features, which will specialize or slightly improve the present application.
13.2.1 Critical
Publication of video files
That has been the Achilles heel of Perla. In spite of some progress with
reference to this feature, I could not get a real solution. The true way to achieve
this task would entail a complete review of the elements involved in creating the
video pipelines and a more accurate study of the used technology.
User feedback
In order to achieve one of the non functional requirements (Perla must be
easy to use), a better way of communicating with the user must be developed. At
the moment, warning messages and errors are written on the standard error
output. This is correct as long as Perla is called by the terminal application, but a
non-technical user will call the application through its icon, so graphical ways of
doing so are more than desirable. It is also recommended to allow users to cancel
some heavy operations, such as downloading videos from YouTube or broadcasting
(but without losing his local reproduction)
88
Development of a multimedia application
Improve full screen mode
The present implementation can be taken as a prototype or a first step of
what full screen really means. The official version of Perla must let users employ
100% of the screen with a certain stream.
Auto detection of web cam
As was said before, Perla wants to reach the widest range of users possible
and the majority of them will not even know what a device is (I do not wish to
demean them, but the simpler the application the more used it will be). So it is
recommended that our system detects the type of device and its data and then
displays it instead of the present option. That will also help the user avoid possible
problems.
13.2.2 Desired
Super server
At the moment, every set server has its own information. This is not really a
problem at this stage, but if the usage of the application grows, it is desired that
several servers are spread as these sort of applications do. In order to have a
common database of clients and sources, a new implementation of the server must
be developed, using two levels of hierarchy: a low-level server, where the streams
are really published and whom the client attends, and a high-level server, or super
server, where the list of clients are placed and where other servers attend to
retrieve the information asked by the clients.
Migrate
Server to a relational database: As previously stated, if the amount of users
grows, the capacity of the server system is seen to be jeopardized. At the moment,
Perla servers handle the client information in arrays. This implementation has
several set backs: the server needs to be up uninterruptedly and, if a certain
number of clients are registered in this server, the supporting system cannot give
enough memory, and it breaks. Carrying it to a relational database, such as MySQL,
solves both problems at once.
Seeking
The developed display of the time line is an understatement. It is not a critical
part for the official version because web cam streams are not allowed to use it, but
89
Development of a multimedia application
it is more than desirable that a video player should have this feature.
Choose the audio source for web cam
At the moment, if a video is chosen, the audio comes from the source itself; if
it is a web cam which is playing, the source from the ALSA driver is set as its audio.
As long as we want to use cameras far from our computer (connected somehow
with it) or mix audio and video, the audio track has to be freely set, or not even set
at all.
Other special sources
It was considered one of the first main features that Perla must have.
Nowadays Perla supports videos from YouTube and from an URL, but it has to
improve in some ways. For instance, adding some other sites, such as GoogleVideo
or adding Live TV applications, such as Zattoo.
13.2.2 Optional
New Metadata
One of the personal purposes for which I use Perla is as an alternative to
going to the cinema or watching a film. So it must deal with greater metadata than
the present one: copyright, pass code...
Design
In order to give Perla its own personality, a set of icons (at least, the defining
one) are recommended to be drawn.
90
Development of a multimedia application
14. Bibliography
1. [5]GNOME Documentation Library. GTK + Reference Manual
http://library.gnome.org/devel/gtk/unstable/
2. Tim-Philipp Müller. GTK+ 2.0. Tree View Tutorial
http://scentric.net/tutorial/
3. GTK+ IRC Channel. Queries and advice
irc://irc.freenode.net/#gtk+
4. Thomas Vander Stichele. Streaming
http://thomas.apestaart.org/log/?p=168
5. [7]Gstreamer. Application Development Manual
http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/index.html
6. Gstreamer. Overview of all Plug-ins
http://gstreamer.freedesktop.org/documentation/plugins.html
7. Gstreamer. Core Reference
http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/
8. The OLPC Wiki. Gstreamer
http://wiki.laptop.org/go/GStreamer
91
Development of a multimedia application
9. Botánica. streaming de video usando gstreamer y v4l
http://www.inventati.org/communa/platanal/2006/12/16/streaming-devideo-usando-gstreamer-y-v4l/
10. Gstreamer IRC Channel. Queries and advice
irc://irc.freenode.net/#gstreamer
11. ICECAST. Icecast 2 Basic Setup
http://www.icecast.org/docs/icecast-2.1.0/icecast2_basicsetup.html
12. Ubuntu forums. Howto make debian standard debs from scratch
http://ubuntuforums.org/showthread.php?t=51003
13. John
R.
Sheets.
Writing
GNOME
Applications.
Creating
Your
Own
Configuration
http://developer.gnome.org/doc/books/WGA/creating-configuration.html
14. [3]WIKIPEDIA. Skype
http://en.wikipedia.org/wiki/Skype
15. [1]WIKIPEDIA. VLC Media Player
http://en.wikipedia.org/wiki/VLC_media_player
16. [2]TOTEM. The Media Player
http://www.gnome.org/projects/totem/
17. [4]Daniel G. Siegel. CHEESE
http://live.gnome.org/Cheese
92
Development of a multimedia application
18. Joan Biscarri Rodríguez. Àlbum de fotos digital
Final degree project
19. Teresa Calveres Nanivés. Eina d'ajut al buidatge de llibres històrics
Final degree project
93