End User Mashup
Programming Environments
Oleg Beletski
HUT, Telecommunications Software
and Multimedia Laboratory
28.03.2008
Content
Introduction
Approach – mashup methods and ways to
compare
Case study – overview of mashup
environments
Results + Discussions
Conclusions
Introduction
Mashup – web application that combines data
from online sources of data and perform new
tailored task
Mashup creation
– Hard way – programmer skills required
– Easy way – use end user programming tools and
visual environments
Approach
Find environments for a comparison
Define criteria for comparison
Analyze each site
Sum-up and analyze results (drop if
environment does not fit well to our scope)
Make conclusions
Approach – criteria for comparison
Lifecycle
Richness of functionality
Usability
Target type of users
Publishing formats
Technologies
Data sources
Case study – list of environments
Yahoo! Pipes
Microsoft’s Popfly
Marmite
Dapper
Google mashup editor
Mashmaker
QEDWiki
Case study – Yahoo! Pipes
Pipes is a powerful composition tool to aggregate,
manipulate, and mashup content from around the web.
Visual drag and drop environment
Does not require knowledge of programming languages
Requires good understanding of a data formats.
Sample application:
– Combine many feeds into one, then sort, filter and translate it.
– Geo code your favorite feeds and browse the items on an interactive
map (use location extractor)
– Power widgets/badges on your web site.
– Grab the output of any Pipes as RSS, JSON, KML, and other formats.
Case study – Yahoo! Pipes
Case study – Microsoft Popfly
Microsoft® Popfly™ is a web site and tool to help people create and share
web sites, mashups, and other kinds of experiences. Popfly is the fun and
easy way to build and share mashups, gadgets, and Web pages.
It’s made up of online visual tools for building Web pages and mashups
and a social network where you can host, share, rate, comment and even
remix creations from other Popfly users.
It is fun, it is simple
Uses Silverlight runtime
Has a lot of predefined modules for existing services and social network
Cool visualizations
Case study – Microsoft Popfly
Case study – Microsoft Popfly
Case study – Marmite
Marmite, a tool that will let everyday end-users create mashups by making it easy to extract
content from web pages, process it in a data-flow manner, integrate it with other data
sources, and direct it to a variety of useful sinks, such as databases, map services, and
compilable source code that can be further customized.
This proposal focuses on three high-risk issues:
–
–
–
making it easy to select what content to crawl
developing a hybrid dataflow / spreadsheet UI that shows what content has been extracted and how
that content is transformed
developing techniques for handling exceptions in the dataflow.
Structure comprises of sources, processors, sinks
Sources: add data into Marmite by querying databases, extracting information from web
pages, and so on.
Processors: modify, combine, or delete existing rows/columns (geocoding, filtering).
Sinks: redirect the flow the data out of Marmite. (showing data on a map, saving it to a
file/web page)
Ownership: Jason I. Hong
Human Computer Interaction Institute
School of Computer Science
Carnegie Mellon University
Case study – Marmite
Case study – Dapper
Dapper stands for data mapper. The main purpose of the service is to
convert any type of content into standard form that can be reused (RSS,
XML). It also has the set of publishing features that turn that content into
Google Gadget, Netvibes Module, iCalendar, Flash widgets and so on.
Visually it run is a wizard mode
Results is to create a dapp (data imported).
Dapps can be shared and there are many of then for popular services
Dapp often is sufficient and can be tuned in map mashup or image loop
Output or visualization is defined by user
Next level of development is to combine those dapps into aggregator
service. The typical example is to combine search result from several
search engines or video clips from alternative video services similar to
magg movie aggregator (http://www.dapper.net/dapplications/Magg/).
Case study – Dapper
Case study – GME
Google Mashup Editor is an AJAX development framework
and a set of tools that enable developers to quickly and easily
create simple web applications and mashups with Google
services like Google Maps and Google Base. Google Mashup
Editor is a great tool for grabbing information from feeds and
letting users see and manipulate it.
Creating applications with Google Mashup Editor is “simple”
for developer
This is a software developer tools and not an end user
mashup environment.
Google mashup editor will be excluded from the section
where we compare different mashup environment because it
did not quite fit to out purpose.
Case study – GME
Case study – Mashmaker
Intel® Mash Maker is an extension to your existing web browser that allows you to
easily augment the page that you are currently browsing with information from
other websites. As you browse the web, the Mash Maker toolbar suggests
Mashups that it can apply to the current page in order to make it more useful for
you. For example: plot all items on a map, or display the leg room for all flights.
Community can contribute to mashup DB. Any user can teach Mash Maker new
mashups, using a simple copy and paste interface,
New user created mashups are suggested to other users.
Relies on the community to teach it about the structure and semantics of web
pages, using a built in structure editor.
There is no dedicated page on the web where to have to go and construct the
mashup application. All what user has to do is to install the toolbar and start
browsing pages. So the paradigm could be called “annotate and mix while
browsing”.
Users themselves tell to Mashmaker servers about the page structures that they
visit and purpose content from other sites to mix with current page.
The service is in beta and not available to wide public at the moment.
Case study – Mashmaker
Case study – QEDWiki
QEDWiki is a browser-based assembly canvas used to create simple
mashups.
The same steps: Assemble, Wire, Share
A mashup maker is an assembly environment in which the creator of a
mashup uses software components (or services) made available by
content providers.
Wiki framework in that it provides both Web users and developers with a
single Web application framework for hosting and developing a broad
range of Web 2.0
Sample applications
– Web content management for a typical collection of Wiki pages
– traditional form processing for database-oriented CRUD
(Create/Read/Update/Delete) applications
– document-based collaboration
– rich interactive applications that bind together disparate services
– situational applications (or mashups).
Case study – QEDWiki
Results + Discussions
Programming paradigm
Richness of functionality
Technology
Ease of use
Openness for integration
Results table 1/2
Pipes
PopFly
Marmite
Dapper
QEDWiki
Mashmaker
Paradigm
Data flow
Data flow
Flow + spreadsheet
Flow + wizard
Application on
canvas
Browsing
enrichment
Maturity
Mature
Mature(beta)
Research project
Mature
Mature
Close, beta
Functionality
Rich but
techy
Rich, integrated with
services
Poor
Limited
Rich
N/A
Easy of use
For
coders,
not for an
end users
Simple and friendly, makes
a suggestion about linking
parts together
Cryptic
Easy, but still not
100% end user
tool
Easy, lazy
developer tool
Toolbar is simple to
use if site structure
know
Technology
Standard
web, YUI
Silverlight, proprietary
Browser plug-in, runs in
own windows, allows
point at the parts of web
pages to scrap
Standard and/or
popular WEB
technologies
Standard
In browser mashing
up, needs a toolbar,
enabled by
community
maintained DB of
sites structure
Results table 2/2
Openness for
integration
Html code to embed in
blog, RSS,
JSON
HTML code to
embed
n/a
New API creator.
Export to
Google
gadget,
RSS,
XML,
Flash
Widget
Extensibility
Yes
Yes, user can
contribut
e
n/a
Dapps can be
reused by
other
dapps
Extension by
JavaScri
pt
?
Advanced features
Other
Claim to have unique
web page
scrapping
approach
Can embed
public
WEB
services
and
widgets
N/A
New sites and
widgets can
be added
Results – programming paradigm
Data flow with various visual coding
– Wizard based
Dapper
– Visual connected functional
Pipes
Popfly
– Spreadsheet
Marmite
Visual widget assembly
Mashmaker
QEDWiki
Results – richness of functionality
Data sources
– RSS, ATOM feeds (low level that is supported by all tools)
– Page scrapping (Most tools support, except Mashmaker & QEDWiki)
– Service wrappers (Pipes, POPfly)
Data processing options
–
–
–
–
Feed merging
Location extraction
Numeric operation
String manipulation
Visualization of results and output formats
–
–
–
–
Multiple visualization alternatives (POPFly only)
Visual appeal
Export to external services via RSS/ATOM feeds, XML, JSON
Blog integration
Results - technology
AJAX – standard web technologies, runs in
any browser (Pipes)
Silverlight – proprietary but highly visual
technology from Microsoft (Popfly)
Browser add-on, dependent on browser
(Mashmaker, Marmite)
Results – ease of use
User roles
– Mashup author
– Mashup user
– Environment developer
Mashup author is the user of tools we analyze
– Average user that uses provided blocks
– Advanced user that adds modules and new services to the
environment
Goal – make system as simple as possible for use
by average user
Results – openness for integration
Can results of the mashup be reused?
Popfly – supports only html code for adding to
external blogs
Pipes are more open -RSS, KML, JSON
Dapper is designed for integration - XML,
RSS, Google Gadget, Flash, Google Map,
Image Loop, iCalendar
Mashmaker and QEDWiki is a bit out of scope
Conclusions
Mashup environments target different user groups –
programmers, technology enthusiasts, non technical
users
Best visual tools for non technical users hide data
exchange formats and provide blocks for every
service that user is likely to integrate
Advanced functionality will keep environments on the
float by letting users provide support for new
services
Standard technologies are preferred
New direction – enhancing browsing experience