Llifelog:
Discovering and Evaluating Research Projects
through Peer-to-Peer Exchange
ARCIVES
MFA(SHUSETTS 'NSTITUTE
oR mEoo
SJl7
by
Li Bian
LBRARE_
Submitted to the Program in Media Arts and Sciences,
School of Architecture and Planning,
in partial fulfillment of the requirements for the degree of
Master of Science in Media Arts and Sciences
at the
OF TECHNOLOGY
INSTITUTE
MASSACHUSETTS
June 2012
0 2012 Massachusetts Institute of Technology. All rights reserved
Author
Li Bian
Program in Media Arts and Sciences
May 5, 2012
Certified by
U
(j
enry Holtzman
Research Scientist
Media Laboratory
Thesis Supervisor
Accepted by
Mitchel Resnick
LEGO Papert Professor of Learning Research
Program in Media Arts and Sciences
1
2
Llifelog:
Discovering and Evaluating Research Projects
through Peer-to-Peer Exchange
by
Li Bian
Submitted to the Program in Media Arts and Sciences,
School of Architecture and Planning,
in partial fulfillment of the requirements for the degree of
Master of Science in Media Arts and Sciences at the
Massachusetts Institute of Technology
June 2012
Abstract
Llifelog is a web platform for discovering, evaluating, and exchanging research projects among
students and researchers from both academia and industry. The goal of Llifelog is foster a
community of critique among students and researchers in a project market. Those who engage in
the market answer the call "find out how your work, projects, and ideas are valued in the
community." The unique contribution of the Llifelog system lies in designing market-based
mechanisms that stimulate the project exchange while maintaining information confidentiality,
and devising a peer-to-peer, direct communication channel between project creators and project
viewers. In this thesis, I explain how such market mechanism results more objective evaluation
compared to other existing rating mechanisms online. A virtual credit system, Truons, is used in
this web platform to facilitate the project evaluation and exchange.
The website was built using Java, MySQL, Apache Tomcat, HTML, CSS, and Javascript and
hosted on an Ubuntu server. It also uses a Natural Language Processing engine, AlchemyAPI,
and various image processing techniques. The platform was tested among about 305 users with
74 projects within 3 weeks. User feedback was also gathered after the testing to further
understand the context and rationale for user behavior on the web platform.
Thesis Supervisor: Henry Holtzman
Title: Principal Research Scientist
3
4
Llifelog:
Discover and Evaluate Research Projects
through One-on-One Peer-to-Peer Exchange
by
Li Bian
The following people served as readers for this thesis:
Thesis Reader
Pattie Maes
rofessor of Media Arts and Sciences
Program in Media Arts and Sciences
Thesis Reader
Sep Kamvar
Associate Professor of Media Arts and Sciences
Program in Media Arts and Sciences
Thesis Reader
Joichi Ito
Director
Media Laboratory
5
6
Acknowledgements
This thesis project represents the evolution of my passion and thinking since entering the Media,
Arts, and Sciences program for research almost 2 years ago. Without an open environment,
interdisciplinary culture, and tremendous creativity such as that of the Media Lab, I would not
have been able to reach this stage today and this project would not have come to fruition. Thus, I
would like to express my deep gratitude for the following people:
Henry Hotlzman, my advisor and thesis supervisor. Thank you for giving me the freedom to
stretch my imagination as far as it could reach and providing me immense research resources. I
will never forget the long-hour discussions we had about various research topics, gadgets,
fashion, and about how I'd fail the Turing Test as an android. Thank you for your tremendous
support for my China Initiative activities outside the "normal" research scope. But believe it or
not, my initial inspiration for the market-based project evaluation platform has come under the
grilling inquiries from the sponsors about our projects.
Pattie Maes, thank you for being an inspirational figure in my life since I first took your class in
undergrad of MIT. Your open yet pragmatic approach to solving problems has taught me more
than just how to do research in school but also how to manage the ups and downs in life.
Henry Lieberman, Sandy Pentland, Sep Kamvar, and Joichi Ito, thank you for all your
advice and feedback on my thesis project.
Thank you for pushing me hard to think more
critically about many important aspects of this research.
Matt Hirsch, Greg Elliot, David Carr, Dan Schultz, Arlene Ducao, and Rob Hemsley, thank
you for being such wonderful group members!
You, and bubble tea, were the spice of my
research life at the Media Lab.
Wei Pan and Dawei Shen, thank you for being the big brothers in my life in the past two years.
You were always there when I needed prompt help or advice. I feel fortunate to have known
7
both of you, worked hard on the China Initiative with both of you, and receiving guidance from
you.
All the Media Lab sponsors who have given me the chance to demo projects to you, thank you
for coming!
You were the best channel for me to reach out to the real world for the most
practical feedback.
Special thanks to Hisense, for being one of the test beds for my thesis
project. I owe you much for your support and collaboration.
All the friends in the Media Lab community, Jinha Lee, Santiago Alfaro, Xiao Xiao, Lining
Yao, Roy Shilkrot, Valentin Heun, and many others whose faces appear in my mind vividly
now, it is a pleasure to have known all of you. Thank you for all the fun time together and the
serious discussions about research together!
Lastly, a big thank you to my friends across the globe who helped me test my thesis project
website. That includes my high school / undergraduate friends who now go to other universities
or work at top technology companies such as Google, my former managers and colleagues at
Microsoft and Nokia, and numerous friends that I met by luck in life throughout the years.
Thank you all for using the website and providing me the most valuable insights for how to
improve the work in the future!
8
9
Contents
A b strac t .......................................................................................................................................
3
A cknow ledgem ents ......................................................................................................................
7
1. Introduction .............................................................................................................................
13
1.1 Motivation ...................................................................................................................
14
1.2 The V alue Proposition Dilemm a ...........................................................................
16
1.3 Approach ..............................................................................................................
16
1.4 Contributions ...........................................................................................................
17
1.5 Outlinend .Rela ek........
20
.............................................................................................
2. Background and Related W ork................................................................................
2.1 Relatedw ork....................................................................................................
..... 21
...21
......................................................................................................
28
3.1 Design M ethod..........................................................................................................
28
3.2 Process .......................................................................................................................
29
3. Design..........
4. Implem entation ....
....................................................................................................
36
4.1 The Prototype ..............................................................................................................
36
4.2 Overview ..............................................................................................................
45
4.3 System Architecture..............................................................................................
47
5. Evaluation
and. u...........
...............................................................................................
49
5.1 U ser Testing.. ................................................................................................................
49
5.2 Data Analysis...............................................................................
52
5.3 U ser Feedback ............................................................................
56
6. Conclusions and Future Work......................................................................
59
Bibliography ..............................................................................................
62
A ppendi.x ......................................................................................................................................
71
A . Final Round Survey Questionnaire ...............................................................................
71
10
List of figures
Figure 2.1 Quora's interface for asking and answering a question with Quora Credits............22
Figure 2.2 Barter's home page listing all information shared on the website....................23
24
Figure 2.3 Barter's "Economy" page .................................................................
Figure 2.4 Types of rating activities for the three datasets used in the study....................26
Figure 2.5 Dataset from CouchSurfing.com.............................................................27
Figure 3.1 User-Centered Design Process...............................................................29
Figure 3.2 Two rounds of iteration for the development of the project..............................31
Figure 4.1 The home page that a user lands on as soon as she signs in..............................37
Figure 4.2 The preview of a project on the market...................................................39
Figure 4.3 The rating of a project.....................................................................40
Figure 4.4 The display of full content of a project......................................................42
Figure 4.5 The project portfolio page of a user......................................................43
Figure 4.6 Simple and easy browsing of projects on the portfolio page...........................44
Figure 4.7 Built-in feedback system on the website....................................................45
Figure 4.8 An overview of Llifelog web platform.....................................................46
Figure 5.1 Website traffic monitored through Google Analytics.................................
50
Figure 5.2 Geographic usage monitored through Google Analytics..............................51
Figure 5.3 Number of exchanges versus number of images in a preview for a project
....... 53
Figure 5.4 Project rating versus the number of images in a preview for a project ................. 55
11
12
W
Introduction
"Information Wants To Be Free. Information also wants to be expensive.. .That tension
will not go away."
-- Stewart Brand, The Media Lab: Inventing the Future at MIT. 1988
This thesis is at the heart of that tension. We live in an information society under an
economic system of capitalism. For information markets to flow, we need to be able to quantify
and preserve the value of information, yet once we hand over information, we risk having lost
control of it. The fickle nature of information poses challenges to creating online information
markets: reveal too much of the information in advance of a transaction and the buyer could get
all she needs for free; present too little and risk nobody showing interest. Furthermore, the value
of information is often greatly affected by how it is framed or organized.
The author presents a novel approach to organizing information for exchange over an
information market, a method for previewing the information and its organization without
diminishing the information's value, and a rating system for assigning quantitative value to the
information to ensure that the value of the information can be accurately predicted from the
preview. This contribution has been implemented and evaluated in the context of an example
information market, a market for research projects in computing technology fields. This market
is not about the exchange of money, but rather aims to foster a community of critique around
student and workplace projects. Those who engage in the market answer the call "find out how
your work, projects, and ideas are valued in the community."
13
1.1
Motivation
We live in a world of abundance in information.
Our generation has witnessed the
evolution from the World Wide Web to the "Web of the World" in which mobile
communications, social technologies and sensors are connecting people, the internet and the
physical world into one interconnected network. Our identities are authenticated and recorded,
our location tracked, our speech translated and gestures recognized; we are scrutinized by
today's information technology.
With Facebook [13], Linkedin [22], FourSquare [14], and
Amazon [4], there seems to be no aspect in our life that is not already recorded in real time,
stored in the cloud, and analyzed for better, "personalized" services. To make the case more
sound, the more active information citizens among us faithfully and tirelessly contribute to the
continued digital encyclopedia of human knowledge, by sharing our ideas, questions and answers
on Ask.com [6], Yahoo Answers [33], Quora [25], Wikipedia [31] and the likes. Among the
technical savvy ones, making what one has built "open source" is what is popular.
And so we hear shout-outs from the overwhelmed crowd, "Information is free!"
Quietly yet swiftly, another movement emerges. Diaspora [11], a notable attempt launched
in 2010 that allowed every individual person to take control of his or her own information, rose
among the numerous Kickstarter [20] backed start-ups as the highest funded one-it even got
funding from Facebook CEO Mark Zuckerberg. Following Diaspora, a new wave of start-ups is
thriving on the premise that users want to take more control over the information they have been
freely sharing. Some start-ups take an even more pragmatic approach: letting users trade their
data for monetary value.
But these start-ups aren't innovating new concepts: academia for
several years has been analyzing the dynamics of personal data privacy and information
confidentiality and producing prototypes for trading such information with virtual or real
currency.
For example, Barter [7], a wisdom exchange platform from Viral Communication
Group of the MIT Media Lab, has already run beta testing among 5000 users around the globe in
the course of 2 years. A series of work on Reality Mining [26], spearheaded by Nathan Eagle
and Alex Pentland over the past decade from the Human Dynamics Group at the Media Lab, has
seen its derivatives in the privacy control and information trading domain in the past few years
since the rise of online social media and networked data. A comprehensive summary of this
14
ongoing phenomenon is provided in the white paper of the World Economic Forum in 2011,
which systematically linked the social, economic, and technological impact as a result of the
" 2 1st century's new oil", data [32].
"Information wants to be free. Information also wants to be expensive... That tension will
not go away."
In 1988, Stewart Brand famously quoted this in his book The Media Lab:
Inventing the Future at MIT [8]. As of today, in the year of 2012, this still appears to hold true.
The MIT Media Lab, founded in 1985, manifests this tension itself. Numerous projects coming
out of the Lab have benefitted the world greatly as open source. For example, Processing, a
programming language, development environment, and online community that since 2001 has
promoted software literacy within the visual arts, was created by Ben Fry and Casey Reas in
2001 while both were John Maeda's students at the MIT Media Lab [27]. DepthJS, a recent
open-source project ran by Aaron Zinman, Doug Fritz, Greg Elliott, and Roy Shilkrot
successfully connected Microsoft's Kinect motion-sensing technology with Javascript to allow
gesture-based web browsing [10]; the source code is hosted in github, an open source code
sharing website for software developers [16]. The list of Media Lab open source projects goes
on, but the majority of the Lab's projects are not open source. The sponsorship model of the
Media Lab requires each member company pay a minimum of $200,000 annual fee to access the
rest of the Lab's project information, ideas, source code, and publications, not to mention
engagement with the Lab's most valuable asset, the talent pool [21]. At first glance, the Media
Lab is like a vertical Google of research projects, except it resembles one with a "freemium and
premium" model where people need to pay money to use this information platform, for unlimited
research discovery, evaluation, and exchange.
My deep involvement in some of the Lab's
sponsor engagement activities, especially that from the Media Lab China Initiative [23], has
inevitably forced me to question critically the validity and practicality of the campaign of both
"free information" and "monetization of information".
And it has come to no surprise that
talking with different advisors at the Lab about this issue, I have identified a wide range of
thoughts and opinions across the whole spectrum from free to not free.
15
1.2 The Value Proposition Dilemma
As the connection between information and its economic potential becomes more evident
in today's big data world, businesses and research organizations both face an inevitable value
proposition dilemma: which direction to focus now to embrace tomorrow's information
transformation: direct financial return from information trading markets, or collective evaluation
of information to unleash the potential of new business and research opportunities? While the
former seems immediately foreseeable, the latter is an unknown land waiting for exploration.
Many have failed before in trying to combine the two. For example, Google Answers was an
online knowledge market released by Google in 2002 that allowed users to post bounties for well
researched answers to their queries [18]. Asker-accepted answers cost $2 to $200 and Google
retained 25% of the researcher's reward and a 50 cent fee per question. However, the service
was shut down by Google in the end of 2006 possibly due to decreasing number of users. Quora,
another Q&A website launched in 2010, allow its users to spend Quora Credits to get answers to
questions and earn Quora Credits when others user give credits to request answers or when their
content gets up-votes. As of today, Quora has not allowed cash-in or cash-out of the Quora
Credits system yet, but instead has been using the system to direct the site's question and answer
flow and maintains the quality of content on its website.
1.3 Approach
Llifelog borrows ideas from the real world economy and tries to apply market
mechanisms in a digital information exchange platform with virtual currency. Contrary to
Google Answers, Llifelog does not encourage the culture of expert promotion through real
money transaction; instead, it fosters a community of crowd-sourced critique on research
projects. At the same time, Llifelog upholds the value of quantitative rating beyond qualitative
rating, due to the fact that a quantified system often provides more stringent metrics for
evaluation.
However, assigning a numerical rating to creative expression, such as a project
description, is difficult, if not only novel. Moreover, digital information exchange pertains to the
intrinsic atomic nature of information, i.e. giving too much to lose selling power or giving too
little to lose user incentive. Mimicking existing real life examples in the music, movie, and
16
publication industry, Llifelog adopts a preview method to allow users to get a glimpse of the
information they are about to receive. Same as in the music, movie or publication industry, the
quality of a preview can prove to be a good summary and reference to the actual content or leave
the audience uninterested. The author believes that the preview method is the right way to go but
that there needs to be many iterative testing and tweak before getting the preview right.
1.4 Contributions
Motivated by such problem and opportunity, the author designed and implemented a web
platform, Llifelog, which allows students and researchers from both academia and industry
across the US-and some other parts of the world-to discover, evaluate, and exchange their
research projects with one another. The goal of this platform is to foster a community of critique
around student and workplace research projects. Those who engage in the market answer the call
"find out how your work, projects, and ideas are valued in the community." Before jumping into
descriptions of what this platform is or does, I find it necessary to make a list of things that this
platform is not and does not do, lest any confusion comes up later:
1. Exchange of money. Llifelog is not a money exchange platform. It does not allow or
facilitate buying or selling of anything with real currency on the website. The market
mechanism and virtual credit system used on the website is for the purpose of
stimulating project exchange and encouraging project sharing with a gamification
flavor. In Chapter 2, Background and Related Work, I will give examples of other
web platforms that also use market mechanism and virtual credit for similar purpose
for different content sharing.
2. Intellectual Property (IP) trading. The author has been contacted by various peoplegovernment IP trading officers, business professionals, the website's own userswhether IP trading is possible on the website. This thesis project done at the Media
Lab, Llifelog, is not for IP trading and does not facilitate or take responsibility for
that.
3. "Yet another academic research archive"-Yes and no. Yes, because the author did
spend quite a lot of effort in figuring out an interface that allows users to more easily
17
document, upload and save their projects online. No, because this web platform is
not for archive purpose, but designed for better discovery, peer-evaluation, and
exchange.
As mentioned above, Llifelog is a web platform for students and researchers from both
research academia and industry to discover, evaluate and exchange their research projects with
one another.
In order to achieve this goal, a complete package of web features need to be
provided, so that the users get to experience the entire interaction flow and the author gets to
collect real time data for analysis for this thesis project. The major contributions of this whole
package include:
1. Providing a vertical search web platform dedicated to research project discovery
across different research organizations. Allowing the users to search based not only
on the content of projects but also on meta-data such as topic, project creator,
organization, and location. Although with only a few hundred users on the website
from the 3-week user testing due to the scope of this thesis project, the diversity of
users-from MIT, CMU, Stanford, Qinghua, Singapore University of Technology
and Design, Hisense, Microsoft, Nokia, Toshiba-has made it interesting for project
discovery and comparison.
2. Designing a minimal user interface for content creation and sharing, as well as a sleek
interface for displaying a user's project portfolio. This part of the interface design
has received positive feedback from many users, which demonstrated that with
today's advanced and mature web technology, designing a tool that is useful, usable,
and desirable is more important than the technology tool alone.
3. Devising a peer-to-peer, direct communication channel between project creators and
project viewers. This private messaging system for project exchange has saved users'
time and effort in searching online for the best way to reach out to one another. It
also functions as the backbone for project evaluation as it facilitates back-and-forth
discussion and assessment about a project.
4. Designing
market-based mechanisms
that
stimulate project
exchange
while
maintaining information confidentiality. A virtual credit system, Truons, is used in
this web platform to help users quantitatively measure and assess the value of each
18
and every project. The exchange and evaluation statistics are anonymously shared for
every project so that each user gets to know what the rest of the community
collectively thinks of a project.
Such quantitative evaluation also comes with
qualitative feedback provided through the private messaging system described in #3.
5. Automatically generating a preview based on a project's full content. After users
create their project content using the interface described in #3, the system
automatically generates a preview of the project using natural language processing
and image processing techniques. It then renders the preview on HTML5 canvas.
Previews are what go on to the project market when a user shares a project. This way,
we allow the students and researchers to take more control of and be aware of what
they are sharing with the public on the project market.
Llifelog was tested in two rounds, first in October 2011 before the system was completed
with 120 users over one week, and second in February 2012 when the system was complete with
300 users over three weeks. The user study and testing provided much insight into the research
community's problems and concerns with sharing their work using existing tools and platforms,
their desire for more efficient sharing with a targeted audience while maintaining information
confidentiality, as well as how the market-based platform can help them better evaluate one
another's projects which are not always conducted for the purpose of conference paper
publication. Another interesting phenomenon observed is that the platform's user base seemed
to quickly get adapted to using the virtual credit system, Truon, for measuring their project value
quantitatively. The pattern of average scoring of a project given its content quality and quantity
will be discussed in details in Chapter 5.
19
1.5 Outline
This thesis has six chapters.
The first chapter, Introduction, provides a general
introduction and motivation behind the project along with the list of major contributions. In
Chapter 2, Related Work, we will show examples of related work and web platforms that are
similar to Llifelog. Chapter 3, Design, depicts the whole design process and work flow of this
project which has taken almost a full academic year. Chapter 4, Implementation, provides the
technical details of the software system and how each of the five contributions listed above was
implemented. Then in Chapter 5, Evaluation, we take a look at users' onsite behavior patterns
and statistics from a system backend perspective as well as using data from Google Analytics.
We also share the feedback that users have provided during and after the testing. Lastly, Chapter
6, Conclusion and Future Work, presents conclusive remarks regarding Llifelog and discusses
future possibilities for this project.
20
E
Related Work
"It was six men of Indostan, To learning much inclined, Who went to see the Elephant,
(Though all of them were blind), That each by observation, Might satisfy his mind."
--John Godfrey Saxe
2.1
Related Work
Quora, the question-and-answer website mentioned in the previous section, is a self-
contained market platform that scores every question and answer. Each user is given 500 Quora
Credits to spend upon signing up. In order to earn more Quora Credits, the user has to post
answers good enough that other users would send request to him and possibly give the answers
up-votes. Figure 2.3 shows an example of the Quora interface for asking the question "What is
Ashton Kutcher's diet?" and the list of answers along with their creators and asking credits.
Such mechanism forces the users to think more when they choose the answer(s), which should
help with evaluation of the answers and improving the site's content quality in general.
However, as we could see in the screenshot below, there is too little information displayed about
the answers or the creators of the answers for a user to decide which answer to give credits to.
Because of this extreme lack of meta-information, it is difficult to make sense of the difference
implied by the credits "415", "50", and "25". A glimpse of the answer as well as the answerer's
background is needed in this case to facilitate an asker's choosing process. In addition, it is
unclear how many people have asked the same question and how many times each answer has
been given. There is also no space for negotiation or bargain, so for each answer, the asker is left
in only a choose-or-not choose situation that does not stimulate further conversation exchange
between the asker and the answerer.
21
Quora ISearch your boards and all of Quora
Ashton Kutcher Diets
/
Et
VWhat is Ashton Kutchers diet?
-I
Add Question Detais
W Comment
Post Wiki Options PeJirect Question
You have 461 credits
Ask to Answer - Find People
Jason Putorti, Co-Founder, Votizen
1 Answer in Ashton Kutcher
Brian Wang, Experienced in a a
2 Answers in Diets
...
Johanna Brewer, co-founder of frest...
415 credits to
Ask
25 credits to
Ask
50 credits to
Ask
View More Suggestions
Figure 2.1 Quora's interface for asking and answering a question with Quora Credits.
A more sophisticated knowledge sharing web platform similar to Quora is Barter by
Dawei Shen from the MIT Media Lab, Viral Communication Group. Barter's goal is "to design
and create a market platform that incentivizes decentralized information creation within a
community." This platform has been deployed at the Media Lab's Media Ventures class to help
students share their venture ideas and exchange knowledge. Comparing with Quora, Barter's
interface is more explicit in giving the users a glimpse of the full content that they are about to
view or interact with through the content's title and a meta-description field called "details"
filled out by the content creator. It also shows the statistics of how many times a file or idea has
been "purchased" through an on-click popup. However, a close investigation of the listed files
and ideas shared reveals that the "details" that the content creators have filled out are mostly
subjective and often times self-promotional. In some cases, the users reveal everything in the
content in the "details" field, which defeats the purpose of using the site's virtual credits for
exchange, as no one would still have the incentive to give out credits when she already gets
every piece of the information. Additionally, Barter is a general knowledge exchange platform,
22
as we can see on the top right navigation menu. One can share ideas, Q&A, documents, and
The "Economy" page shown in Figure 2.5 visualizes a user's credit history and
news.
distribution within the Barter system in studious details. While the numerous functionalities
contribute to the technical merit of the platform, from a user interaction design perspective, much
would need to be truncated or simplified for a narrowed focus of content type. In the next
Chapter, Design, the author will go through the present thesis project's design process and how
Llifelog started broad as well and iteratively narrowed down its focus through multiple rounds of
user testing and numerous brainstorming, design, and redesign.
*QS
9
*Search
Al
DISPLAY: BOTH -
Vote for Funf! The Funf guys are trying to qualfy for
Knight Foundation funding
Ui im
Idea
SORT: HOT*
Sud"
Ners shaed byManuLohiya
$10 - 4 +1 - Details- 22 days ago
Q&A
Doc
Auction
News
Fix
My Stats
Tt"
BonUS
Eunried
F m,
My Recent Transaictions
Tribs
U ianis not n anytribes.
Al Tribes Overview
Top Tags
http:// radar.oreiny.com/ 2012/03/planning-bigdata.html
Idea shared byTomSnith
0 - Details- 26da y ago
00 ,
OnMotion: real-time tremor monitoring and feedback for
Parkinson's patients
(41)
ACOVy
Pric
+3
13mns ago
Login
to Barter
+3
14 days ago
A montlysusidy
amount
+200 14 daysago
27dav ago
Price
shared byJennfwe Jordan
3 - 4, +5 - Detaiis - 4 daysago
Idea
(23)
34 daysago
(15)
(14)
(14)
%arowaeTags
Anwao
Lognto arter
NoMyTransct!on
-onowa
Mobie Gaming for National Security
Idea shared bySi Young
1 -i +3 - Detais
.
52 days ago
Price
+
Top Users This Wedi
No top eswsyeV
Figure 2.2 Barter's home page listing all information (ideas, Q&A, documents, and news) shared
on the website
23
IbortSr
0
Ea
Economy
Idea
Money
Supply
Distrbution of Wealth
Dstribution of Earning
zoo
250
Q&A
Doc
News
Auction Fix
WeathEawg
ZOO
ISO
1%5
100
100
0
20
0
400
600
S00
100
1200
1400
g00
1000
Weath
Transactions by Market
120D
1400
160
100
Earig
Transactions by Type
Q&A
Q&A T-Qu-
Figure 2.3 Barter's "Economy" page which visualizes for the user her credit distribution
over time and various activities on the site.
While Quora and Barter relate to Llifelog by both using market-based mechanisms for
information sharing, a key difference between these two systems and Llifelog lies in the focus of
the content and user base. Llifelog is a platform dedicated to research projects, and such shared
projects are limited to first, the computing technology industry and digital design/art field instead
of any field such as business or biology; second, work that has already been done or is ongoing
instead of ideas, thoughts, questions which often times are too premature to measure, and third,
users who are either graduate students and faculties in the research academia or industry
employees in the research division of their companies such as Microsoft Research instead of
Microsoft's development sector.
24
From the angle of research work sharing, we can also find some existing platforms
related to Llifelog. For example, Academia.edu is "a platform for academics to share research
papers" [1], and ResearchGate.org is "a free social networking site for scientists and researchers
to share their research, experience, and publications [28]." While both platforms serve over 1
million users, the websites are designed to encourage sharing of mostly academic publications in
the field of engineering, sciences, and social studies.
Geekli.st [15] is an achievement-based social portfolio builder where "all bad-ass code
monkeys around the globe can communicate, brag, build their street cred and get found."
Software developers post summaries of their work on Geekli.st, such as an Android application
they have written, a website they have built, the technologies they have used or acquired through
these experiences. Similarly, StackOverflow is a well-known Q&A website within the software
development community that uses a set of gamified mechanisms to keep the high quality of the
site content, such as different levels of virtual badges, certifications, points, and thumbs up or
down [29]. It is interesting to note that StackOverflow also uses virtual credits and market-based
mechanisms to allow users to trade software code that takes much more effort to produce than
just a few lines of answer.
Although "thumbs up or down" type of mechanisms works well on StackOverflow, it is
found to be too subjective a rating method to indicate the actual quality of the content objectively
on most of the other websites. In the use case of StackOverflow, a question is usually specific
enough to demand a particular working solution, and the answers that get thumbs up are usually
well-written software code that can be tested to work-make a software program runimmediately. The use cases on the other web platforms, such as giving a "thumbs up", "like", or
"+" to a photo album, a research document, or a description of technical expertise, give little
indication of such action and effect relationship. In I rate you. You rate me. Should we do so
Publicly, Chun-Yuen Teng et. al exam the several possible factors that contribute to how and
why users give ratings, such as age, gender, geography, anonymity, level of trust and friendship
[30].
To study these factors, they conducted extensive analysis with three different datasets:
collections of user-to-user ratings from the CouchSurfing [9] social network, user-to-user and
user-to-article ratings from Epinions [12], and user-to-product reviews from Amazon.
Their
25
analysis found the near absence of negative rating compared with the distribution of positive
rating especially when users of these websites were required to use their real identity for all
rating and feedback activities. This insight echoes with that of another study, Rating Friends
without Making Enemis, by Lada A Adamic et. al., where the authors found that "to users,
providing faceted ratings presents challenges, including differentiating and quantifying
inherently subjective feelings such as friendship and trust, concern over a friend's reaction to a
rating, and knowledge of how ratings can affect others' reputations [2].
Amazon: user-to-product ratings
revtew
Anonymous
or
idenwed
---------------------------------------
Epinions: user-to-user ratings
user-to-article ratings
raung of trust
In artce
raung of trust M user
-------------------
CouchSurfing: user-to-user ratings
L.doo. ratng of fAnemshlp
and trust
rating of fMndship
and trust
Figure 2.4 Types of rating activities for the three datasets used in the study. From the
paper I rate you. You rate me. Should we do so publicly? by Chun-Yuen Teng et. al
26
8
8
0-
extremely
negative
negative
III
neutral/
no reference
positive
extremely
positive
Figure 2.5 Dataset from CouchSurfing.com shows the distribution of positive and
negative references. From the paper I rate you. You rate me. Should we do so publicly?
by Chun-Yuen Teng et. al
27
Design
"Design is not just what it looks like and feels like. Design is how it works."
--Steve Jobs
3.1
Design Method
The process of this thesis project follows a participatory User-Centered Design approach,
where a great number of ideas were flushed out quickly in the beginning with the famous "No
But" rule, followed by sketches or paper prototyping, which soon evolves to become "highresolution, high-fidelity" digital prototype. Users are invited to evaluate an evolving prototype
for participatory design until it reaches a release version. Further user testing is then conducted
and usage data and user feedback analyzed for the next round of iteration, which begins again
with a set of "why" and "what if' concept design questions.
While User-Centered Design
methodology is widely used across the world, Participatory Design, or sometimes called
Cooperative Design, is one school of User-Centered Design originated and popular in
Scandinavian region. Participatory Design is an approach to design that actively involves all
stakeholders-especially users and customers-as early as possible in the design process in order
to ensure that the product designed meets their needs.
28
AK&Lvss
USER-CENTERED
DESIGN PROCESS
DESMG &
Figure 3.1 User-Centered Design Process
3.2 Process
Llifelog followed the User-Centered Design approach and quickly iterated two rounds
within 6 months of development. It began with an ideation stage where the author brainstormed
with her advisor, lab colleagues, friends, potential users, and undergraduate students working
with her. The initial brainstorming conversations were high level, sporadic, and scattered around
many observations and topics, including first, big data, second, online social media as platform
for collaboration, third, unbalanced distribution of information, and fourth, the trend of
monetization of personal data. Following the first round brainstorming, a low-resolution, lowfidelity web prototype was built to test some of the ideation assumptions and speculations, as
well as opening up opportunities for unexpected insights and feedback from users.
The 120
users in this 1-week testing round were mostly people from the author's close network, and
provided informal or written feedback through phone calls, emails, and in-person conversations
after the testing. Such feedback was crucial and inspiring for the second round iteration.
The second round began with a close examination of the feedback from users from the
first round, coupled with a brainstorming that is much more focused comparing with that in the
29
first round. Specifically, particular user need and desire was identified in information sharing.
For example, it is during this brainstorming that the author narrowed down the scope of
information content on the website from all life experiences to just research projects related to
the computing industry.
In addition, the targeted user group was narrowed down to graduate students and
researchers from academia and industry, where a diverse set of academic laboratories and
institutes in both the computing and design disciplines were identified as potential user testing
bases.
A high-resolution, high-fidelity prototype was built including all major functionalities,
such as search, user portfolio, project documentation, one-on-one, peer-to-peer exchange system,
messaging and scoring system for each project exchange. Additional features such as a "Send
Us Feedback" label and a referral system were built to better facilitate the engagement with users
as extra user needs were identified throughout the development process.
The second round user testing took 3 weeks with about 300 users and 50 projects
uploaded to the website.
User behavior was closely tracked through monitoring from the
website's backend and with the help of Google Analytics [17]. More details on user testing and
data analysis will be discussed in Chapter 5. Although the author actively asked for more
feedback after the second round user testing, most of the feedback was received during the
testing, as a result of the "Send Us Feedback" function which allowed the author to make
changes and improvement of the website in almost real time according to user feedback.
30
1"round brainstorming
Topics: big data, social media platform as a collaboration tool,
unbalanced distribution of information, and trend of monetization of
information
1 st
round prototyping
Low-resolution, low-fidelity, website features not complete
1 st
round user testing
120 users within 1 week
2 "dround
brainstorming
Topics: how did users evaluate their content with virtual currency in the
What
1 st round? What content seems most valuable for exchange?
features to include in the website to facilitate that exchange?
2 nd
round prototyping
High-resolution, high-fidelity website features complete
2 nd
round user testing
Over 300 users within 3 weeks
Figure 3.4 Two rounds of iteration for the development of the project.
As mentioned previously, the initial brainstorming during the first iteration was around
four different topics, big data, online social media as platform for collaboration, unbalanced
distribution of information, and the trend of monetization of personal data. Each of these topics
has many researchers working on it already with concentrated effort, so the goal of the
brainstorming was not to pick one of them and go deep down for incremental research value
added in that field. Instead, the goal was to look at the big picture, trying to understand each
topic area and figure out how they are inter-connected and affecting one another on the entire
chess board of global information economy. In the middle of a big data era, many people are
working on data mining, data visualization, trying to come up with the best algorithm for
organizing data and getting the juicy bite. At the same time, others are focused on trading of
such data and figuring out what the best monetary exchange scheme is.
An interesting
phenomenon is observed that while in some geographical regions and industries, people
complain about being overwhelmed by information explosion, in others, people's knowledge and
understanding of "viral growth", "terabyte storage" seems to still stay in the Stone Age. We
have a lot of information; we also do not have much information. Information differs in its
distribution, as well as its value with regard to time, geography, and different people.
The
31
complexity of such a networked problem domain is not what one can break through with one
magical computer algorithm. Instead, it will take a heuristic approach for the entire network of
people, both data producers and consumers, to collectively come up with a network of
organization framework of information.
Such brainstorming process led the author to believe that the toughest bottleneck problem
in understanding big data is the unbalanced distribution of information and that the crux of
solving this problem is by channeling its flow from higher density to lower density, in order to
reduce the "information potential"-in similar way that electrons flow from high voltage
potential to low voltage potential, or channeling the flow with an "like attract" approach by
helping people discover projects related to what they are working on. Market-based information
exchange comes into the picture when we analyze both social and economic incentives of
individuals within the network as part of the information channeling process. As Dawei Shen
has pointed out in Barter, a complete value system should take into consideration both aspects to
effectively incentivize users of an information platform. Such incentives are not only crucial for
keeping the information flow but also the driving force for information evaluation, which circles
back to affect users' motivation for progressing or blocking the information flow. In the case of
Llifelog, market-based incentives are more appropriate than those based on social bonds alone
because the type of project information that users share does not come as effortlessly as that they
share on Twitter or Facebook, but is rather time and resources consuming to the users. Whereas
a simple tweet of "I went to lunch with my best friend from high school" comes from an hour's
lunch, a full documentation of full-length research project might come from a few months or
years of painstaking research, development, testing, and analysis. Additionally, a favor economy
works well when the network community knows one another well, especially at a personal level.
In Llifelog, a professional project exchange platform, most of the users come to discover
research projects for work related purposes, instead of trying to find friends or reconnect with old
friends as people do on a Facebook kind of social network platform.
The targeted users of
Llifelog, graduate students and researchers, often find themselves put in the position of having to
connect with strangers because of the need for research exploration.
A purely favor-based
platform may not work as efficiently in helping the users achieve their research goal.
32
So how shall we effectively merge both social incentive and economic incentive into one
platform? Or if the incentives already co-exist in a particular context, how can we utilize both to
achieve the effective information channeling? Although the first round user testing brought back
more questions than answers, it helped the project to be narrowed down to a particular type of
information content, i.e. research projects in the design and technology industry, and a particular
user base, i.e. graduate students and researchers in both the academia and industry.
In this
context, there are existing social incentives for project discovery, evaluation and exchange:
students and researchers want their work to be discovered and it is mandatory for them to be
aware of similar work in their field. A community evaluation helps them get feedback on their
research, keep track of their progress, and establish a desired reputation. In fact, in both rounds
of user testing, the users have indicated that discovery and evaluation are two important factors
that they value highly in their research process. When asked to rate from 1 to 10 the importance,
1 being the lowest and 10 being the highest, an average of 8 was given from all the user survey
participants. Note that the "discovery" in the user feedback entails "exchange", which the author
has separated purposely here in this thesis writing for mapping them to indicate different
technical features on the website. There are also existing economic incentives for the users in
this context: researchers want to find potential collaborators who can help advancing their
project faster and better; students want to be recognized and hired by their potential employers.
After the first round user testing, users from both Hisense and MIT Media Lab have demanded
that the author add a feature on to the site in the future, such as listing their project collaborators'
name in a project description, as well as being able to receive recommendations from people who
are working on projects similar to theirs from outside of their own organizations.
Based on initial brainstorming ideas, first round user testing and feedback, and a constant
feedback mechanism built into the website for the second round iteration, the author filtered
through a plethora of functionalities and features for the website, and finalized on the following
list:
1. Search, which allows users to discover projects by content relevancy, creator
information, geography, and company or school name.
2. Scoring system, which allows the users to rate each project with a quantitative
score. To help users get started with thinking about how to evaluate a project,
33
the scoring system lists two aspects to which users can give rating,
"presentation" and "content".
The total score for a project is the sum of the
scores for both presentation and content of the project. This resembles the
existing conference paper peer-review system, where a paper is usually
scrutinized from both a technical perspective, where theory, data analysis etc.
is one aspect and English grammar, layout, etc. is another. Given the fact that
the Llifelog platform gathers projects from both technology and design fields,
its scoring system also resembles that for sports such as figure skating, where
the total score of a performance is the sum of scores for technicality (number
of jumps, rotations) and presentation (artistry).
3.
Private messaging system, which allows users to chat with each other in real
time about a project. This messaging system is placed right below the scoring
system so that it is easier for users to relate both the quantitative evaluation
with the qualitative evaluation of the project.
4. A "bargaining" system was built on top of the scoring system to facilitate a
constant back-and-forth "negotiation" interaction between the project creator
and the project seeker. In the process of a project request, when a user gives a
project a rating, the other user can then respond by either accepting the rating,
rejecting the rating, or suggest a different rating. A project's market rating is
calculated by dividing the sum of all the ratings in successful project exchange
by the number of times of exchange for the project. The market rating will be
visible to any future viewer who discovers the project. In this way, the system
discourages the users from "selling" projects for whatever is offered just to
earn the virtual credit.
5. An automatically generated preview for every project that users upload. A
preview is consisted of three parts: the first 250 characters of a project text
description, the first three images extracted from the project documentation
with additional image processing technique to blur and resize the images, and
a WordCloud that visualizes the key phrases within the actual documentation
and their frequency of occurrence. A preview to project documentation serves
two purposes. First, it allows users to quickly glance through many projects
34
without spending unnecessary time on the details before focusing on the
projects that interest them. Second, it eases the concerns of many users who
worry about the confidentiality of their projects when they share on the market
and who, at the same time, desire their projects to be discovered. A project
preview resembles a movie trailer, which is an abridged version of the whole
content, sufficiently representative yet not revealing all the details, in order to
stimulate user interest in purchasing the whole movie.
It also parallels an
abstract to a paper in online publication platforms, where a user gets to browse
through many paper abstracts before deciding which ones are relevant or
interesting to him.
35
E
Implementation
"Usually, if you have a new idea, you very rarely break through to anything like
recognizable development or implementation of that idea the first time around - it takes two or
three goes for the research community to return to the topic."
--Martin Fleischmann
4.1
The Prototype
Given the nature of this project, it is difficult to understand the concepts without a
concrete prototype.
Therefore, in this section, without further delay, we go through the
interaction flow of the website that has been built for this project in the course of 6 months. This
website was a necessary tool for the author to conduct user testing and pragmatically understand
the users' behavior in a context of exchanging research projects using a market-based web
platform. Screenshots are provided for some of the important features of the website, coupled
with explanation of the interaction as well technology used in the particular part when needed.
Figure 4.1 shows the home page that a user sees as soon as she signs in, i.e. the home.jsp
page. This page is divided roughly into two portions, 40% left and 60% right. The left side
displays a stream of briefings of the user's own project, and on hovering shows the entire project
documentation. The right side shows the market, where other users on the site have shared their
projects.
Each project on the market comes with an image extracted from the project
documentation-if no image from the project document then the user's own profile image is
displayed instead, the project title, creator's name, the creator's organization, the project's
upload time and the current market rating for the project.
36
LI Blan
Advanced Search 0
Research Assistant
M[T Media Lab
MatchMaker: Scal TV meets Friend
Recommendaion:
MatchMaker is an automated collaborative
filtering system that recommends friends to
people on Facebook by analyzing and matching
people's online profile with the profiles of TV
characters. The goal of MatchMaker is to produce
ustmldaslmin
dtI
BIDI Screen Matt Hirsch
8:03AM 04/11/2012
200R
Open Event Interpreter Dustin Smith MIT
8:07PM 03/07/2012
1001
DepthJS roy shilkrot MIT Media Lab
EyeRing: A compact, intelligent vision
s... Suranga Nanayakkara SUITD
A Mobile Tool for Language
Suranga Nanayakkara SUTD
Festival of Learning Sayamindu Dasgupta
dg
truons: 911
8:o7PM 03/07/2012
21
8:O7PM 03/07/2012
13R
8:07PM 03/07/2012
201Z
8:07PM 03/07/2012
101Z
8:07PM 03/07-2012
s Projet ColSensei:
Just acollection of projects I worked on in the year of
2011.
to
vjiew
-My
8:o7P 0 7 2012
The Queen's New Clothes:
Inspired by the Danish fairy tale The Emperors
New Clothes and Lady Gaga's Orbit dress we
designed and implemented a costume. The
Queen's New Clothes, which appears plain to
naked eyes but exhibits changing patterns on
i"E 29i
-
8:07PM 0307,2012
Figure 4.1 The home page that a user lands on as soon as she signs in. On the left side
is a stream of project briefing of the user's own project, and on the right side is the market
of projects that other users have uploaded.
Please note that the upload time for all the projects on the market in the screenshot was
set to "8:07PM 03/07/2012" due to an inevitable database revamp during the recent development
process. It has caused the wrong display time for all the projects uploaded before the revamp,
but does not affect any projects that are uploaded after the database change. The 40% and 60%
division of home page was designed not for the look and feel of the home page, but rather, for
allowing the user to be able to engage with some of the core functionalities of the website
without having to navigate through multiple pages.
For any website design, the navigation
architecture is a key that can either help the users focus on their goal at the moment or confuse
the users with extraneous, unintuitive, or even illogical redirections. During the initial design
process before implementation, the author went through most of the major social networking and
knowledge-sharing websites, such as Facebook, Twitter, Linkedin, and Quora, and did repeated
comparison and analysis of both these websites'
page layout and functionalities.
The
37
comparisons and analysis led the author to the design decision that there was a need to reduce the
page navigation,
simplify navigation menus, and draw out the most frequently used
functionalities of a website to one place while balancing the information density through careful
aesthetic design. This home page of Llifelog gives a summary of the user's own projects and the
market's most recently projects, provides search functionality of projects and project creators,
and allows easy access to the user's history of project exchange. Admittedly, this design is not
perfect. For example, the author has been trying to come up with a better design for further
simplifying the user's history of project exchange but has not been able to fix on one that
convinces herself yet.
Both the 40% left side and the 60% right side of the home page allows the user to use
independent infinite scroll bar to retrieve more information. As soon as the scroll bar reaches the
bottom of the page, home.jsp sends an HTTPRequest to the backend asking for ten more projects.
AJAX technique is used here to make the user feel that there is almost a continuous retrieval of
more projects whereas in fact it is ten projects at a time. The number ten is also a result of
technical balance between page loading speed and the hardware configuration, among other
issues encountered throughout the development process. Additionally, AJAX is also used for
allowing reuse of the area where market projects are displayed on the screenshot.
The user's credit is displayed on the top right above the market. With each market
project's rating in display, this credit label gives the user a quick assessment of how many
projects she could potentially request from the other people.
When the user clicks on a project on the market, a collapsible <div> tag expands and
shows the preview of the project, as demonstrated in Figure 4.2. Preview is used throughout the
website for projects that the user does not have permission to view the full version yet, or for the
user to be aware of the amount of information she is sharing with the other people on the market
with regard to her own projects.
38
Circular sensor motion photography: removal of motion blur in arbitrary directions
Market Rating: 50 truons
Created By: Yosuke Bando
8:07PM 03/0712012
Motion blur is one of the sources of image degradation. It can severely attenuate high frequency
image content to an unrecoverable extent. if motion direction were known beforehand, one could pan
the camera to track that motion (follow shot). If not, or there are multiple objects moving in different
directions,
arbitrary directionsattenuateblurry
camera Imagecircular sensor motion
different directionsfrequency image content
- image degradationimage
middle imagemotion
sensor
blur
motion camera imagemotion direction
multiple objectsoptical axisprototype camera
standard camera imagetilted acrylic plate
unrecoverable extent
Presentation:
4
30
0
Content:
4
0
Your Rating: S
40
Yosuke Bando's Rating: 50
Li Bian:0
Figure 4.2 The preview of a project on the market. Each preview is composed of three
parts: the first 400 characters of a project's text description, the first three images from
the project documentation, and a WordCloud consisted of the key phrases from the
project's full text description.
The preview of a project is composed of three parts: the first 400 characters of the full
project text description, the first three images extracted from the full project documentation, and
a "WordCloud" that displays the key phrases from the full project description. The images
extracted from the full project documentation are algorithmically pixelated and resized to 100
pixel by 100 pixel for each image. Thus, in Figure 4.2 where three images are displayed for the
project, the preview indicates that this project's creator has included at least three images within
his project documentation for explanation. In Figure 4.3 we shall see an example where there is
only one image in the preview, which leaves empty space for two images. The lack of images
gives the users an indication that the actual project documentation will be mostly text. The
39
WordCloud is created by using a Natural Language Processing engine called AlchemyAPI [3]
provided by Yahoo. AlchemyAPI is a cloud-based text mining platform that uses deep linguistic
parsing, statistical natural language processing, and machine learning to analyze web content and
extract semantic met-data in order to provide semantic tagging.
To create WordCloud on
Llifelog, we use AlchemyAPI on the backend to scan through the entire document of each
project that users upload, extract key phrases from the documentation, and on the front end, we
create nested <div> for each extracted key phrase in order to resize the phrases based on their
frequency of occurrence in the full project documentation. The more key phrases extracted and
displayed in a preview, the longer the actual project documentation in text volume; the bigger
size of a phrase in the WordCloud display, the higher the frequency of occurrence of this phrase
in the documentation.
EyeRing: A compact, intelligent vision system on a ring
Market Rating: 10 truons
iyakkara
EyeRing: A compact,
intelligent vision system on
EyeRing is a wearable intuitive interface that allows a perse Creator's Rating: 18 truons
more information about it. We came up with the idea of a n Exchanged: 1 time
finger with a button on the side, which can be ...
0O2012
3ar
index
buttonfinger-wom configuration
future versionsindex finger
informationmicro camera
mobile phonemyriad
non-visual data capturepicture
possible appicationsSenSOrSthumb
uservisual formvisually impaired
wearable intuitive interface
FPresentation:
fl
Content:
4
0
Your Rating: 1
Li
19
19
Suranga Nanayakkara's Rating: 18
Bian: 0
Figure 4.3 The rating of a project depends on both the user's subjective rating when he
created the project documentation and the market's accumulative rating based the user's
exchange history with the other users of this project.
40
Preview is not the only channel through which a user gets a glimpse of the full content of
a project documentation. The market rating and number of exchanges for a project also indicates
to the user the popularity of the project. The market rating is calculated by dividing the sum of
all ratings for a project through successful exchanges by the number of successful exchanges. As
the example in Figure 4.4 shows, the project EyeRing was evaluated by the project creator at 18
Truons, where Truon is the unit of virtual credit in the system and 18 is the sum of the project's
presentation rating, 9, and content rating, also 9, given by the project creator, as we can see on
the two scroll bars. It has been successfully exchanged with someone once, and as a result, its
current marketing rating has dropped from 18 Truons to 10 Truons. When another user comes to
request for this project, she looks at both the preview-which contains only one image-and the
successful exchange and rating history of the project, and decides that she'd probably give the
project a rating similar to the previous user who has requested the project. The number of
images in a previous gives a hint of the project's presentation, and the WordCloud gives a hint of
the project's content. In the example above, the user who is requesting for the project EyeRing
agrees to give the project creator 9 Truons for the project content but is only willing to give 3
Truons for the project presentation instead of 9 Truons due to the lack of enough visual elements.
This will add up to a total sum of 12 Truons that she offers to the project creator. The user can
explain her rationale for her rating to the project creator or ask him questions about the project
through the private messaging box right below the rating scroll bars. When she presses "Submit
Rating", her rating as well as her message will get sent to the project creator, who can then
respond to the user accordingly. The Llifelog system facilitates such communication not only
through the Llifelog website messaging system but also by sending email notifications and
summaries of project request, inquiries, and comments to the users' email inbox.
The triangle buttons on both sides of the scroll bar are for changing the range of the
rating. Because Llifelog adopts a market approach for its project ratings by using virtual credits,
theoretically there is not upper limit for a project rating as there is for, say, five-star rating. At
the same time, the lower boundary of the rating is 0, since virtual credits can only be nonnegative.
Practically, on the market when the user can only view the preview of a project, her rating cannot
go above the project creator's rating, because otherwise that would create an efficient market, i.e.
the buyer is willing to give the seller higher price for an item than the seller is selling for.
However, since the purpose of the Llifelog platform is for project evaluation instead of trying to
41
fit projects into a market setting, the system gives the user a second chance for rating when she
successfully obtained the full content of a project, as shown below in Figure 4.5. The full
content of the project may contain detailed description of the research purpose, design,
implementation diagram, more information or outside links to the same project, and images of
the finished prototype. When the user obtains the full content of a project, she can then evaluate
the project again by giving a "full version" rating. The full version rating is also divided into
two aspects, presentation and content. The difference between a preview rating on the market
and a full version rating on the full content of a project is that there is no upper bound for the full
version rating for the evaluator whereas the upper bound for the preview rating for the evaluator
is the project creator's initial self-rating of the project. The evaluator also gets a second chance
to comment on the project. Upon the evaluator pressing on "Submit" at this stage, Llifelog
system sends the project creator a summary email about the project, which contains, from the
same evaluator, her preview rating, preview comments, full view rating, and full view comments
for comparison.
Figure 4.4 The display of full content of a project that the user has successfully obtained
from another user.
42
Ientrepreneur.
am an interaction designer, an engineer, and an
I like bilding things and make things
U Sian
Ii_chakuh@yahoo.com
happen.
Research Assistant
Mif Media Lab
617-335-3781
Projects
NJ
Figure 4.5 The project portfolio page of a user.
If the user is interested in viewing more projects by the same project creator, she can
either search the project creator's name on the home page or simply clicks on the project
creator's name on the project documentation, which leads her to the creator's project portfolio
page. A project portfolio showcases all the projects by one user, as that in Figure 4.5. A user
can click on each image extracted from the projects to view either the preview or the full content
of that project, depending on whether she has access permission to it. For example, a user can
view the full content of projects on her own project portfolio, or she can view the full content of
projects which she has already successfully exchanged for with her virtual credits. Figure 4.6
gives the screenshot of the former case.
Figure 4.7 shows a screenshot of the dropdown feedback box that allows the users to
communicate with the author more easily and efficiently than through emails or only through a
post-prototyping user survey. Every time when a user sends a feedback, the author gets it
43
instantly in her Gmail inbox. As a result, over the course of user testing, the author received
many emails from users in the targeted testing organizations across the globe. The emails
include reports of bugs which the users have identified on the website, complaints about the page
loading speed, suggestions for new features and how to further improve existing features, and
"thank you" emails. In the next chapter, we will discuss in details the user testing and user
feedback results.
Figure 4.6 Simple and easy browsing of projects on the portfolio page.
44
>ep
wmtkm
(rn?
VI
Figure 4.7 Built-in feedback system on the website for more efficient communication
with users.
4.2 Overview
The Llifelog website was hosted on an Ubuntu server with 1GB RAM and 25GB storage
space. Ubuntu is a free computer operating system based on the Debian Linux distribution. The
languages used to build the website include Java, MySQL, HTML, CSS, and Javascript, where
MySQL-My Structured Query Language-is the world's most used relational database
management system that runs as a server providing multi-user access to a number of databases.
The programming environment was Eclipse, a multi-language integrated software development
environment and an extensible plug-in system. Eclipse is mostly written in Java.
As Figure 4.8 shows, when a user interacts with a webpage of Llifelog, either by
uploading a project, sending a search query, or requesting for a project exchange, home.jsp, the
website's main JavaServer Page, communicates with Apache Tomcat, which sends requests to
MySQL database through Java servlets. Apache Tomcat is an open source web server and
servlet container developed by the Apache Software Foundation [5].
Apache Tomcat
implements the Java Servlet and the JavaServer Pages specifications, and provides a pure Java
HTTP web server environment for Java code to run. The Llifelog system uses Apache Tomcat
version 6.0. MySQL provides a query processing layer that pulls requested data from the
database and sends Java servlets back to Apache Tomcat, which sends the data back to home.jsp.
45
In order to render the webpage as desired with the data, home.jsp also pulls in data from four
files: homeini.js, a Javascript file that initiates the page, home.css, a Cascading Style Sheet that
determines the look and feel of the page, home.js, a Javascript file that sets the basic frontend
dynamic interaction with the user on the page, and item.js, a Javascript file that sets the dynamic
interaction between the user and projects on the page.
For easy development purpose, a simple interface, phpMyAdmin, was used to look up
data in the database. PhpMyAdmin is an open source written in PHP intended to handle the
administration of MySQL with the use of a web browser [24]. It can perform various tasks such
as creating, modifying or deleting databases, tables, fields or rows; executing SQL statements; or
managing users and permissions.
ltemjs
111feloa
user
I 4e0evit
-M&ySQLON
servlet
AM
..
4servIet
6.-
wow
Linux
J
Se rver!
Figure 4.8 An overview of Ilifelog web platform.
46
4.3 System Architecture
The Llifelog system
adopts client-server
architecture,
with two
distinct but
interconnected tires: the back-end, and the front-end. As mentioned in 4.2 Overview, the backend, which is composed mainly of a series of Java servlets powered by the open-source Apache
Tomcat server, interacts with the MySQL database through a JDBC (Java Database Connectivity)
driver. JDBC is an API for the Java programming language that defines how a client may access
a database. It provides methods for querying and updating data in a database.
The MySQL database itself contains the majority of the data stored on the website: the
users of the website are stored in a table called users, and the projects that these users create are
stored in a table called items. Each entry in the item table has a foreign key to the users table;
this foreign key represents the owner of the object. This allows easy lookup of all the items
belonging to a specific user through a simple SQL SELECT query.
The database also contains a table called categories, which represents the tags that are
attached to each project. The mapping of tags to projects is done in a separate table called
items tocategories, which is a table with an ID field and two foreign keys-one for the item
and the other for the category. This structure allows us to both look up projects associated with a
specific category and the categories associated with a specific project without switch tables.
A slightly more complicated structure is found in the "trades" table, which represents the
exchanges of projects among users on Llifelog. The trades table has two foreign keys for the
users' performing the project exchange, as well as another foreign key for the project being
exchanged, and some extra fields that provide meta-data about the on-going trade. These extra
fields include a field for the status of the trade, the notification preferences of each user, and the
different ratings and point values set forth by both the creator and the "buyer" of the project.
The trades can be augmented with messages in the trade-messages table, which has a tradeid
foreign key representing the trade to which the message pertains.
These messages are the
communication among users on Llifelog when they discuss about a particular project under
potential exchange.
The front end of the website is implemented through JavaServer Pages (jsp), which is
HTML for front end layout with inline Java code for server-side logic. Most of the dynamic
47
interactions are coded in Javascript using the popular open source library called JQuery [19], and
then loaded into the pages dynamically as needed. Each front end JSP page has a corresponding
Javascript file and a CSS file which define the dynamic interactions and styling that are possible
on that page, respectively.
One issue that all websites have to deal with sooner or later is the issue of loading timethe time it takes for the webpages on that website to load. On the web, the faster the pages load,
the more satisfied users will be with their experience on the website. Llifelog tackles the loading
challenge in several ways. First, all the Javascript logic and CSS stylesheets are kept in separate
files, so that they can be cached by the browser and do not have to be loaded on every page
reload.
Additionally, Llifelog employs a set of widespread techniques called AJAX
(Asynchronous Javascript and XML) to decrease loading time for the users of the website.
AJAX programming takes into account the fact that when a user is browsing the website, a lot of
the things on each page do not need to be reloaded each time the user sends a request-they are
just static content.
When performing different search queries on the project database, for
example, there is no need to reload the whole webpage and send request for all the external style
sheets and Javascript files all over again.
Instead, we can make a specific Javascript
XMLHttpRequest to the server to request just the desired data, and then dynamically update the
website with a callback function that gets called whenever the data is fetched from the server.
Llifelog employs these AJAX techniques to speed up the "feel" of the website and to
reduce discontinuity between users' actions and their effects. For example, on the home page, the
search results are loaded completely via AJAX. The same holds true for the all of the different
project request views and even the profile edit view; the use of AJAX simply speeds up the
interaction by avoiding all extraneous header and Javascript/CSS file requests.
48
M
Evaluation
"An approximate answer to the right problem is worth a good deal more than an exact
answer to an approximate problem."
--John Tukey
5.1
User Testing
In Chapter 3, Design, we gave a brief overview of the six months development and user
testing process of the Llifelog system. In this chapter, we discuss the detailed results from user
testing and user feedback, as well as analyzing the usage data we have collected.
As mentioned before, the Llifelog system was developed and tested in two quick
iterations within 6 months. In the first round iteration, the system was not completed yet. As a
result, the website was made public for only one-week testing among 120 users, with 2/3 of the
users from MIT and 1/3 of the users from Hisense based in Qingdao, China.
The website
allowed the users to exchange any type of life experiences with one another using virtual credits.
The goal of this round of user testing was to find out whether people would be able to measure
and evaluate digital information quantitatively through a market-based exchange platform.
Users expressed a lot of confusion, as expected, but also gave many good suggestions that helped
the author narrow down the scope of the site content and user base in preparation for the second
round user testing.
The second round user testing involved 305 registered and verified users as recorded in
our system database. Accordingly to Google Analytics, however, there were 690 unique visitors
49
who visited the Llifelog website 1318 times with 4362 pageviews during the period of February
3 to March 3, as shown in Figure 5.1. One possible reason for this discrepancy in the number of
"users" is that Google Analytics relies on each unique IP address for counting "unique visitors",
whereas in the Llifelog database, we count each unique user through verified school or corporate
email address as well as other personal information. A closer investigation in Google Analytics
also reveals that the users have been accessing the website using various mobile devices beside
their laptop or desktop computers. Figure 5.1 gives a clear visualization of the daily traffic in the
30-day period. One thing to note is that the official user testing for the second iteration lasted
from February 3 to February 25, when the author actively advocated this website within her
social network. In Figure 5.1, we see a surge of traffic on the website toward the beginning of
March when the author has stopped advocating the website for a week; this is possibly due to the
network effect that let the word spread out.
Feb 3, 2012 -Mar 3, 2012 -
Visitors Overview
Advanced Segments
f
EmaUSETA Export-
Add to Dashboard
100.00% oftotal visits
overview
Visft -
vs.
Select
amtric
Hourly DWy Week Month
0 Visits
200
100
Feb8
Feb15
Feb22
Feb29
690 people visitd Usi sie
d\~~J Visits:1,318
,4/\.-J Unique Visitors: 690
/4M.
Pageview: 4,362
PageeMVit: 3.31
V.%A
' Avg. Visit Duration: 00:05:47
m 50.23%
New vso
=Vzawt
U 49.7.
Returning visitor
Bounce Rate: 48.25%
Figure 5.1 Website traffic monitored through Google Analytics.
50
Figure 5.2 lists the geographic distribution of users during the second round user testing.
As we can see, over 60% of the users were from the United States, about 35% of the users were
from China, and the rest from various parts of the globe. The wide spread of the geographic user
distribution added to the effect of information gap and weak social bond, which affirms the need
for a market-based platform for project exchange, instead of a purely favor-based one.
countrylTerrory
Visits
4,
Pagesvsit
Avg. Visit Duration
% New
viait$
ounceRate
1. United States
894
3.71
00:06:21
4508%
49.55%
2. China
383
2.56
0005:01
59.79%
4334%
3. Canada
9
1.33
00:00:30
66.67%
77.78%
4.
8
1.38
00:01:08
50.00%
62 50%
5
2.00
00:0053
60.00%
60.00%
United Kingdom
5. Brazil
6.
Hong Kong
3
1 33
00.00:07
10000%
66.67%
7
India
2
2.00
00:01:43
100.00%
50.00%
8. Japan
2
200
0000:19
100.00%
50.00%
9. Norway
2
3.50
00:01:32
50.00%
5000%
10. Taiwan
2
1.50
00:01:54
50.00%
50.00%
Showrows: 10
Goto: 1
1- 10of 18
Figure 5.2 Geographic usage monitored through Google Analytics.
Among all these users, about 2/3 were university graduate students and research staffs
and 1/3 were from the industry, such as Hisense, Microsoft, Nokia, and Google.
The 2/3
academia users contributed 95% of the research projects uploaded on Llifelog website and only a
few projects were shared by the industry users. In the user testing process, many users expressed
concerns over the confidentiality of their projects; and this is especially the case for the industry
users, who sent the author feedback throughout the testing that they loved using the site to
discover what the academia had been working on but were hesitant to share their projects which
were likely to become their company's revenue stream. About 50% of the academia users who
shared their research projects also sent the author inquiries regarding the confidentiality or IP
issue.
Besides responding to the users about their concerns, the author quickly added new
features on the website during the user testing period to make the users feel more comfortable to
share their projects there. For example, after a user finishes documentation of a project, he can
51
click on a "preview" button to view what other users on the market can see about his project, and
can modify his documentation accordingly if he wants. When a user receives a project exchange
request, he can also hover over the project creator's name to view the project creator's
professional background such as school or company, and geographic location. In about 10 cases
when the author was observing the users using the website in person, the author noticed that
these users almost always hovered over or clicked on the other users' name to find out more
information about them before making a project exchange decision. Most of these users also
clicked on the "preview" button to see what they had shared on the market, and in a few cases,
the users went back and added more content to their documentation to make a richer presentation
of their project preview. Such targeted sharing mechanisms give the users more control over
who they want to share project information with and what they want to share. Although the
project previews appear to be an encryption tool of projects, creating project previews and
allowing users to control what others see about their projects through these previews encouraged
more project-sharing on the website.
5.2 Data Analysis
There were 305 unique users visiting the Llifelog website during the second round user
testing period. Among them, 32 users uploaded 74 research projects, with 42 projects being
exchanged at least one time. The total number of exchanges is 139. Thus, the content producer
ratio on Llifelog was:
(32 / 305) * 100% = 10.5%
The average project upload rate per content producer was:
74 / 32 = 2.3 project / user
The average project exchange rate among all the projects on the website was:
(139 / 74) = 1.9 exchanges / project
In Chapter 4, Implementation, we discussed in details the three components of each
preview, which was automatically generated for every project that a user creates on Llifelog
52
website. One of the most indicative components is the image extracted from each project. If a
project has at least three images in its full content, three images will be displayed in the preview;
if a project has less than three images in its full content, 0, 1, or 2 images will be displayed in the
preview. Because images are often an essential visual element that indicates the presentation
quality of a project as well as the stage of progress of the project, we compare the number of
exchanges of each project with the number of images contained in each project's preview in
Figure 5.3. Due to the fact that there can be multiple factors affecting a user's project exchange
decision, such as the number of images in preview, the project's rating, and the private messages
exchanged, in Figure 5.3 we only use the data from projects that have been exchanged at least
once, i.e. the 56.8% of all the projects uploaded onto the website. The smaller dots represent 1
or 2 projects and the bigger dots represent 3 or 4 projects at the same coordinate, since most of
the exchanged projects fall into the number of exchange range from 1 to 4, with two exceptions
exchanged 8 and 9 times respectively. As we can see in Figure 5.3, the more images there are in
a preview, the higher the average exchange rate of a project-as indicated by the blue curveand the higher total number of exchanges of projects as indicated by the red line. There is a
decrease in the average exchange rate from no image to one image due to extreme low initial
rating of projects with no image. We will show this through Figure 5.4.
# of exchanges 9
9 8
0
7
6 5 4 -
1
0
0
1
2
3
# of images in preview
Figure 5.3 Number of exchanges versus number of images in a preview for a project.
53
Figure 5.4 is a visualization of the rating for all successful project exchanges.
The
horizontal arrows indicate that the project requesters and project creators agreed at the same
rating initially set by the project creators. The vertical arrows indicate that the rating of a project
dropped from the project creators' initial rating (the arrows' starting point) to mutually agreed
ratings (the arrows' ending point). All the arrows are facing downward because the system was
designed in such a way that no project requester's rating will exceed the project creator's initial
rating, lest there be an inefficient market.
As we can see in Figure 5.4, when the rating for a project is lower than 20, the number of
images inside a preview has almost no effect on the exchange rate.
In fact, many projects
without any images in a preview but with ratings under 20 were successfully exchanged. So we
can say that the users' exchange behavior was inelastic to the quality of a project, when a project
rating is sufficiently low and does not cost the users much to obtain. We also see an exponential
decrease of rating drop as the number of images increases in the preview. This indicates that the
higher the initial rating and the fewer images in the preview for a project, the more dramatic drop
of rating upon a successful exchange. This implies that the users do care about the quality of a
project and behave reasonably rationally in terms of using the market rating to measure the value
and worth of a project based on their estimation of the project quality. As the number of images
increases in a project preview, we see more "stable" project exchange in the region from rating
40 to 100. A stable project exchange is when the project was successfully exchanged at its initial
rating set by its project creator. After manually going through almost every project's full content
on the Llifelog website, the author observed that most of the projects successfully exchanged at a
rating between 40 and 100 include 2 to 5 images and about 2000 to 2500 words. The author also
noticed that most of the projects that had below 20 ratings fell into two categories, first, they had
very little content-so that the project creators probably did not feel confident enough to give
high evaluation themselves, second, they were created very early on during the user testing-so
the initial users were probably still trying to figure out how to use the system and waiting to get a
few quick back-and-forth interaction with the other users on the market. But as time goes on, by
mid-week in the second week of the user testing with about 150 users on the website and 1 or 2
project upload per day, the users started getting familiar with this market environment and
getting to learn how to rate each project better with virtual credits.
Thus we started seeing a
convergence of rating versus project quantity and quality then.
54
Scanning through all the private messages among the users engaged in project exchanges,
the author found that for projects with rating lower than 20, there was very little conversation, for
projects with high rating but relatively fewer images and phrases in the WordCloud, the
messages seemed to be focused more on bargaining for a mutually-agreeable rating, and for
projects whose rating fell in the range from 40 to 100, the private messages were mostly focused
on discussions and inquiries of the content of the project. For example, "Do you have X-Ray
specs?" "Interesting application, may I look at it?" "Hi? How to know the details? Thanks!"
A comparison of rating and messages exchanged at preview versus rating and comments
left at full content for each project reveals that in only a few cases did the user change rating
after receiving the full project content, when the project's preview rating is within 40 to 100
Truons. Most of the users did not change the rating, or seem satisfied with the fact that what
they got was expected. Outside the range of 40 to 100 Truons, however, whether it is rating
below 20 or above 150 Truons, the user comments suggest that the preview did not do a good job
conveying the quality and quantity of the project documentation. For example, for projects with
initial rating below 20 Truons, some users requested them immediately only to be disappointed
at how little description there is. This indicates that there is still much space for tweaking the
preview to ensure accurate depiction of the actual content.
Project rating
200
180
160
140
120
10080
40
20
0
0
-
.
- - -.
.-.-.
1
2
3i--------------------
2
3
# of images in preview
Figure 5.4 Project rating versus the number of images in a preview for a project
55
Although it is common for people to try to game markets like this, and the existing
prototype does have many loopholes and shortcomings, it is fortunate to note that out of 305
users, only two people seemed to have tried to game the system by exchanging projects among
themselves in an attempt to horde virtual credits. Unfortunately for them, the only way to horde
virtual credits in the system right now is by creating multiple accounts per person and the email
verification process in the system has prevented most people from signing up with multiple
school or corporate email addresses. Another phenomenon that could theoretically happen is
users reselling other people's projects that have already been sold on the website. However, this
has not happened on the website so far, possibly due to the academic integrity and social
reputation concerns of the targeted testing user base.
5.3 User Feedback
After finishing the two-round iterations, the author conducted a survey (Appendix A)
among Llifelog's existing users to further understand the users' background and determine the
usefulness of the website to them. 20 users responded to the survey. The questions cover six
categories of user bahavior- frequency of web use, response to market rating system, motivation
for research, vehicle for project evaluation, time used for similar project search and Llifelog user
experience. Through analysis of the user feedback, the author concluded that the website found
its target audience and met the needs of the target audience, despite barebones set up.
All of the people surveyed reported to be web users who had used Facebook, Google,
Linkedin, Youtube, Amazon or their Chinese equivalent. 70% used Facebook, Youtube and
Google at least once a day and most of them spent more time browsing than posting. Thus, the
people who were using Llifelog for project exchange were already well-integrated into the digital
world.
80% did not find that any of those platforms would be good places to post their research
projects. 70% found the feedback systems in sites such as Facebook and Youtube to be lacking
in objectivity. They found these sites ineffective in giving relevant feedback besides personal
support. One user wrote: "I nearly never post researchprojects on Renren [Chinese Facebook]
because I have only friends there, not professional connections.
We only share fun and
56
emotional things; the occasional times when I've posted work photos I only expected others to
say 'cool!' but did not expect any discussion." Llifelog market system allows users to evaluate
objectively how much a project is worth and not merely express support for a friend's experience.
All users wanted evaluation and feedback on their projects but found that the traditional
conference publication peer review method insufficient. Only 10% selected peer review as their
method for evaluation and feedback. Because creating, evaluating and improving projects did
not allow them to receive timely feedback from the paper peer review process-as papers are
published after the project is polished, 90% received feedback and evaluation on their projects
through user research, study, and testing and talking to advisers, colleagues, and friends.
Projects aimed for demonstration or commercialization purposes do not fit the traditional mold
of conference publication peer review because projects need timely review and usually quick and
multiple rounds of development iterations.
On the goal of research, 60% of the surveyed users reported that it was because they
wanted to create project prototypes, and among the rest the author found a variety of motivations
including calling for attention for their research field, being able to share their research findings
and becoming pioneers in their research field. None wanted to publish papers as his or her main
objective, but 20% wanted that as their second or third. Creating demonstrative projects that
demand effective feedback and sharing systems require a new platform that is supplied by
Llifelog.
100% of the survey respondents reported that they had not used any system similar to
Llifelog for exchanging research projects before. 1/3 found the website to be "a little bit easier",
1/3 "no difference", and 1/3 "a little bit more difficult", when asked to compare Llifelog's
existing search system with Google search for research projects. This implies the lack of critical
mass of content on Llifelog, as well as Google search's low relevancy on finding research
projects aside from the ones that already have paper publications.
Among the survey
respondents who have used Llifelog's private messaging system during project exchange, 50%
found it to be "very efficient", 40% found it "somewhat efficient", and 10% found it no
difference from existing peer review system for paper publication. A key improvement for the
system to be more satisfying for all the users, as the survey suggests, is gathering sufficient
57
number of users and projects. One user recommended, "Considering linking to external sources
for projects." Adding a recommendation system was also on top demand from users who wanted
even more targeted research results on top of the existing vertical search system.
58
E
Conclusion and Future Work
"Logic will get you from A to B. Imagination will take you everywhere."
- Albert Einstein
In this thesis, the author presented Llifelog, a market-based web platform for research
project discovery, evaluation, and exchange.
The goal of the Llifelog system was to cultivate a
community of critique among students and researchers in both the academia and the industry on
their work. Those who engage in the market answer the call "find out how your work, projects,
and ideas are valued in the community."
The motivation and inspiration that lie behind this
project came from the author's crossroad engagement with the research academia and industry,
as well as curiosity amongst the tension of information value proposition. A web prototype was
built, deployed, and iterated twice among 305 students and researchers from all over the world in
the course of six months with thousands of site visits.
Natural Language Processing, image
processing, and web development techniques such as AJAX were used in composing the core
functionalities of the website.
Through the Llifelog website, the author observed users' behavior and interaction with
one another in real time on this information market focused on research projects. The observed
user behavior pattern shows that it took some time for the users to get familiar with the
mechanisms in the system before they were able to assign quantitative value to the digital
documentation of research projects. During the
2
"dround user testing period, the first week and
half seemed to be the most confusing time when users gave random ratings to projects and it was
difficult to draw any correlation between these ratings and the quality of the projects.
Soon
59
afterwards, however, we started seeing a converging trend in this quantitative evaluation process.
Specifically, most projects with at least 3 images and a text description at 2200 to 2500-word
range were rated with a score between 40 Truons and 100 Truons. A private messaging system
allowed users to communicate with one another about the project quality, content, and rating. It
was observed that most of the conversations happened for projects with rating between 40
Truons and 100 Truons. Projects with initial rating below 20 Truons by the project creator were
usually requested without any bargaining between the creator and the viewer. In addition, the
higher a project's initial rating, the sharper the project's market rating drops after exchange.
One of the challenges in creating Llifelog was devising a project preview that can
accurately convey to the users the content and presentation of a project yet does not reveal too
much information, lest the users lose interest to exchange with virtual currency. Based on user
testing data, we can see that the current preview in Llifelog seem to provide users sufficient
information for the full content, when a project's initial rating is between 40 Truons and 100
Truons. However, when the rating is outside this range, such as below 20 Truons or above 150
Truons, the comments that users left after having received a project's full content suggest that
full content did not meet their expectation. This indicates that there is further space to tweak the
project preview so that it helps the user reach an adequate level of expectation in all project
rating cases. In a few cases, users gave higher rating to a project after having received the full
content. Due to the limitation of user testing data, it is difficult to draw any conclusion from this.
However, these rare cases might deserve further study with more user testing, since in real world
market, people generally do not give more money to a product than what was asked from them.
The current Llifelog system only allows exchange with virtual currency; it would be interesting
to see how users' rating behavior changes, if it does, when the same system allows users to
convert virtual currency with real currency. Additionally, while most of the user behaviors seem
to form a correlated pattern for exchanging projects rated within 40 Truons to 100 Truons range,
it will be necessary to test if this was affected by the initial 500 Truons given to every user upon
signing up and if this range still exists when the system gives every user different amount of free
Truons to spend. Nonetheless, within the 40 to 100 Truons range, we see a positive correlation
between the amount of information revealed in previews of projects and the projects' market
rating.
60
From the user feedback conducted in the end, we see that the majority of students and
researchers working on projects constantly value the importance of feedback on their work, and
that existing content-sharing social networking platforms such as Facebook and Linkedin do not
provide an environment suitable for research project sharing.
Although the current Llifelog
platform does not fully serve users' need for such sharing, it is evident that a platform with more
standardized evaluation mechanism and more efficient communication and exchange features is
in demand.
Although the current thesis work shed light on a future promise of wider information
market, much remains to be done in order to further validate the patterns observed from the
existing system. A critical mass is yet to be reached before we can generalize the insights across
the information spectrum and among different user groups.
To aggregate content onto the
website more efficiently, linking to external sources and allowing networked sharing could help
with the viral growth of the market. For continued iterations, machine learning techniques could
be adopted and applied in project search, recommendation, ranking and preview generation, in
order to keep up with the veracity of market dynamics.
61
Bibliography
[1] Academia, http://academia.edu/.
[2] Adamic, L.A., Lauterbach, D., Teng, C. and Ackerman, M. Rating Friends Without
Making Enemies, AAAI Publications, Fifth InternationalAAAJ Conference on Weblogs and
Social Media, July 2011.
[3] AlchemyAPI, http://www.alchemyapi.com/.
[4] Amazon, http://www.amazon.com/.
[5] Apache Tomcat, http://tomcat.apache.org/.
[6] Ask.com, http://www.ask.com.
[7] Barter, http://barter.mit.edu/.
[8] Brand, S. The Media Lab: Inventing the Future at MIT, Penguin Books, 1987.
[9] CouchSurfing, http://www.couchsurfing.org/.
[10] DepthJS, http://depthjs.media.mit.edu/.
[11] Diaspora, https://joindiaspora.com/.
[12] Epinions, http://www.epinions.com/.
[13] Facebook, http://www.facebook.com/.
[14] FourSquare, https://foursquare.com/.
[15] Geeklist, http://geekli.st/beta.
[16] github, https://github.com/.
[17] Google Analytics, http://en.wikipedia.org/wiki/GoogleAnalytics .
[18] Google Answers, http://en.wikipedia.org/wiki/GoogleAnswers .
[19] JQuery, http://jquery.com/ .
[20] Kickstarter, http://www.kickstarter.com/.
[21] Levels of Sponsorship, http://media.mit.edu/sponsorship/funding-levels .
[22] Linkedin, http://www.linkedin.com/.
62
[23] Media Lab China Initiative, http://china.media.mit.edu/.
[24] PhpMyAdmin, http://www.phpmyadmin.net.
[25] Quora, http://www.quora.com/.
[26] Reality Mining, http://reality.media.mit.edu/publications.php .
[27] Reas, C. and Fry, B., Getting Started with Processing, MIT Press, August 2007.
[28] ResearchGate, http://www.researchgate.net/.
[29] StackOverflow, http://stackoverflow.com/.
[30] Teng, C., Lauterbach, D. and Adamic, L. A. I rate you. You rate me. Should we do so
Publicly, WOSN'10 Proceedings of the 3rd conference on Online social networks, Boston,
MA, 2010.
[31] Wikipedia, http://www.wikipedia.org/.
[32] World Economic Forum, Personal Data: The Emergence of a New Asset Class, Jan. 2011.
[33] Yahoo Answers, http://answers.yahoo.com/.
63
Appendix
Final Round Survey Questionnaire for Llifelog
A Market-Based Web Platform for Discovering, Evaluating, and Exchanging Research Projects
The purpose of this survey is to better understand the context of user behavior on information sharing platforms,
and the rationale of users sharing and rating information. The goal is to draw insights from such collective
feedback in order to improve an existing market-based web platform that we have built for discovering, evaluating,
and exchanging research projects within the academia and industry. Allfeedback response provided through this
survey will be kept confidential and only used for analytic purpose for this academic study. Any personal data
provided will be kept anonymous.
Tell me a little bit about you.
1. You are a
o graduate student/ academic researcher
2.
Oother (specify)
How old are you?
o Oto 20
3.
0 industry researcher
0 20 to 25
025 to 30
030 to 40
040 to 0 5
0 0above
You are
0 male
O female
4. You mainly do your research at
o office/School
Ohome
Usage of related web platforms
5. Which one of the following do you use? (Check all that apply)
E Facebook
E Google+
E Twitter
0 YouTube
E Linkedin
E Amazon
E Renren (AA)
E Sina Weibo (,9krd
t)
64
E YouKu ('Uii)
EDTaobao (N0)
El none of above
6.
Please check the frequency at which you use/visit the above websites.
a few times a
a few times
a few times
a few times
(almost)
day
a week
a month
a year
never
Facebook
Google+
Twitter
YouTube
Linkedin
Amazon
Renren
Sina Weibo
YouKu
Taobao
7.
How would you describe your activities on these websites? (Check all that apply)
50% time posting;
20% time posting; 80% time 80% time posting;
browsing others' posting
20% time browsing
others' posting
50% time browsing
others' posting
Facebook
Google+
Twitter
YouTube
Linkedin
Amazon
Renren
Sina Weibo
Youku
Taobao
8.
How often do you rate other people's post, such as "like" or
"+"
someone's status, give
5 stars to an item for sale, write a customer review?
o frequently
0 occasionally
0 almost never
9. How often do you get rating from other people on your post?
0 frequently
0 occasionally
0 almost never
65
10. How indicative do you find the ratings regarding the posts' content quality on social
networking sites such as Facebook, Google+, or Twitter?
o very objective
o somewhat subjective
0 somewhat objective
0 very subjective
11. How likely will you post your research projects on these platforms for documentation
and sharing purposes? Why?
O Unlikely
O maybe
O Very likely
Reason:
Tell me a little bit about your research
12. What motivates you the most in your research? (please rank the following, start from 1
being the most important)
0 building a working prototype for demonstration
0 seeing the commercial impact of your research
E getting paper publications
E call for attention for a research field or being able to share your research findings
E become the pioneer in a research field
13. On a scale of 1 to 10, how much do you care about evaluation and feedback to your
research? (1 being the least and 10 being the most)
1
2
3
4
5
6
7
8
9
10
14. How do you usually receive evaluation and feedback on your research? (Check all that
apply)
E peer reviews in the process of submitting papers to conferences
E user research, study, and testing
0 talking to advisers, colleagues, and friends
0 I usually don't get / ask for evaluation or feedback on my research
15. How much time do you usually spend finding out relevant, related, or similar work to
that one you are working on?
0 a few months
0 a few weeks
0 a few days
0 a few hours
0 no time
66
16. How much do you care whether someone has worked on / is working on something
similar to what you are working on? (1 being the least and 10 being the most)
1
2
3
4
5
6
7
8
9
10
Usage of Llifelog system (URL: Llifelog.media.mit.edu)
17. Have you used something similar to the system before for sharing research projects?
0 yes (please specify
f)
o no
18. How easy was it for you to find projects of interest using this website's search feature
comparing with, say Google, or your university/company's project listing page?
o much easier
o a little bit easier
o no difference
o a little bit more difficult
o much more difficult
19. Have you used the website's one-on-one private messaging system? If so, how efficient
do you find this communication method in terms of finding the creator of a project,
sending inquiries, exchange project feedback?
o very efficient
o somewhat efficient
o no more efficient than existing paper review systems
o somewhat less efficient
0 very inefficient
o I haven't used the website's private messaging system
20. Have you used the website to find projects from universities/companies other than your
own? If so, how useful or important do you find being aware of other institutes'
research through this website?
o very useful / important
o somewhat useful / important
o somewhat not useful / important
o not useful / important
0 I haven't found projects from other universities / companies on the website
67
21. The web platform is still at its early stage of development and more user data and
projects can definitely provide more value and better user experience for you. Imagine
if we had gathered millions of user data and projects already, what improvement would
you like to see on the website? What new features / functionalities would you suggest?
68