CS8803-ADVANCED INTERNET APPLICATIONS AND DEVELOPMENT
[APARNA RAMAN : 902537338]
[ARVIND SUBRAMANIAN : 902423426]
[RADHIKA PARTHASARATHY: 902500361]
[RASHMI SUBRAMANIAN : 901973190]
MOTIVATION :
No girl would be caught dead shopping alone for anything even as small as a
toothbrush! You always need to have a buddy to tag along and offer expert
opinion on what you're buying and whether the price is right and whether you
could get something else instead! So why not have a shopping buddy online too? And why not take
that one step further and let your buddy make suggestions for your shopping list and tell you where
you could get what you're looking for?
WebGenie is an application which does just that. Extracting information from your wishlist, it
suggests what you could or should buy, where you can get it and displays reviews and information
for the same. One could say that WebGenie is for products as Pandora is for music. Using heuristics,
WebGenie gets you what you want and tells you where to find it. With Earth's Biggest Selection,
webGenie has everything you could possibly ask for--so ask for it! It's easy to create a list of what
you're hoping for: as you shop, click the add button on the page of any item that strikes your fancy.
Make the list publicly searchable and presto--you're simple to shop for.
What we set out to do
Find out what the user wants
Tell him where to get it
Tell him what his friends want
2
EXISTING APPLICATIONS :
Applications available currently on social networking sites work on the Push—paradigm,waiting
for the user to enter products of interest and allowing the user to rate and review the items they
add. Applications such as My Listo on Facebook allow users to maintain lists of items they are
interested in owning and to see what their friends are interested in. Reviewing of items is also a
feature present in some applications but it is mostly limited to reviews within the social network.
But these applications need active triggers from the users to be of use. They do not intelligently
determine user tastes or provide suggestions based on such analysis. Shopping sites like Amazon
and eBay suggest related products to the user. However, this is based on the purchase made by the
user and other products purchased by buyers of that product.
FEATURES :
The two special features of our application are
•
Wishlist Monitor
•
Dynamic Community Watch
Wishlist Monitor –
The members can add this application to their face-book profile, which lets us pull all the
information from their profile. Now we perform the basic functions of the wish-list such as user can
manually add and remove items from the wish-list. The user can edit their profiles. They can add
items to their wish lists. The items added in the public list can be viewed by all the users who have
added our webgenie application. The Friends List items can be viewed by all exactly like public
pages. They can also search for the friend and view the friends wish list. When one wants to look at
a particular friend’s wishlist , the application allows you to click on the friend’s tab, which would
take him to his friend’s wishlist . You can see the current deals related to the items on the friend’s
wishlist and could buy his most favourite gift at the best deals offered. As of now, the search engine
has been set in such a way that it gives us the top 5 – most highly rated best-seller + best
price(lowest –highest).
3
Dynamic Community Watch This feature is the core of the feature, which would tell the user what he might like instead of user
telling the system. Here we analyse the user’s profile, crawl through his communities and find the
user’s interests. This is fed into the search engine as keywords and the best deals from the market
are presented to the user. This would make a casual facebook user to perform an exciting shopping.
Since the results are basically automated, the user may like it or not like it. So we give him an option
by which he could either increase the probability of his favorite results being displayed by clicking
on “I like it” button and reduce the probability of results he did not like by clicking on “ I don’t’ like
it” button. This trains the system according to user’s taste.
4
ARCHITECTURE :
This shows an abstracted level of the system architecture. The WebGenie application talks to the
Facebook API and extracts user information from it. This information is now used to suggest
possible products of the users choice. These keywords are stored in the SQL database. Now these
keywords are retrieved and searched for in the Amazon and eBay API and presented to the user.
5
The above figure zooms into what makes up WebGenie. The user interface is used to take inputs for
the user’s wishlist as well as display his wishlist and suggestions. The wishlist manager manages
the addition and deletion of items from the user’s wishlist. The suggestion engine intelligently
analyses user information and suggests products of the user’s choice. Finally the deal finder talks to
the APIs and presents the user with the best matches currently available for that particular
keyword.
IMPLEMENTATION
We have used Facebook application development API to implement our project.
Facebook offers two different ways of rendering applications using their API •
Iframe
•
FBML ( Facebook Markup Language)
6
We chose to use Iframe application development because it enables development on localhost,
which was not the case in FBML. We identified that development on Localhost would be the best
way to prototype and experiment faster. We also observed that the functionalities provided by both
these methods were the same.
Platform : The development environment was windows- vista /Apache/MySQL/PHP
Firstly, the Facebook client is installed on the webserver. The server code that we develop uses the
Facebook client internally. When we register our application with Facebook, we are provided with
an application ID, and a secret key. This are passed as parameters when
The server code is organized in the following fashion internally :
index.php : This has the UI Javascript code of the application homepage, which includes Tabs for
seeing the user's wishlist, and the user's friends who have added the application. The functions to
handle addition /removal of items in the wishlist are present in this file ( addclick(itemname) ,
removeclick(itemname) . Also present are the FQL( Facebook Query Language) code to extract the
user's friends who have added our application, and the MySQL queries to extract the items present
in a user's wishlist. When the user clicks on application homepage, the request is handled functions
present in ebayproductpage.php
ebayproductpage.php :
This file contains the implementation of the Wishlist manager and Suggestion Engine . These
encapsulate calls made to the API of the Ebay.com and SemanticHacker.com.
WishList Manager :
ItemSearch(itemname) accepts a product to be queried and searches the Ebay , with the search
tagged by , biased towards best price by it returns the maximum number of items. In our
application, we chose to display the top five results.
Suggestion Engine :
This uses the function ItemSuggest1()
7
This function implements the following algorithm :
1. Extract the following information from the user's profile.
1. Favorite books
2. Favorite movies
3. Favorite music
4. Interests
5. Group names of which the user
6. Group description
2. This information is analysed semantically and keywords are extracted from the data.
3. These keywords are inserted into the database.
4. Each keyword is assigned a weight of 5.
5. Choice of keyword
1. A random number between 1 and 10 is generated.
2. If this random number is below 3 a keyword of weight < 3 is selected.
3. If the random number is above 3, a keyword of weight > 3 is selected.
6. The selected keyword is used to query eBay and the results are displayed.
7. The user likes or dislikes the suggestion. Based on this the weight of the keyword is
incremented or decremented respectively.
8. When the weight of the keyword reaches 0, it is removed from the database.
SEMANTIC SEARCH API
In our project we have used the API provided by semantichacker.com . This provides the ability to
do extract the following :
a) Concepts – Given a text input, it extracts the important keywords that describes the content
best
b) Similarity Search : Finds out semantic similarity among paragraphs of the same text file, or
across text files
c) c)Categorization : Finds out the best category to describe words that contained within the
text.
In this project we have used (a) - extracting concepts from a text. To the API, we pass the group
names, and group descriptions of all the groups that a user is a member of.
8
While hakia also gives a similar semantic search feature, we found the semantichacker's api to be
better suited to our purpose.
EBAY API
The eBay API is used to provide links to products the user might be interested in purchasing. We
use the BestMatchPlusPrice search to provide optimum relevance as well as price. Also, the
number of results displayed is attenuated to 5 to ensure a not so cluttered interface.
Database Design
The database consists of a set of two tables associated with each user. The first is the wishlist table
and the second is the keywords table.
Table: wishlist
Used to store the wishlist associated with each user registered with our appplication
Fields:
uid – User ID
itemname – Name of the item as entered by the user
Table: keywordrank
Used to keep track of keywords accociated with a user and the ranks he/she has assigned to them
Fields:
Uid – User ID
Keyword – Keyword associated with that user
Weight – Weight of the keyword
9
EVALUATION
Methodology
A number of different users were used as a sample space for the evaluation of our project. The
variation in users was used for different amount of profile information. Thus the more data a user’s
profile has, the higher the response time of the application owing to the time taken for analyzing
and organizing the keywords specific to the user’s tastes.
The performance of our facebook application, disregarding network bandwidth issues, depends
on the following :
1) Performance -response time of FQL queries
2) Performance of API calls to the Semantichacker
3) Performance of API calls to Ebay
4) Number of groups which a user is a member of / length of Group Description
To minimise the database query time on application server , we decided that the table would be
populated with keyword tags from the user's profile , the first time when he uses the application .
This is because the API calls to semantichacker took a relatively long time to process (in the order
of 5-8 seconds) . The application would asynchronously crawl and populate the changes made to
the user's profile over a period of time. Hence, though the first access was costly, the subsequent
accesses of the user to the application would be addressed faster ( sub-second)
To observe the impact of performance effects to API calls of Ebay, we conducted the following
experiment .
•
At a regular interval of time, we sent queries returning different number of elements (5,10,
20, 50,100,200,500) .
•
For each value of the number of results, 4 requests were made , to level out the random
delays. The average of these values gives the delay happening for each value of query size.
•
When we plotted the following graph [Fig 2] we found that there was variation with the
time of conduction of the experiment. However, we found that there was not a great
variation among the values we observed.
10
The time taken includes the time of retrieval and the time of rendering the output on the user
screen
ITEM SEARCH -- Fig 2 ( X axis : query return size , Y axis : Response time in)
INITIAL LOGIN AND SETUP TIME WITH WEB GENIE Fig 3 : X axis time in seconds
Fig 3 pertains to the analysis of time response of semantichacker api. We observed that
contrary to the expectation that large text would imply larger time to process, the api returned
with more or less constant delay , of about 8 seconds. This delay occurs only once when the user
is a new user, and his profile details/ group membership details are not processed semantically.
To do this, we first deleted from the table keywords pertaining to the logged-in user. Then,
11
when the user clicked on 'I'm feeling lucky', it forces the invocation of the semantic hacker API,
fed in with group descriptions, and user interest profile information. Different users had
different number of groups when the experiment was conducted. User 1 had joined no groups.
The other users had 4,6, 10 groups. However, the difference among the delays observed for
different members were in the same range. ( 7 to 9)
ITEM SUGGEST -- Fig 4 : Response time/ per user , subsequent usages, different times
We analyze the time delays encountered for subsequent visits of a user to the application, when
he clicks on 'I'm feeling lucky' . In this case, the database is already populated with keywords
pertaining to the user. We found that subsequent visits had a constant time of response. This is
because, the retrieval is taking place from the database . Our results indicate that this is
normalized, and it is independent of the number of groups , the user is a member of.
IMPACT OF NUMBER OF GROUPS ( PERTINENCE) :
When a user is a member of few groups, then the liklihood of him seeing the same results when he
clicks of 'I'm feeling lucky ' is more, despite the randomization of our algorithm. However we
could observe that when a group description was small, the results tended to be more pertinent.
This was because a one line group desciption would invariably refer to the the group itself, and thus
the semantic search tended to be more accurate, free of errors. In contrast, a large description can
pick up wrong keywords at times.
12
FUTURE SCOPE
Categorization of search results, improved interface to accommodate price comparison of other
vendors are some of the features which would be taken into account in subsequent versions of our
application.
13