Enhancing the Web
With End-User Programming
Tak Yeon Lee, Ben Bederson
Outline
Problem
Inefficiencies on the Web
Challenge
Building extension is very hard
Our goal
Expressive and Easy Programming Env.
- Improves a wide range of tasks
- No programming skills required
Snapshot
ANY WEBSITE
Inefficiencies on the Web
1. Missing relevant information
“Is it a good value?”
 Kelly’s Blue book
“Is the model reliable?”
 Auto forums
“I want to see the pictures.”
 Detail page
“How far is this place?”
 Google Map
2. Removing unwanted information
AD
AD
2. Removing unwanted information
3. Repetitive Operations
“I want to download all PDFs named as
[first author]-[title]-[year].pdf ”
10.1.1.154.444.pdf
Howe-TheRiseOfCrowdsourcing-2006.pdf
Kittur-08-001.pdf
Kittur-CrowdsourcingUserStudies-2008.pdf
60947(1).pdf
Brabham-CrowdsourcingAsAModelFor-2008.pdf
Alonso2008.pdf
Alonso-CrowdsourcingForRelevanceEvaluation-2008.pdf
4. Finding a needle in a haystack
Grading Procedure
1. Finding mistakes
Takes 4-8 hours
to grade 70 students
2. Add comments
3. Calculate the total points
4. Submit to grade server
Summary of
Potential Mistakes
Grading with an extension
MyDouble.zero
2. Click to add
predefined comment
MyDouble.zero
1. Click to scroll to the
code line of the mistake
“Previously it took several hours, but now I
can finish in 40 minutes.”
- Lecturer
“Three times faster than before, at least!”
3. Setting partial
-Grading
TA
credit
MyDouble.zero
4. Report total score
to another server
Browser Extensions
add new features and personalize experience.
Over 50,000 on Chrome Web Store
AdBlock
Photo Zoom
Print All
>15 million users
>2 million users
E-mails in a folder
Building an extension requires
• Programming knowledge (HTML, CSS, and Ajax)
• Time and effort of development
• The grading example took >40 hours
OUR GOAL IS TO BUILD
Expressive and Easy programming env.
The toolkit should support tasks like...
•
•
•
•
Automating repetitive jobs
Integrating data from multiple sources
Custom filtering
Information Summary
We defined a domain language model having
Pick, Change, Add operations
as generic functionalities of the extensions.
Pick Change Add model
Pick
extract information from web pages or elements.
<A> tags
The rise of
crowdsourcing.pdf
Crowdsourcing user
studies...
Crowdsourcing as a model
...
URLs
http://sistemashumano.com...
http://dl.acm.org/citation.cfm
?id=1357127
http://con.sagepub.com/...
Pick Change Add model
Change
manipulates dataset.
[b]
3
[a]
1
• Arithmetic operations
• +, -, /, %, ...
• sum, avg, count, ...
2
3
• String manipulation
[a] * [b]
3
6
9
• concatenate, split, find, replace, ...
• Filter
• Sort
Pick Change Add model
Change
manipulates dataset.
[b]
-
[a]
I
• Arithmetic operations
• +, -, /, %, ...
• sum, avg, count, ...
am a
boy
• String manipulation
Join [a] with [b]
I-am a-boy
• join, split, find, replace, ...
• Filter
• Sort
Pick Change Add model
Change
manipulates dataset.
[a]
a.jpg
[b]
png
• Arithmetic operations
• +, -, /, %, ...
• sum, avg, count, ...
b.gif
c.png
• String manipulation
Filter [a] having [b]
c.png
• join, split, find, replace, ...
• Filter
• Sort
Pick Change Add model
Change
manipulates dataset.
[a]
Toronto
• Arithmetic operations
• +, -, /, %, ...
• sum, avg, count, ...
Seoul
Paris
• String manipulation
Sort [a]
Paris
Seoul
Toronto
• join, split, find, replace, ...
• Filter
• Sort
Pick Change Add model
Add
modify the original web pages.
• Create new elements
• image, text, button, ...
• Modify element’s style
• Show / Hide, changing color, fonts, ...
• Triggering events
• click, ...
• Type text
Pick Change Add model
e.g. Removing AD banners
[1]. Pick ad banner elements
[2]. Hide [1]
AD
AD
Pick Change Add model
e.g. Downloading files with custom name pattern
[1]. Pick title text
The rise of crowdsourcing.pdf
[2]. Merge [1] with
name pattern “.pdf”
Crowdsourcing user studies with Mechanical Turk.pdf
[3]. Pick download links
Crowdsourcing as a model for.pdf
[4]. Add [2] to ‘download’
attribute of [3]
Crowdsourcing for relevance evaluation.pdf
OUR GOAL IS TO BUILD
Expressive and Easy programming env.
• Requires no programming knowledge
• Programming-by-Example techniques
• Telling computer what I want, without worrying
how to do.
Programming-by-Example technique
INPUT
PROGRAM
OUTPUT
Traditional Programming
Programming-by-Example
?
1, 2, 3
int sum=0;
for(int v in input)
{
sum=sum+v;
}
return sum;
?
!
6
Programming-by-Example technique
Programs with multiple inputs
e.g. Adding suffix to strings
[1]
[2]
Paris
Input
.jpg
NewYork
Seoul
Tokyo
Merge [1] with [2]
Paris.jpg
Output
NewYork.jpg
Seoul.jpg
Tokyo.jpg
Programming-by-Example technique
Multi-step programming
e.g. Counting items of [1] that exist in [2]
[1]
[2]
Paris, France
Korea
Istanbul, Turkey
Japan
Seoul, Korea
Osaka, Japan
Filter [1] exist in [2]
[3]
Seoul, Korea
Tokyo, Japan
Count [3]
2
Programming-by-Example technique
Multi-step programming
e.g. Counting items of [1] that exist in [2]
[1]
[2]
London, UK
UK
LA, US
Canada
Beizing, China
China
Toronto, Canada
Filter [1] exist in [2]
[3] London, UK
Beizing, China
Toronto, Canada
Count [3]
3
Demo time
Conclusion
A new opportunity for end-users to personalise the web by
creating their own extensions.
Programming-by-Example techniques
with Pick, Change, Add model
Future work includes
• Usability Tune up
• Evaluation of the system’s learnability
Please email us if you have an idea for extensions
Tak Yeon Lee <tylee@umd.edu>