An Agent Capable of Learning to
Create and Maintain Websites
Anthony Tomasic, Ravi Mosur
Alex Rudnicky, Raj Reddy, John Zimmerman
Carnegie Mellon University
18 April 2003
Outline
• Project vision
• Problem
– Assumptions
– Inputs
– Outputs
•
•
•
•
•
Missing capabilities
Our approach
Impact
Evaluation
Conclusion
Project Vision – Honeydew
• Honey – An agent
– Learns by observation
– Obtains advice and consent
– Creates and maintains public project websites
Project Vision – Honeydew
• Honey Can Perform Tasks
– Organize, manage and update a complex project website
– Delegate tasks
– Generate periodic briefing folders
• Email extracted material and online documents with planning and
summarization capabilities
– Respond to specific information queries
– Extract relevant information
• WWW & mailing lists
– Perform teaching
– Communicate with other Honeys and EPCAs
Problem Assumptions
• Scope of a “project” website is predefined and not
learnt:
–
–
–
–
–
–
–
–
–
Publications
Presentations
Milestone status
News updates
Demonstrations
Links to collaborators
Software releases
Documentation
FAQs
Problem Assumptions
• Things that Honey will not do
– System administration
• Capacity planning
– Graphic design
• Font selection
• Site design
• Some layout design possible …
– Content creation
Inputs to Honeydew
• E-mail messages with updates to website
– Volunteered and solicited information
• Minutes from project meetings
– Tracking project participants and events
• Queries from external sources
– Inferred information needs
– Click sequences
• Publicly visible events, not explicitly provided to
webmaster
– Conference appearances, news stories, etc.
Inputs to Honeydew
• Sequences of UI actions performed
– Receive e-mail request to add paper to WWW site
– Extract title, author, abstract, publication forum,
funding agency
– Think up file name
– Copy attachment to conference paper directory
– Update WWW page with info and link
– Notify user of change
– React to advice from user about change
Expected Outputs
• Project websites (5 subprojects, 1 project)
• Report generation
– Overviews of activity over time (e.g., quarterly reports)
• Briefing generation
– Overviews of current project activities
• Question-answering agents
– Google-like search, summarization in response to
specific queries
• Semi-automatic FAQ generation
Expected Outputs
•
•
•
•
•
•
Shared knowledge base of learned tasks
Toolkit for rapid construction of new assistants
Assistant Monitoring and Interaction GUI
Requirements for “Assistant Aware Applications”
Stream of papers
Stream of masters and Ph.D. students
Missing Capabilities – What
• Identify significant webmastering events
• Represent webmaster activities through
generalizable descriptions
• Create consistent and complete task
representations
• Formulate key clarification dialogs
• Adapt to errors in task execution
Our Approach
• Ethnographic study of Webmasters
• WoZ system for domain definition
• Human webmasters with Honey observing
activities
• Information-sharing among EPCAs
Our Approach
•
•
•
•
In-line data labeling by humans
Interactive clarification of human actions
Lightly-supervised learning
Generalizable and sharable representations
of activities
• Learn by being told
• Enthnographic study of Honey users
Impact
• Relieve the user of routine maintenance
tasks associated with web pages
• Illustrate portability by using in other
WWW task domains
– HCII web page
– Pittsburgh Post Gazette web page
– Workflow systems
Impact
• Dramatic reduction of human effort in
construction and maintenance of web sites
– Improve time productivity by 100 to 500%
– Little or no loss in quality of site
• EPCAs that can learn the skill of cooperatively
structuring and managing information
• Dramatic reduction in the cost of construction of
assistants
– Reduce size of backroom knowledge engineers
• Assistants become trainers of new Web masters
Evaluation: mid-term and finals
• Honey performance to be compared with 5 human subjects
– 5 other human coaches to be used in providing data and inputs
needed for the Honey to learn from Experience
– Operational Version 0 in 3 months
• Evaluation – 6 months
– Honey performs 50% of tasks correctly
• Evaluation – 12 months
–
–
–
–
Elapsed time from e-mail to WWW update improved by 1.3
Elapsed time to assemble report improved by 1.3
Honey performs 75% of tasks correctly
Quality of WWW site and reports comparable to human
Infrastructure Input
• Users
• Architecture
• Functional
Specification
• Text
– Filtering
– Summarization
– Extraction
• Quality Assurance
• Quality Assurance
• Evaluation
• Knowledge
Representation
• Dialog
Conclusion
• Key emphasis
– Learning
– Coaching
– Retargetting
•
•
•
•
Potential huge impact to Webmaster job
Large amount of shared infrastructure
Many similar problem domains
Many, many research problems
Research Issue
• Architecture
Execution
Recommend
Learning
User GUI
Coaching GUI
KB GUI
Monitor/Do GUI
Matching
Summarization
KB
Event Stream
Monitor/Do
E-Mail Server
E-Mail GUI
Editor
File System
Command