Centralizing POMDP Solvers
via Web Services
BY
Roi Ceren
Muthukumaran Chandrasekaran
Background
POMDP
Models a single agent’s decision process in which the agent
doesn’t directly observe the underlying state
Exact solution yields the optimal action for each possible belief
over the states
Sequence of optimal actions is known as the optimal policy of
the agent and can be represented as a decision tree
In practice, POMDPs are often computationally intractable
to solve exactly
Thus, the need for approximate solutions
Several exact and approximate methods for solving
POMDPs exist
No centralized test-bed available
Background
Web services
Applications can be broadcast as a service on a
web server, such as Apache
A wrapper, such as Axis or Tomcat, can be used to
execute these applications
Using standardized protocols, such as SOAP or
REST, servers can communicate with one another
to access these services
In this way, we can allow disparate platforms to
interoperate
Proposed Problem
Many programs exist to implement POMDP
resolutions, their subsets, and approximations
techniques
Robust applications streamlining and centralizing
these functions and programs are largely not
available
We propose a web application that allows
users to present problem domains and, in one
location, use POMDP and various
approximations to resolve them
Approach
Expose at least one exact and one approximate
POMDP programs on a THINC Lab PC
Cassandra organization source code (exact)
http://www.cassandra.org/pomdp/code/index.shtml
SARSOP via APPL toolkit (approximate POMDP
planning)
http://bigbird.comp.nus.edu.sg/pmwiki/farm/motion/index.php?n=Site.PomdpPlanning
Publish access to these functions via web services
SOAP or REST for communication
Build a web interface that utilizes these web
services
Approach
Users will interact with the web interface to resolve
POMDP problems
1.
2.
3.
4.
User provides a .pomdp or .pomdpx file via the user’s browser
File is passed to the POMDP solver with the selected parameters
Policy is retrieved from POMDP solver
Policy is displayed graphically as a policy tree in a graph applet via
the user’s browser along with time of computation
Browser
Communication
Protocol
.pomdp(x)/parameters
policy
POMDP Solver
Approach (Web Interface)
Input: AJAX Form
POMDP file uploader
Parameter input, examples
Time limit
Horizon
Discount factor
Stopping criteria
Solver type
Browser
POMDP SOLVER
POMDP File Location
ALGORITHM
Upload
Select Algorithm
V
PARAMETER 1
PARAMETER 2
PARAMETER 3
…
RESULTS
Output: Applet
time in ms
Resulting policy tree
Computation time
Download
Print
Approach (REST/SOAP)
Apache Axis C++ module will be configured on
the POMDP solver server
Will serve as a wrapper for the various
implementations of POMDP solvers
Either REST or SOAP will be used to communicate
REST calls will be made in JavaScript methods on the
client side after form submission
SOAP calls will be made using the PHP module kSOAP
Policy data will be collected, parsed, and, using
JavaScript, passed to the Applet
Demo Overview
Website will be loaded during presentation
Problem domain file will be uploaded and
parameters configured
.pomdp/x file
Discount factor, horizon, etc.
Applet will display output information
Policy will be parsed as a decision tree
Computation time will be overlaid
Demo Details
Implement a working demonstration of the
following methods
Exact POMDPs
Enumeration
Incremental Pruning (if time permits)
Approximate POMDPs
SARSOP
Sample Domains
Tiger Problem
Deliverables
Central location for toolkits
Located on Muthu’s THINC lab machine
e.g. Cassandra program/APPL toolkit
Web interface for POMDP solver
Form and decision tree applet
URL is TBD on Muthu’s cs.uga.edu website
Responsibilities
Muthu
Configuring server with POMDP toolkits
Web page design
Roi
Implementing RESTful web services
Joint
Web interface
AJAX web forms
Graph applet