0
1
Pete Bonasso
Mark Boddy (Adventium)
Dave Kortenkamp
Scott Bell




We have PRIDE & PRL for procedure authoring,
display and execution
We have augmented PRIDE & PRL to author &
display planning information associated with
procedures that can be used by automated planners
The major stumbling block to realizing the
widespread use of automation techniques for
operations is capturing and maintaining the domain
models needed to support such techniques
This project seeks to develop a framework for
consistent ontological modeling across domains,
that can be exploited by planners and executives
currently being developed for NASA
2
Ontological Development Environment
Planning
Actions
• Procedures
• Plan Actions
• Models
Procedures
&
Planning
Actions
PRL
PRIDE
Procedure info
to executives
T
r
a
n
s
l
a
t
e
Plan Actions
Goals
Planning
Algorithms
Planning Aid
Plans
OWL
Database
Ontology
Editor
XML,RDF
http
Telemetry
(ISS Configuration)
*
*
Design Docs
Configuration
snapshots
CAMMP & IMS
(ORU locations)
3

Develop an authoring system for domain models
(PRONTOE – PRIDE ONTOlogy Editor)

Develop the ORU location kernel for EVA/ROBO (use
core kernels from Phase 1)

Develop interfaces to online model data

Develop the ORU LOCation (ORLOC) application

Identify methods to integrate data with the models

Test PRONTOE and ORLOC
4

Classes, subclasses and relations among them
• truss-segment, ORU-bag, ceta-light
• can-hold (medium-ORU-bag, ceta-light)

Class instances
• LAB-ceta-light1, S1
• located (LAB-ceta-light1) = S1

Directives
• Preconditions and effects, e.g., crew must possess a PGT before
unbolting a CETA light
• Axioms for bookkeeping and physics, e.g., if a container is
located at A, so are its contents
• Constraints (includes flight rules), e.g., if a ceta-light heater is
not on then its lamp must be on

Data - facts that represent a specific configuration of all
the instances, e.g., switch settings, modes, locations
5

The base
• “Floor” of the ontology = classes and relations based on modeling
choices, e.g.,
• Includes axioms
• Not editable by users
• We have an “all domains” base & an “ISS-base”

The kernel extensions
• Per flight discipline, e.g., EPS, C&DH, EVA
• A starting set of classes and instances of the ISS to be edited and
expanded by users

The ontological data
• Collection of facts that represent a specific configuration of all the
instances
• Can be updated both manually and electronically
6







User doesn’t need to know about ontologies
Add/delete/modify data to support procedure authoring
Navigate graphically through the ontological hierarchy
Invoke ontological reasoners to speed relation entries
Author and save what-if snapshots to use for planning
future activities
Integrate with PRIDE Planning Wizard
• Classes, instances and relations
• Axioms for bookkeeping and constraint management
Output to OWL files
• To be read and updated by planning systems
• To be read and updated by procedure execution
systems
7
Ontology Editor

















DC Switching Unit – DCSUs
Main bus switching unit – MBSUs
DC-to-DC Converter Unit – DDCUs
Remote power controller module – RPCMs
Multiplexer/Demultiplexer – MDMs
Flex hose rotary coupler – FHRCs
InterFace Heat Exchanger – IFHXs
Battery Charge Discharge Unit – BCDUs
Control Moment Gyro – CMGs
Cargo Transport Container – CTCs
Nitrogen tank assembly – NTAs
Ammomia tank assembly – ATAs
Trailing Umbilical System Reel Assembly – TUS-RAs
Pump Module Assemblies – PMAs
Utility Transfer assembly – UTAs
YAW Joints – YAW-Jts
Solar Array Wings – SAWs
“Big 12” ORUs
9







OTSD – ORU Temporary Stowage Device
MUT-EE – multi-use tether end effector
FGB/AGB – fixed/adjustable grapple bar
Ballstack
LDTDT – Long Duration Tie-Down Tether
RET/adjustable – Retractable Equipment Tether, also adjustable
CETA-swing arm
•
•
•


















configuration
WIF – worksite interface
ORU tethers
Not in 1074
Not in ExCATT
Needs work
SPDM OTP (ORU and tool platform)
APFR – articulating portable foot restraint
SPD – Spool Positioning Device
Safety tether
Thermal booties/FPP (floating potential probe)
QD (quick disconnect)/Vent tools (bags)
Toolbox
MLI (multilayer insulation) shrouds
Tool bag
Truss stowage bin – capacity, contents, launch locks
GAP spanner
Fluid jumper
OIH – On-orbiter Installable Handrail
Tether shuttle
OIWIF – On –orbiter installable WIF
Rail stop (MT – mobile transporter, ETR (– EXPRESS Transportation Rack), tether shuttle
Scoop
D-handle
10





Quick graphical/textual search for items by class/subclass
User editing of item state, e.g., location, settings, anomalies
Include URI to Mission Integration Database Applications
System (MIDAS) (https://isswww.jsc.nasa.gov/midasagnt/plsql/midas_home_oas/)
Include on-orbit media links (io.jsc.nasa.gov/app)
Single locus of ORU data
• Flight controllers prepare EVA activities and associated
equipment location changes (EVA Xfer Cue Card)
• Flight controllers update models via completed EVA tasks
• Changes auto-generated and sent to external databases (IMS,
EXCATT) and others (ROBO, Pointing, etc.)


Auto-generation of reports by equipment type or location
Author and save what-if snapshots to use for planning future
ROBO/EVA activities
11
Ontological Data Connections
• Procedures
• Actions
- Preconditions
- Effects
T
r
a
n
s
l
a
t
e
PRL
(XML)
• instances
Planning
Wizard
PRONTOE
• classes/subclasses
• instances
• relations
• resources
• classes/subclasses
• instances
• relations
• resources
• axioms
• constraints
to planners
• classes/subclasses
• instances
• relations
• axioms
• constraints
Procedure
Authoring
PRIDE
Planning
Actions
OWL
Database
(XML,
RDF)
*
*
updated
properties http
ORLOC Tool
Configuration
snapshots
display
processing
Design Docs
instances
Telemetry
instances
changes
CAMMP & IMS
DBs (ORU
locations)
Reports
12
JSON
Event
Files
ExCATT
Query for differences
OWL
Database
ORLOC
JSON
Event
File
Post after validating
Changes from EVAs
Query for differences
Changes from EVAs
API
Change
Form
IMS
Post after validating
13
Base
- Abstract Entity
- Commands
- PRIDE:CDRSarmCmd
- PRIDE:MDMrtEnable
- Telemetry
- PRIDE:smokeDetActiveBitStatus
- PRIDE:MDMframeCount
Derived from PRIDE sysref
14
Telemetry Updates
*
OWL
Database
(XML,
RDF)
*
Config
Snapshots
Data
Actions
ExCATT Files
• Create Configuration Snapshot
• Merge Configuration Snapshot
• Create dynamic configuration (telemetry)
Captured
• Update from ExCATT
ISP
Tele
metry
updated
properties http
display
processing
Design Docs
Telemetry
Config Plugin
• ISPresso
• REStful
ISP
15
1
Year 1*
Task 1:
Task 2:
Task 3:
Task 4:
Task 5:
Task 6:
Task 7:
Principal
STAFF
Quarters
TASKS
2
3
4
5
6
7
Refine Phase 1 base ontology
Develop editor with Phase 1 Models
Design and develop data interfaces
Extend editor to support data editing
Iterate editor development with SMEs
Build AP interface with OWL models
Test models using Phase 1 scenario
8
Bonasso,Boddy,
All
Bonasso
All
Bonasso, Boddy, SMEs
Bonasso
All
Year 2
Task 1: Use editor to complete ORU models
Task 2: Work with SMEs to select a new scenario
Task 3: Demonstrate plan generation for new scenario
Task 4: Develop ORLOC tool with SMEs
Task 5: Demonstrate ORLOC in new scenario
Task 6: Document results and deliver software
Bonasso, SMEs
Bonasso, SMEs
All
Bonasso, SMEs
All
All
Rpt
Rpt
Rpt
Test Rpt
Rpt
Rpt Demo
*Overrunning resources
Editor1
Rpt
Editor2
ORLOC- Final Report
alpha
ORLOC-beta
16

Ontologies
• Phase 1 ontology for core systems verified
• Redesign of location modeling for Big 12 ORUs complete
• Phase 2 initial modeling complete

PRONTOE 0.5 – complete
• Completed first round of PRONTOE demos to interested parties
• IWPSS paper accepted/presented*
• Modeling changes made based on user feedback

ExCATT
• Phase 2 ontology used ExCATT part catalog and event files
• Can compare ontology with ExCATT files
IMS – waiting on API (Russell Knight’s work?)
 ISP – have a preliminary design

*Jeremy wants to meet
17

Develop PRIDE interfaces to Phase 2 ontology
1) Commands and telemetry
2) Procedure editing
3) DOUG – develop 1st cut interface (today’s show and tell)

Integrate with planner – Develop I/O API for Phase 2
ontology
• Initial OWL->PDDL completed and translation plans achieved
• Working on axioms next
• PPW integration deferred to next year
ExCATT, IMS – attend “stakeholders” meetings
 Paper to AI in Space workshop

18


Limit one user read/write at a time
User uses PRONTOE to edit kernel and run reasoner
• Launching PRONTOE will trigger reasoner
• Inferences to other kernels are posted
• User corrects problems to achieve a consistent ontology

Other users use svn status to check for changes to their
kernels
• Review the changes in PRONTOE
• If accepted commit to repository
• If rejected => meet with other users to resolve differences
19




Updates affect all kernels
Reasoners must reach a fix point => no more inferences
can be made
Updates from planners can expect to be logically ordered
Asynchronous updates (e.g., ISP) will be
problematic
• Define a logical endpoint to an update (e.g., some quiescent state
of the ISS)
• Run reasoner to a fix point and check for consistency
• Accept inconsistencies from soft constraints
• Run “hard” constraint violations to ground

Is this an SBIR topic?
20
Direct subclasses => types/subtypes
 Class assertions => instances
 Object properties => predicate definitions
 Object property assertions => predicate
assertions

*PDDL-e
= PDDL 2.1 + axioms + hierarchical actions
21
(define (domain ISS-BASE)
(:comment "ISS-base Ontology")
(:extends base)
(:types
station-object stowage-structures orbital-replacement-unit - physical-entity
iss-computer - station-object
oru-role - role
sarj-mode us_gnc-mode device-mode - mode
iss-schematic-location - schematic-location
iss-computer - computer
oru-location iss-location - physical-location
truss-location module-location external-stowage-platform-location ex-press-logistics-carrier-location
airway - iss-location
)
(:constants
esp0306-aft-stb-nad esp0206-aft-prt-zen esp0206-fwd-prt-nad esp0204-aft-stb-zen - external-stowageplatform-location
active non-active stand-by - device-mode
auto-track blind directed-position - sarj-mode
Z1B00F03ZS Z1B00F03NS Z1B00F03NP Z1B00F03ZP P3B02F01NP P1B06F04MS S1B05F04MP
P1B06F01MP S1B05F01MS S5B26F04MM S5B29F01MM S6B53F04PB S6B53F01PB S1B07F03MP
S0B04F01MS S1B11F01MS P1B12F01MP P1B08F01MS S1B07F01MP P1B16F01NS S1B15F01ZP
S0B00F06ZP S0B00F06ZS S0B01F01NS S0B01F01ZS S1B13F01ZP S0B00F06MP S0B02F02MS
S0B02F02MP S0B00F06MS S0B02F01MP S0B01F01MS S0B01F01MP - truss-location
elc0307-aft-prt-nad elc0109-aft-stb-zen elc0109-fwd-prt-zen elc0102-fwd-prt-zen elc0201-aft-prt-nad
elc0205-aft-stb-nad elc0106-fwd-stb-zen elc0206-aft-stb-zen elc0107-fwd-stb-nad - ex-press-logistics-carrierlocation
CMG cmg_ta drift gnc-standby udg wait - us_gnc-mode
intra-vehicle - iss-location
airlock ulab-aec-stb nod2aec-zen nod2aec-nad nod2fec-nad nod2fec-zen ulab-aec-prt - module-location
spare-oru used - oru-role
ISS-base.pddl
)
(:predicates
(has-oru-role ?oru - orbital-replacement-unit ?or - oru-role)
(fills-oru-role ?or - oru-role ?oru - orbital-replacement-unit)
(is-attached-to ?so - station-object ?so - station-object)
(has-iss-location ?so - station-object ?il - iss-location)
(has-oru-location ?oru - orbital-replacement-unit ?ol - oru-location)
)
)
22
Container(?x), PhysicalEntity(?y),
ISSlocation(?z), contains(?x, ?y),
hasISSlocation(?x, ?z)
->
hasISSlocation(?y, ?z)
(:axiom
:vars (?x - container
?y - physical-entity
?z - iss-location)
:context (and (contains ?x ?y)
(has-iss-location ?x ?z))
:implies (has-iss-location ?y ?z)
)
<DLSafeRule>
<Body>
<ClassAtom>
<Class abbreviatedIRI="base:Container"/>
<Variable IRI="urn:swrl#x"/>
</ClassAtom>
<ClassAtom>
<Class abbreviatedIRI="base:PhysicalEntity"/>
<Variable IRI="urn:swrl#y"/>
</ClassAtom>
<ClassAtom>
<Class abbreviatedIRI="ISS-base:ISSlocation"/>
<Variable IRI="urn:swrl#z"/>
</ClassAtom>
<ObjectPropertyAtom>
<ObjectProperty abbreviatedIRI="base:contains"/>
<Variable IRI="urn:swrl#x"/>
<Variable IRI="urn:swrl#y"/>
</ObjectPropertyAtom>
<ObjectPropertyAtom>
<ObjectProperty abbreviatedIRI="ISS-base:hasISSlocation"/>
<Variable IRI="urn:swrl#x"/>
<Variable IRI="urn:swrl#z"/>
</ObjectPropertyAtom>
</Body>
<Head>
<ObjectPropertyAtom>
<ObjectProperty abbreviatedIRI="ISS-base:hasISSlocation"/>
<Variable IRI="urn:swrl#y"/>
<Variable IRI="urn:swrl#z"/>
</ObjectPropertyAtom>
</Head>
</DLSafeRule>
23
(:axiom
:vars (?a - agent
?o - physical-entity)
(define (domain nasa-domain)
:context (and (possesses ?a ?o)
(:requirements :domain-axioms :multi-agent :durative-actions)
(not (= ?o nothing)))
(:extends nasa2i)
:implies (possessed_by ?o ?a)
(:types body-size - abstract-entity
:comment "possessed_by inverseOf possesses")
crew flight-controller - Agent
(:axiom
cato phalcon adco odin thor - flight-controller
:vars (?a – Agent ?t1 ?t2 - physical-entity)
safety-tether-pack - container)
:context (and (contains ?t1 ?t2)
; put least-used first
(:constants
(possesses ?a ?t1)
sally joe bob - crew
(not (= ?t2 Nothing))
small medium large - body-size)
(not (= ?t1 ?t2)))
(:predicates
:implies (possesses ?a ?t2)
(body-size - fact ?c - crew ?s - body-size)
:comment "possesses is transitive over contains")
(ingressed ?c - crew
(:axiom
?o - (or spdm_arm
:vars (?o1 ?o2 - physical-entity)
crew_&_equipment_translation_aid_[ceta]
:context (and (contains ?o1 ?o2)
articulating-portable-foot-restraint))
(not (= ?o2 Nothing)))
(too-far ?l1 ?l2 - location)
:implies (contained_by ?o2 ?o1)
(extracted-item-to ?ev - crew ?item - station-object
:documentation "contained_by inverseOf contains")
?container - container)
(:axiom
(retrieve-item_a ?ev - crew ?item - station-object)
:vars (?a – Agent
(tether-swapped ?ev - crew ?st1 - safety-tether)
?h - (or spdm_arm crew_&_equipment_translation_aid_[ceta])
(tethered_to ?st - safety-tether ?ev - crew)
?l - iss-location)
)
:context (and (ingressed ?a ?h)
(has-iss-location ?h ?l))
:implies (has-iss-location ?a ?l)
:documentation "you are located wherever the thing you're ingressed to is located.")
(:axiom
:vars (?o1 ?o2 - physical-entity)
:context (not (contains ?o1 ?o2))
:implies (not (contained_by ?o2 ?o1))
:documentation "when one is negated, the other must be as well")
)
agents&axioms.pddl
(owl:Restriction
(owl:Restriction
(owl:Restriction
(owl:Restriction
'possesses 'owl:cardinality :multi-value)
'possessed_by 'owl:cardinality 1) ; predicate default is :multi-value
'contained_by 'owl:cardinality 1)
'contains 'owl:cardinality
24
base.owl
base.pddl
(+ axioms)
ISS-base.owl
ISS-base.pddl
(+ axioms)
nasa2i.owl
nasa2i.pddl
Instances & assertions
eva.pddl
actions from
PPW
agents&actions.
pddl
initial situation
nasa-domain
= extends
25
26
actions & a problem
(define (durative-action Translate-by-handrail)
(define (durative-action Retrieve-item)
:parameters (?ev - crew
:parameters (?ev - crew
?end-loc - iss-location)
?item - (or luminaire__ceta_light
:vars (?start-loc - iss-location)
control-panel-assembly
:condition (at start (has-iss-location ?ev ?start-loc))
power-cable
:effect (at end (has-iss-location ?ev ?end-loc))
space-positioning-device))
:duration time-from-path
:vars (?container - (or oru-bag fish-stringer)
:comment "?ev travels by handrail from ?start-loc to ?end-loc")
?loc - (has-iss-location ?item))
:expansion (sequential
(define (durative-action Extract-item-to-bag)
(possesses ?ev ?container)
:parameters (?ev - crew
(has-iss-location ?ev ?loc)
?item - (or luminaire__ceta_light
(extracted-item-to ?ev ?item ?container)
control-panel-assembly)
)
?container - oru-bag)
:effect (at end (retrieve-item_a ?ev ?item))
:vars (?pgt - pgt-with-turn-setting
:comment "?ev picks up ?container, travels to ?item's loc,
?l - (has-iss-location ?item))
unmounts and stores ?item in ?container and returns.")
:duration 12.0
:condition (and (at start (has-iss-location ?ev ?l))
(define (problem bob-get-light)
(at start (possesses ?ev ?container))
(:domain nasa-domain)
(at start (= (possessed_by ?pgt) ?ev))
(:situation phalcon-eva)
(at start (bag-size-for ?item ?container)))
(:deadline 100.0)
:effect (and (at end (extracted-item-to ?ev ?item ?container))
(:init (has-iss-location bob airlock)
(at end (contains ?container ?item))
(possesses bob oru-bag_1)
)
(possesses bob pgt_1))
:comment "crew removes ?item at ?l and stows in bag."
(:goal (retrieve-item_a bob LUMINAIRE_3))
)
)
27
plan achieved
purpose-established RETRIEVE-ITEM0: RETRIEVE-ITEM_A(BOB,LUMINAIRE_3)
Thus ends the plan-generation phase of BOB-GET-LIGHT ...
-------- plan: RETRIEVE-ITEM0 -------RETRIEVE-ITEM0: RETRIEVE-ITEM_A(BOB,LUMINAIRE_3)
sequential
BOB-GET-LIGHT_IS: POSSESSES(BOB,ORU-BAG_1)
TRANSLATE-BY-HANDRAIL13: HAS-ISS-LOCATION(BOB,S1B07F03MP)
EXTRACT-ITEM-TO-BAG56: EXTRACTED-ITEM-TO(BOB,LUMINAIRE_3,ORU-BAG_1)
probability-of-success = 0.902
duration = 27.00
+nodes-examined+ = 4
+constraints-tested+ = 0
+axioms-applied+ = 3
0.05 CPU seconds [0.012 CPU sec/node]
RETRIEVE-ITEM0
28
29
more actions & a
(define (durative-action install-tether&swap)
:parameters (?ev - crew
?st1 ?st2 - safety-tether)
:vars (?l - (has-iss-location ?ev)
?stp - safety-tether-pack)
:condition (and (at start (not (= ?st1 ?st2)))
(at start (tethered_to ?st1 ?ev))
(at start (contains ?stp ?st2))
(at start (has-iss-location ?stp ?l))
)
:effect (and (at end (not (contains ?stp ?st2)))
(at end (tethered_to ?st2 ?ev))
(at end (has-iss-location ?st2 ?l))
(at end (tether-swapped ?ev ?st1)))
:duration 5
:comment "?ev installs a second tether and then swaps
his current tether for the new one.")
nd
2
problem
(define (durative-action Translate-by-hr&swap)
:parameters (?ev - crew
?end-loc - location)
:vars (?start-loc - (has-iss-location ?ev)
?mloc - (intermediate-loc-for ?start-loc ?end-loc)
?st - safety-tether)
:condition (and
(at start (too-far ?start-loc ?end-loc))
(at start (not (= ?start-loc ?end-loc)))
(at start (tethered_to ?st ?ev))
)
:expansion (sequential
(has-iss-location ?ev ?mloc)
(tether-swapped ?ev ?st)
(has-iss-location ?ev ?end-loc)
)
:effect (at end (has-iss-location ?ev ?end-loc))
:duration time-from-path
:comment "?ev travels by handrail to ?end-loc via ?mloc")
(define (problem bob-get-light2)
(:domain nasa-domain)
(:situation phalcon-eva)
(:deadline 100.0)
(:init
(has-iss-location bob airlock)
(possesses bob oru-bag_1)
(possesses bob pgt_1)
(tethered_to 85-ft_tether_3 bob)
(possesses bob stp_1))
(:goal (retrieve-item_a bob LUMINAIRE_2))
)
30
nd
2
plan achieved
purpose-established RETRIEVE-ITEM0: RETRIEVE-ITEM_A(BOB,LUMINAIRE_2)
Thus ends the plan-generation phase of BOB-GET-LIGHT2 ...
-------- plan: RETRIEVE-ITEM0 -------RETRIEVE-ITEM0: RETRIEVE-ITEM_A(BOB,LUMINAIRE_2)
sequential
BOB-GET-LIGHT2_IS: POSSESSES(BOB,ORU-BAG_1)
TRANSLATE-BY-HR&SWAP3: HAS-ISS-LOCATION(BOB,P3B02F01NP)
sequential
TRANSLATE-BY-HANDRAIL78: HAS-ISS-LOCATION(BOB,P1B10F01MM)
INSTALL-TETHER&SWAP135: TETHER-SWAPPED(BOB,85-FT_TETHER_3)
TRANSLATE-BY-HANDRAIL182: HAS-ISS-LOCATION(BOB,P3B02F01NP)
EXTRACT-ITEM-TO-BAG213: EXTRACTED-ITEM-TO(BOB,LUMINAIRE_2,ORU-BAG_1)
probability-of-success = 0.815
duration = 32.00
+nodes-examined+ = 8
+constraints-tested+ = 0
+axioms-applied+ = 20
0.19 CPU seconds [0.024 CPU sec/node]
RETRIEVE-ITEM0
31
MBSU at S0B02F02MP
DDCU at S4B23F04ZM
32
FHRC at ESP0207aftPrtNad
PMA at ELC0107FwdStbNad
DCSU at ESP0102aftPrtMid
33
View for items on any forward face for
nod2 cole jemp jmlm pma2 pmml
IFHX at "COLEprtFwd"
34
View for items on any nadir face for
pma2 nod2 nod3 cole ulab jemp pmml alck
PDGF at ”UlabFwdNad"
35
Aft end cones have special views
Port side of Aft end cone of Node 1
36
Specific views for specific locations, e.g., Node 2 aft end cone
IFHX at ”Nod2AecNad"
37
38
Discussion