An Assessment of ITO Projects for Potential ISO Transition: Preliminary Leo Obrst, Ph.D. July 24, 1998 Intelligent Decision and Training Systems MITRE 2 09/10/97 10:08 AM Outline Background Assessment Approach Presentation of results Conclusions MITRE 3 09/10/97 10:08 AM Background In 1997, MITRE reviewed the ITO Human Systems Integration area In 1998, MITRE asked to review the following areas of the ITO research program: – CORBA-related projects in all of ITO but focusing on Quorum (C&N) – Other Quality of Service projects in Quorum – Intelligent Systems & Software: Evolutionary CORBA:Other Evolutionary Design of Complex Software (EDCS) Design of Complex Software Intelligent Systems & Software – Computing & Networking: Global Mobile Information Systems (GloMo) Quorum CORBA:Other Computing & Networking CORBA: Quorum & QoS GloMo MITRE 4 09/10/97 10:08 AM Reason for Assessment ITO research is resulting in demonstration systems that are being used in various places Would like to facilitate transition mechanism of projects – ITO -> ISO( -> ACTD) -> JPO -> DISA – May not go through all steps Difficult for ITO PMs to make all connections with appropriate ISO PMs Difficult for ISO PMs to identify those ITO projects applicable to their program MITRE 5 09/10/97 10:08 AM Improve the Technology Transition Path DII ISO AITS JPO ITO Technologies AITS Integration Environment DII GCCS GCCS GCSS GCSS DARPA C4I Programs APII LES Service Service C4ISR C4ISR JTA emerging standards DII COE recommendations DARPA DISA MITRE 6 09/10/97 10:08 AM Improve the Technology Transition Path (concluded) Benefits to ISO – Quick reference to research being conducted that can potentially transition into programs – Identify appropriate PMs, PIs to include in discussions – Determine holes in the program not being addressed by research program – Identify commercial solutions that may solve the need Benefits to ITO – Get feedback on transitionability of projects – Get feedback on how to improve research – Identify holes for potential new research areas Benefits to both – More efficient mechanism than having every PM trying to connect with correct PIs MITRE 7 09/10/97 10:08 AM Assessment Approach Two-sided approach - ISO/DoD program technical requirements - ITO project assessment ISO program requirements - Rely on current, published knowledge - Rely on MITRE personnel, when available - Talk with DARPA PMs and contractors as appropriate MITRE 8 09/10/97 10:08 AM Approach (concluded) ITO project assessment - Technology description - Applicability to program requirements To fill a technological need To demonstrate a potential operational solution - Maturity Paper/theoretical (e.g., algorithm analysis) Prototype (laboratory vs. demonstrated vs. tested) Operational (freeware or component of commercial product) - Applicability to development cycle in a prospective ISO new start (for EDCS) - Heuristics for assessment of projects MITRE 9 09/10/97 10:08 AM Overall Assessment Results on IS&S and C&N Research Areas (Partial) Key: Color-coded by Program and Maturity Level 18 projects ITO Research Programs In Study (194 projects total, concentration on EDCS & Quorum:120 projects) 21 projects 22 projects 18 21 22 Evolutionary Design of Complex Software CORBA:Other Intelligent Systems & Software 61 23 CORBA: Quorum & QoS CORBA:Other Computing & Networking 14 14/59 GloMo P r o g r a m 36 MITRE 10 09/10/97 10:08 AM Heuristics for Assessment of Projects When did project start? If new start, probably not ready. If a restart or change in direction, probably not ready. Prototype available yet? Extent of prototype, I.e., versions, applications being built by universities? Industry? If set of tools, general maturity of individual tools and integration? Used in any EDCS integration efforts? MITRE 11 09/10/97 10:08 AM Potential Immediate Transition Projects (22) Aleph/Desert, Evolutionary Design of Complex Systems, Brown University Ensemble, ML/Nuprl Logical Environment: Construction Methodologies for Improving Distributed System Security, Cornell University Orbit: Collaboration Support for Complex Systems, University of Illinois [see Atlantis] Atlantis: An Open Architecture for Synergy of Process-Centered Environments and Computer-Supported Cooperative Work, University of Illinois [see Orbit] STeP: Software Development Technologies for Reactive, Real-Time, and Hybrid Systems, Stanford University UniCon, etc.: A Technology Investigation Supporting Software Architecture and Analysis for Evolution, Carnegie Mellon University RAPIDE: Architecture Definition Languages, Stanford University DylanWorks: A Dylan Development Environment, Harlequin, Inc. PVS, Murphi: An Integrated Environment for Efficient Formal Design and Verification, Stanford University MITRE 12 09/10/97 10:08 AM Potential Immediate Transition Projects (continued) I-SPECS: Integrated System for the Predicable Evolution of Complex Software Systems, Software Productivity Solutions, Inc. (an integration project) ADAM: Architectural Design, Analysis, and Measurement, Lockheed Martin C2 Integration Systems (an integration project) QUEST: Quality Enabling Software Technologies, Microelectronics and Computer Technology Corporation DEMETER: Evolution of Software Via Adaptive Programming, Northeastern University CPAS: Capability Packaging for Avionics Systems, Northrop Grumman Corporation (an integration project) TAOS, ProDAG, Chimera: Research in Advanced Environments, University of California, Irvine Endeavors, EDEM (Expectation-Driven Event Monitoring): Open Technology for Software Evolution: Hyperware, Architecture, and Process University of California, Irvine MITRE 13 09/10/97 10:08 AM Potential Immediate Transition Projects (concluded) Flavers, JIL, Julia, IRIS, Pleiades, JESS - UMASS ARCADIA: Support for Advanced Software Engineering Environment, University of Massachusetts WinWin, A4/ACT-I, AAA: Collaborative WinWin System Definition Technology, University of Southern California, Center for Software Engineering ACME, USC/ISI MediaDoc, USC/ISI MICE: Model-Integrated Computing Environment, Vanderbilt University SECC: Software Evolution by Component Consolidation, Xinotech Research, Inc. MITRE 14 09/10/97 10:08 AM Actively Monitor for Near-Term Transition Projects (21) Arcadia: Software Environments in Support of Wide-Area, Development University of Colorado ADC/Rapide: Architecture Driven Construction and Monitoring of Distributed Object Systems, Stanford University Sanctuary: Evolving Large Information Layers, University of Colorado at Boulder Maude: Semantic Interoperation of Open Systems, SRI International FAMILIAR: Formal Alternative Management Integrating Logical Inference and Rationales, Knowledge Evolution CHAIMS: Compiling High-Level Access Interfaces for Multi-Site Software, Stanford University EVOLVER: Evolutionary Views of Lifecycle Versions, Elements, and Rationales, Lockheed Martin (an integration project) AppletMagic for EDCS, Intermetrics, Inc. MITRE 15 09/10/97 10:08 AM Actively Monitor for Near-Term Transition Projects (continued) OzWeb: A New Infrastructure for Evolutionary Design and Implementation, Columbia University CLOS as a Substrate for EDCS, Franz, Inc. CLIM 3.0: A High Level User Interface Definition, Dynamic Object Language Labs, Inc EMMA: Evolution-Memory Management Assistant, CoGenTex, Inc. FLEA (Formal Language for Expressing Assumptions): Software Evolution Through Automatic Monitoring, Cs3: Computing Services Support Solutions MORALE: Mission Oriented Architectural Legacy Evolution, Georgia Institute of Technology RTDL: Real-Time Dynamic Languages for Embedded Processors, IS Robotics EBPSDR: Evolution Based on Precise Semantic Design Records, Kestrel Institute (SpecWare 2.02 immediately available) MITRE 16 09/10/97 10:08 AM Actively Monitor for Near-Term Transition Projects (concluded) SADL (Structural Architecture Description Language), IVM (Internet Virtual Machine): Incremental Evolution of Network-Wide Dynamic Systems, SRI International EASOF (Execution-time Analysis of Specification-based Oracle Failures): Perpetual Testing, University of California, Irvine Jakarta: A Tool Suite for Constructing Software Generators, University of Texas at Austin SE-HOT: Software Evolution Using HOT Language Technology, Yale University MetaH: High Assurance Evolution of Software Architectures, Honeywell Technology Center MITRE 17 09/10/97 10:08 AM Monitor for Long-Term Potential Projects (18) Metaphor: Technology for Analyzing Change & Composing Reusable Real-Time Software Components and Applications, Carnegie Mellon University, Robotics Inst. GWYDION: An Integrated Environment for Evolutionary Software Development & Maintenance, Carnegie Mellon University ACT: Adaptation and Commitment Technology for Evolutionary Software Development, Carnegie Mellon University INSERT: Incremental Software Evolution for Real-time System, Carnegie Mellon University EXPRESS: A Programming Environment for Evolutionary Software Development, MIT MULTI-GEN: Multi-Targeted Program Generators, USC/ISI SSC: Software Security through Certification, Cornell University FABASIT-UCol: A Formal Architecture-Based Approach to Software Integration Testing, University of Colorado, Boulder SE&I: Software Evolution and Interleaving, Georgia Institute of Technology MITRE 18 09/10/97 10:08 AM Monitor for Long-Term Potential Projects (concluded) DECR: Dynamically Extensible Computational Reflection, Indiana University HASE: High-Assurance Software Evolution, Kansas State University PT/P: Perpetual Testing, Purdue University Darwin/Moses: Architectural Invariants of Evolving Systems, Rutgers University EDCRTS: Evolutionary Development of Complex Real-Time Software, Charles Stark Draper Laboratory, Inc. FABASIT-UCI: A Formal Architecture-Based Approach to Software Integration Testing, University of California, Irvine PT/UMass: Perpetual Testing, University of Massachusetts AOP: Aspect-Oriented Programming, Xerox Palo Alto Research Center F-HOT: Foundations of HOT Languages and Software Evolution, Yale University MITRE 19 09/10/97 10:08 AM Potential Reassignments of Maturity To Immediate: – From Near: FAMILIAR: Formal Alternative Management Integrating Logical Inference and Rationales, Knowledge Evolution MetaH: High Assurance Evolution of Software Architectures, Honeywell Technology Center – From Long: INSERT: Incremental Software Evolution for Real-time System, Carnegie Mellon University To Near: – From Long: AOP: Aspect-Oriented Programming, Xerox Palo Alto Research Center GWYDION: An Integrated Environment for Evolutionary Software Development & Maintenance, Carnegie Mellon University ACT: Adaptation and Commitment Technology for Evolutionary Software Development, Carnegie Mellon University MITRE 20 09/10/97 10:08 AM Simulation & Modeling Knowledge Management & Representation Ontology Induction Learning Knowledge Discovery/Acquisition Filtering Mining Intelligent Information Processing Categorization Translation Fusion Summarization Extraction Debugging Predictive Modeling Resource Security Risk Composition Replication Behavior Storage Performance Adaptability CORBA, DCOM, etc. Object Network Management Monitoring Communication Repository and Tools Evolution Infrastructure Communications Interface Mobility Virtual Network Adaptability/ Security Error Correction Wireless Operating System Real-Time OS Data Management Metadata Management Shared Data Access (e.g., Location, Mediation, Replication/Distribution, Security/Integrity) Physical Data Management Data Base Segments Legacy Data Network Management Network Services Message Processing Event Processing SignalProcessing Hardware Sentinel Services Interface Common Services Smart Object Coordination, Assessment, Planning, Communication, & & Decision Support Control Applications Applications Verification Reasoning Agent Environment Mission Applications Database Engineering/Migration Proc Shared Applications Dialog Manager Multi-Modal Interaction Distributed Object Services Shared Documents User Modeling Speech and Language Server Agent Interaction Human Computer Interface Agent Controller Visualization and Presentation Process Management Applications (e.g., workflow, monitoring, CASE) Analysis Multipoint Communications Collaborative Environments Context Manager ITO Projects in a Technology Program Framework MITRE 21 09/10/97 10:08 AM Mission Applications Infrastructure Communications Interface Mobility Virtual Network Ensemble/ Adaptability/ Security Error Correction Wireless Nuprl Operating System Real-Time OS Shared Data Access (e.g., Location, Mediation, UniCon Replication/Distribution, Security/Integrity) Physical Data Management Data Base Segments Agent Controller Metadata Management Smart Object Resource Security Risk Composition Replication Behavior Demeter Storage Performance Adaptability DylanWorks Rapide CORBA, DCOM, etc.AppletMagic Object Network Data Management Endeavors EDEM Legacy Data Network Management Network Services Message Processing Event Processing SignalProcessing Hardware Sentinel Multipoint Communications CPAS Integration Database Engineering/Migration Proc MediaDoc Distributed Object Services Umass Arcadia Evolution Monitoring Communication AppletMagic Predictive Modeling Debugging Demeter Management Shared Documents Aleph/Desert Services Interface Common Services Repository and Tools Shared Applications Collaborative Environments Umass CPAS TAOS Process Management TAOS ACME Coordination, DylanWorks Atlantis/Orbit Arcadia Assessment, Planning, Communication, & Applications (e.g., Integration ProDAG ADAM ProDAG & Decision Support ADAM Atlantis/Orbit Control Applications workflow, monitoring, CASE) Rapide PVS/Murphi Ensemble/ WinWinIntegration Endeavors Quest Integration Analysis Verification Reasoning UniCon Applications STeP UniCon Umass Nuprl EDEM Aleph/Desert Simulation & Modeling Knowledge Management & Representation Umass Rapide CPAS Arcadia SECC MICE ACME MICE Discovery/Acquisition Ontology Induction Learning Knowledge Arcadia Integration MediaDoc Filtering Mining Intelligent Information Processing SECC Categorization Translation Fusion Summarization Extraction WinWin Agent Interaction Dialog Manager Agent Environment Context Manager CPAS TAOSIntegration UniCon Endeavors Interface Human Computer SECC ProDAG Visualization EDEM Speech and Multi-Modal Aleph/Desert Atlantis/Orbit and Presentation Language Server Interaction Umass MediaDoc Endeavors Arcadia EDEM WinWin User Modeling Technology Program Framework: 22 Immediate EDCS Projects MITRE 22 09/10/97 10:08 AM MORALE EVOLVERCLIMHuman Computer Interface FLEA 3.0 integration Visualization EMMA Speech and Multi-Modal and Presentation OzWebLanguage Server FAMILIAR Interaction User Modeling Context Manager Technology Program Framework: 21 Near Term EDCS Projects Dialog Manager Mission Applications Shared Data Access (e.g., Location, Mediation, Sanctuary Replication/Distribution, Security/Integrity) Physical Data Management Data Base Segments Agent Interaction Agent Controller Smart Object Management Metadata Management Legacy Data Infrastructure Network Management FLEA Communications Interface CHAIMS NetworkSADL Services Mobility Virtual Network UColorado Adaptability/ Message Processing Security Error Correction Wireless Arcadia Event Processing MetaH Operating System Real-Time OS RTDL SignalProcessing Hardware Sentinel Replication Behavior OzWeb CLOS UColorado Storage Performance Adaptability substrate Arcadia EVOLVER Sanctuary CORBA, DCOM, etc. ADC/Rapide CHAIMS Object Network integration Repository and Tools Jakarta Monitoring Communication ADC/Rapide MetaH Predictive Modeling Debugging CHAIMS Resource Security Risk SADLComposition Database Engineering/Migration Proc Shared Applications Shared Documents Multipoint Communications Collaborative Environments Distributed Object Services UColorado Evolution Arcadia Agent Environment RTDL EBPSDR Coordination, Process Management UColorado Assessment, Planning, Communication, Applications (e.g., EBPSDR & FLEA OzWeb Arcadia & Decision Support FAMILIAR Control Applications workflow, monitoring, CASE) EMMA EMMA FAMILIAR MetaH ADC/Rapide Maude Applications Analysis Verification EASOF Reasoning SADL SE-HOT MORALE ADC/Rapide CHAIMS MetaH UColorado Simulation & Modeling Knowledge Management & Representation EMMA Ontology Induction Learning EASOF Knowledge Discovery/Acquisition EVOLVER Arcadia Jakarta Filtering Mining Intelligent Information Processing integration EVOLVER Categorization Translation Fusion Summarization Extraction EMMA integration Services Interface OzWeb Data Management Common Services OzWeb MITRE 23 09/10/97 10:08 AM Infrastructure Communications Interface Mobility Virtual Network Adaptability/ Security Error Correction Wireless Operating System Real-Time OS INSERT EDCRTS Physical Data Management Data Base Segments User Modeling Agent Interaction Smart Object Shared Data Access (e.g., Location, Mediation, Replication/Distribution, Security/Integrity) Agent Controller Metadata Management ACT Darwin/ Moses Legacy Data Network Management Network Services Message Processing Event Processing SignalProcessing Hardware Sentinel Shared Documents Data Management Agent Environment Distributed Object Services Database Engineering/Migration Proc Services Interface Common Services Darwin/ Evolution MULTIMoses Monitoring Communication PT-UMass PredictiveGEN Modeling Debugging EDCRTS Metaphor Resource Security Risk Composition HASE SE&I Replication Behavior Storage Performance Adaptability CORBA, DCOM, etc. Object Network Multipoint Communications Dialog Manager Gwydion Mission Applications Coordination, Process Management FABASIT PT-UMass Assessment, Planning, Communication, & Applications (e.g., MULTIPT-P FABASIT -UCol & Decision Support Control Applications EDCRTSmonitoring, workflow, CASE) GEN HASE -UCol Applications Analysis Verification Reasoning ACT SSC INSERT Metaphor EDCRTS PT-UMass Express FABASIT Simulation & Modeling Knowledge Management Representation Gwydion & SE&I Discovery/Acquisition AOP Ontology Induction Learning-UCI Knowledge Darwin/ DECR F-HOT FABASIT Filtering Intelligent Information Processing Moses Mining -UCI Categorization Translation Fusion Summarization Extraction Management Gwydion Human Computer Interface EDCRTS Visualization PT-UMassSpeech and Multi-Modal PT-P and Presentation Language Server Interaction ACT Repository and Tools Shared Applications Collaborative Environments Context Manager Technology Program Framework: 18 Long Term EDCS Projects MITRE 24 09/10/97 10:08 AM ITO Projects in a Development Cycle Framework: EDCS Cluster Overview Rationale Capture & Software Understanding EDCS Architecture/ Generation High Assurance & Real-Time Dynamic Languages Formal Verification Design Management Information Survivability Wrappers & Composition MITRE 25 09/10/97 10:08 AM Development Cycle Framework Full Cycle Testing Design Requirements Implementation Maintenance Deployment Run-Time MITRE 26 09/10/97 10:08 AM Development Cycle Framework with EDCS Clusters Full Cycle Testing Design Rationale Capture &Architecture/Generation Formal Software Verification High Requirements Understanding Implementation AssuranceDeployment Design & Real-Time Wrappers & Management Composition Dynamic Languages Maintenance Run-Time MITRE 27 09/10/97 10:08 AM Development Cycle Framework: 22 Immediate EDCS Projects Aleph/Desert Orbit Endeavors/ EDEM UMass/ Arcadia I-SPECS CPAS integration integration TAOS/ ProDAG Demeter ADAM integration Full Cycle Testing Design Atlantis MediaDoc MICE Quest TAOS/ ProDAG PVS/Murphi Rationale Rapide Endeavors/ MediaDoc Capture &Architecture/Generation Formal STeP AppletMagic EDEM Software Verification High Ensemble/ WinWin Requirements Understanding Implementation AssuranceDeployment Nuprl Design ACME & Real-Time Wrappers & Management Composition DylanWorks SECC Maintenance Dynamic Languages Run-Time MITRE 28 09/10/97 10:08 AM Development Cycle Framework: 21 Near Term EDCS Projects Sanctuary EMMA EVOLVER integration Full Cycle Testing Design OzWeb CLIM 3.0 EVOLVER integration Rationale Jakarta EBPSDR ADC/Rapide Maude EASOF EMMA & Capture Formal Architecture/Generation MetaH Verification FAMILIAR MORALE Software High UColorado FAMILIAR Requirements Understanding Implementation AssuranceDeployment Arcadia Design UColorado SADL MetaH & Real-Time Jakarta Arcadia Wrappers & MORALE Management EASOF Composition CHAIMS UColorado Arcadia MORALE Maintenance SE-HOT CLOS Dynamic substrate Languages RTDL CLIM 3.0 Run-Time FLEA MITRE 29 09/10/97 10:08 AM Development Cycle Framework: 18 Long Term EDCS Projects Gwydion Full Cycle EDCRTS Testing Design FABASITUCI Metaphor FABASITPT-UMass PT-P Rationale UCol MULTI-GEN Capture &Architecture/Generation Darwin Formal DECR ACT Moses SSC Software Verification High F-HOT Express Requirements Understanding Design Management Implementation AssuranceDeployment HASE Wrappers ACT & Composition SE&I MULTI-GEN ACT Maintenance INSERT & Real-Time INSERT Gwydion DynamicExpress AOP Languages EDCRTS DECR Run-Time MITRE 30 09/10/97 10:08 AM Development Cycle Framework: Detailed Design Testing Requirements Implementation Deployment Full Cycle Maintenance Run-Time Testing Design Software Analysis Software Design & Architecture Meta-modeling Conformance (mapping) Object-based Modeling Generic ADL Behavior-based Modeling Domain-Specific ADL Interchange/neutral ADL Simulation Animation Requirements Generative Executable Verification(pre) Multiple Styles Constraint-based Model Checking Model Analysis Component Analysis Implementation Static Analysis Legacy Support Dataflow Analysis Connector Analysis Deployment Theorem-proving Run-timel Analysis Regression Testing Event-pattern Rules Run-time Support Validation (post) Event-monitoring Legacy Analysis Implementation, Configuration & Deployment Dynamic Upgrading Maintenance Real-time Support Run-Time Adaptive/Aspect Programming Reengineering Dynamic Language Alternatives/Change Management Rationale Management Code-generating Run-time Constraints Optimization Reverse Engineering MITRE 31 09/10/97 10:08 AM Development Cycle Framework: Detailed Immediate Design Testing Requirements Implementation Deployment Full Cycle Aleph/Desert Orbit MICE Atlantis MediaDoc Testing Design Maintenance Run-Time Software Analysis Software Design & Architecture TAOS/ ProDAG Conformance (mapping) Rapide UniCon Aleph/Desert UniCon Verification(pre) Validation (post) UMass/ Quest PVS/Murphi Arcadia UniCon Model Checking Model Analysis Dataflow Analysis UniCon Interchange/neutral ADL Rapide Rapide Constraint-based Component Analysis Connector Endeavors/ Analysis Ensemble/ Endeavors/ Endeavors/ MICE STeP Simulation Requirements Implementation Deployment Generative EDEM UniCon PVS/MurphiNuprl EDEM EDEM TAOS/ Theorem-proving Static Analysis Run-timel Analysis TAOS/ Rapide UMass/ Rapide ProDAG Executable MICE Regression Testing Event-pattern Rules ProDAG Animation Rapide Aleph/Desert Quest Arcadia Legacy Analysis Event-monitoring SECC AppletMagic Legacy Support Run-time Support Aleph/Desert Quest Meta-modeling Endeavors/ Aleph/Desert EDEM Rapide Demeter Rapide Object-based Modeling Behavior-based Modeling Endeavors/ UMass/ UniCon EDEM Arcadia Generic ADL Domain-Specific ADL ACME Multiple Styles ACME MICE Rapide Implementation, Configuration & Deployment WinWin AppletMagic Dynamic Upgrading Maintenance Real-time Support Run-Time Ensemble/ Adaptive/Aspect Programming Nuprl Demeter MICE SECCReengineering UMass/ Endeavors/ Dynamic Language Arcadia DylanWorks EDEM MediaDoc Alternatives/Change Management Rationale WinWin Management Code-generating Demeter Run-time Constraints Ensemble/ Demeter Nuprl Optimization UniCon Reverse Engineering MITRE 32 09/10/97 10:08 AM Summary of UniCon Visualization and presentation - graphical editor for ADL modeling and taxonomy for many visualization techniques for architectures Data services - connections for database components to applications Object-based information representation with "drill down” Tools to better manage and visualize workflow Generic architectural modeling Generative modeling capability Model analysis Verification Validation Reverse Engineering Technology program framework ISO program requirement Operational context Development cycle MITRE 33 09/10/97 10:08 AM Conclusions These are preliminary findings Study undertaken in necessarily breadth-first fashion If you have additional information or question where your project is wrt maturity level or technical or lifecycle judgment, please contact me Intended as a continuing process for technology transition - Provides feedback to ITO PMs on benefits of research program - Provides ISO PMs quick access to upcoming research Immediately applicable to DARPA architectural efforts Real possibility this could engender a new start in ISO next year: a focused application program which would employ methods and tools from EDCS for full development cycle MITRE