Coarse-grained COTS Software Component Architecture, Integrated
by Programmable Middleware; New Human Computer Interactions.
John A. Bayless, Ph.D.
bayle_j@cs.odu.edu
Introduction
This paper describes Software Component Architecture using programmable middleware
to integrate entire COTS (commercial-off-the-shelf) software products as components. It
possesses new qualities of HCI (human-computer-interactivity) synthesized by properties
of the architecture, and supports new actionable cognitive-behavioral HCI possibilities,
not anticipated before they emerged during system observation. These HCI phenomena
have become visible for the first time, having apparently been initialized in the concrete
realization of the architecture, because of properties specific to the inclusion and role of
programmable middleware in the design.
Initial objectives of the project were to [1] greatly simplify design and development of
software to do systematic, asynchronous, user-driven work, such as research users need
in long involved tasks requiring frequent expert user judgment, [2] enable computercapable, but non-computer science professionals, to design and build software suites to
do complex practical tasks advancing their work. The model is a workbench, and bench
tools devised and needed by intellectual craftsmen, for idiosyncratic, personal, expert
professional use. Version 1.0 described here is one example. Other COTS components
can be quickly installed to create radically different suites, the major objective of the
method. Executable software components and elements already in the user’s repertoire
can be integrated into an overall architecture with the COTS components as well.
The architecture concept was successfully built, and the software and methodology were
exercised over 10 following months of producing experimental application prototypes.
The unanticipated actionable properties of the architecture were noted and observed,
which are of interest in computer science and computational cognitive science. They
appear to be a fruitful new research area, and are discussed in the last two sections.
History, Rationale, and Methodology
The writer became interested in the potential of programmable middleware to accomplish
the practical objectives stated above in mid 1995, on being exposed to a primitive early
version of such software, and began monitoring emerging products, and computer science
and engineering literature. These objectives were refined and a development strategy
formulated. A proposal presented to US Army Logistics Integration Agency in February
1998 was funded in August 1998. The concept, middleware, and components were
demonstrated in November 1998, prior to an integration accomplishment and architecture
build. Work funded privately in April 1999 enabled reduction to practice and prototype
development. It was rolled out at a Federal CIO Council Expo February 23, 2000.
John Bayless
Page 1
3/8/2016
Rationale for a middleware approach to a COTS-component architecture was visualized
almost immediately upon initial exposure in 1995. Although the component architecture
model initially appeared in terms of objects as components, the main reason discouraging
consideration of stand-alone commercial products as components has been the daunting
complexity of structural architecture matching and mating such components. In systems
dealing with user interaction, the programmable middleware sidesteps this issue. Second,
‘products’ are unambiguous concepts to users for whom the model was evolved. Third,
the practical technology for facilitating application products in large computer systems
evolved as middleware. Its functionality varies, but in client-server settings middleware
provides services between operating systems and application software including [1]
internal messaging and communication, [2] environment application links analogous to
APIs, [3] transaction monitoring or auditing, [4] gateways to other services and
applications, and [5] data connection. Services vary as one goes up or down scales of
cost and scope, but these are typical. The writer’s insight was that programmability could
potentially extend any attribute of architecture virtually, into a particular middleware’s
other native capability.
Programmable middleware as such, receives very scant mention in computer science
literature, which was a major surprise. A review in 2000 by Bachman, et al., of the SEI
at Carnegie-Mellon, cites only 2 (!) articles, a 1995 paper by J. Bergstra at the University
of Amsterdam describing a scripted “toolbus” in an early stage of design at the time, and
a 1998 paper by Berry on Finesse, a language for manipulating abstract bindings among
large scale components. The writer verified in Mar 2001 by a search for ‘programmable
middleware’ and related terms on multiple search engines, that this is still the case.
A research commitment was to create the entire architecture model infrastructure from
COTS products. Inventing infrastructure and new tiers in infrastructures is too difficult.
Paramount questions are: Is this objective practical? Is it feasible to hope to empower
technically capable, non-computer science professionals thus, given the technology limits
of the market? Is architecture possible within those limits? The critical path had to lead
to COTS infrastructure with features supporting interesting workbenches and commercial
vitality to ensure practical longevity and successive technology upgrades.
Analysis of OEM (original equipment manufacturer) mid tier products revealed products
with some or all integrating functions of middleware not described as ‘middleware’ by
OEMs for business model reasons. Products with long user experience, stable platform
support, and ubiquity, possessing integrating functions, plus programmability, did exist.
The COTS product selected was a programmable support platform tier produced by the
OEM as infrastructure for their own manufactured large-scale enterprise software. The
OEM business model focuses on sale of their vertical market products supported by the
platform, and only incidentally on sale of the middleware itself. The term ‘middleware’
is not used in their product literature, but the descriptions following will make clear how
it meets the rule. Personal communication between the writer and many of the OEM top
technical and engineering design managers agree this is an accurate characterization.
Four Large-grained COTS Components were selected. They include [1] a portal to data,
work groups and workflow, [2] support for geospatial work display and logistics user
John Bayless
Page 2
3/8/2016
workflow cognition, [3] EDI mapping with services across many commercial domains,
and [4] automated records management (which is DoD Standard 5015.2 compliant.)
Choices reflected the writer’s desire to create a demonstration that could be programmed
to show off applications to a wide audience. A capable-user should be able to produce
specifications from the user’s domain of applications, at the same level of abstraction.
Design at the level of selecting COTS products determines the features and code base of
the system. The end user thereby has detailed specification influence not typical of
“big” software design.
With infrastructure and components chosen, their resource connections are now the nodes
of many possible user layouts, with fluid native capabilities. This is the desired result.
Design is now adaptable to the capable user’s expertise: Design always structures the
user’s process. Instead of conforming user process detail to structural detail necessity,
this process known best to the user can be laid out in large fluid chunks by the user. This
design model now accepts user choice uncertainty. The user can now think about
interventions, and create required access to the run-time process. The product evaluation
is for suitability to user process, rather than infrastructure plumbing. The expert user’s
workbench preferences can now structure a bench-tool process design, almost completely
idiosyncratically.
The following figure summarizes the architecture as defined after the technology review,
at the end of the design phase.
Coarse-grained component architecture
With “shrink-wrapped” COTS applications as components,
integrated by programmable middleware, and centrally
served user-interactive process automation workflow . . .
•
Middleware centrally builds and executes automated just-in-time
data messaging between integrated components and processes,
confluently compiled into centrally executed user applications.
•
Users interact with and control the system in real time, via web
browser interface receiving interactive web pages
•
Integrated COTS software applications & product-based services:
– EDI mapping and e-commerce automation embedded in the business
process automation
– Electronic records management integrated into the work flow and the
process application builder
• DoD Standard 5015.2 compliant
– Enterprise portal access to both data and workflow
– Geo-spatial workflow and data presentation
Figure 1. Characteristics of the Suite at the End of the Design Phase
John Bayless
Page 3
3/8/2016
The first three bullets characterize the design attributes as ‘system’ in summary. Bullets
at the bottom list Coarse-grained Architecture Component capabilities. COTS product
components and summaries of their features are identified in Figure 2.
Architecture Overview
Architecture is designed structure and infrastructure that ‘stands up’ components of an
entity and permanently establishes relationships between them. In software engineering,
architecture applies to tiers of permanent logical and electronic structure for logical data
and meta-data ‘piping’ between software and hardware components. The following is a
summarized structural image of the suite. It is an as-built example of the possibilities
afforded by programmable middleware integrating COTS components. This approach to
establishing the architecture, successfully avoided the problem of designing and building
a new tier of permanent structural logic to mate plumbing and wiring differences of fully
completed COTS software components. It also has synthesized and virtual aspects that
support new HCI qualities, which appear to have been synthesized from within the suite
as built, and establish additional value, or raison d’être, for application and study.
Figure 2. The Coarse-grained COTS Component Architecture Suite As Built
The architecture components are named here, as well as characterized, to emphasize that
this is truly an architecture of COTS commodity software products. These are out-of-theshrink-wrapped-box, unmodified, standalone, commercial products. They do not need
each other, or the middleware, in their regular business lives. Not included here, but part
John Bayless
Page 4
3/8/2016
of the system, is Microsoft Back Office, mounting the middleware and products on NT
Server 4.0, providing MS Exchange e-mail services, Outlook e-mail client, Microsoft IIS
web server, Internet Explorer 5.5 browser client, and SQL Server, data base to workflow
resources, all of which are also employed in the suite. This was a convenience since NT
4.0 Server platforms the middleware. Other e-mail servers, ODBC databases, web
servers, and browsers are equally workable.
Drawing on the functionality of the four integrated components, a runtime engine native
to the middleware, serves process-centric, agent-triggered, middleware-built, workflow
automation to the users of the system,. With different COTS software components using
the same middleware and methods, the suite functionality would be totally different.
Many possibilities are afforded by the model. New levels of access to design, software
system building, and application development is now exposed to capable users who are
not software engineers or computer science professionals. Self-initiated capable-users
are not required to see interior plumbing to add components, or with minimal assistance
and mentoring, to specify, design, build, program, and use a kind of en suite software
based on attributes of the architecture, provided by the middleware.
The architecture’s centrally positioned attributes, native to the middleware, include:
 Resource links to data anywhere in the world, reachable through LAN or WAN and
TCP/IP, or dialup telephony.
 Links that integrate external application software—the COTS components.
 Application building/programming environment creating ‘suite-oriented’ application
building perspective. The application builder has a view from which to program all
integrated COTS products simultaneously, from the highest level in the system.
 Comprehensive suite applications use and control the executable resources of all
COTS software components at runtime, from one central middleware runtime engine.
 Centralized workflow programming tools in the application builder, referencing the
entire data environment, enable the embedding of inter-component runtime data
messaging within every user application program.
 End-to-end process automation service to all users, draws on any linked data and
integrated software. Different users instantiating different applications software are
served data messaging and inter-component workflow automatically.
 User interaction GUIs are web browser screens. The middleware application builder
authors web sites and built-on-the-fly interactive web pages, sent by the runtime
engine to an IIS or Netscape web server, supporting runtime user interaction
 ActiveX controls and Javascript are not required or used for web automation by the
middleware application builder. They are available as extra automation features, or
may be omitted entirely, should user security so require.
 Computer-telephony integration also provides voice-enabled services, optionally
including high-end voice, speaker, and natural language recognition,.
John Bayless
Page 5
3/8/2016
A. The Middleware View
The middleware view is in the bottom half of the following graphic.
Figure 3. Summary Structure and Features of the Programmable Middleware.
The two outside blocks in the top row are user connection functionality. The two inside
blocks are data connection resources and a software agent pool, respectively.
 The middleware creates user interaction GUIs through a native web site publishing
capability, producing built-on-the-fly pages, with control, text, and data input for realtime user interaction and manual workflow intervention. Pages served to the user’s
PC browser provide the primary GUI interface and user screens. They afford users
text and graphic work output from the functionality created by the entire suite and its
applications, and interactive input controls and data entry fields for entering human
judgments and interventions in the runtime automation.
 The application builder includes a web page editor among its tools. Most third party
tools such as DreamWeaver can also be used for authoring. Their HTML files are reopened in the native editor to have middleware-specific control tags added. The use
of outside tools especially, affords the capable user/designer state-of-the-art control
over every multimedia detail of interactive system GUIs. The middleware publishes
pages at run-time to IIS or Netscape web servers.
John Bayless
Page 6
3/8/2016
 In addition to this PC-and-browser interface, the middleware provides computertelephony integration and has telephony, fax, pager, WAP resource connections, and
optional high-end interactive voice recognition and text-to-speech capability. The
voice recognition features include speaker recognition, vocabulary to 200,000 words
and phrases, and recognition through regional accents and imperfect grammar. In a
group research setting the workflow can generate and complete remote telephone or
pager messaging to team members who may be across a campus or across the world,
in real time, in correlation with workflow events. For example, to wake up and logon
to observe crucial experimental events. Fax and pager text can be sent in the same
way, composed dynamically by the workflow and transmitted to a single recipient or
to a hundred-member research team.
 ODBC database client and host-terminal access to hosted legacy data are provided.
Major RDBMS products such as Oracle can be accessed directly with middleware
registered clients. Product with registered ODBC drivers, from Excel spreadsheets to
statistical analysis products such as SPSS or Systat, can be reached as data sources,
or to receive computed data from the full suite. Terminal emulation products such as
NetManage provide mainframe host access via 3270, 5250, VT-100, other emulated
terminals. The system learns and retains host screen navigation to provide networked
hosted data and application connections.
 Basic software agent services are allocated from a pool to monitor web job logon
pages, allocated telephony job phone and fax lines, clocks, database conditions, the
integrated application conditions, other triggering events. When an event such as a
user logon occurs, the agent instantiates and runs assigned stored workflow services,
initiates services and actions requested by the user, interprets user trigger events such
as the mouse clicks on web page controls, and supervises jobs internally according to
the evolving conditions created by the user. When simultaneous users are logged on,
separate agents are assigned to each user, instantiating entirely separate conditions
and jobs for each user. In a single middleware-server node system, up to 30 users can
be accommodated simultaneously. In multiple node systems each middleware server
node can accommodate from 250 to 280 simultaneous users. Some large commercial
OEM installations have in excess of 850 or more nodes, deployed worldwide, able to
serve well over 225,000 simultaneously logged-on, distributed users..
 Integrated COTS product architecture components are lumped in the red block.
Although this effort focused on COTS product components, any well-behaved
executable software reachable through the system links can be integrated, including
legacy, custom-built, or experimental software in the user’s portfolio, and COM
objects.
B. Component Mating and Data Messaging
Each product is linked to the middleware, which functions as the backbone of the suite,
through an idiosyncratic combination provided by the product’s native APIs, shared
databases, and the DLL and NT Command objects of the of the application builder and
runtime engine. Registering COTS product DLLs and their functions in the middleware
library services, allows the application builder to reference, and the runtime engine to
John Bayless
Page 7
3/8/2016
directly access, product DLLs and their functions through DLL objects in compiled
automated workflow jobs, created in the application builder. This software application
integration linkage, analogous to an API, is common to all application software in the
middleware environment, i.e., the COTS product components
Similar application builder reference and runtime engine access is provided by an NT
Command object. Command objects in the workflow stream cause NT commands to be
executed, and make it possible to wrap command line scripts within Command objects.
This enables construction of original command line scripts embedded in compiled C++,
callable as primary tasks, or as workflow subroutines.
Abstract and practical design factors arise from the locale of DLL integration and
Command Line objects as resources situated in the middleware application builder and
runtime engine, as follows:
[1] They enable just-in-time coupling of component resource connections,
[2] to data connections allocated just-in-time by a runtime code stream,
[3] exchanging application run-time values between components and workflow
[4] in data objects native to the middleware,
in a component workflow process that would otherwise be provided in a messaging tier.
This virtualizes all the literal structures of logical abstraction or integration plumbing. A
recurring software engineering caution or advice encountered during the project was that
such a loose coupling could result in non-robust, inefficient, architecture failure points,
but in experience, this was not the case. All actual connection and data messaging among
the COTS components is an on-the-fly, just-in-time process, as summarized below, rather
than permanently embodied structure such as tiered difference-matching abstraction and
metastructure.
 Natively embedded workflow tasks, procedural steps, tools and rules needed at each
step, source and destination resource connections, input and output data objects,
application builder, and runtime engine, make it possible to construct virtually in the
middleware, all the inter-component data and meta-data messaging results that would
be conventionally performed in a new permanent structural architecture tier.
 During the application design process, the processing features and services to be
provided by the diverse COTS products are identified. Accessing and using them as a
programming activity is performed in the application builder. This will typically
require data input to the product feature or service, the service or feature operation on
the data, and data exit to a destination. This process and operations are specified in
the application builder with tools mentioned above. Ultimate sources and
destinations of data will be databases and/or other applications reachable by the
middleware on the TCP/IP network, referenced by IP address and data name in the
application program. Local or proximal sources and destinations at runtime are
discussed below.
 Data messaging and other inter-component workflow required for each application is
interleaved and confluent with normal procedural steps of the functional application
John Bayless
Page 8
3/8/2016
programs, such that just-in-time data messaging workflow is compiled into every
application program. The total compiled object code is a bi-level application that
executes its own messaging as a just-in-time process function, concurrently with
application code procedures, via the runtime engine. This virtualization of the
architecture’s results, avoids the requirement for a permanent structural messaging
tier, and associated runtime processing overhead.
 Accessing and using data links at runtime is done by the runtime engine running the
compiled application. The proximal data sources and destinations, when the runtime
engine is processing, are varied. Sometimes database connections or other ODBC
sources themselves are allocated in-line during the workflow. This is especially true
during short workflows involving many independent users, like standard transaction
processes. With a long complex workflow for a single user, the proximal data
sources and destinations are internal middleware data objects: typed objects, list
objects, package objects, complex objects, etc, native to the workflow process,
located in RAM, on-board cache, or page addresses known in the runtime stack.
Intermediate data objects will be used repeatedly in longer workflows.
 Automated just-in-time workflow linkage is to application resources throughout
application domains ranging from a single room to worldwide. Workflows can be
catenated end-to-end to create complex messaging extended through the domain.
Especially in systems where interacting human behavior latencies predominate there
may be nothing practical to improve by architecture matching and mating structural
software components, and very little to risk relying on loose coupling. Benefits of loose
coupling are high: Conceptual clarity, fast assembly and minimal cost , all for the
capable-user-designer-developer. It is a capable-user-friendly strategy.
Product APIs provide access to product functionality through the manufacturers APIs
(application programming interfaces). These are accessible from the users application
programming perspective. This subtle but necessary conceptual difference addresses the
bi-level nature of the compiled executable application code. Two interleaved correlated
programs are compiled together. One executes the user’s functional application, whether
it be related to conduct of research, scientific data analysis, combat management, or
commercial activity. The other executes correlated component data messaging, building
structural connections just-in-time, de-allocating after use.
User application connections will be through the COTS product APIs in most cases,
accessing application-value features. Workflow and data messaging connections will
always be through the middleware facilities because of their logical hub location in the
architecture.
No logical or locale necessity prevents application connections from the middleware
facilities, however. Registering product DLLs and their functions in the middleware,
allows the application builder to reference, and the runtime engine to directly access
product DLLs and their functions through workflow DLL objects. To the OEM, these
hooks facilitate integration of the middleware smoothly into customer environments.
They allow designers of customer installations to hook seamlessly to existing customer
software and pass transaction values back and forth between already installed functional
John Bayless
Page 9
3/8/2016
customer software and processes, and the workflow mediated processes of the new OEM
vertical products installed. However, these application builder and runtime engine DLL
object hooks mean that all process resources of a COTS product can be introduced as
user workflow functionality, not merely that which incidentally facilitates middleware
OEM vertical product integration. Users can write and register their own DLLs to access
product functionality in any idiosyncratic way facilitating user workbench purposes.
Similar application builder reference and runtime engine access is afforded via the NT
Command object. Each NT Command object in the workflow contains one NT command
to be executed, which is the way they are typically used by OEM designers hooking up a
customer workflow integration to pass and receive values between installed customer
software and workflow controlled OEM software. It is also possible to chain Command
objects, enabling construction of command line scripts called from objects in compiled
C++, as primary tasks, or workflow subroutines.
Other integration linkages include
 Shared product and/or any other databases to which the middleware connects via a
registered database client, (e.g., Oracle) or an ODBC client. Best-practice use of the
middleware also includes a database as a central working repository. All component
software has access to this database.
 Access to IBM, UNIX, other hosted data and applications via terminal emulation.
 The runtime engine runs the compiled C++ bi-level workflow/functional applications
for the suite, created by the native application builder. Collections of these jobs
comprise functional application services of the system, instantiated and run by the
system’s software agents. This is discussed more under Workbench Job Descriptions.
Because of the OEM’s strong focus on other attributes, a number of the most interesting
structural and functional results that stem from technical properties of this middleware
had not been investigated before the writer’s search for COTS workbench infrastructure.
Many of the notable phenomena possible to create and support with programmable
middleware flow from these uninvestigated properties,
Research Workbench Requirements and Middleware Affordances
For capable users envisioned by the research, these are actionable properties and hooks
provided by this Large-Component methodology. They point ahead to other extensions,
not now part of the prototype version.
A. General Middleware Extensibility
General extensibility is an undocumented actionable property of the middleware afforded
by application builder and runtime engine DLL and NT Command objects. Using them
this way enables the middleware to be an architecture ‘shape-shifter,’ one of the primary
practical attributes required by the ‘Research Workbench’ project.
As immediate design and programming affordances, integration of complete multiple
products through these application builder and runtime engine DLL and Command Line
John Bayless
Page 10
3/8/2016
objects, means that all the process resources of the COTS product components can now
be introduced as user workflow functionality.
B. User-Designed Functions, Features, and Component Code Base
In this regard the affordances of a middleware-integrated research workbench make it
possible to take advantage of thousands of COTS products that are highly specialized
niche items designed to stand alone, or user require customization. This means being
able to buy and use the same code over and over, just as a homeowner uses browser and
e-mail. Middleware and architecture allow the user to assemble a re-usable commodity
product code-base quickly and inexpensively, rather than slowly build an expensive code
base with little long term potential.
The workflow code itself, derived from the application builder is only 1 to 2 percent of
any Workbench Job application’s total compiled code—The rest is the middleware
connection infrastructure code that comes out of the box, plus Component functionality
code, which also comes out of the box. The small amount of code actually originally
written for a given Job Description is leveraged by 50 to 100 times as much re-used
infrastructure code, which is unchanged from job to job. The resulting improvement in
immediate efficiency and practical utility and value of computers in the world of the
individual researcher is immediately obvious.
C. The Capable-User Software Builder and Self Developer
It is entirely practical for a computer-capable person willing to give two weeks to formal
application-builder training, and a little more in practice, to be a self-developer.
In commercial practice customer applications are built from flow-charted designs evolved
together by a customer-user and trained developer. The middleware’s application builder
was designed to create workflow applications originally called Job Descriptions, a name
inspired by early 90’s software agent technology. Agents as designed then, monitor IVR
telephone lines, trigger workflow applications requested by callers, interpret interactive
user-originated events, then trigger appropriate system workflow responses, until the user
session is closed out, hence these were visualized as a software agent’s job description.
Those features were extended to the web extensions introduced in 1996, to monitor users
signing on to web logon pages, and user page events.
The Job Description concept is apt for Workbench applications, so these are called
Workbench Job Descriptions, as a metaphor for connectable knowledge work elements in
the human researcher’s job. In the demonstration, the term Job Description applies to
application builder programs doing rule-based work based on user-added knowledge.
The workbench model is not limited to rule-based work, however, even in its present
iteration. In-line inferential algorithms and complete AI software applications can be
integrated into the architecture today. Inferential and AI functionality is immediately
available to the Workbench Job Description application builder, just as is rule-based
logic. This is expanded in the next section.
The application builder uses drag and drop elements—objects—mapped to a multilevel
GUI flow diagram, to develop the most often used actions of typical workflow process,
such as database transactions. Each object exposes built in methods to the developer in
menus and dialog boxes. COM object automation can also be included in a workflow.
John Bayless
Page 11
3/8/2016
Finally, the entire analytical and developmental power inherent in the C++ programming
language is represented in the application builder by objects called Expression Forms,
thus enabling a capable user to build and include logic and analytical computation to any
level of detail required by the researcher’s task at, or for, the Workbench.
Completed flow diagrams that comprise Job Descriptions are compiled into segmented
C++ executables, run by basic software agents that monitor the triggering events in the
system, such as a user signing on to a job logon page. Users control the suite through an
exchange of built-on-the-fly web pages with the middleware’s native web site, which
present GUIs to the user, including text boxes for natural language entry. Optional voice
and speaker recognition capability provides spoken text entry, although the current demo
was not equipped with this feature for cost reasons. Interpreted natural language, written
or spoken, allows modification of the GUI controls.
Human-Computer-Interactions in Programmable Middleware Architecture
After the software component suite was reduced to practice with the programmable
middleware, it was observed in action for 10 months, being used to produce prototype
application trials. Several interesting, unanticipated, new attributes were seen. None are
counterintuitive, now that they have been noted, but few were predicted either, because
they are a different kind of affordance phenomena than the writer had been predisposed to
imagine through earlier experience with the software. Post-integration observation and
reflection provided a deep “Aha” experience. They present new issues in computer
science and computational cognitive science.
A. Suite-Programming Point Of View
A new suite-programming point-of-view has been introduced by the architecture. The
application/builder runtime engine locale is also the physical means for performing (and
observing) what is described next. In affordance terms, runtime engine and application
builder are a single locale. The application builder is the user interface to the runtime
engine, thus also exposing its affordances. Extension of the middleware is done here by
using these loose-coupling links to access properties of external COTS products
integrated as components. Other links may exist and be used as well but the application
builder links tie directly to component executables and DLLs.
Initializing and linking the architecture and suite is as follows, in summary: The
middleware platform is installed on a TCP/IP network. The data sources must be
accessible from the network, with optimum access deployment being on the same LAN
or WAN. The initial middleware install is a multi-step, but well documented process. A
pre-planned default initial install is perhaps a 1-day process. Initial configuration is a
similar multi-step process that can take 2 to 4 more days.
COTS software is installed according to the instructions in the boxes, on the TCP/IP
network with the middleware. Their IP addresses are registered with the middleware
during its configuration.
Installation and configuration per OEM instructions, of middleware, database access, and
all COTS software, plus installation or connection and configuration of Microsoft Back
Office or equivalent services, verification of all IP addresses to the Domain Name Server,
John Bayless
Page 12
3/8/2016
and final verification of TCP/IP services, completes initial installation of the architecture,
components and means for runtime connection are concerned. This is the system build
on-site, a process of following OEM and default InstallShield procedures that a capable
user can do in person.. Network configuration is also standard, but typical institution
procedures will involve using the in-house network technicians who are responsible for
maintaining IP addresses and domain configuration. It will be a completely routine ‘allin-the-day’s-work’ task for them.
The rest of ‘standing up the architecture’ takes place during application design and at
runtime, performed in the application builder and runtime engine, which concretely
complete the ‘loose-coupling’ definitions, and their on-the-fly connections at runtime.
A subtle new actionable property has now been created. As a structural architecture
backbone, programmable middleware makes integrated COTS software programmable
also, in ways not fully anticipated. When a purposefully selected suite of COTS products
is made the basis of actual application building, something new happens, that does not
happen when using the links in incidental customer application integration, as practiced
by OEM application designers following the OEM business model, even though the
capable-user is reading the same product application user guides.
Each separate component is a fully developed commercial product of high value to users
who buy it as a standalone item. The product actions and services are features producing
task value for the user. Those task values are already in place for the user, or the product
obviously would have no market. What is programmed through the application builder to
be run by the runtime engine are these in-front-of-the-product task values, features that
would be available to a user with mouse and keyboard, and product menu bars, menus,
commands and options, for each separate product as a whole. This is not like an OEM
application programmer extracting from a hosted data-base with a Host Terminal object
to load a List object to send to a table in a Page object. The capable-user application
builder working 5 big software products, with an in-front-of-the-product viewpoint, has
an absolutely different vision of the affordances at hand, and goes about everything
differently. It is a researcher’s viewpoint, not a technician’s
B. Architecture-Enabled Suite-Programming Synthesizes Procedure Automation
To Users From COTS Products Without Native Procedure Automation Features
The component-extended middleware synthesizes workflow automation in a new way, not
available in non-extended middleware, which concretely realizes suite-programming.
The application developed from this viewpoint is then compiled as a C++ executable. At
run-time the runtime engine invokes the integrated COTS software as required to perform
the application as programmed. Then another new actionable property appears that is
established by the architecture. The runtime engine is now centrally serving the
programmed application automation to the extended-middleware integrated component
software to perform. COTS product components with no native workflow automation
features now perform the automated application task flow by virtue of loosely coupled
integration in a programmable architecture—a thought-provoking result.
John Bayless
Page 13
3/8/2016
C. Architecture Creates New User Cognitive and Control Perspectives
Several other new actionable properties now appear due to the architecture, which
provide new cognitive experiences for users.
[1] The user (or workgroup) perceives an entire suite of separate COTS products as a
single multi-featured task entity, when presented by the middleware at electronic speeds,
It is user-engaged as a single centrally-controlled unit, experientially.
Running Job Descriptions written in the application builder, the runtime engine is serving
programmed instruction to each product, as the human developer would use them as
independent enterprise-level desktop applications, but now in interactions interleaved
among the components at electronic speed. The suite-as-a-whole, under control of the
middleware with all the rest of its infrastructure and parts, serves these programmed
results on to the interacting user, along with the user’s interactive input as computed in
real-time.
 From a user’s cognitive perspective the architecture centrally serves previous userprogrammed workflow automation, combined with present user interaction, together with
free-running product background automation results, to a user’s perception and response
options, simultaneously, from all the product-components of the suite. As do most such
stand-alone COTS products, the products provide a mix of background and user-operated
services. For example, the COTS electronic records management component can be set
up to automatically index and file in its own locked repository, as an ongoing background
process, any defined document that appears in the workflow, e.g., e-mail with a particular
addressee, or sender, or containing particular keywords in the subject line, or in the body
of the message. It will also allow the user on-line at the GUI to declare official records
on an ad hoc basis, or to declare new classes of records, such as .doc items with different
addressee or subject text. There is a large ad hoc feature repertoire designed for user
interaction at the desktop, available to the application builder and programmable control,
via the NT Command object. Likewise the geospatial display system allows the on-line
user creation of new display dimensions and features. All these user-performed features
are now automated by the workflow and performed at electronic speeds by a software
agent running the workflow.
 Suddenly, in the perceptual experience of the user, these products en suite are now
performing the users formerly manual task stream as a synthesized combination of [a]
their own workflow-automated previously-planned behavior, occurring at electronic
speeds, served back to them as if by thought, [b] the product-contributed background
automation, and [c] their own in-line, real-time behavior at normal user performance
speeds, all in relation to outside event sources as well. Users perceive interestingly
variable, diverse event-source dimensionalities, bearing information about related events,
from a new synthesized perspective. This is a new cognitive computation affordance
[2] This observation about suite-synthesized input to user perception also obviously
applies to controls. If components provide synthesized input to independent user
perceptual modalities (auditory, tactile, multidimensional visual, etc), and at the same
time other components receive output from appropriate response option devices, new
learned multidimensional user responding and synthesized distribution possibilities, can
be created via the centrally programmable architecture as well.
John Bayless
Page 14
3/8/2016
The suite-programming viewpoint, together with the quality that interactively combines,
controls, and adds process automation to previously unconnected processes, and further
together with a new perceptual-cognitive perspective, creates a new actionable cognitiveresponse possibility of harmonious control of multiple distributed activities as a user’s
physical and cognitive en suite affordance of intention .
New visualizations come to mind immediately and will multiply in the minds of others:

A battle commander controlling a suite of mobile weapons in the field, in the air,
in space, or under water as a singly-intended, multi-dimensional implement;

A surgeon controlling several distinct micro-tools as a singly-intended, multidimensional surgical implement;

A student entering a multidimensional instructional environment controlled
simultaneously in real time by singly-intended teachers,

A researcher creating a singly-intended multidimensional research environment,
dynamically controllable in all dimensions, in real time, including data recording;

Multiple AI applications in synchronous use by singly-intended user

Multiple agent communities under singly-intended experimental manipulation
 Synthesized values produced by all components can be perceived via multimedia
display elements and features that can be multidimensional and extremely sophisticated.
This leads to other new aspects such as: [1] Possibility for a researcher to comprehensibly
perceive multiple synchronous research data sources simultaneously, for understanding
by a single experienced mind, [2] the possibility for new comprehensible cognitive
learning experience from more sophisticated multidimensional ‘surround’ data display/
exhibition devices, and acquisition of entirely new multidimensional cognitive skills.
 The system as a whole can be configured as entirely rule based, as rule-based plus
input of user present-judgment-based control, or as including inferential AI components,
either in-line in the workflow stream, or in the resource product suite. The possibility for
creating inferential workflow, or workflow containing AI elements of choice comes first
from the fact that they can be included as components in the suite. Multiple AI elements,
models and agent communities, can be set up within a single server based suite, as well as
interactions with parallel interacting suite systems on separate servers, interacting as one
chooses, by remote workflow calls. Combination AI and human user interaction through
dual port video displays is also possible in the current implementation, so human users
may observe uninterrupted AI output on one screen while interacting via the workflow
through another. Finally, in addition to the drag-and-drop objects that enable the speed
and capable-user friendliness of the application builder, it also contains the complete
capability of the C++ programming language, enabling the coding and compilation of inline inferential logic within the Workbench Job Description.
These possibilities which became apparent in a working system, appear to be new
actionable qualities of computer-interactive cognitive experience. The results noted
earlier were concretely observed to be new action properties of computer utilization in
general. It is possible to visualize that the architecture could become a privileged source
John Bayless
Page 15
3/8/2016
of many new computer use affordances given imaginative and novel integrated COTS
components that others will doubtless bring to it. It clearly widens the avenues of both
abstract and applied research in computer science and computational cognitive science.
John Bayless
Page 16
3/8/2016