Cognitive Assistance in Government
Papers from the AAAI 2015 Fall Symposium
Cognitive Assistance at Work
Hamid R. Motahari Nezhad
IBM Almaden Research Center
San Jose, CA 95120 United Stated
motahari@us.ibm.com
their main communication paradigm and information sharing,
5) Vast amount of information that are produced,
communicated, processed and needed to be managed by a worker in the work context in order to
perform work effectively;
6) And, the fast pace of the work that has led to
shaping new interaction and communication patterns and habit, and in particular the increasing
use of instant and real-time communication as integrated part of work productivity tools. There are
already startups and enterprise applications that
innovate by bringing messaging tools and integrating it into enterprise collaboration platforms.
As the result of these trends, we witness an enormous
shift in the collaboration platforms which is being replaced
with integrated, social and stream-driven collaboration
environment where all information such as messages, files
apps are reporting into an integrated view, and more importantly (cognitive) agents (automated bots) are participating in conversations among human to facilitate work.
In particular, the growing trend in bringing conversational virtual agents into work context presents unique opportunities and challenges. The opportunities are enormous
in terms of capitalizing on these agents to assist humans in
be more productive, and have them to perform jobs that a
machine would be superior and better than a human, but in
a way that complements human worker. The challenges
include characterizing the type and abilities of a human
and machine, and investigating the art of possible in technologies to have the two (human and machine) to effectively communicate and collaborate.
One other typical challenges of AI, and for building
cognitive assistant [EDWARD A. FEIGENBAUM 2003]
has been characterized as the ability of machines to build a
model of world, their human subject and themselves in a
scalable, adaptive and dynamic manner. The huge amount
of data available in the work context, generated as the results of human interactions, workflow, enterprise applications, databases and knowledge bases and the advances in
methods for processing and building knowledge models
out of unstructured information in systems such as IBM
Abstract
Today’s businesses, government and society work and services are centered around interactions, collaborations and
knowledge work. The pace, amount and veracity of data
generated and processed by a worker has accelerated significantly to the level that challenged human cognitive load
and productivity. On the other hand, big data has provided
an unprecedented opportunity for AI to tackle one of the
main challenges hindering the AI progress: building models
of world in a scalable, adaptive and dynamic manner. In this
paper, we describe the technology requirements of building
cognitive assistance technologies that assists human workers, and present a cognitive work assistant framework that
aims at offering intelligence assistance to workers to improve their productivity and agility. We then describe the
design and development of a set of cognitive services offered by the framework, based on advanced NLP and machine learning methods. The cognitive services help workers
in processing and linking information and identifying and
tracking work items over interactions in communication
channels such as email, social conversations and media,
chats and messaging and calendar applications. These cognitive services are designed to be adaptive, online and personalized so that over time adapt to changing environment
and knowledge, and the models become personalized
through learning preferences and working language and
style of the subject worker.
Introduction
The nature of work is undergoing tremendous transformation. The key drivers of this transformation are as follows:
1) Globalization and geographical distribution of
workforce (remote and global workforce);
2) Faster business cycles and business agility due to
faster product and innovation cycles, and shorter
time to market and time to service;
3) Mobility and the wide adoption of mobile technology among workforce with bring your own device becoming the mainstream;
4) Millennials getting into the workforce, who have
grown up with mobile and social networking as
Copyright © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
37
Watson [AAA 2010] has opened up a new era for AI, and
for enabling technologies for cognitive assistants, towards
building such models of world, human subject and cognitive agents. Nevertheless, this is not a solved problem, and
the problem of developing cognitive assistants requires
taking incremental steps in understanding the technology
need, limits and problems that are enabler of building cognitive assistant.
In this paper, we investigate the problem of building
cognitive assistants that help and complement human
workers. A human worker spends on average of 28 hours
on collaboration, coordination and communication activities (email –reading and responding-, calendar and meetings, communicating on social media and other communication tools). This amounts, on average, for 70% of a normal 40-hrs work week for a knowledge worker in private
enterprise and also in government sectors. To improve with
productivity in this space, we focus on the problem of offering cognitive assistance over conversations and work
organization for a knowledge worker. We start by the generic problem of offering cognitive assistance, characterizing the cognitive abilities of a human and machine (and the
ones that each excel at), and then continue by introducing
innovative technologies methods for the design and development of a cognitive productivity assistant for human
workers.
current AI technologies. Examples of this category include
engaging in creative conversations, emotional intelligence,
etc. Symbol manipulation also happens in the lowest level
of hierarchical structure of brain function. The higher levels of hierarchical structure of brain function involve
emergent concepts where higher level concepts/ideas combine, and form complex organisms (take an analogy with
‘cloud’, a whole, relation to air and water molecules, component). Arguably, there is a subtype of synthetic cognition
that relies on fast and efficient processing of a large
amount of information, which is out of power of human
intelligence, while machines excel at those type of cognition. It is natural to observe that this is one area where machines can complement humans. This is where we define
the scope for cognitive assistance to a person, and in work
context, in particular.
A software program (agent) can be characterized as intelligent if it can employ computational intelligence techniques in order to define a model of the world that facilitates synthetic cognitive tasks. However, the challenge in
achieving this type of intelligence is the building of the
knowledge model of the world and the domain that allows
understanding of the meaning of the intellectual tasks.
In artificial intelligence literature, there has been considerable amount of research in knowledge representation and
knowledge acquisition technologies through reading text
and other data types, and methods based on semantic and
symbolic representations flourished by the concept of
building ontologies in the context of semantic web, and
linked data. Though, most approaches have not been able
to demonstrate the process of knowledge building can be
done in a scalable, adaptive and dynamic manner when
entering new domains or adding new information to the
base model. Though, cognitive computing methods, and in
particular those methods developed in the context of Watson including deep learning methods, for processing unstructured information and building models of the world
(for specific tasks or domains) can be considered as the
first step towards building intelligent methods that can
build models of the world in a scalable manner by processing unstructured information (without the need for
manual crafting of models – e.g. ontologies). Though,
these models yet have not been shows to be adaptive and
dynamic.
In this paper, we investigate the problem of offering
cognitive assistance to enterprise workers by offering analytical cognitive capabilities that supports synthetic cognitive tasks in the context of human collaboration and communication tasks, and organizing work and performing
work for human workers to support higher productivity and
agility.
There has been a tremendous progress in the development of personal assistants in the industry. In particular,the
most popular personal assistants include Apple
Siri,
Problem Statement and State of the Art
A cognitive agent (CA) is defined as a software tool that
augments human intelligence [Engelbart 1962]. To achieve
this objective, a CA should offer complementary cognitive
capabilities to a human by picking up those cognitive tasks
that are time consuming, daunting or require high computational and cognitive power beyond human intelligence,
but in which a machine is greater than human. Let us characterize the overall cognition capabilities into those that are
main differentiator of a machine and human. Cognition
capabilities are categorized into two major types [Eric
Lord, Science, Mind and Paranormal Experience, 2009]: (i)
analytical cognition, and (ii) synthetic cognition.
Analytical cognitive skills include those that the machines excel at them but would take a lot of intellectual
efforts from human. Examples of these capabilities are
mathematical calculations, making logical decisions in
complex situations requiring a series of computation, and
chess all of which are recognized as computational intelligence. This type involves manipulation of symbols (symbolic processing) through algorithmic information processing. At this level, the processing units does not know
or care about the “meaning” of symbol. On the other hand,
the second type of cognitive capabilities are those that human performs effortlessly but are hard for machines with
38
Google Now, Microsoft Cortana, Amazon Echo, Braina,
Samsung's S Voice, LG's Voice Mate, SILVIA, HTC's
Hidi, Nuance’ Vlingo, AIVC, Skyvi, IRIS, Everfriend, Evi,
and Alme (patient assistant). There are also productivity
focused agents including Amy (x.ai), Genee for scheduling
assistant, which focus on one specific domain. These applications offer conversational interfaces for human starting by voice (or some text) which is then processed by
these agents to offer services. A related paradigm is the
notion of offering intelligent conversational assistance as a
service. A number of these platforms include Assistant.ai
(human speech conversational services), ChatBots (chatbots.io), telegram platform for including bots in chat conversations and Watson’s Dialog Service in IBM BlueMix/Watson Developer’s Cloud. All these offer enabling
technologies for a personal assistance however, none, yet,
addresses the problem of supporting the human workers in
getting work done and improving their productivity. This is
the focus of this work to offer such cognitive assistance.
tems, knowledge basis in forms of structured and unstructured information.
In order to support a worker in above three scenarios, we
define a Cognitive Work Assistant as an intelligent software for the work context that offer cognitive services to a
worker following the mythology of: monitor, process, recommend and act. In particular, it would offer the following
capabilities: Understands human language; Monitors collaboration channels including email, calendar, chat and
enterprise information sources; Builds a model of the user
and the world (work context), and is situational aware
(context); Offer assistance by pre-processing information,
and presenting information in human understandable format; Categorizes and filters information; Gathers and organizes related information; Schedules meetings and manages the time on behalf of its human subject, Identifies
requests, and organizing to-dos of its human subject, Assists in performing tasks such as organizing events, travel
assistant; And, suggests taking certain actions to its human
subject that supports increasing productivity, and growth.
We also recognize that there are different roles in the enterprise that may benefit from such a work assistant: managers (people who have a human personal assistant), employees (who do not have a dedicated human personal assistant), and human personal assistants themselves in get-
Cognitive Work Assistant
We envision that need for cognitive assistance of a
knowledge worker at three different domains in private or
government sectors: (1) the assistance in on-boarding,
Employee Cognitive
Agents
Community of cognitive agents that
collaborate effectively with one
another to support human activities.
Assistant’s Cognitive
Agents
Expert Cognitive
Agents
Cognitive Assistant Platform
Interactions types need to be supported:
• Cog-to-Cog interactions,
• Human-Cog interactions, and
• Cog-backed human-to-human interactions
Figure 1. Different types of cognitive work assistants form a community, each specializing in offering one type of service
orientation and growth, which entails ability to proactively
point the worker to the right information and material at
the right time, (2) assistance in off-loading work entailed in
communication and interactions, which account for a significant part of a worker, as it was pointed out earlier in
this paper. In this context, the worker would interact using
synchronous and asynchronous communication channels
such as email, calendar, texting/messaging and network
information diffusion and consumption. And, (3) the work
and project management context in which the worker
would need to interact with organization application sys-
ting their job more efficiently and effectively. Therefore,
we define the following three types of personal assistants:
Cognitive Employee Assistant: These assistants would
have access to the data space (and applications) that their
human subject (employees and managers) has access to
with the same level of visibility, and offer cognitive work
assistance to them.
Assistant’s Cognitive Assistant: An assistant to a human
assistant helps them to become more productive, and focus
on work that require human judgment, while more routine
requests to them is handled by the cognitive assistant
themselves.
39
Expert Process Assistants: This type of assistants are
er of an email, participant in a chat session, or receiver of a
experts in a specific domain such as
travel, human resources (HR), etc.
and are accessed by personal assistants to serve their human subjects.
Delivery Platforms
In Figure 2, we present an architecture for a Cognitive Work Assistant Platform targeting offering
Cognitive Work Assistant Platform
cognitive services for improving the
Cognitive Services APIs
Conversation Interface Bot (voice, text)
productivity over communication
Context-aware
Calendar and
To-do, Task
Email Analytics,
Personal
and collaboration tools and platInformation Finder
Scheduling
and Process
Auto-Response,
Model Builder
Assistant
Assistant
Classification
forms in the enterprise. The key
focus is on offering actionable insights integrated into collaboration
Watson Apps or Services on BlueMix
tools such as emails, chats, workspaces and other work management
Concept and
Knowledge
Questions
Natural Language Parsing
Watson Dialogue
Relationship
Graph
and Answer
Toolkit (PoS Tags, and
Service
systems through integrating an inExtraction
Builder
Dependicy Parser)
telligent agent that processes unstructured (text) and structured inEnterprise Repositories, Applications and Data Sources
formation and identifies action item
(or tasks) that a worker is assigned
Feeds
Document
Repositories
collections
(requests), or commitments and
promises that he makes to others.
Figure 2. The Cognitive Work Assistant Framework and Platform
Assisting a worker requires personalizing all such services, and therefore the “personal model builder”
text/sms, etc.).
module is responsible for scanning and preparing a model
Action recommendation. This module’s functionality is
of the world and the user’s work by monitoring the work
to determine the type of the actionable statement, and make
environment including people he is interacting with, his
recommendation to the subject to take an action. Examples
communication and conversation, and key entities includincluded adding to to-do list, or updating the status of an
ing to-dos, follow-ups, calendar invites, and processes that
action in the to-do list, scheduling a meeting, canceling or
the person is involved in.
rescheduling a meeting. The system has a number of action
The cognitive work assistant is architected as an agent
types that it recognizes, and a template language that althat can be offered as a mobile app that can interact with
lows definition of new action types and corresponding logusers using a voice activated or messaging interface (a chat
ic to hand the action by the agent.
bot) that can participate in chat sessions. In addition, it
Context-aware information finder. This module is an
offers the same services via API calls. At a lower level, the
advanced search module that proactively scans the inforcognitive assistant benefits from Watson services on
mation space of a user and makes links and connections
Bluemix, and a natural language parser that delivers part of
among the entities (emails, calendar invite, files, people
speech tags, word dependencies to verbs, on top of which
and other supported entities). For identifying such links,
an advanced natural language model builder extract key
the information within the content of the entities, and text
elements for actionable statements within conversations,
and the role of the target users will be considered.
and are delivered to the higher level modules within the
cognitive work assistant. In the following, we provide an
overview of key components of the cognitive work assisReferences
tant.
EDWARD A. FEIGENBAUM, Some Challenges and Grand
Actionable Insight Analytics over Conversations. The
Challenges for Computational Intelligence, Journal of the ACM,
conversation among workers include exchange of work
Vol. 50, No. 1, January 2003, pp. 32–40
item and action requests and promises. The module related
David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan,
to actionable insight over Email, Chat and other communiDavid Gondek, Aditya A. Kalyanpur, Adam Lally, J. William
Murdock, Eric Nyberg, John Prager, Nico Schlaefer, and Chris
cation channel focuses on identifying from the text, actionWelty, Building Watson: An Overview of the DeepQA Project
able statements that are of type request, promise or quesAAAI Magazine, 2010.
tions addresses to the audience of the conversation (receiv-
40