See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/229060177
Guideline to Efficient Smart Home Design for Rapid AI Prototyping: A Case Study
Conference Paper · June 2012
DOI: 10.1145/2413097.2413134
CITATIONS
READS
37
1,401
3 authors:
Kévin Bouchard
Bruno Bouchard
University of Québec in Chicoutimi
University of Québec in Chicoutimi
136 PUBLICATIONS 1,204 CITATIONS
174 PUBLICATIONS 2,039 CITATIONS
SEE PROFILE
Abdenour Bouzouane
University of Québec in Chicoutimi
154 PUBLICATIONS 2,008 CITATIONS
SEE PROFILE
All content following this page was uploaded by Bruno Bouchard on 19 May 2014.
The user has requested enhancement of the downloaded file.
SEE PROFILE
Guidelines to Efficient Smart Home Design for Rapid AI
Prototyping: A Case Study
Kevin Bouchard, Bruno Bouchard and Abdenour Bouzouane
LIARA Laboratory
Universite du Quebec a Chicoutimi (UQAC)
Chicoutimi, G7H 2B1, Canada
{kevin.bouchard, bruno.bouchard, abdenour.bouzouane}@uqac.ca
ABSTRACT
Advances in ubiquitous technology have moved us towards the
dream of creating intelligent houses that can help human in their
everyday life. The next step in the completion of this vision is to
make major breakthroughs in artificial intelligence. In fact, it is
the key component for allowing sensors and effectors to give
useful services when it is appropriate. In consequence, researchers
need to conduct more experiments in realistic setting (e.g. smart
home). In order to face this challenge, many research teams try to
build new experimental infrastructures without any background
experience, guidance or even a real idea of their research needs
and issues. Our team is composed of specialists in AI for
cognitive assistance and has worked with four major smart home
infrastructures. From that experience, we propose, in this paper, a
set of guidelines for designing and implementing an efficient
smart home architecture on both hardware and software
perspective. This paper aims to be a major step toward the AI
development (rapid prototyping) and smart home research.
Moreover, we share our recent experience with the construction of
a new smart home and clinical trials conducted at our laboratory
with real Alzheimer’s subjects.
Categories and Subject Descriptors
D.2.11 [Software Architectures]: Data abstraction, Domainspecific architectures, H.1.2 [User/Machine Systems], J.3 [Life
and Medical Sciences]: Health.
General Terms
Algorithms, Design, Reliability, Experimentation, Human Factors,
Standardization.
Keywords
Smart home design, cognitive assistance, architecture, guidelines,
prototyping, hardware, software
1. INTRODUCTION
Smart home has become a very active topic of research in ambient
intelligence in the last decades primarily due to advance in
engineering and technology [1]. Scientists and private corpo-
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, to republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
PETRA'12, June 6 - 8, 2012, Crete Island, Greece.
Copyright © 2012 ACM ISBN 978-1-4503-1300-1/12/06... $10.00
rations around the world are working on this paradigm either to
make life easier for habitants or to provide services to a particular
target audience such as Alzheimer patients. Many believe, such as
our team at the LIARA laboratory, that smart home could be
exploited to provide support services to vulnerable persons and
those struck by cognitive impairment. However, smart homes that
are implemented and used in real contexts are actually not that
“smart”. To enable intelligence in smart homes, researches on
Artificial Intelligence (AI) have first to tackle many challenges. In
fact, we believe that the required technology, including sensors
and effectors, to achieve the ultimate objective of technological
assistance at home is currently available on the market. AI is the
key component missing to turn that technology into useful
services [2]. More precisely, we lack a brain to interpret raw data
perceived from environmental sensors in order to target the
assisting needs of the resident and to be able to select an adequate
technology adapted to assist him considering his profile.
To accomplish the objectives of AI, researchers will require to
conduct larger experimentations inside real smart homes and to
improve the collaboration of teams around the world in order to
incrementally progress through the visionary goal. The design and
implementation phase of smart home is quite complex. These very
expensive projects are generally completely built from scratch and
are often the first attempt of laboratories (lacking experience)
resulting in repetition of design errors. Our team, composed
primarily of AI specialists, has implemented projects through the
years on four major smart environments infrastructure worldwide
[3-5] including our own recently built full size smart home. From
that experience, we propose, in this paper, a set of guidelines for
designing and implementing an efficient smart home architecture,
both hardware and software, specifically oriented for cognitive
assistance our main application background. Our objective is to
facilitate research by investigating these three points:
1. How to design hardware for rapid AI prototyping?
2. How to design software for rapid AI prototyping?
3. Promote collaboration and exchange for incremental
advances.
We think that not only these guidelines will prevent research
teams from repeating certain design errors, but it will as well
shorten the conception phase. In this paper, we also propose to
study the case of our own intelligent house and to present the
experiments that were conducted at the laboratory from the
contextual point of view. To improve rapid prototyping of AI
algorithms, we will describe the best architectural choice, the
technological options and the software techniques we used. Our
Legend –
Installation
Data
complexity
Videocamera
Microphone
Electromagnetic Contact
IR Motion Sensors
RFID Tags
Pressure Mat
Light sensors
Flow Switch
Temperature sensors
Ultrasonic sensors
Invasiveness
To define guidelines for smart home design, we must first
pinpoint what we mean by this term. Originally, a smart home
was simply a house with automated environmental systems such
as lightning and heating control features. Nowadays, almost any
electrical house component can be included in the system [2].
Smart houses are used for several purposes. They can improve
comfort at home, reduce energy consumption and enable
automation of household chores. They can provide a better-quality
entertainment while adapting to the preferences of residents.
Many scientists believe, however, that the field of smart homes
will reach its full potential by providing health assistance to
impaired persons. This very interesting application would help
residents to remain autonomous at their home for an extended
time. It would reduce the workload of natural caregivers and ease
the anxiety of families while not available to help. Such
technology would not only help directly the resident but also
produce reports for physician or allow instant monitoring. In this
paper, we will mean by smart home an enhanced house designed
to assist cognitively impaired persons and to help researchers in
their work to address the two questions presented.
Precision
2. SMART HOME
Robustness
Table 1: Comparison of most popular sensors
Cost
contribution is, therefore, not only theoretical but also practical
and experimental. Finally, we will describe our open tools [6]
project that serves to the promotion of international exchange and
collaboration.
B
B
A
A
B
C
B
B
A
B
B
A
A
A
B
B
B
B
A
B
B
B
A
C
C
A
B
B
B
B
E
C
A
B
B
B
A
A
A
A
B
B
B
A
D
A
C
B
C
C
E
D
A
A
B
A
A
B
A
B
A: Best
to E: Worst
In the next subsections, we delve into two important elements that
are too often ignored by research teams at the step of final
sensor’s types’ selection that we thought they deserved more than
a line or two to talk about.
3.1.1 Energy efficiency
To build a smart home, we have first to choose the kind of
technology and hardware we will integrate in it. Of course, there
are many ongoing intelligent house projects around the world
such as MavHome [7], the eHome [8] and the House_n project [9]
from which we could learn. The problem is that none clearly
provides information about their creation process and their
hardware choices. Furthermore, it is unrealistic to assume, as
many teams do, that smart home will only be new house
construction. In this section, we will focus the guidelines on the
material aspects while keeping that criterion in mind.
A thing that many researchers fail to recognize is the importance
of energy management. There are many reasons why we should
choose sensors and devices that minimize energy consumption.
First, it matters a lot for the resident. Of course, if we want to
spread smart home utilization in the consumer market, we will
have to prove them as economically viable technology. That is
because the electricity bill is important for the user, and,
furthermore, he might also consider the environmental issue. We
should then prioritize sensors that do not require too much
electricity and certainly try to avoid those that use disposable
batteries (no user like to buy and change batteries). Moreover, the
latest are big trouble for assisting smart homes, since we expect
the smart home to be completely independent and autonomous.
3.1 Choosing the Sensors
3.1.2 Perception of the resident on sensors
3. HARDWARE ARCHITECTURE DESIGN
In order to design better smart home, a very important phase is the
selection of sensors and technology that will be exploited. To do
so, we must first identify criterions that will be used for the
selection. We have to consider two important things. We have to
evaluate from the user point of view and from the system
perspective. On the user side, we want to implement smart home
at a reasonable price, so we need to prioritize low priced
technology. On the other hand, the resident certainly does not
want an unreliable system. Putting bottom-of-the-range residential
home automation equipments is, therefore, not an option. We need
to use rugged sensors that can withstand daily use. On the system
side, we want to have easy-to-install sensors that we would be
able to put in every house without much difficulty. This is
important to be flexible since real smart homes will be often
installed in old building. Finally, we must consider the precision
of sensors and the complexity of the information which they
transmit. It is evident why the first is important, but the reasons
for the second are somewhat more obscure. Data complexity is
important for two reasons. First, we must be able to get useful
information from a sensor if we want to use it. If data is complex
to model, we won’t be able to achieve rapid prototyping. Second,
it is important for the smart home to act fast when the user needs
its assistance. The table 1 lists the different sensors that can be
used and allows comparing them on the important criterions basis.
Another point that is often minimized is the perception of the
sensors and habitat by the resident. Various researches have
shown through time that residents that feel observed and invaded
in their private life have a lower quality of life [10]. In addition, if
a resident suffers from a cognitive affliction, his state might
worsen significantly as a consequence. That is directly in contradiction with the goal we try to achieve by assisting Alzheimer’s
subjects with smart home technologies. Therefore, it is important
to choose carefully the sensors and the effectors of the smart
home in order to minimize the negative impact of invasiveness.
We should also install sensors with effort to hide them from the
view of the resident in the house.
3.2 Communication Technologies
Another key element in smart home design is the choice of
communicating technology [11]. Sadly, there is no standard
nowadays and smart home builder need often to deal with
compatibility problems. The most extended technologies in smart
home networks are wired technology where X10 dominate as a
Power Line Carrier (PLC) standard that uses home existing
electric wiring to operate. The disadvantage is that not all
components are compatible with the same PLC technology.
Moreover, installing these technologies in older building is often
complex and thus wireless technology should be prioritized when
possible. Notably, many are familiar with Wifi, Bluetooth and
RFID. There is also UWB the wireless version of USB [12] that
should allow communication between our current USB objects at
short distance (approximately 10 meters). Finally, one of the most
promising wireless standards is certainly ZigBee [13] an open
platform based on IEEE specification for personal networking. It
takes its name from the message transmitted like bees fly (in
zigzag), looking for the best path to the receiver.
3.2.1 Centralized or decentralized processing
When designing a new smart home a choice comes from using
classical centralized communication through a server or trying to
decentralize the communication decentralize communication as in
the vision of ubiquitous computing. In a centralized system,
components are dumb; they transmit directly their input to a
server. On the other hand, in a decentralize system, components
communicate with each other trying to take decisions and
collaborate on services. The major drawback of such a system is
the design complexity. Therefore, our proposition is to put our
effort in the creation of a working centralized solution and create
a decentralized version when researches will be advanced enough.
Furthermore, nothing prevents someone from conceiving multiagents software on centralized hardware architecture.
3.4 LIARA’S Smart Home Case Study
The LIARA laboratory recently conceived and implemented a
new cutting edge smart home infrastructure that is about 100
square meters and possesses around a hundred of different sensors
and effectors. Among the sensors, there are infrared sensors,
pressure mats, electromagnetic contacts, various temperature
sensors, light sensors and eight RFID antennas. We also have
many effectors, including an Apple© iPad, many IP speakers
around the apartment, a flat screen HD television, a home cinema
theater and many lights and LEDs hidden in a strategic position.
The figure 1 shows a cluster of images from different parts and
angles of our smart home.
3.3 Choosing the Right Effectors
The collection of information on the resident is very important,
but it would be for naught without methods to provide assistance.
Smart home predominantly uses verbal prompts with little
knowledge of their effectiveness [14]. To be effective, it is
important to use prompts that are optimized with the profile of the
resident and the characteristics of the tasks. That is why part of
our team is investigating effectors' efficiency. We can identify
four main categories of prompts: auditory (verbal, musical,
sound), pictorial (photographic, textual), video (pictorial, modeling) and light. Experience has shown that each of these has
contextual specialty. Therefore, we recommend including enough
effectors in the smart home to be able to send all types of prompt
everywhere it is relevant. That work was concluded with the
creation of guidelines that could be used by smart home
researchers toward assistance of cognitively impaired persons. It
shows the effectiveness of each type of prompt according to
significant individual profile and with tasks' characteristics. The
table 2, presented below, is a reduced version of the real one in
[14].
Table 2: Prompt efficiency according to the profile and tasks
Figure 1: LIARA’s Smart home
The main image is the kitchen. At the bottom from left to right
you can see: a tagged cup (RFID tag), the dining room, an RFID
antenna, the HD television. From top right to bottom can be seen:
the server, the bathroom and the library. The server is a Dell©
industrial blade computer and it is the one in command of
processing the information. We also have an AMX© system to
control multimedia hardware such as the DVD player, the
television and the IP speaker. As shown on figure 1, the iPad is
embedded in the refrigerator. It controls the habitat for the
experiments and can be used to test the equipment or to assist the
resident with the help of videos when he is located in the kitchen.
Talking about assistance, the television can also be remotely
controlled from computer (or AMX) for that same purpose. Our
respective offices and a meeting room are built around the
intelligent home. In addition, we can see inside the apartment with
the windows mirror specially designed for our experiments.
3.4.1 General architecture
Legend: HI=highly ineffective; I=ineffective; M=moderate;
E=effective; HE=highly effective
The LIARA’s hardware architecture follows the proposed
guidelines. It has been conceived sturdy enough to support real
intensive daily use. For that purpose, we did installed industrial
grade material. We want to avoid hazardous situation as most as
possible. For example, some cheaper automation systems that
control the house lighting provoke undesired results when a
problem occurs. Imagine the situation where the resident cannot
turn on the light due to a system fail. That is undoubtedly to be
avoided.
In our architecture, the various sensors and RFID antennas are
connected to four independent fault-tolerant islands. If a block
falls, only the sensors of that zone will be affected. An APAX5570 automata harvests information in real time and sent it on the
central computer to a SQLServer database. Thereby, this transfer
hides the heterogeneity of the information coming from sensors
and resolves potential communication incompatibilities between
various communication standards (which there are not in
LIARA’s smart home). On the server of the habitat, an application
reads this data every 100ms, and copies it to another identical
database that is here for third-party applications. Thus, the
database system is impervious to external users and cannot be
damaged during the experiments. The application read RFID
antennas each 500ms because it is the optimal refresh time for
them. The figure 2 shows the hardware architecture of the
laboratory.
turn any Ethernet device in a wireless one at any time. Thus, it is
realistic to consider that our systems will operate in old building.
3.4.3 RFID technology
Since we work in assistive technology, we need to recognize the
activities of daily living (ADLs) of the resident. To do that, we
chose to use RFID technology to detect object’s location and
movement. It is not an easy task to choose the right set of RFID
and configure them for best performance. We chose to use passive
RFID tags since we needed to put them on everyday life objects.
These are small tags that are cheap to buy (often less than a
dollar) and require no other power than a radio pulse from a
nearby antenna. Of course, when comparing to active counterparts
that use their own power to emit and that are always awake,
passive tags have a reduced range and precision. Nevertheless,
with proper adjustment, they give good results and they are robust
(sometime even washables). To obtain a good precision, we must
adjust antenna signal power (range). The lesser the range is; the
better the precision is. We also need to limit as much as possible
the quantity of visible tags. We can accomplish this by shielding
the storage areas (such as cabinets) with aluminum foil. Finally, a
good smart home should minimize the pollution from radio waves
by turning off radio emission when not required.
4. SMART HOME SOFTWARE
Figure 2: Hardware architecture
3.4.2 LIARA’s sensors and effectors
In the design stage of our new smart home, we considered the
various criterions enunciated in the previous sections to choose
the technology to integrate in our infrastructure. First, we chose to
integrate classical infrared motion sensors that provide simple to
process binary information on the presence of activity in a zone or
not. These sensors are not only little invasive (due to their
physical appearance) but also cheap to buy. We also decided to
use electromagnetic contacts that give binary information on the
state of the two part of the sensor (touches, or not). Although they
are wired, they can be completely hidden from the view of the
user due to their very small size. We use them mostly on doors
and panel. On few strategic locations, we decided to integrate
pressure mats that also give a binary information (pressed, or not).
The cost of this type of sensor is significantly higher, so we try
not to rely on them too much. In addition, the installation requires
the alteration of the environment (the floor) which may be
unacceptable to some residents.
Among other types of sensors, we use light detection, temperature
and RFID technology but no video camera or look alike
technology. That is because cameras are too much invasive and
almost always rejected by people. That is true even when the
residents are told that only the system will ever access the image.
Moreover, computer vision is far from the capacity to obtain all
the information from a complex video camera output in
reasonable computational time [15].
In our new smart home, we have taken control over the complete
light system by installing simple tri-way switches. The lightning
system (including the LED) can be controlled thanks to the
installed wires. For upgrade compatibility purpose, we decided
that all the wired effectors (light system, speaker, television, etc.)
and sensors would implement the Ethernet protocol that needs no
more to be presented. Using that protocol ensures compatibility
with a wide range of networking products. Plus, we can easily
Building a smart home is more than choosing which technologies
it should implement. In between the materials and the AI
assistance, there is software implementation that provides an
abstraction layer to work with the infrastructure and sometime
useful service to enhance the control flexibility over the smart
home. In this section, we will discuss the main dilemmas of smart
home software such as the middleware issue, the calculation
complexity and the method for fast prototyping. The second half
of this section will be entirely devoted to review the case of our
new smart home. As we will see, smart home software can be
pushed a step further to provide useful services that will enhance
the prototyping capability and speed. Finally, we will discuss our
architectural and functional choice and their implication in our
experiments.
4.1 Tips for Software Conception
Software side of a smart home has the important role of creating
uniformity in the various heterogenic technologies of the house.
Traditionally, this problem has been addressed by the development of middleware that was given input of various sources and
changed them into uniform output [16]. We believe, however, that
middleware is not useful for every situation. In our case, the
database plays the role of providing a uniform access method to
data. While it is true that a database requires slow writing and
reading access to hard drives, it is no very significant since the
volume of data is generally not very high in comparison with the
amount of calculations to compute them and for AI execution.
Furthermore, we can synchronize the reading and writing of all
sensors reducing the complexity to a constant time O(1). The
choice of using middleware or not is even more important for the
future software development. Despite the uniformity of data
provided from middleware, the lack of concrete files to access
them might limit the choice of language to use in development.
On the other hand, database allows using multiple programming
languages and development platforms.
4.1.1 Calculation complexity
We already covered the complexity of data in the hardware
section. However, from the software point of view, it is rather
more important. In particular, the artificial intelligence of a smart
hhome must be responsive
r
and very fast to be respected by th
he
residents and to be regarded ass intelligent. A slow system will
w
result in delayed interaction with
h the resident. Leet just imagine th
he
ccase where we haave an Alzheimeer’s patient and the
t system always
pprompt him long
g after he has alrready committed
d a mistake due to
hhis impairment. It will certainlly not help him
m and might eveen
cconfuse him even
n more. It is to avoid
a
such situattions that we neeed
too evaluate the co
omputational loaad in the design and maximize th
he
uusage of non greeedy technologies for reduction
n. In fact, a smaart
hhome AI should
d be able to pro
ocess all the in
nformation almo
ost
innstantaneously on
o a human timee scale. It is eveen more importaant
inn the context off assisting techno
ology. Of coursee, one could argu
ue
thhat it is not sign
nificant since computer power iss relatively cheaap,
bbut if calculatio
on complexity is greater than quadratic, addin
ng
m
more processing power might no
ot be enough. Moreover,
M
we waant
too minimize thee space required
d for computer systems at hom
me
ssince it might be limited in som
me existing buillding. To achiev
ve
thhat, memory space can be reaso
onably sacrificeed. Memory is not
n
aan issue on mod
dern personal co
omputers in norm
mal programmin
ng
ssituations, and th
hus it is realistic.
4.2 L
LIARA Softtware Archiitecture
We alreeady aborted thee subject of our hardware architeecture at the
LIARA
A’s smart homee laboratory. L
Last time we sstopped the
descripption at the databbase level wheree the software taakes over. A
softwarre was designedd on the smart hhome to enhancce flexibility
and robbustness. This ssoftware read the database in reeal time and
copies the data on a second iddentical databasse for the
commuunication with AI (AIDB). T
This is importaant since it
protect s the real data ffrom being modiified from third party users.
Neverthheless, the reasoon we implementted this was mosstly to allow
easy reerouting of thee data source. IIn consequencee, we could
changee the source withhout third party aapplications everr noticing. It
would also work as thhe contrary: routte the main dataa to another
Besides, that muulti layered architecture allows uus to add AI
place. B
modulee that can proviide services to ttransform raw ddata in high
level innformation. The software architeecture can be seen on figure
4. Moree details will be given in the folllowing sections.
44.1.2 Designing for the reesident
O
One of the mostt important thing
gs in designing good smart hom
me
ssoftware is the effects
e
on its ressident and the peerception it givees.
C
Cognitively impaired persons need
n
to be challeenged in order to
sstimulate their brain
b
activity an
nd slow down th
he degradation of
thheir states. Thee smart home should
s
encourag
ge its resident to
pperform tasks by
y himself rather than
t
automating
g them (that wou
uld
bbe often easier to
o realize). For instance, if a win
ndow needs to be
b
cclosed because it has begun raaining outside, the house shou
uld
innfluence the resiident to go aroun
nd the window and
a to close it.
W
We must also co
onsider the charaacteristics of residents targeted by
b
oour habitat in th
he design processs. For examplee, a person profiile
m
may require a hig
gher audio volum
me. For elderly, control interfacces
sshould be intuitiv
ve, simple and th
he graphical useer interface shou
uld
bbe conceived with
w
big button
ns, a legible ty
ypeface and hig
gh
ccontrast colors. Moreover, we could facilitatee task for eldeers
(w
without automaating them): butttons to close the windows, voice
ccontrol for the execution
e
of task
ks, etc. Another point that shou
uld
bbe considered when
w
conceiving
g smart home applications
a
is th
he
nnotion of controll. It is importantt that the residen
nt feel empowereed
bby the smart ho
ome in his activ
vities but that th
he utmost contrrol
remains in his haands. It must no
ot feel like the decisions
d
are takeen
bby the smart hom
me and all rem
mains for him is to execute them
m.
M
Moreover, on lon
ng term, an algo
orithm that makees suggestions caan
ddegrade gracefullly but an algoritthm that takes deecisions will mo
ost
pprobably end being overlooked as
a dumb [9].
44.1.3 Energyy managementt
W
We already talk
ked about the importance of choosing energ
gy
eefficient hardwarre. For the resideent, it might be a crucial issue an
nd
aan important argument to convin
nce him of adoptting a smart hom
me
innfrastructure. Therefore,
T
we should
s
go on a step further by
b
inncorporating adv
vanced conceptss of energy man
nagement. A smaart
hhome should alsso possess an AI
A that tries to save energy [9] in
eeveryday activity
y. For example,, the AI of a sm
mart home shou
uld
ppredict the weath
her and optimize A/C or heaterr activation. If th
he
resident is sleep
ping, AI could let
l cold air enteer to get the beest
teemperature as possible
p
during the day. Similaarly, the AI cou
uld
oopen automaticallly the store to warm
w
naturally th
he house while th
he
resident is away
y. Again, the point here is that it might be
b
mportant to preepare the groun
nd for AI rapid
d prototyping by
b
im
ddesigning softwaare that allows abstraction of the
t complexity of
ccontrolling differrent devices.
Figuree 4: Software Architecture
4.2.1 Smart Homee Visualizatioon Tool
In ordder to facilitatee testing, we developed a ssmart home
visualizzation software. A screen shot oof this software showing the
overall smart home ccan be seen onn figure 5. Thhe graphical
interfacce of this softw
ware allows us too see different parts of the
smart hhome or the oveerall picture. In eeach of these innterfaces, we
can seee the state of m
many sensors succh as infrared seensors, light
sensorss, etc. We alsoo can see an appproximate locaation of the
objectss in the smart hhome (rounded rectangle, onlyy appears if
RFID antennas are acctivated) and thhe current posiition of the
residennt (in front of thee kitchen counteer on the right part of figure
5). Theese functionalitiees are very usefful when conduccting experiments since we can aanalyze what w
went wrong by reproducing
sensorss' activation andd double checkking if the matterial works
properlly. In addition, iit allows manuaal testing of effeectors of the
smart hhome, includingg the televisionn, the oven andd the audio
system .
4.2.2 Scenarios reecording
The moost important appplication of our system is unequuivocally the
scenariio recording fuunctionality enaabled by the m
multilayered
architeccture of our dataabase. In fact, inn the visualizatioon software,
we cann create life scennarios from a sinngle button (seee the bottom
right coorner of figure 5). When recording is activatedd, the smart
home central applicattion copies thee data gatheredd to a third
databasse layer that is iddentical to the m
main one. It does not stop the
Figure 5: The smart home visualization software
redirection to the AI database. As a result, third-party software
(the AI, the visualization tool, etc.) can continue their proper
execution. Once recorded, scenarios can be replayed from the
visualization tool. To do so, user needs only to select his
scenario’s name and the central application will retrieve the
record and redirect the flux of data from the scenario DB to the AI
DB (see figure 4). In other words, from the third party
perspective, there is no difference while being in playback mode
compared with normal activity. This functionality is especially
useful to compare different algorithms since we can test them on
exactly the same execution sequence.
4.2.3 Clinical trials at the laboratory
Perhaps the use of the scenario feature is not so obvious. We
believe that it would be important to mention the recent clinical
trials conducted at the LIARA. The objective was to pass a
neurological test called the Naturalistic Action Test (NAT)[17] to
several peoples in order to evaluate their cognitive impairment, to
identify common mistakes they do in their everyday life and to
evaluate the effect of different types of prompt [14]. This test,
consisting of three activities of daily living, was held in the
laboratory and all participants were required to travel to the scene.
We needed both normal subjects and patients with Alzheimer's
disease from mild to moderate state of the disease. Thus, to
implement the whole process, we had to fill out forms and pass
the ethics committee which is responsible for ensuring that the
experiments comply with human rights and pose no danger to the
well-being of individuals. Preparing and conducting such
experiments is tedious and requires considerable time as you can
imagine.
We were able to reuse the results of these experiments in other
prototype tests because we had recorded them on video camera.
However, we were unable to test our latest algorithms directly on
the actions of real subjects; it was always via an extrapolation of
the records that we were able to proceed. Of course, it is
unrealistic to hope for bringing patients at the laboratory to
validate each of our prototypes. Now that our new home
incorporates features for recording, we could repeat infinitely the
action sequence of subjects who came at the LIARA in order to
benchmark the various prototypes. That is why we say that this
feature is inestimable and that any research team should consider
its integration in the construction process of an intelligent home.
4.3 Supplementary AI
As we discussed earlier (see Figure 4), it is possible to add
artificial intelligence modules on the server to process a
transformation on raw data into useful high level knowledge. That
functionality is due to the choice of implementing a multilayered
database instead of middleware. On the current system, we have
two additional modules that provide general services to third-party
applications. The first deals with the raw information from RFID
tags in order to alter them into useful data. For example, it
associates the unique identifier of the tag to a meaningful name
(cup_X) and gives the approximate position of the associated
physical object in the home. For its part, the second module is a
small inference engine that intercepts information from motion
sensors to infer the approximate position of the resident in the
home. The advantage of these services is twofold. On the one
hand, the prototypes can be developed and tested more quickly or
chose to implement from the raw data. In addition, it allows more
flexibility since the algorithms on the server can be easily
modified without affecting the service provided and, therefore,
without affecting third-party applications that use it.
4.3.1 Locating the resident
One of the most active problematic of smart home is the location
of the patient in the house. To begin with, many use wearable
devices for this purpose [15]. We believe it is unrealistic to expect
the resident to always wear them, especially if he is afflicted by
cognitive impairment. To create a positioning system, we must
first define our needs in matter of precision. For most smart home
applications, approximate position is enough (at the scale of large
part of room). This is why we separated our system into logical
zones that we can directly interrogate to know if there is a
presence or not. Our system is primarily based on motion sensors.
The consequence is that we cannot always locate the resident with
a hundred percent certainty. However, it is not necessary. We
improve the certainty when there is detected activity in the smart
home by considering every sensor’s activation in the house. There
is also the issue of multiple presences. If the resident receives a
visitor, the system should just decrease its assistance since it will
be less required. It can easily notice there is one more person so it
should adjust its intelligence to consider that fact. The smart home
does not need to differentiate between the two persons; it might
assist either one that needs it.
5. PROMOTING COLLABORATION IN
SMART HOME RESEARCH
At the beginning of this paper, we stated that one of our objectives
was to promote collaboration and exchange for incremental
advances in smart home researches. Many research teams are
working in agreement with that goal. For example, Cook & al.
[18] from the Washington State University share the data set from
their smart home and Giroux & al. [4] share their real case
scenarios. On our side, to achieve this goal, we are pursuing a
project of collaborative tools in synergy with our researches that
we want to distribute freely to help researchers in their works.
Moreover, these tools would not only increase the collaboration
between researchers but also help them to speed the prototyping
process and the validation of theories.
recorded in the same way our real smart homes do it. SIMACT
will save the sensors firing into a database accessible from thirdparty applications. The consequence is that communication with a
real smart home is simple to realize. SIMACT could put
information from scenario to the smart home database or read
information from it and show what happen in the 3D frame.
The scenarios are special since they are built from simple to use
XML language. The software works as an interpreter and
recognize the different tags of a scenario. If a scenario is
grammatically correct, many actions can be performed automatically in the 3D environment such as object movement or
rotation. In addition, like a multimedia player, the scenario
execution can be played, paused, stopped and even rewound. The
figure 7 shows a scenario editor we are working on.
5.1 SIMACT
Recently, we created a smart home simulator in Java programming language that was named SIMACT. This simulator was
based on a recent 3D engine and was designed to easily
conceptualize a smart home in 3D. To do so, the user only needs
to upload his 3D drawing to the software via an editor. To
simplify the life of the user, SIMACT comes with a smart home
kitchen that was built from the free objects' library of modeling
software named Sketchup from Google. SIMACT can be seen on
figure 6.
Figure 7: SIMACT Scenario Editor
5.2 Open database
Figure 6: SIMACT in action
SIMACT [6] was primarily made to successfully experiment
without requiring a real smart home infrastructure. We had this
idea because many researchers work in small university or simply
do not have the financial support to build a full-sized home.
However, it was built in the goal of providing high flexibility, and
it could certainly be used with little modification for smart home
sensors' visualization. Researchers can do whatever they want
with the code except commercialize it.
5.1.1 SIMACT architecture
SIMACT was designed in such a way as to separate code from
every dynamic aspect of the software. Basically, everything is
configurable in the software. It works a little bit like a multimedia
player. When the 3D environment is loaded (done automatically at
the startup), it allows to load scenarios that will then be played by
the simulator. These scenarios are normal ADLs or ADLs with
mistakes that will simulate exactly the action as it would do in a
real smart home; by firing sensors. The difference here is that the
sensors are virtual conception. However, the information is
With SIMACT, we had the idea of building an open database for
ADLs scenarios. We already built few scenarios that are
distributed with SIMACT, and we are working to create few
more. We want to provide real case scenarios from Alzheimer
experiments in order to have a fundamental basis for experiments
and results comparison of activity recognition algorithms and
assisting algorithm. The experiments we conducted at the
laboratory were conducted with this second goal in mind. We
recorded every bit of information to build realistic scenario and
share the date with the community. Similarly, we hope that teams
will also contribute to this open knowledge by sharing their own
experiments. A second part of this project will be the sharing of
our experimental data. Of course, it is already done via scientific
publication, but due to limited space it is often reduced and
lacking details.
5.3 Prompting tool
We worked to create a prompting tool for the experimentation
with real subjects. This tool is a simple software that enhances the
evaluation methodology. It allows an assistant to send from a
distant computer a prompt if required (e.g., if the participant uses
the wrong utensil). With a simple click on a button, a chosen form
of prompt (i.e., verbal, modeling video without sound or modeling
video with sound) can be sent for a specific step of the task
through a computer screen and speakers placed in front of the
participant. Besides, the software allows the evaluator to comment
the results, the erroneous steps and the type of problem (e.g.,
omission, inaction, substitution) through a simple dynamic text
area box. It is also possible to get a percentage of the task
completed. Moreover, it allows us to save each session separately,
the type of prompt sent, the completion time of the task and other
notes that are relevant. From these sessions, it generates a simple
text report for further analysis.
6. CONCLUSION
In this paper, we proposed a set of guidelines for the conception
of smart home for cognitive assistance on both hardware and
software perspective. The main idea was to provide basics
information about how to efficiently design and implement a
smart home infrastructure for faster AI prototyping. We presented
our own architectural and design choices that were focused
toward that goal. The paper was aimed directly at researchers that
plan to conceive new smart home infrastructure. In that way, we
hope that it will be a foundation to the establishment of an
incremental process toward smart home implementation.
A secondary goal of this paper was to promote the collaboration
and exchange between research teams. We hope to have convinced the reader of the importance of sharing knowledge to
advance farther in the field with the Open Tools project of the
LIARA and the examples we gave. We think that by achieving
those two points, faster/easier experimentation and collaboration
between laboratories, smart home assistance will evolve as a real
applicable solution in the consumers market someday. In future
works, we invite other researchers to improve these guidelines
with their own experience such as construction of smart home, use
of a specific technology, etc. Another aspect that needs to be
covered is the comparison technique of smart home algorithms. In
activity recognition for example, there are several algorithms that
are developed each year and thus is it hard to compare the
performance of each of them. We believe that it would be useful
to work on standardization of metrics, and test protocols for better
evaluation.
7. ACKNOWLEDGMENTS
We would like to thank our main financial sponsors: the Natural
Sciences and Engineering Research Council of Canada, the
Quebec Research Fund on Nature and Technologies, the Canadian
Foundation for Innovation. We would like to thank our health
regional center for providing us the Alzheimer participants.
Finally, special thanks to our neuropsychologist partner and her
graduate students who indirectly worked on this project by
supervising the clinical trials with patients.
8. REFERENCES
[1] Ramos, C., Augusto, J. C. and Shapiro, D. "Ambient
Intelligence: the Next Step for Artificial Intelligence", IEEE
Intelligent Systems, 23, pp. 15-18, 2008.
[2] Robles, R. J. and Kim, T.-h. "Applications, Systems and
Methods in Smart Home Technology : A Review", Int. Journal of
Advanced Science And Technology, SERSC, pp. 37-48, 2010.
[3] Phua, C., Biswas, J., Tolstikov, A., Foo, V., Huang, W.,
Jayachandran, M., Aung, A. P. W., Roy, P. C., Aloulou, H., Feki,
M. A., Giroux, S., Bouzouane, A. and Bouchard, B. "Plan
Recognition based on Sensor Produced Micro-Context for
Eldercare", IEEE, Toyama, Japan, pp. 39-46, 2009.
View publication stats
[4] Roy, P., Bouchard, B., Bouzouane, A. and Giroux, S.
"Challenging issues of ambient activity recognition for cognitive
assistance", Ambient Intelligence and Smart Environments, IGI
global, pp. 1-25, 2010.
[5] Bouchard, B., Roy, P., Bouzouane, A., Giroux, S. and
Mihailidis, A. "An Activity Recognition Model for Alzheimer's
Patients: Extension of the COACH Task Guidance System", In
Proceedings of the ECAI. IOS Press, Grece, pp. 811-812, 2008.
[6] Bouchard, K., Ajroud, A., Bouchard, B. and Bouzouane, A.
"SIMACT: a 3D open source smart home simulator for activity
recognition", In Proceedings of the UCMA, Springer-Verlag,
Japan, pp.524-533, 2010.
[7] Cook, D. J., Youngblood, M., Edwin O. Heierman, I.,
Gopalratnam, K., Rao, S., Litvin, A. and Khawaja, F. "MavHome:
An Agent-Based Smart Home", In Proceedings of the PerCom,
IEEE Computer Society, pp.521-524, 2003.
[8] Kaila, L., Mikkonen, J., Vainio, A.-m. and Vanhala, J. "The
eHome – a Practical Smart Home Implementation", Sydney,
Australia, pp.35-46, 2008.
[9] Intille, S. S. "Designing a Home of the Future", IEEE
Pervasive Computing, IEEE, pp. 76-82, 2002.
[10] Weiser, M. "The computer for the 21st century", Scientific
American, 265, 3, pp.66-75, 1991.
[11] de Vicente, A. J., Velasco, J. R., Marsa-Maestre, I. and
Paricio, A. "A Proposal for a Hardware Architecture for
Ubiquitous Computing in Smart Home Environments", In
Proceedings of the ICUC, Alcalá de Henares, Spain, 2006.
[12] Ricquebourg, V., Durand, D., Delahoche, L., Menga, D.,
Marhic, B. and Loge, C. "The Smart Home Concept : our
immediate future", In Proceedings of the ICELIE, IEEE, pp.2328, 2006.
[13] Poole, I. "What exactly is ZigBee?", Communications
Engineer, pp.44-45, 2004.
[14] Van Tassel, M., Bouchard, J., Bouchard, B. and Bouzouane,
A. "Guidelines for Increasing Prompt Efficiency in Smart Homes
According to the Resident's Profile and Task Characteristics", In
Proceedings of the ICOST, Canada, Springer, pp.112-120, 2011.
[15] Patterson, D. J., Lin, L. A., Fox, D. and Kautz, H. "Inferring
high-level behavior from low-level sensors", In Proceedings of the
UbiComp, Springer-Verlag, USA, pp.73-89, 2003.
[16] Coyle, L., Neely, S., Rey, G., Stevenson, G., Sullivan, M.,
Dobson, S. and Nixon, P. "Sensor Fusion-Based Middleware for
Assisted Living", In Proceedings of the ICOST, IOS Press, UK,
2006.
[17] Schwartz, M. F., Segal, M., Veramonti, T., Ferraro, M. and
Buxbaum, L. J. "The Naturalistic Action Test: A standardised
assessment for everyday action impairment", Psychology Press,
Hove, UK, pp. 311-339, 2002.
[18] Cook, D. J. and Schmitter-Edgecombe, M. "Assessing the
quality of activities in a smart environment", Methods of
Information in Medicine 2009.