adaptive user interfaces
advertisement
Adaptive User Interfaces Based on Models and Software Agents Víctor M. López Jaquero Escuela Politécnica Superior de Albacete Departamento de Sistemas Informáticos Universidad de Castilla-La Mancha Campus Universitario, s/n. 02071 – Albacete (SPAIN) Email: victor@info-ab.uclm.es Supervisors: Dr. Pascual González López Dr. Antonio Fernández Caballero October, 2005 (Albacete) 1 CONTENTS 1. 2. 3. 4. 5. October, 2005 (Albacete) Introduction State of Art in Adaptive User Interfaces Design AB-UIDE: A Method for Adaptive UIs Design A MAS Architecture for UI Adaptation Final Remarks 2 CONTENTS 1. Introduction 1. Motivation 2. Objectives 2. 3. 4. 5. October, 2005 (Albacete) State of Art in Adaptive User Interfaces Design AB-UIDE: A Method for Adaptive UIs Design A MAS Architecture for UI Adaptation Final Remarks 3 MOTIVATION Interaction is changing, and it will keep on changing ... Different Platforms Macintosh Wintel October, 2005 (Albacete) Different Capabilities PDA Different Contexts Different Users In the streets Expert users At Home Rookie users PC 4 MOTIVATION Interaction is changing, and it will keep on changing ... Different Platforms Macintosh Wintel Different Capabilities PDA Different Contexts Different Users In the streets Expert users At Home Rookie users PC There are different platforms October, 2005 (Albacete) 5 MOTIVATION Interaction is changing, and it will keep on changing ... Different Platforms Macintosh Wintel Different Capabilities PDA Different Environments Different Users In the streets Expert users At Home Rookie users PC They have different capabilities October, 2005 (Albacete) 6 MOTIVATION Interaction is changing, and it will keep on changing ... Different Platforms Macintosh Wintel Different Capabilities PDA Different Enviroments Different Users In the streets Expert users At Home Rookie users PC They are used in different environments October, 2005 (Albacete) 7 MOTIVATION Interaction is changing, and it will keep on changing ... Different Platforms Macintosh Wintel Different Capabilities PDA Different Environments Different Users In the streets Expert users At Home Rookie users PC They are used by different users October, 2005 (Albacete) 8 MOTIVATION Interaction is changing, and it will keep on changing ... Different Platforms Macintosh Wintel Different Capabilities PDA Different Environments Different Users In the streets Expert users At Home Rookie users PC We need to face designing user interfaces able to work under these different situations (adaptive user interfaces) October, 2005 (Albacete) 9 MOTIVATION Design for different situations – One interface per situation considered • High monetary cost • High maintenance cost • Impossible to consider all the possible situations! – A single user interface able to adapt to all (or at least many) situations • Easy to keep consistency between versions • Lower maintainace cost • Lower monetary cost October, 2005 (Albacete) 10 MOTIVATION Design for different situations – One interface per situation considered • High monetary cost • High maintenance cost • Impossible to consider all the possible situations! – A single user interface able to adapt to all (or at least many) situations • Easy to keep consistency between versions • Lower maintainace cost • Lower monetary cost October, 2005 (Albacete) 11 MOTIVATION Hardcoded adaptation vs. Engineered adaptation – Hardcoded adaptation rules • • • • Adaptation knowledge reusing is hard Difficult to modify adaptation rules Hard to apply methodological processes Multi-platform development is almost handmade – Engineered adaptation Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> Inquiry <<extend>> Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) • • • • Adaptation knowledge can be reused A standard manner of editing adaptation rules Adaptation can be included within a methodological process Adaptation code can be automatically generated 12 MOTIVATION Hardcoded adaptation vs. Engineered adaptation – Hardcoded adaptation rules • • • • Adaptation knowledge reusing is hard Difficult to modify adaptation rules Hard to apply methodological processes Multi-platform development is almost handmade – Engineered adaptation Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> Inquiry <<extend>> Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) • • • • Adaptation knowledge can be reused A standard manner of editing adaptation rules Adaptation can be included within a methodological process Adaptation code can be automatically generated 13 OBJECTIVES Design adaptive user interfaces able to adapt to: – – – – User’s skills, preferences or characteristics The platform where the application is running on The physical environment where the interaction takes place The adaptation process should preserve the usability Include adaptation within a development process – Reuse adaptation knowledge – A standard manner of editing adaptation rules – Adaptation code can be automatically generated An architecture for adaptive user interfaces execution – Execute designed adaptive user interfaces – At any time apply the best possible adaptation – Support multi-platform development October, 2005 (Albacete) 14 CONTENTS 1. Introduction 2. State of Art in Adaptive User Interfaces Design 1. 2. 3. 4. 5. User Interfaces Design Model-Based User Interfaces Design (MB-UID) Adaptation in MB-UID Adaptation process Software Agents in User Interfaces Design 3. AB-UIDE: A Method for Adaptive UIs Design 4. A MAS Architecture for UI Adaptation 5. Final Remarks October, 2005 (Albacete) 15 STATE OF ART User interfaces design approaches Hard and tedious programming task. Very low level of abstraction. – Language based approaches: the user interfaces is built by programming it using a general purpose language (C/C++, Java, Pascal, etc). – User interfaces integrated development environments: they allow the design of the user interface interactively by means of graphical tools (Borland Delphi, Borland JBuilder, Microsoft Visual Basic, ...). – Model-based user interfaces development environments: they allow the specification of user interfaces out of a set of declarative models. October, 2005 (Albacete) 16 STATE OF ART User interfaces design approaches Hard to apply a methodological approach. Low level of abstraction. – Language based approaches: the user interfaces is built by programming the user interface using a general purpose language (C/C++, Java, Pascal, etc). – User interfaces integrated development environments: they allow the design of the user interface interactively by means of graphical tools (Borland Delphi, Borland JBuilder, Microsoft Visual Basic, ...). – Model-based user interfaces development environments: they allow the specification of user interfaces out of a set of declarative models. October, 2005 (Albacete) 17 STATE OF ART User interfaces design approaches – Language based approaches: the user interfaces is built by programming the user interface using a general purpose language (C/C++, Java, Pascal, etc). – User interfaces integrated development environments: they Higher level of abstraction. Easier to maintain. allow the design of the user interface interactively by means of Automatic code generation. graphical tools (Borland Delphi, Borland JBuilder, Microsoft Methodological approach. Visual Basic, ...). – Model-based user interfaces development environments: they allow the specification of user interfaces out of a set of declarative models. October, 2005 (Albacete) 18 STATE OF ART Model-based user interfaces design – Based on a set of declarative models • Task model • Domain model • Context of use model – user, platform and environment • Abstract user interface model • Concrete user interface model • Final user interface model – The models are transformed into an executable/interpretable presentation automatically or semiautomatically. October, 2005 (Albacete) 19 STATE OF ART Adaptation in model-based user interfaces design – Each approach allows the adaptation of some specific features: • • • • Context aware help systems Look & Feel Navigation Many of them support no adaptation. – Most of the adaptations are personalizations – No language to specify new adaptations – No intelligent adaptation process October, 2005 (Albacete) 20 STATE OF ART Adaptation process Adaptation process can be fired by either the user (adaptability) or the system (adaptivity). Initiative stage Detect Platform Changes Detect User Changes Detect User’s Goals Detect User’s Needs Detect Environment Changes Proposal for Adaptation Proposal stage Select Adaptation Decision stage Execute Adaptation October, 2005 (Albacete) User initiated adaptation Execution stage 21 STATE OF ART Adaptation process Adaptation process can be fired by either the user (adaptability) or the system (adaptivity). Initiative stage Detect Platform Changes Detect User Changes Detect User’s Goals Detect User’s Needs Detect Environment Changes Proposal for Adaptation Proposal stage Select Adaptation Decision stage Execute Adaptation October, 2005 (Albacete) User initiated adaptation Execution stage 22 STATE OF ART Adaptation process Propose feasible adaptations given the current situation Detect Platform Changes and state of interaction. Detect User Changes Initiative stage Detect User’s Goals Detect User’s Needs Detect Environment Changes Proposal for Adaptation Proposal stage Select Adaptation Decision stage Execute Adaptation October, 2005 (Albacete) User initiated adaptation Execution stage 23 STATE OF ART Adaptation process Select the best adaptations among the proposed Detect Platform Changes adaptations. Detect User Changes Initiative stage Detect User’s Goals Detect User’s Needs Detect Environment Changes Proposal for Adaptation Proposal stage Select Adaptation Decision stage Execute Adaptation October, 2005 (Albacete) User initiated adaptation Execution stage 24 STATE OF ART Adaptation process Initiative stage Detect Platform Changes Detect User Changes Execute the selected adaptations. Detect User’s Goals Detect User’s Needs Detect Environment Changes Proposal for Adaptation Proposal stage Select Adaptation Decision stage Execute Adaptation October, 2005 (Albacete) User initiated adaptation Execution stage 25 STATE OF ART The process requires reasoning about which adaptation to fire, choose the best adaptations, .... Adaptation process Multi-agent systems Initiative stage Detect Platform Changes Detect User Changes Detect User’s Goals Detect User’s Needs Detect Environment Changes Proposal for Adaptation Proposal stage Select Adaptation Decision stage Execute Adaptation October, 2005 (Albacete) User initiated adaptation Execution stage 26 STATE OF ART Software agents in user interfaces – Interface agents dwell in the user interface to improve user’s interaction experience. Our agents use BDI mental model. BDI model is a natural manner to deal with the required decision mechanism to execute adaptive UIs. October, 2005 (Albacete) 27 STATE OF ART Software agents in user interfaces – The design of multi-agent systems require new methodological approches: We used Oriented Prometheus/because: • Extensions of Object Knowledge • It supports the whole software life cycle. Engineering methods and techniques • Widely used. – – – – – – – – – October, 2005 (Albacete) • It provides a visual design tool. Tropos • Code generation for JACK and JADE. Gaia AUML OASIS Prometheus Desire MAS-CommonKADS INGENIAS ... 28 CONTENTS 1. Introduction 2. State of Art in Adaptive User Interfaces Design 3. AB-UIDE: A Method for Adaptive UIs Design 1. 2. 3. 4. 5. A Study Case: ATM UI Requirements Analysis stage Analysis stage Design stage Implementation stage 4. A MAS Architecture for UI Adaptation 5. Final Remarks October, 2005 (Albacete) 29 AB-UIDE: A Method for Adaptative UIs Design AB-UIDE (Agent Based User Interface Development Environment) extends usual model-based user interface development methods to support the development of adaptive user interfaces in a seamless way. Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) – User-centred approach – Iterative – Covers the whole development life cycle of the user interface – The adaptive user interfaces designed are executed on an agent-based adaptation engine. 30 AB-UIDE: A Method for Adaptative UIs Design AB-UIDE stages: REQUIREMENTS Platform Model Caso 1 User model Environment Model Actor 2 Actor Caso 2 Use Case Model ANALYSIS role 1 class 1 role 2 class 2 role 3 class 3 Roles Diagram Domain model Adaptation usability trade-off Deposit Logout <<extend>> <<extend>> DESIGN <<extend>> Withdraw <<extend>> Bank Customer Transfer Login Task model <<extend>> <<extend>> uses (task and actions temporal relationships) <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance class 2 class 1 presented as class 3 CUI AUI AIO CIO AIO AIO CIO CIO Interaction objects Adaptivity rules start error IMPLEMENTATION <<action>> InvalidAmount <window> <button> .... <window> October, 2005 (Albacete) Agent programming language XML UI specification Adaptation engine User interface 31 AB-UIDE: A Method for Adaptative UIs Design A study case: ATM UI – ATM UI is a user interface for an automatic teller machine, where the user can: • • • • • • Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance Withdraw money Make a deposit Transfer money Recharge a cell phone Get the account statement Change the preferences for the application start error <<action>> InvalidAmount October, 2005 (Albacete) – The user interface should be able to run on different platforms (bank platform and mobile platform). 32 AB-UIDE: A Method for Adaptative UIs Design Requirements analysis stage – Use case model Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw • Use case sequence diagram : Bank Customer : System 1: Logout <<extend>> Transfer Login Bank Customer 2: Logout OK <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences – “Static” context of use model • User • Platform • Environment Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> Task name Login Description The user logs into the system providing a login and a password. Type Abstract Frecuency High Precondition NULL Postcondition Participant.checkLogin() Presentation FreeContainer Action name Input Login Description The user types in the login. Type Input Frecuency High <<extend>> <<extend>> Inquiry Precondition NULL Postcondition Participant.login!=“” Action name Input Password Description The user types in the password for the login. Type Input Frecuency High Precondition Participant.login!=“” Postcondition Participant.Password!=“” Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) 33 AB-UIDE: A Method for Adaptative UIs Design Requirements analysis stage – Use case model Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw Deposit Logout <<extend>> <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> Bank Customer <<extend>> <<extend>> Inquiry Transfer Login Recharge cell phone Change preferences <<extend>> <<action>> Enter amount start <<action>> Show balance start <<extend>> error <<action>> InvalidAmount <<extend>> Inquiry Recharge cell phone Change preferences October, 2005 (Albacete) 34 AB-UIDE: A Method for Adaptative UIs Design Requirements analysis stage – Use case model • Use case sequence diagram : Bank Customer : System 1: Deposit 2: Show balance 3: Withdraw 4: Show balance : Bank Customer : System 1: Login 5: Transfer 2: Login OK 6: Show transfer info Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login 3: Perform operation <<extend>> 7: Inquiry <<extend>> <<extend>> 8: Show info Inquiry Recharge cell phone Change preferences 9: Recharge cell phone <<action>> Enter amount start <<action>> Show balance start 10: Show recharge info error <<action>> 11: Change preferences InvalidAmount 12: Show preferences 13: Logout 14: Logout OK October, 2005 (Albacete) 35 AB-UIDE: A Method for Adaptative UIs Design Requirements analysis stage – “Static” context of use model • User • Platform • Environment Platform Bank ATM User Customer platformId 0025P platformName BankATM Deposit Logout <<extend>> <<extend>> Platform PDA userId 0025U platformId 0026P userName BankCustomer platformName PDA01 <<extend>> Withdraw isColorCapable true <<extend>> Bank Customer Transfer Login Font Arial isColorCapable true <<extend>> isImageCapable true <<extend>> <<extend>> Inquiry Recharge cell phone FontColor Black screenSize 400x350 Change preferences <<action>> Enter amount start <<action>> Show balance isImageCapable true Modality Graphical screenSize 220x144 start error <<action>> InvalidAmount Environment Bank Environment Street environmentId 0028E environmentId 0029E environmentName BankOffice lightingLevel low isNoisy false October, 2005 (Albacete) environmentName InTheStreets lightingLevel average isNoisy true 36 AB-UIDE: A Method for Adaptative UIs Design Analysis stage – Domain model role 1 – Roles model role 2 role 3 Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> – Usability trade-off Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) 37 AB-UIDE: A Method for Adaptative UIs Design Analysis stage – Domain model Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) 38 AB-UIDE: A Method for Adaptative UIs Design Analysis stage – Roles model BankCustomer Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login MyBankCustomer <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) 39 AB-UIDE: A Method for Adaptative UIs Design Analysis stage – Usability trade-off Usability criteria to be preserved. Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) BANK ATM PDA Weight of the criterium in the usability trade-off. 40 AB-UIDE: A Method for Adaptative UIs Design Design stage – Adaptivity rules Watch speedometer Watch RPM – Task model Watch Oil Temperature Watch lighting indicators Watch system messages – Interaction objects specification Deposit Logout <<extend>> <<extend>> AIO <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> AIO AIO – Abstract User Interface (AUI) <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount – Concrete User Interface (CUI) CIO CIO CIO Connector model October, 2005 (Albacete) 41 AB-UIDE: A Method for Adaptative UIs Design Design stage Transitions are labelled to specify the dialog. – Task model end Deposit Logout start Login ( 1,* ) start Perform operation <<extend>> <<extend>> <<extend>> Withdraw Deposit Logout <<extend>> <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> Inquiry Transfer Login Bank Customer <<extend>> <<extend>> Recharge cell phone : Bank Customer : System 1: Login Change preferences <<extend>> <<action>> Enter amount start <<action>> Show balance start 2: Login OK <<extend>> error <<action>> <<extend>> Inquiry 3: Perform operation InvalidAmount Recharge cell phone Change preferences October, 2005 (Albacete) 42 AB-UIDE: A Method for Adaptative UIs Design Design stage Actions and tasks – temporal relationships are described by using LOTOS operators (as defined in CTT). end Task model start Login ( 1,* ) start Perform operation Deposit <<action>> Enter login/card Withdraw <<action>> Enter pin Transfer Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login Statement <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone <<action>> Change preferences Tasks and actions are described specifying a set of properties to help on UI generation. <<action>> Enter amount start <<action>> Show balance start Enter amount start <<action>> Show balance start Recharge cell phone error <<action>> InvalidAmount error <<action>> Change preferences InvalidAmount Logout October, 2005 (Albacete) 43 AB-UIDE: A Method for Adaptative UIs Design Design stage – Interaction objects specification <<action>> Enter login/card Login <<action>> Enter pin (1,1) Deposit Logout <<extend>> Pin <<extend>> <<extend>> Withdraw Name Login Type String <<extend>> Bank Customer Transfer Login (1,1) <<extend>> <<extend>> <<extend>> Initial value Inquiry Recharge cell phone “” Change preferences <<action>> Enter amount start <<action>> Show balance start error Name Pin <<action>> InvalidAmount Type String Initial value Password October, 2005 (Albacete) “” true 44 AB-UIDE: A Method for Adaptative UIs Design Design stage – Abstract User Interface (AUI) The AIOs are grouped in containers, that help on deciding a good final layout for the UI elements. <<action>> AUI consists of AIOs: Inputters, Displayers, Editors, ActionInvokers and Selectors. Enter login/card Login <<action>> (1,1) Enter pin Deposit Logout <<extend>> <<extend>> Pin <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> (1,1) <<extend>> <<extend>> Inquiry FreeContainer Login Login Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start Login error <<action>> InvalidAmount Pin OK October, 2005 (Albacete) 45 AB-UIDE: A Method for Adaptative UIs Design Design stage – Concrete User Interface (CUI) FreeContainer Login CUI consists of CIOs. CIOs are derived from AIOs and Interaction Objects. Login Login Pin OK Deposit Logout <<extend>> <<extend>> Withdraw <<extend>> Transfer Login <<extend>> <<extend>> <<extend>> box Login textComponent textComponent textComponent textComponent Name=“loginLabel” isEditable=false defaultContent=“Login” <<extend>> Bank Customer Window Login Name=“Login” isEditable=true defaultContent=“” Inquiry Recharge cell phone Change preferences <<action>> Enter amount error <<action>> InvalidAmount start <<action>> Show balance start Events represent the behaviour of the system. October, 2005 (Albacete) Name=“pinLabel” isEditable=false defaultContent=“Pin” button Name=“OK” Name=“Pin” isEditable=true defaultContent=“” Event Name = “event010” postcondition Name=“postcondition004” device=“mouse” eventType=“OnClick” expression=“Customer.checkLogin()” 46 AB-UIDE: A Method for Adaptative UIs Design Design stage – Conectors model Graphical relationship. • Automatically generated Mapping between CUI and AUI. Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount Mapping between AUI and Domain model. October, 2005 (Albacete) 47 AB-UIDE: A Method for Adaptative UIs Design Design stage Sensors model the information captured from the context of use. – Adaptivity Rules Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) 48 AB-UIDE: A Method for Adaptative UIs Design Design stage Context events are produced by one or several sensors. They trigger adaptivity rules. – Adaptivity Rules Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) 49 AB-UIDE: A Method for Adaptative UIs Design Design stage – Adaptivity Rules Adaptivity rules will be available to be applied if the context precondition is met. The “real” adaptation is described by means of graph grammars transformations rules. Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) 50 AB-UIDE: A Method for Adaptative UIs Design Design stage – Adaptivity Rules SENSOR <softwarePlatformSensor id="SPS001" name="screenResolutionPlatformSensor" dataName="/contextInfo/platformInfo/hardwarePlatform/screenResolution" dataType="_2x2_" description="Senses whenever screen resolution changes."/> CONTEXT EVENT <softwarePlatformSensor id="SPS001" name="screenResolutionPlatformSensor" dataName="/contextInfo/platformInfo/hardwarePlatform/screenResolution" dataType="_2x2_" description="Senses whenever screen resolution changes."/> ADAPTIVITY RULE <adaptationRule id="AR002" name="radioGroupsOrientationChange" description="When screen resolution is reduced, tries to fit the presentation on screen by changing the orientation of the radio button to 'vertical'." > <contextEvents> <contextEvent id="CE001"/> </contextEvents> <contextPrecondition id="CP001" name="screenResolutionShrinked"> <logicalSentence id="LS001" name="OldGreaterThanNew" sentence="SPS001.oldValue > SPS001.newValue "/> </contextPrecondition> <transformation> Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences radioGroupsOrientationChange <<action>> Enter amount start <<action>> Show balance start error Parte izquierda (LHS) <<action>> InvalidAmount Parte derecha (RHS) </transformation> </adaptationRule> October, 2005 (Albacete) 51 AB-UIDE: A Method for Adaptative UIs Design Implementation stage – User interface specification • User Interface eXtensible Mark-Up Language • Stores the whole user interface specification • The specification is rendered for the target platform Deposit Logout <<extend>> <<extend>> <<extend>> Withdraw <<extend>> Bank Customer – Adaption engine Transfer Login <<extend>> <<extend>> <<extend>> Inquiry Recharge cell phone Change preferences <<action>> Enter amount start <<action>> Show balance start error <<action>> InvalidAmount October, 2005 (Albacete) • Multi-agent system based architecture • Takes advantage of the user interface specification for the application of adaptations • The architecture applies the adaptation facilities defined in the design process 52 CONTENTS 1. 2. 3. 4. Introduction State of Art in Adaptive User Interfaces Design AB-UIDE: A Method for Adaptive UIs Design A MAS Architecture for UI Adaptation 1. 2. 3. 4. 5. Initiative stage Proposal stage Decision stage Execution stage Implementing the MAS architecture 5. Final Remarks October, 2005 (Albacete) 53 A MAS ARCHITECTURE FOR UI ADAPTATION ::= Context Adaptivity rules Usability trade-off Platform Environment Current UI Multi-Agent System User Task October, 2005 (Albacete) Adapted UI 54 A MAS ARCHITECTURE FOR UI ADAPTATION Multi-agent system goals Adaptation process stages have been refined to design the goals for the multi-agent system. October, 2005 (Albacete) 55 A MAS ARCHITECTURE DispatcherAgent and AgentContextOfUse sense the context of use by means of the designed sensors. FOR UI ADAPTATION Multi-agent system overview October, 2005 (Albacete) 56 A MAS ARCHITECTURE FOR UI ADAPTATION Multi-agent system overview AgentContextPlatform, AgentContexUser and AgentContextEnvironment process the incoming data and produce the context events. October, 2005 (Albacete) 57 A MAS ARCHITECTURE FOR UI ADAPTATION The MAS uses all the Multi-agent system overview knowledge about the UI collected at design time. AgentAdaptationProcess proposes the plausible adaptations, selects the best ones and executes them. October, 2005 (Albacete) 58 A MAS ARCHITECTURE FOR UI ADAPTATION Initiative stage – Adaptation can be initiated by: • The user (adaptability) • The system (adaptivity) – System initiated adaptation • Sensing the context of use – Sensors detect the events produced in the context » Software sensors » Hardware sensors • Detecting the user’s current goal – What is the task the user is carrying out at a moment? – Recurrent task sequences – Heuristics based on the interaction data collected October, 2005 (Albacete) 59 A MAS ARCHITECTURE FOR UI ADAPTATION Proposal stage – A set of plausible adaptations for the current situation is proposed. – The possible adaptations to be applied are those adaptivity rules specified at design time. Adaptivity rules – The adaptation applicable given a context of use change are those that: • Are fired by the context events produced by the changes in the incoming sensors data. • The context precondition is met. October, 2005 (Albacete) 60 A MAS ARCHITECTURE FOR UI ADAPTATION Decision stage – How to choose the best adaptation (plan) among the proposed ones • Compute how good or bad an adaptation (plan) is for the user: – Migration cost: represents the physical, cognitive and conative effort the user needs to apply in order to migrate from one context to another. – Adaptation benefit: represents how good an adaptation will be for the user in the new context. • Choose the one that maximizes: Adaptation benefit – Migration cost October, 2005 (Albacete) 61 A MAS ARCHITECTURE FOR UI ADAPTATION Decision stage – Migration cost: represents the physical, cognitive and conative effort the user needs to apply in order to migrate from one context to another. User’s mental effort required to resume the task that was carrying out before adaptation took place. Migration cost Discontinuity Physical effort Cognitive load Cognitive effort Preferences Conative effort October, 2005 (Albacete) 62 A MAS ARCHITECTURE FOR UI ADAPTATION Decision stage – Migration cost: represents the physical, cognitive and conative effort the user needs to apply in order Amount of information the user to migrate from one context to another. needs to understand to perform the tasks using the adapted user interface. Migration cost Discontinuity Physical effort Cognitive load Cognitive effort Preferences Conative effort October, 2005 (Albacete) 63 A MAS ARCHITECTURE FOR UI ADAPTATION Decision stage – They are assessed by means of based on the empirical Migration cost: represents metrics the physical, cognitive results from GOMS-based and conative effort the user(Goals, needs to apply in order Operators, Methods, Selection rules) evaluations. to migrate from one context to another. Migration cost Discontinuity Physical effort Cognitive load Cognitive effort Preferences Conative effort October, 2005 (Albacete) 64 A MAS ARCHITECTURE FOR UI ADAPTATION Decision stage – Migration cost: represents the physical, cognitive and conative effort the user needs to apply in order to migrate from one context to another. Migration cost Preferences modify the other two parameters evaluation. Discontinuity Physical effort Cognitive load Cognitive effort Preferences Conative effort October, 2005 (Albacete) 65 A MAS ARCHITECTURE FOR UI ADAPTATION Decision stage – Adaptation benefit: represents how good an adaptation will be for the user in the new context. When a context of use situation is often found, the cost should be reduced since it will allow dealing with common situations. Adaptation benefit Context frequency User Feedback The adaptations can be rejected by the user. The more times the user rejects an adaptation the less likely that adaptation will be. October, 2005 (Albacete) 66 A MAS ARCHITECTURE FOR UI ADAPTATION Proposal stage – Because of the limitations of the model the system needs to evolve at run time to improve adaptation process. This evolution has been included as Bayesian learning (as in antispam filters, for instance). – The formula below will be applied for each selectable adaptation (producing a ranking of rules). P(R|S) quantifies the compatibility between the hypothesis (the adaptation selection) and the contents of the adaptation (the adaptation itself). P(S) represents the a priori probability that R is selected to be applied. P( R | S ) * P( S ) P( S | R) P( R) P(S|R) represents the probability that when R is applicable, R is choosed. October, 2005 (Albacete) P(R) represents the probability of R. 67 A MAS ARCHITECTURE FOR UI ADAPTATION Execution stage – The system executes the first adaptation in the ranking by means of the transformation engine. – The system checks that the application of the adaptation doesn’t violate the usability trade-off for the current platform profile created at design-time. – If the adaptation violates the usability trade-off • Undo last adaptation • Repeat the execution and usability trade-off checking processes for the next adaptation in the ranking until: – One adaptation meets usability trade-off – Adaptation ranking list is empty (no adaptation could be applied) – An adaptation is found where ranking value is too low October, 2005 (Albacete) 68 A MAS ARCHITECTURE FOR UI ADAPTATION Execution stage Translate the UI graph representation into usiXML syntax. Translate the XML specification into a graph representation. Apply the graph grammar transformations on the UI graph representation. October, 2005 (Albacete) Render the usiXML specification for the target platform. 69 CONTENTS 1. 2. 3. 4. 5. Introduction State of Art in Adaptive User Interfaces Design AB-UIDE: A Method for Adaptive UIs Design A MAS Architecture for UI Adaptation Final Remarks 1. Conclusions 2. Contributions 3. Future work October, 2005 (Albacete) 70 FINAL REMARKS Conclusions & Outcomes – An adaptive UI design method (AB-UIDE) – Adaptive user interfaces execution October, 2005 (Albacete) 71 FINAL REMARKS Conclusions & Outcomes – An adaptive UI design method (AB-UIDE) • A specification to capture context data through sensors modelling. • A metamodel for adaptivity rules to provide a common syntax for adaptations specification. • A runtime quality model (usability trade-off) to preserve usability while adapting the user interface. • A task model enriched with dialog specification. • An abstract user interface (AUI) model and a set of heuristics to transform the domain and task/dialog model into the AUI and the CUI. • A graphical syntax for model-to-model mapping that allows preserving traceability in the development process. – Adaptive user interfaces execution October, 2005 (Albacete) 72 FINAL REMARKS Conclusions & Outcomes – An adaptive UI design method (AB-UIDE) – Adaptive user interfaces execution • An architecture based on multi-agent system for adaptive user interfaces execution. • The integration of the adaptation facilities designed following AB-UIDE within the architecture in a seamless way. • A model to assess how good or bad an adaptation is given a context of use state. • The implementation of the MAS architecture proposed. • The implementation of a tool for the transformation of user interfaces specifications by means of graph grammars transformations rules. October, 2005 (Albacete) 73 FINAL REMARKS Acknowledgements – This work has been supported by • The spanish grants: – CYCIT TIN2004-08000-C03-01 project – JCCM PBC-03-003 project • European networks – SIMILAR Network of Excellence – Seven month stay at BCHI (Belgian laboratory of Computer Human Interaction). October, 2005 (Albacete) 74 FINAL REMARKS Contributions – Topics • Adaptive user interfaces development related papers • Multi-agent systems related papers • Study cases related papers October, 2005 (Albacete) 75 FINAL REMARKS Contributions – Adaptive user interfaces development related papers (i) • López-Jaquero, V., Montero, F., Molina, J.P., González, P., Fernández-Caballero, A. A Seamless Development Process of Adaptive User Interfaces Explicitly Based on Usability Properties. Proc. of 9th IFIP Working Conference on Engineering for Human-Computer Interaction jointly with 11th Int. Workshop on Design, Specification, and Verification of Interactive Systems EHCI-DSVIS’2004 (Hamburg, July 11-13, 2004). Lecture Notes in Computer Science, Vol. 3425, Springer-Verlag, Berlin, 2005. • López Jaquero, V., Montero, F., Fernández Caballero, A., Lozano, M.D. Towards Adaptive User Interfaces Generation: One Step Closer to People. In Enterprise Information Systems V. Kluwert Academia Publishers, Dordrecht, Holanda, 2004. pp. 226-232. ISBN: 1-4020-1726-X. • López Jaquero, V., Montero, F., Molina, J.P., Fernández-Caballero, A., González, P. Model-Based Design of Adaptive User Interfaces through Connectors. Design, Specification and Verification of Interactive Systems 2003, DSV-IS 2003. In DSV-IS 2003 : Issues in Designing New-generation Interactive Systems Proceedings of the Tenth Workshop on the Design, Specification and Verification of Interactive Systems. J.A. Jorge, N.J. Nunes, J. F. Cunha (Eds). Springer Verlag, LNCS 2844, 2003. Madeira, Portugal June 4-6, 2003. • López Jaquero, V., Montero, F., Fernández, A., Lozano, M. Towards Adaptive User Interface Generation: One Step Closer To People. 5th International Conference on Enterprise Information Systems, ICEIS 2003. Proccedings of 5th International Conference on Enterprise Information Systems, ICEIS 2003, vol. 3, pp. 97103. Angers, France, April 23-26, 2003. • Montero, F., López Jaquero, V., Molina, J.P., González, P. An approach to develop User Interfaces with plasticity. Design, Specification and Verification of Interactive Systems 2003, DSV-IS 2003. In DSV-IS 2003 : Issues in Designing New-generation Interactive Systems Proceedings of the Tenth Workshop on the Design, Specification and Verification of Interactive Systems. J.A. Jorge, N.J. Nunes, J. F. Cunha (Eds). Springer Verlag, LNCS 2844, 2003. Madeira, Portugal June 4-6, 2003. October, 2005 (Albacete) 76 FINAL REMARKS Contributions – Adaptive user interfaces development related papers (ii) • Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., López-Jaquero, V., UsiXML: a Language Supporting Multi-Path Development of User Interfaces, Proc. of 9th IFIP Working Conference on Engineering for Human-Computer Interaction jointly with 11th Int. Workshop on Design, Specification, and Verification of Interactive Systems EHCI-DSVIS’2004 (Hamburg, July 11-13, 2004). Lecture Notes in Computer Science, Vol. 3425, Springer-Verlag, Berlin, 2005, pp. 207-228. • Montero, F., López-Jaquero, V., Vanderdonckt, J., González, P., Lozano, M.D., Solving the Mapping Problem in User Interface Design by Seamless Integration in IdealXML. 12th International Workshop on Design, Specification and Verification of Interactive Systems (DSV-IS’2005), Newcastle upon Tyne, England, July 13-15, 2005. Springer-Verlag, Berlin, 2005 (in print). • Montero, F., López-Jaquero, V., Lozano, M., González, P. A User Interfaces Development and Abstraction Mechanism. Artículo seleccionado en el V Congreso Interacción Persona Ordenador para su publicación en Springer-Verlag, Berlin, 2005 (in print). October, 2005 (Albacete) 77 FINAL REMARKS Contributions – Multi-agent systems related papers • López-Jaquero, V, Montero, F., Molina, J.P., González, P., Fernández-Caballero, A. A Multi-Agent System Architecture for the Adaptation of User Interfaces. 4th International Central and Eastern European Conference on Multi-Agent Systems (CEEMAS 2005). 15-17 September 2005, Budapest, Hungary. In Multi-Agents Systems and Applications IV. M. Pechoucek, P. Petta, L. Zsolt Varga (Eds.) LNAI 3690, Springer-Verlag, Berlin. • López-Jaquero, V., Fernández-Caballero, A. Métricas de Usabilidad y Sistemas Multiagente en Hipermedia Adaptativa. XIII Escuela de Verano de Informática. Tendencias Actuales en la Interacción PersonaOrdenador: Accesibilidad, Adaptabilidad y Nuevos Paradigmas. ISBN: 84-921873, pp. 21-34, Albacete, España, 2003. • Fernández-Caballero, A., López Jaquero, V., Montero, F. , González, P. Adaptive Interaction Multi-agent Systems in E-learning/E-teaching on the Web. International Conference on Web Engineering, ICWE 2003. In Web Engineering: International Conference, ICWE 2003, Oviedo, Spain, July 14-18, 2003. Proceedings. J.M. Cueva Lovelle, B.M. González Rodríguez, L. Joyanes Aguilar, J.E. Labra Gayo, M. del Puerto Paule Ruiz (Eds.). Springer Verlag, LNCS 2722, pp. 144-154. ISSN:0302-9743. Oviedo, Spain, June, 2003. • López Jaquero, V., Montero, F., Fernández, A., Lozano, M. Usability Metrics in Agent-Based Intelligent Tutoring Systems. Human-Computer Interaction: Theory and Practice (part 1). J. Jacko, C. Stephanidis (Eds.). Lawrence Erlbaum Associates. Londrés, Reino Unido, 2003. ISBN: 0-8058-4931-9. pp. 539-543. October, 2005 (Albacete) 78 FINAL REMARKS Contributions – Study cases related papers • Robles, A., Molina, J. P., López-Jaquero, V., García, A. S. Even Better Than Reality: The Development of a 3-D Online Store that Adapts to Every User and Every Platform. HCI International 2005, Las Vegas, Nevada, USA, July, 2005. Volume 7 - Universal Access in HCI: Exploring New Interaction Environments. • López-Jaquero, V., Fernández-Caballero, A., Montero, F., Molina, J.P., González, P. Towards Adaptive Elearning / E-teaching on the Web. International Conference on Technology-Enhanced Learning (TEL 2003). Procedings of International Conference on Technology-Enhanced Learning (TEL 2003). Milán, Italia, noviembre, 2003. • González, P., Montero, F., López Jaquero, V., Fernández, A., Montañés, J., Sánchez, T. A Virtual Learning Environment for Short Age Children. IEEE International Conference on “Advanced Learning Technologies”, ICALT 2001. Proccedings of the IEEE International Conference on Advanced Learning Technologies, ICALT 2001, Okamoto, T., Hartley, R., Kinshuk, Klus, J. (eds.). IEEE Computer Society, Los Alamitos, CA., Agosto 2001, pp. 283-285. ISBN:0-7695-1013-2. Madison, USA, August 6-8, 2001. October, 2005 (Albacete) 79 FINAL REMARKS Future work – Collaborative adaptive user interfaces – Adaptation in virtual environments – Visual and intuitive graphical tools for adaptivity rules design – Creating a corpus of adaptiviy rules big enough – Porting the multi-agent system to a open source agent language like JADE. – Adding user modelling techniques to make user model evolve automatically by inference. – Make some more usability tests for adaptive user interfaces designed and executed by using our approach. October, 2005 (Albacete) 80 QUESTIONS & ANSWERS Adaptive User Interfaces Based on Models and Software Agents Víctor M. López Jaquero Escuela Politécnica Superior de Albacete Departamento de Sistemas Informáticos Universidad de Castilla-La Mancha Campus Universitario, s/n. 02071 – Albacete (SPAIN) Email: victor@info-ab.uclm.es Thanks for your attention October, 2005 (Albacete) 81
