Human Computer Interface
Goutam Datta
UNIT 1 : What is HCI
• The study of how computer technology affects human labour and
activities is known as Human–Computer Interaction (HCI).
• The word "computer technology" currently encompasses a wide
range of devices, from traditional computers with displays and
keyboards to smartphones, home appliances, in-car navigation
systems, and even integrated sensors and actuators like automated
lighting.
What is HCI….
• HCI is linked to a design field known as Interaction Design or UserCentered Design, which focuses on how to make computer
technology as simple and enjoyable to use as possible.
• The concept of "usability," which is generally characterized in terms of
efficiency, effectiveness and satisfaction, is an important part of the
design profession.
• The study of creating computers and technologies to best serve their
users is known as human-computer interaction (HCI) (i.e. humans).
HCI Contd.
• The Human computer interaction is the practice and study of
usability.
• It is about the relationship between a human and a computer,
their mutual understandings and by creating a software which
would ease the work of a human and people would love to use,
and would be able to use it.
• It may also be said that it is a study of how humans use
computers to perform certain tasks and use it in such a way that
the interaction is being enjoyed and effective.
HCI-Interdisciplinary
•
•
•
•
Computer Graphics
Artificial Intelligence
Computer Vision
Ergonomics and Human Factor :Greek word ergon-work, nomos-laws i.e.
“law of work”.
• Ethnography : Study of individual culture.
• Psychology :the mental characteristics or attitude of a person or group.
• Cognitive science/ergonomics:Cognitive ergonomics is concerned with
mental processes, such as perception, memory, reasoning, and motor
response, as they affect interactions among humans and other elements of
a system.
HCI Contd.
• It involves the sketching of low and high fidelity, i.e., the degree of
exactness a thing is being reproduced.
• HCI -------• Human : The humans are good at performing fuzzy/hard
computations.
• Computer : The computers are used for interaction with the users as they
have special components that can interact with the
users.
• Interaction : It is the interaction between a computer and a human to
produce
an effective output. The interaction is a two- way process between a user
and a computer.
HCI Design approaches
• 1. Anthropomorphic Approach:
This approach involves designing human interface such as to
produce human like characteristics.
• 2. Cognitive Approach:
This approaches used to develop a user interface that supports the
end user and considers the abilities of human brain and sensory
recognition.
• 3. Empirical Approach:
This approach is used for examining and comparing the usability of
multi-conceptual designs.
HCI design approaches
• 4. Predictive Modelling Approach:
GOMS method is used for examining and takes into consideration, user’s experience in terms of time taken by a
user to efficiently and effectively complete a goal.
GOMS stands as g stands for goals, o for operators, and m for methods and s for section rules.
The definite measurements of human’s performance are used to calculate the time taken by it to accomplish a particular
goal.
Applications of HCI/Widely used devices
• Every day life : ATM machine, Smart Phone,Railway Ticket selling
machine, Hot beverage vending machines
• In Industry and business : Power plants, air traffic control etc.
• Other applications : Virtual Reality.
Goals of HCI
Usability
• Crucial issue in this area!
• Combination of
– Ease of learning
– High speed of user task performance
– Low user error rate
– Subjective user satisfaction
– User retention over time
Characteristics of Graphical User Interface
• Sophisticated Visual Presentation :
• Pick-and-Click Interaction :
• Restricted Set of Interface Options : WYSIWYG
• Visualization : Visualization is a cognitive process that allows people
to understand information
Effective visualizations can facilitate mental insights, increase
productivity, and foster faster and more accurate use of data.
• Use of Recognition Memory : Continuous visibility of objects and
actions encourages to eliminate “out of sight, out of mind” problem
• Concurrent Performance of Functions :
Characteristics of Web User Interface
• Webmail,
• online shopping,
• online document sharing/Online form filling,
• social media, .
User Interface Design Process
User-Centered Design :
• Analyze user’s goals & tasks
• Create design alternatives
• Evaluate options
• Implement prototype
• Test
• Refine
Above All Else…
Design Evaluation
Important Parameters to measure Usability
Use Case- ATM transaction
Use case ATM-Contd.
Use case ATM- Contd.
Use case ATM-Contd.
Direct and Indirect Interaction with Interface
• Virtual Reality can be viewed as a field which can draw upon the
principles of direct manipulation for Human-Computer Interaction
(HCI) design or as an example or extension of direct manipulation
itself. In VR, not only can task objects and actions be naturally
represented, the task environment can be naturally represented as
well. With either view, an understanding of direct manipulation
principles is essential for the successful design of human computer
interfaces in virtual environments.
• An example of direct manipulation is resizing a graphical shape, such
as a rectangle, by dragging its corners or edges with a mouse.
• Example : Zoom in /Zoom out of image
Direct Manipulation Contd.
• The system is portrayed as an extension of real world.
• There is continuous visibility of objects and actions.
• Actions are rapid and incremental with a visible display results.
• Incremental actions are easily reversible.
Indirect Manipulation
• CUI : Character user interface, Typing commands etc. to perform any
work.
Qualities of a good interface design
Qualities of a good interface design
Simplicity :
• User Interface design should be simple.
• Less number of mouse clicks and keystrokes are required to
accomplish this task.
• It is important that new features only add if their is compelling need
for them and they add significant values to the application
Software Engineering User Interface
User interface is the front-end application view to which user interacts
in order to use the software. The software becomes more popular if its
user interface is:
• Attractive
• Simple to use
• Responsive in short time
• Clear to understand
• Consistent on all interface screens
User Interface design Process : A spiral model
User Interface design process: A spiral Model
• User, task, environmental analysis, and modeling: Initially, the focus is based
on the profile of users who will interact with the system, i.e. understanding, skill and knowledge,
type of user, etc. based on the user’s profile users are made into categories. From each category
requirements are gathered.
• Interface Design: The goal of this phase is to define the set of interface objects and actions
i.e. Control mechanisms that enable the user to perform desired tasks. This phase serves as the
foundation for the implementation phase.
• Interface construction and implementation: The implementation activity begins with the
creation of prototype (model) that enables usage scenarios to be evaluated.
• Interface Validation: This phase focuses on testing the interface.
Waterfall model
Expert System : Inferencing
Interface design: Prototyping
• Essential element in user centered design :
- Is an experimental and partial design
-Helps involving users in testing design ideas without implementing a
full-scale system
HCI : Prototyping
In HCI, prototypes take many forms :
• – A storyboard (cartoon-like series of screen sketches)
• – A power point slide slow
• – A video simulating the use of a system
• – A cardboard mock-up
• – A piece of software with limited functionality
• – Even a lump of wood
HCI : Prototyping
We can categorize all these different forms of prototypes in three
groups
• – Low fidelity prototypes
• – Medium fidelity prototypes
• – High fidelity prototypes
Low Fidelity Prototypes
• Basically paper mock-up of the interface look, feel, functionality :
– Quick and cheap to prepare and modify
• Purpose :
 Brainstorm competing designs
 Elicit user reaction (including any suggestions for
modifications)
Low Fidelity Prototypes
Medium Fidelity Prototypes
• Prototypes built using computers :
– More powerful than low fidelity prototypes
– Simulates some but not all functionalities of the
system
– More engaging for end users as the user can get better
feeling of the system
– Can be helpful in testing more subtle design issues
Hi Fidelity Prototypes
• Typically a software implementation of the design with full or most of
the functionalities :
– Requires money, manpower, time and effort
– Typically done at the end for final user evaluations
Inferencing , Bayesian Belief Network etc.
CBIR
iHCI Design : User Model Acquisition and
Representation, Inference based approaches
• Use models : acquisition and representation : Some of the interfaces
along with case studies : Decision support system , Knowledge
discovery in data mining,
• CBIR : Interface for content based image retrieval
• Inference based approaches : Expert system design
• Fuzzy logic based models
• Bayesian strategies for data entry
Design Evaluation : An heuristic based
approach
• The final stage of assessment is performing the heuristic evaluation
performed on the developed product.
• With heuristic evaluation we can be assured that the interface
designed satisfies all the usability requirement given by Nielsen’s
usability.
Heuristic Evaluation Criteria : Nielsen’s
usability criteria
• Visibility of a system’s status : For example interface designed for an
app…..Status of app in terms of online ,offline, user’s progress etc.
• Match between the system and the real world : match between real
world scenario and the terminologies used. Easy to comprehend
language to use that system/app etc.
• User control and freedom : This heuristic puts the user in charge and
gives them the ability to do things freely.
• Consistency and standards : There are some things that need to be
conventional for a user. For example : buttons, keywords, highlighted
links etc.
Heuristic Evaluation Criteria : Nielsen’s
usability criteria
• Error prevention : Error prevention with options to delete and undo etc.
• Recognition rather than recall : It is much easier for users to recognize
options rather than recall from memory. The entire system depends on this
heuristic as the idea is to lessen recall load for users by giving them options
to recognize.
• Flexibility and efficiency of use: Regular routes are listed on maps, but
locals with more knowledge of the area can take shortcuts. Example :
Keyboard short cut, touch gestures etc.
• Aesthetic and minimalistic design : minimalistic design is about making
sure that everything presented to the user is necessary and useful.
Aesthetic considerations are where visual designs come into play. It has
been made a priority to provide only necessary information to users
through simple yet appealing designs.
Heuristic Evaluation Criteria : Nielsen’s
usability criteria
• Help and documentation: Help and instructions are frequently
needed by the user. They should be accessible, understandable and
accurate.
Virtual Reality, Augmented Reality
• Virtual Reality (VR) is a computer-generated environment with
scenes and objects that appear to be real, making the user feel they
are immersed in their surroundings. This environment is perceived
through a device known as a Virtual Reality headset or helmet.
• Examples : stereo headphones, head-mounted stereo television
goggles, and data-gloves.
• Augmented reality (AR) is an interactive experience of a real-world
environment where the objects that reside in the real world are
enhanced by computer-generated perceptual information, sometimes
across multiple sensory modalities, including visual, auditory, haptic,
somatosensory and olfactory.
Interface Designer to user experience(UX)
designer.
• Interactive/interaction designer : People involved in the design of all the
interactive aspects of a product.
• Usability Engineers : People who focus on evaluating products, using
usability methods and principles.
• Web designers : People who develop and create the visual design of
websites such as layouts.
• UI designers : People experienced in user centered design methodologies.
• UI design engineers : People who develop and model the end user
experience, using task, workflow analytic methods, and low and high
level prototyping tools.
Jakob Nielsen’s ten rules of thumb!
• Jakob Nielsen is a leading web usability consultant. He holds a Ph.D. in
human–computer interaction.
• Nielsen’s heuristics method are empirically based derivations. Widely
used by Usability professionals (which includes Interface designers),
they are a means of quickly identifying likely interface design
problems in an application.
• Method suggested by Nielsen is popular because of its simplicity and
low cost. It is preferred evaluation technique at the preliminary
design stages by HCI professional.
Interface Designer to user experience(UX)
designer.
• Information Architects: People who come up with ideas of how to
plan and structure interactive products, especially websites.
• User experience (UX) designers/architects/researchers : People who
do all the above but who may also carry out ethnographic field
studies to research into users’ needs and convert them into
actionable results.
Jakob Nielsen’s ten rules of thumb!
Conclusion
• These ten heuristics of usability help in refining a potential design into
a good design.
• They ensure that interfaces evolve in the right direction.
• These rules of the thumb act a check list to evaluate a design.
• They also can be used as check list while evaluating any GUI.
ATM -Interface
Study of ATM interface
• Are attractive interfaces more or less usable?
• An original Japanese interface rated high in diagram (a)
• And rated low in diagram (c) on apparent usability and aesthetics.
• Equivalent Israeli interface rated high figure (b) and rated low figure
(d) on apparent usability and aesthetics.
Biological Neuron –Artificial Neuron
Artificial Neural Network
ANN with multiple hidden layers.
ANN
• ANN detail architecture
• Activation function : Liner and Non linear
Hidden user Interface via wearable and
Implanted Devices
Device can be :
• Accompanied : External to body, not attached ,Example : Mobile, Smart
card.
• Wearable : External but attached to the body, Example : Hearing aids,
earpieces, eyetap
• Implanted : This type of device is likely to require a surgical procedure to
install the device inside the body, medical purpose, Example : Computer
Implants..
RFID chips :Microchip inserted under the skin - could be used in the place of
a contactless bank card, to hold medical information, or as ID.
Cochlear implants : An enhanced version of a hearing aid that connects to
the auditory nerve behind the ear. Advancements in these are looking to add
noise cancelling and voice command AI (artificial intelligence).
Hidden user Interface via wearable and
Implanted Devices
• Implanted :A cochlear implant is a small electronic device that
electrically stimulates the cochlear nerve (nerve for hearing). The
implant has external and internal parts. The external part sits behind
the ear. ... A cochlear implant may help someone with hearing loss
restore or improve the ability to hear and understand speech.
UNIT 2 : Brain-Computer Interface
• A Brain Computer Interface (BCI) is a collaboration in which a brain
accepts and controls a mechanical device as a natural part of its
representation of the body.
• A brain-computer interface acquires signals from the brain, analyzes
them and translates them into commands. These commands are then
translated into a signal from peripheral devices to provide the desired
output.
• The primary goal of a BCI is to restore useful function for people who
have developed neuromuscular disorders cerebral palsy, stroke, or
spinal cord injury.
GENERAL PRINCIPLE
• In healthy subjects, primary motor area sends movement
commands to muscles via spinal cord :
(b) In paralyzed people this pathway is interrupted.
(c) Computer based decoder translates this activity into
commands for muscle control
Background
• Signals from an array of neurons read.
• Cerebral electric activity recorded.
• Signals are amplified.
• Transmitted to computer
• Transformed to device control commands.
• Using computer chips and programs.
• Signals translated into action.
BCI
Brain Computer Interface
Brain Computer Interface Types
There are two kinds of Brain-Computer Interface:
• Non-Invasive Brain-Computer Interface and
• Invasive Brain-Computer Interface.
Non-Invasive Brain-Computer Interface
Non invasive BCI
• As the name suggests, a non-invasive brain-computer interface is
one that can work without intrusive procedures into the brain. A nonIntrusive Brain-Computer Interface mostly works on the principles of
EEG (Electroencephalography).
• Electroencephalography is mostly used in the medical field to see and
analyze the brain wave activity of patients.
• The way an EEG is performed in a medical setup is by attaching
multiple electrodes to the scalp of the patient. EEG measures the
voltage fluctuations in the brain caused by the ionic current within
the neurons of the brain. An EEG records the spontaneous electrical
activity of the brain over a certain period.
Non invasive BCI
• Most Non-Invasive BCI uses the concept of EEG in their models.
• The most famous among them is Neurosky, a consumer-friendly
product that uses the concept of EEG.
• It has various modes where one can test one’s level of attention,
mental effort, and meditation level. It consists of one EEG sensor that
touches the front left upper part of the skull, just above the left
eyebrow.
• The applications are only limited by the user’s own ability to use the
product.
Non invasive BCI
• The advantages of a non-invasive brain-computer interface stem from
the fact that it is much cheaper to work with and heavy research
focus is always given to non-invasive BCI.
• Also, multiple people from diverse backgrounds can work on noninvasive BCI.
• In the case of an invasive BCI, a medical professional is always
needed.
Invasive BCI
Invasive BCI
• An invasive brain-computer interface involves the surgical
implantation of a device into the skull of the user.
• There are two kinds of Invasive BCI that have been tried and tested
thus far.
Invasive BCI
• The first example is ECOG (Electrocorticography), which is when an electrode
plate is kept in direct contact with the brain’s surface to measure the electrical
activity of the cerebral cortex.
• To access the cerebral cortex, a surgeon must perform a craniotomy or open a
part of the skull to expose the brain’s surface. This procedure is usually done
under general or local anesthesia if patient interaction is required. ECoG
electrode arrays typically consist of sixteen sterile, disposable stainless steel,
carbon tip, platinum, Platinum-iridium alloy or gold ball electrodes, each
mounted on a ball and socket joint for ease in positioning.
• These electrodes are attached to an overlying frame in a “halo”
configuration. The grids are transparent, flexible, and numbered at each
electrode contact… The electrodes sit lightly on the cortical surface and are
designed with enough flexibility to ensure that regular movements of the brain
do not cause injury.
Invasive BCI :Cochlear Implant
Invasive BCI
• Finally, we have the Intracortical Microelectrodes, also known as
chronic electrode implants. A chronic electrode implant is an
electronic device that is usually implanted into the brain or issue for
an extended period. It has two significant applications, one for
stimulating and the other for recording. Applications for stimulating
involve sensory prosthetics, such as cochlear implants. A cochlear
implant is a device that provides the sensation of sound to a person
with severe or profound sensorineural hearing loss.
Research in BCI
• Brain-Computer Interface is a cutting edge and relatively new topic.
Ongoing research by prestigious universities and major
corporations will undoubtedly take this technology to a whole new
level in the near future!
Books/Web Links/Research Articles
• Learn Human Computer Interaction Christopher Reid Becker
• Emotions in Technology Design : From Experience to Ethics Rebekah
Rousi, Jaana Leikas, Pertti Saariluoma
• https://www.scienceabc.com/innovation/what-is-a-braincomputer-interface.html
HCI : MCQ
• 1. Identify the evaluation technique that is useful to evaluate early
design : A. heuristic evaluation
B. cognitive walkthrough
C. model based
D. review based
• 2. Who proposed heuristic evaluation?
A.nielson and molich
B. allen newell
C. thomas p.moron
D. stuart k.card
HCI : MCQ
User Interface and interaction for four widely used devices:
• Personal Computer
• Hand held mobile devices for communication
• Remote controlled Audio Video Display
• Games console
• None of the above
• All of the above
HCI : MCQ
• What are the things needed for cognitive walkthrough?
A. prototype of the system
B. description of the task
C. both a and b
D. none of the above
• In which evaluation method "usability criteria" is identified?
A. review based evaluation
B. model based evaluation
C. heuristics evaluation
D. experimental evaluation
HCI : MCQ
• The multiplicity of ways the user and system exchange information.
A. flexibility
B. robustness
C. learnability
D. usability
BCI: Data sets
• Non-invasive Brain-computer interfaces are an exciting new
technology that provide a channel for communication between the
brain and a computer system. They can be used as communication
devices.
• They can be used as communication devices (Chaudhary et al., 2016;
Brumberg et al., 2018), rehabilitation systems (Cervera et al., 2018),
entertainment devices (Gürkök et al., 2017), and for a wide range of
other applications.
BCI : Dataset
• Research in non-invasive BCIs is developing rapidly and is a highly
multidisciplinary field, involving, among others, neuroscientists,
engineers, psychologists, computer scientists, and clinicians.
Continuing development of BCI technology relies on advances made
in each of these fields, which individually and collectively can
contribute to improving all aspects of BCI systems including signal
acquisition, processing, classification, and user interface design.
BCI : Data sets
• Many individual parts of a BCI system are typically first developed and
evaluated on pre-existing datasets. However, there are only a few high
quality publicly available datasets on which new systems, tools, and
technologies can be evaluated and compared. For example, the publicly
available BCI competition datasets (Sajda et al., 2003; Blankertz et al.,
2004, 2006) provide an excellent set of resources for BCI researchers and
have been widely used by numerous researchers to develop and evaluate
new signal processing and classification methods (Arvaneh et al., 2013;
Ghaemi et al., 2017; Lotte et al., 2018; Sakhavi et al., 2018; Zanini et al.,
2018; Zhang et al., 2018). Yet, the relatively small size and number of such
datasets introduce the risk of overfitting to methods developed and
evaluated with these datasets. In other words, the reliability and
reproducibility of BCI research is held back by a lack and sparsity of publicly
available datasets.
BCI Data set
• This special issue provides a collection of descriptions of publicly
available physiological datasets recorded during development,
training, and evaluation of non-invasive BCI systems from BCI
research labs around the world.
• The collected datasets consist of signals recorded via a wide variety of
modalities, including, but not limited to, electroencephalography
(EEG), functional near infrared spectroscopy (fNIRS),
electromyography (EMG), electrocardiography (ECG), galvanic skin
response (GSR), skin temperature measures, respiration rates, and
body movement data. Many datasets include multimodal recordings
with combinations of two or more of these signal modalities.
BCI : Data set
Signal Processing Steps:
• The human motion pattern-recognition system based on EEG signal is mainly
divided into several parts:
• Preprocessing : De noise the signal. Get signal window range etc.
• Feature extraction : There are many methods for feature extraction, including
time domain, frequency domain, time-frequency domain etc.
• In this regard, the feature vectors can be transformed by principal component
analysis (PCA), linear discriminant analysis (LDA) [26], and other methods, which
will result in fewer dimension features
• Pattern recognition : The most important is classifier design. Common classifiers
are Linear Discriminant Analysis(LDA), ANN, Support Vector Machine classifiers.
• post processing :
among which feature extraction and pattern recognition are especially important.
Important Terms Related To BCI
• Bioelectrical potentials recorded from electrodes placed invasively on
the cortex or noninvasively on the scalp are named
electrocorticogram (ECoG) and electroencephalogram (EEG),
respectively.
• ECoG is also an important research tool to identify and study new
cortical areas responsible for language, sensori-information, motor
functions, vision, faces, and symbols and many more functions. These
newly identified regions can be used for real-time control applications
and to improve our understanding of brain function.
Important Terms Related To BCI
• Mental imagery is similar to motor imagery, but instead of imagining
motions, the user performs different types of cognitive activities:
mental subtraction, auditory imagery, spatial navigation, etc.
• Motor imagery is a cognitive process in which a subject imagines
that he=she performs a movement without actually performing the
movement and without even tensing the muscles. It is a dynamic
state during which the representation of a specific motor action is
internally activated without any motor output.
• The term motor imagery (MI) refers to the mental simulation of body
movements.
Hyper Parameter Optimization
• 1. Select the objective function and the performance metrics;
• 2. Select the hyper-parameters that require tuning, summarize their types,
and determine the appropriate optimization technique;
• 3. Train the ML model using the default hyper-parameter configuration or
common values as the baseline model;
• 4. Start the optimization process with a large search space as the hyperparameter feasible domain determined by manual testing and/or domain
knowledge;
• 5. Narrow the search space based on the regions of currentlytested wellperforming hyper-parameter values, or explore new search spaces if
necessary.
• 6. Return the best-performing hyper-parameter configuration as the final
solution.
Motor Imagery Classification..
• Existing motor imagery classification methods can be categorized into
five types [8]: 1) linear classifiers with linear discriminant analysis
(LDA) and support vector machine (SVM) being the representative
ones; 2) nonlinear Bayesian classifiers such as Bayes quadratic and
Hidden Markov Model (HMM); 3) nearest neighbor classifiers like k
nearest neighbors (KNN); 4) neural network methods such as
multilayer perceptron (MLP) [12] and Radial Basis Function (RBF)
neural network [13]; 5) combinations of different classification
methods by boosting or voting
Hyper Parameters of ML Algorithms.
Some of the default hyper parameter configurations of Basic ML algorithms:
1. Random Forest classifiers :

n-estimator=No. of decision trees , default value=100(in scikit-learn)

max-depth= Max. height of the tree upto which it can grow, default
value=None(implies all nodes can continue to grow till all nodes become pure
leaf)

min_samples_split= Specifies the min amount of sample each node can hold to
further split that node. min_samples_split= between 2 and 6. But min value=2

min_samples_leaf=It specifies the minimum amount of samples that a node
must hold after getting split. Default value =1

max_features=Random forest takes random subsets of features and tries to find
the best split.

max_leaf_nodes=It sets a limit on the splitting of the node and thus helps
to reduce the depth of the tree, and effectively helps in reducing
overfitting.
max_samples=This hyperparameter helps to choose maximum number of samples
from the training dataset to train each individual tree.
MI Patterns….
Spike-decoding in BCI
• Spike-decoding has used both linear and nonlinear models to relate
spike trains to actual or intended actions. Model performance is
typically measured by comparing the movement derived from the
spike trains to the actual (or intended) movement and computing
their difference in terms of mean squared error (i.e., the closer to
zero, the better) or a correlation 280 | BRAIN–COMPUTER
INTERFACES coeffi cient (i.e., the closer to + 1, the better) (fi g. 16.7A
).
Long Term Recording Issues
• Since the surgical implantation of foreign objects always carries some
amount of risk, invasive BCI devices, whether implanted within the brain
itself or within the cranium, must last for many years without requiring
replacement or repair.
• Furthermore, the long-term implantation itself must not entail significant
risks of infection or tissue damage. These risks can be minimized by good
choices as to the type and geometry of the implanted materials, by using
proper surgical insertion techniques, by ensuring implant stability, and by
continued monitoring of tissue responses and possible infection.
• Although these factors undoubtedly contribute to risk/benefit assessment
and to the success of long-term recording, they have not been formally
evaluated in a comprehensive fashion. Given the many variables that
distinguish the various sensor approaches from one another, it is difficult to
make direct comparisons with compelling conclusions.
FACTORS THAT AFFECT RECORDING STABILITY
• The variables that most affect long-term iBCI sensor performance can
be grouped into three major classes:
• Movement (the extent to which physical forces cause motion of the
electrodes relative to the surrounding brain tissue)
• Biocompatibility (the ability of the tissue to accept the device, i.e.,
the extent of tissue damage and/or tissue responses that encapsulate
the electrodes and impair their recording capacity)
• Biostability (the ability of the implant to resist damage by the body’s
environment and to resist material failure during long-term use).
Decoding from Local Field Potentials
• While spiking activity originates from a single neuron located close to
the electrode tip, the local field potential (LFP) is the summation of
excitatory and inhibitory dendritic potentials from a large number of
neurons in the neighborhood of the recording site (see chapters 3 and
16 in this volume). Because of the larger volume from which it gathers
electrical information, LFP activity is likely to be less affected by tissue
reactions (see chapter 5) following electrode implantation. It is
therefore logical to ask whether LFPs could be used to control a
neural prosthetic device, and what kind of information might be
carried by such signals. By similar reasoning, signals recorded from
the cortical surface (i.e., electrocorticographic [ECoG] activity
[Chapter 15]) might also be used to decode movement intentions.
Dimension Reduction: Principal Component
Analysis
• Such dimensionality reduction can be a very useful step for visualising and
processing high-dimensional datasets, while still retaining as much of the
variance in the dataset as possible. For example, selecting L = 2 and
keeping only the first two principal components finds the two-dimensional
plane through the high-dimensional dataset in which the data is most
spread out, so if the data contains cluster these too may be most spread
out, and therefore most visible to be plotted out in a two-dimensional
diagram; whereas if two directions through the data (or two of the original
variables) are chosen at random, the clusters may be much less spread
apart from each other, and may in fact be much more likely to substantially
overlay each other, making them indistinguishable
Dimensionality Reduction : PCA
• Similarly, in regression analysis, the larger the number of
explanatory variables allowed, the greater is the chance of
overfitting the model, producing conclusions that fail to generalise
to other datasets. One approach, especially when there are strong
correlations between different possible explanatory variables, is to
reduce them to a few principal components and then run the
regression against them, a method called principal component
regression.
Dimensionality Reduction : PCA
• Dimensionality reduction may also be appropriate when the variables
in a dataset are noisy. If each column of the dataset contains
independent identically distributed Gaussian noise, then the columns
of T will also contain similarly identically distributed Gaussian noise
(such a distribution is invariant under the effects of the matrix W,
which can be thought of as a high-dimensional rotation of the coordinate axes)
• Reduce-Signal to Noise ratio.
HCI : Miscellaneous
• Water fall model
• Deep Neural Network : Perceptron, Fully connected feed forward
NN,RNN, Convolutional NN,LSTM, Transformer etc.
• i-BCI
• Decoding from Local Field Potential : Slide No.95
GOALS IN DEVELOPMENT OF iBCI SYSTEMS
• Because iBCIs use sensors implanted in the brain and record activity
from relatively small populations of neurons, some of the goals of iBCI
research and development are unique to iBCIs. Th e goals of iBCI
development include: to demonstrate the ability of signals recorded from limited neuronal
populations to provide complex control (e.g., of cursor movements or
limb movements) .
 to establish the range of functionally useful actions that iBCIs can
restore to people who are paralyzed
 to identify the advantages and benefits of iBCIs over eBCIs, which do
not require that sensors be implanted in the brain
GOALS IN DEVELOPMENT OF iBCI SYSTEMS
 to develop iBCIs that are safe for long-term use (i.e., years and
decades)
 to develop iBCIs that can function reliably and stably for years
These goals are being addressed through studies in both animals
(primarily monkeys and rats) and, in early-stage human clinical trials in
people with severe motor disabilities. Animal models are important for
development and testing of new sensor designs (including device-tissue
interactions) and decoding algorithms, and they are also contributing
to new understanding of the neuronal control of movement.
Fourier Transform and Wavelet Transform
• Fourier Transform (FT) is one of the methods for signal and image
compression. FT decomposes a signal defined on infinite time interval into
a A-frequency component where A can be either real or complex number.
• Wavelet transform is the latest method of compression where its ability to
describe any type of signals both in time and frequency domain. That
makes WT becomes the most needed tool in signal and image processing
area.
• While the Fourier transform creates a representation of the signal in the
frequency domain, the wavelet transform creates a representation of the
signal in both the time and frequency domain, thereby allowing efficient
access of localized information about the signal.
Fourier Transform in detail
• Types of Fourier transform
• Applications of Fourier transform
Band Pass Filter, Linear and Non linear
feature, Gaussian Mixture,LDA,HMM etc.
• Band Pass Filter : Band pass filtering can be used to enhance edges
(suppressing low frequencies) while reducing the noise at the same
time (attenuating high frequencies)is given as a fraction of the
highest frequency represented in the Fourier domain image.
• Linear Regression is a linear model…Logistic regression is a non linear
model.
Linear Discriminant Analysis
• Linear Discriminant Analysis or Normal Discriminant
Analysis or Discriminant Function Analysis is a dimensionality
reduction technique that is commonly used for supervised
classification problems. It is used for modelling differences in groups
i.e. separating two or more classes. It is used to project the features
in higher dimension space into a lower dimension space.
For example, we have two classes and we need to separate them
efficiently. Classes can have multiple features. Using only a single
feature to classify them may result in some overlapping as shown in
the below figure. So, we will keep on increasing the number of
features for proper classification.
Linear Discriminant Analysis
Hidden Markov Model
• A hidden Markov model (HMM) is a statistical model that can be
used to describe the evolution of observable events that depend on
internal factors, which are not directly observable.
•
Gaussian Model
• Gaussian mixture models (GMMs) are a type of machine learning
algorithm. They are used to classify data into different categories based on
the probability distribution. Gaussian mixture models can be used in many
different areas, including finance, marketing and so much more.
• Gaussian Naïve Bayes is a variant of Naïve Bayes that follows Gaussian
normal distribution and supports continuous data.
• Naïve Bayes are a group of supervised machine learning classification
algorithms based on the Bayes theorem. It is a simple classification
technique, but has high functionality. They find use when the
dimensionality of the inputs is high. Complex classification problems can
also be implemented by using Naive Bayes Classifier.
Gaussian Model
Gaussian Bayesian Network
Gaussian Bayesian Network
Gaussian Bayesian Classifier
Kalman Filter
• For statistics and Control theory , Kalman filtering also known as
linear quadratic estimation(LQE) is an algorithm that uses a series of
measurements observed over time including statistical noise and any
other inaccuracies and produce estimates of an unknown variable.
Image Compression
• Eliminate Redundancy
Activity 1 : Case Study
• One of the main design principles which Nielsen has proposed,
especially for web site design, is simplicity. He proposes that
designers go through all of their design elements and remove them
one by one. If a design works just as well without an element, then
remove it. Do you think this is a good design principle? If you have
your own website, try doing this and seeing what happens. At what
point does the interaction break down?
Activity 1 : Case Study 1
• Simplicity is certainly an important design principle. Many designers try to
cram too much into a screenful of space, making it unwieldy for people to
find what they are interested in. Removing design elements to see what
can be discarded without affecting the overall function of the website can
be a salutary lesson. Unnecessary icons, buttons,boxes,
lines,graphics,shading, and text can be stripped, leaving a cleaner,crisper
and easier to navigate website. However, graphics, shading, coloring, and
formatting can make a site aesthetically pleasing and enjoyable to use.
Plain vanilla sites with just lists of text and a few hyper links may not be as
appealing and may put certain visitors off returning. Good interaction
design involves getting the balance between aesthetic appeal and the right
amount and kind of information per page.
CUI and GUI
• Activity 2
• Give a comparative Analysis.
Activity 3:The Complexity Dimension :
Complex Vs. Simple Movements
• Many everyday movements are complex and can be viewed as either
sequential or simultaneous recruitment of simpler movements. For
example, typing on a keyboard or playing the piano requires a
sequence of finger movements.
• It is also evident in piano playing (Engel et al. 1997 ) and probably in
many other complex movements. These behavioral phenomena
suggest that there are specialized neural circuits that link and
coordinate simple movements into more complex sequences.
Complex vs Simple Movements Contd.
• It has been recognized that the supplementary motor area (SMA) of
the cortex is an important area in the representation of complex
movements involving multiple limb segments.
• The distinction between external and internal movement guidance
can be seen rather dramatically in Parkinson’s disease.
Supplementary Motor Cortex
Primary Motor Cortex
• The primary motor cortex(or just motor cortex) can be defined as the
region of cortex from which movements are elicited with the least
amount of electrical stimulation.
Supplementary Motor Area
• The SMA and premotor cortex are located in the frontal lobe and
implicated in the planning and initiation of voluntary motor
movements.
• Neurons in SMA respond to visual, auditory, and tactile stimuli when
these stimuli are used as signals to start a movement or series of
movements.
• The SMA is involved in self-initiated actions (e.g. throwing a ball,
rising from a chair) rather than movements that occur in response to
an external stimulus or trigger (e.g. catching a ball)
Pre motor area
• The premotor cortex is an area of the motor cortex lying within the
frontal lobe of the brain.
• Nerve signals generated in the premotor cortex cause much more
complex patterns of movement than the discrete patterns generated
in the primary motor cortex.
Neuron Recorded from the Supplementary
motor cortex.
The Source Dimension : External Vs Internal
Movement Initiation.
• A movement can be triggered and initiated by an external stimulus, or it
may be initiated internally by motivation. For example, one may reach for
and grasp a cup among several other visually presented objects after
instruction to do so (i.e., external initiation), or one may simply decide to
reach out for the cup to satisfy a desire for the hot coffee it contains (i.e.,
internal initiation).
• The distinction between external and internal initiation is particularly
evident in the generation of complex movements such as those involved in
playing a musical instrument: a pianist can play from sheet music that
provides visual instructions as to which keys to press, for how long, and in
which order (i.e., external initiation); or, with repeated practice, the pianist
can remember and generate (i.e, internal initiation) the proper sequence of
key presses without the support of external stimuli.
Multimodal HCI
• Multimodal displays providing feedback, supporting different learning
styles, cross modal interactions, Multimodal visualization, Better
realism, Behavioral and physiological evidence, Difference between
sensation, perception, and cognition: Relation of each to HCI design
Physiology of visual system, information transduction, and cortical
representation Perceptual distortions and visual illusions
Multimodal HCI
Multimodal Interaction
• Combination of multiple modalities or usage of more than one
independent channel signals for the interaction between a user and a
machine is termed as multimodal human computer interaction
system.
• A multimodal interface acts as a facilitator of human computer
interaction via two or more modes.
• Input : Voice, pen, facial expression, gesture etc.
• Output : Voice, graphical output etc.
Advantages of Multimodal Systems
• Provide transparent, flexible, and powerfully expressive means of HCI.
• Easier to learn and use.
• Robustness and stability.
Application areas of Multimodal systems
• Driver monitoring
• Virtual Reality
• Intelligent Games
• Pervasive and Ubiquitous computing
Multimodal System
Multimodal Systems
• Epstein [46] and Grifoni et al. [47] have shown that with the
increasing use of smartphones and other mobile devices, users are
becoming more comfortable in experimenting with different new
modalities. After the introduction of voice assistants such as Siri,
Alexa, Cortana and Google Home, some users began to use voice
assistants as an alternative way to communicate with computers and
other digital devices [48,49]. This epitomizes how certain modalities
with contrasting strengths are useful in various situations [50]. Some
other modalities such as computer vision can be utilized to capture
three-dimensional gesticulations using depth cameras, such as
Microsoft Kinect [44].
UML Diagram
Class Diagram
Activity Sequence Diagram
Cross-Modal Interaction Between Auditory and
Visual Input Impacts Memory Retrieval
• It enhances memory retrieval.
• Users experience with different types of auditory input (spoken words
vs. environmental sounds) modulates how humans remember
concurrently-presented visual objects.
Cross-modal interaction
Cross modal interaction
• Possible processing route whereby activation from auditory (e.g., a
meowing sound) or visual (e.g., a picture of a cat) stimuli could spread
to a corresponding conceptual representation (e.g., of a cat) and
associated visual features, thereby strengthening the salience of the
visual object.
Cross-Modal Interaction Contd.
• Many of us have had the experience of feeling transported back in
time upon exposure to a familiar sight or sound – a song on the radio
might conjure the image of your first car, or the sight of a tuba might
invoke the cacophony of your middle school band. Such phenomena
illustrate the essentially multisensory quality of what we experience,
and subsequently, what we remember. While it is simple enough to
intuit that memories made in one sensory modality could become
associated with those in another by virtue of their shared context or
source, here we ask whether the things we hear can directly alter
how we encode and remember the things that we see.
Case Study
• The present experiment was designed to examine the extent and
nature of audio-visual interactivity in visual memory, as well as the
role of experience-dependent associations between cross-modal
stimuli. Memory for objects’ identities (i.e., “what”) revealed that
listening to meaningful spoken words or environmental sounds both
enhanced memory for what visual objects were previously seen.
Sample Questions Discussion
Heuristic Evaluation :
•Developed Jacob Nielsen in the early 1990s.
•Based on heuristics distilled from an empirical analysis of 249
• usability problems.
•These heuristics have been revised for current technology.
•Heuristics being developed for mobile devices, wearables,
• virtual worlds, etc.
•Design guidelines form a basis for developing heuristics
Sample Question Discussion Contd.
• Steps/process of Heuristic Evaluation:
• Each evaluator performs two or more passes through system
inspecting :
•flow from screen to screen ---on each screen :
evaluate against heuristics
find “problems”
Sample Question Discussion Contd.
 each evaluator rates the severity of each problem :
Score
Severity
Fix Priority
4
catastrophic problem
imperative
3
major problem
high
2
minor problem
low
1
cosmetic problem only
0
not a problem
Sample Question Discussion Contd.
• Advantages and Disadvantages of Heuristic Evaluation :
• Fast
• Cheap : suitable for small companies.
• Usable early in development process
Drawbacks :
• Depends on expertise of reviewers
• Selecting right heuristic is difficult.
• Depends on heuristic
Shneiderman(1998) : Interface Design Rules
Shneiderman (1998) devised Eight Golden Rules for interface design :
• 1. Strive for consistency e.g. put File menu on left hand corner
• 2. Enable frequent users to use shortcuts E.g. command keys or function keys
• 3. Offer informative feedback e.g. ‘corrupt file’ not ‘Error 675’
• 4. Design dialogs to yield closure e.g. ‘Printing completed’
• 5. Offer error prevention and simple error handling e.g. try to make it impossible
to get it wrong, make it easy to put it right. For example, flag the text fields where
the users forgot to provide input in an online form.
• 6. Permit easy reversal of actions e.g. undo facility
• 7. Support internal locus of control e.g. make user feel they are in control
• 8. Reduce short-term memory load. E.g. use options rather than make users
remember.
Sample Question Discussion:
• Describe how ethnography and prototyping can be used together to
get information about what people want ?
• Refer to next slide for Discussion…………
Sample Questions Discussion
• Description of ethnography(Cultural) – Human factors, the study of
people carrying out tasks to reach goals.
• Description of Prototyping – a physical design that can be used to
demonstrate and communicate design ideas
• A combination of these approaches would benefit the requirements
process as ethnography allows you to gather detailed requirements
about the tasks users carry out. These requirements can then be
designed into a prototype, a prototype can then be used to validate
the requirements.
Useful Notes
• List of Impairments - Cognitive impairment, includes memory,
perception, problem-solving, conceptualisation and attention deficits.
This may result from a range of conditions such as mental
retardation, autism, brain injury, Parkinson's disease, Alzheimer's
disease and old age.
• Learning difficulties can also affect a variety of memory, perception,
problem-solving and conceptualisation skills.
Learning difficulties include reading problems such as dyslexia,
computational, reasoning and non-verbal learning disorders. These are
sometimes also associated with Attention Deficit Disorder and
Hyperactivity.
Unit IV -Multimodal Interfaces
• As discussed in the class
• Refer to uploaded research papers
UNIT V: HCI and the Web
• GUI and web interfaces
• Translation and Types : as discussed in class
References