This diagram represents our vision of building upon the platform of lifelogging towards creating
persistent personalised lifebraries. The realisation of this vision requires truly cross-disciplinary
serendipitous collaboration including input from technologists, psychologists, sociologists,
philosophers, ethnographers, artists, and ethics experts. We now provide some sample use-case
scenarios highlighting the need for this cross-fertilisation of ideas.
Use Case 1 – Archiving & Preserving Actively Created Personal Digital Material
Luisa Gandarias at the age of 60 is still in the amazement of technological revolutions over
the last 30 years. The memories of early days where the primary form of communication was
used to postal service’s the technological revolutions have made it possible to share
information across the globe with only a few clicks of a mouse. As a technology hobbyist,
she has certainly kept up with what was offered by such revolutions and now she is
surrounded by most of the high tech gadgets ranging from smart mobiles to the use of
Bluetooth. In addition, her interest in recording her activities on day-to-day basis has resulted
in her using wearable devices.
Luisa Gandarias is an environmental consultant about to be retired from the company where
she has been working during the last 28 years. She has combined her professional activity
with the photography, which has been her main hobby during the last 30 years. As she is
contemplating of retiring from a professional environment, she wants to share her legacy with
her friends, family and most importantly her social peers who have cheered on her
achievements and encouraged on her adventurous trips. Her dedicated work ethics and
passion for photography has resulted in a lot of content commonly categorized as “User
Generated Content”.
Her need for assembling an automatic lifebrary, consisting of a collection of her lifeworks
highlighting the source (or point of origin) of the digital media comes from her wish to share
her experiences with multi-view perspective of her life, i.e. family sharing personal
experiences, memories, trips etc and social peers having similar interest in environment
sharing her adventurous journeys through different terrains.
Firstly in this scenario the guidance of librarians and information and library science experts is
essential in helping technologists become aware of the methods that humans use to construct a
catalogue of Luisa’s life. Secondly the guidance of ethnographers is required to help construct an
idea of the relevant social networks that may be important and significant to Luisa. Carrying out a
semantic analysis of Luisa’s online social network interactions may prove a successful path in
attempting to automate this scenario. Thirdly the role of artists in presenting Luisa’s personal
lifebrary in novel fashions which bring her digital preservation and legacy to life is a worthwhile
persuit.
Use Case 2 – Personal Lifebraries for those with dementia
Given the demographic shift towards older populations in the EU and that dementia become
increasingly prevelant as people get older, it is worthwhile to consider this scenario as a
potential benefit that may be offered in moving from lifelogging to lifebraries. Mary was
healthy throughout her whole life, but unfortunately in her early sixties she has developed a
memory impairment. This has been very distressing for Mary as she now very much struggles
to remember events that happened as recently as the previous day. She has been embarrassed
to talk to others for fear of forgetting seemingly trivial pieces of information, e.g. who she
met at church 3 days ago, etc. However of late she has begun to wear the Vicon Revue, a
visual lifelogging device. She now puts it on at the start of every day after letting it charge
overnight when sleeping. Every evening she uploads images to her PC, and our software
automatically processes this data. As Mary's Vicon Revue takes approximately 2,000 images
per day, it would overwhelm herself and her husband John (who is in effect her carer) to look
through all these images. Therefore we automatically summarise this data by exploiting
characteristics of how the human memory operates. An example of this would be segmenting
sequences of the images into distinct events/activities, e.g. when she had breakfast in the
morning, when she went for a walk in the afternoon, when she was talking to Ann next door,
etc. This helps herself and John more easily review what Mary has done in any given day.
Indeed they particularly enjoy reviewing their Grandson's 4th birthday that took place last
Friday. Naturally there were some routine/mundane events in the morning first of all, so
again in exploiting an understanding of the human memory system it is important for our
software to these mundane events appear smaller, while the main event of the birthday
gathering is magnified so as to catch John and Mary's attention. By playing through the
images of this event Mary can remember that she was talking to her friend Jack about how
quickly her Grandson is growing. For certain events John can prompt Mary on what she was
doing, with the SenseCam images being very beneficial. Indeed Mary wonders what other
times recently she was talking to Jack, so our system provides her with the ability to retrieve
other similar events to the one of talking to Jack. She is then provided with a list of other
potentially relevant events, which she quite enjoys looking at as they trigger some memories.
This use case represents a number of accomplished goals:
1. Gathering and storage of passively recorded lifelog data, and considering when it is
ethical to record such information
2. An understanding of how the human mind works (e.g. information stored as events,
associating similar events, distinctive events more strongly remembered, retrieval of
events leads to a stronger encoding, etc.) so as to summarise one’s lifelog information
by presenting the more memorable events
3. Development of information retrieval models (extraction of relevant image features,
good fusion models, BM25/etc. retrieval models) to summarise one’s lifelog and offer
multi-faceted options to retrieve such relevant data
4. Effective presentation of the most important and relevant content to Mary. This
requires the guidance of dementia clinicians which then guides computing researchers
develop effective “personal lifebraries”
Use Case 3 – Mining a community of lifelog data and creating the appropriate lifebrary:
Consider the following scenario as a motivation for the role of lifelogging. Jim decides to go
on a one day holiday to Dublin city centre and he wears his lifelogging device all day long.
That evening he wishes to review the lifelog images of his day. Firstly it will be important for
his viewing application to automatically summarise his content in the appropriate manner and
identifying those events that are more likely to be remembered as important (quite similar to
that described in the previous scenario, and also understanding how self-driven goal events
are strongly encoded in the human mind). While in the city centre he started talking to his
friend Mark (also wearing his miniature wearable lifelogging capture device) and this was a
very important event due to its uniqueness as Aiden had not talked to Mark for quite a while,
especially in Dublin city. That evening Aiden wonders when he was last talking to Mark, and
he uses his personal lifelog application to find previous occasions similar to this. He can see
that he had last talked to John 1 month ago, while it was almost 3 months ago since he talked
to Mark in O'Connell St. However, he also nostalgically notices that he was talking to his
friend Mary in O'Connell St. 2 weeks ago, as this event is visually quite similar (and thus
somewhat associated) to that of talking to Mark earlier today. While reviewing the event of
talking to his friend Mary, he notices that he was at a football match later that evening, which
is a vital memory hook. After browsing through some past memories Aiden then looks at his
favourite picture from his trip today; an image of Trinity College.
In this instance it is of interest to the community that a shared lifebrary is created of this
network of individuals (Jim, Mark, and Mary in this case). This network may be a close-knit
community, a family, or it could also be an entire region or nation. By firstly utilising the
guidance of ethics experts, we can create an environment where people are aware when it is
acceptable to capture lifelogged material, and also in how to share such material. Secondly in
collaboration with memory experts, technologists strive to store events of personal
significance, and through the guidance of ethnographers and sociologists, models can be
conceived to capture and preserve material that is of significance to the relevant social
groups.
In addition to the drivers of the previous use case, in this scenario we show the role of these
additional facets in addressing the challenge of moving from lifelogs to lifebraries:
1. A social networking analysis to find out what information other people find most
interesting about an individual e.g. work colleagues are interested in one’s work
achievements, training partners are only interested in sporting accomplishments,
family most interested in attendance of family occasions, etc. … Here the guidance of
ethnographers is essential
2. Storage and processing of one’s lifelog material so that we can then create a
“lifebrary” of an individual, which then should fit into a community lifebrary. As
different people can have different views of this lifebrary, we require the expertise of
librarians and library information scientists to decide how decisions are made on
different views. They guide our algorithms and selection methods
3. Final presentation of a personalised community lifebrary is very important. This is
where our work will be guided by artists to disseminate lifebrary information of
people and communities in novel and meaningful fashions.