Lab Seminar
2014-11-21
@UVRLab
Sung Sil Kim
mania@kaist.ac.kr
Contents
• Progress and Plans
• Paper Review
HRHRP Goals
• 스마트폰을 사용하는 사용자의 일상 생활 중 웃는 얼굴로 보안을
위한 로그인 패턴을 입력하고, 의도된 불편함을 UI에 포함하여 자주
웃도록 유도하는 스마트폰 표정입력의 패턴 분석을 통한 감정 마
이닝 기술을 연구함.
• 스마트폰의 잠금해제시 얼굴/표정 인식을 통해 보안성을 확보하고,
주기적으로 웃는 표정을 짓도록 유도하여 사용자의 표정 패턴을
수집 및 인식함.
• 동시에 감정 관련 정보를 보조•상호보완적으로 활용하여 개인의
핵심 감정상태를 추출 및 추적하여 사용자의 감정문제를 사전예방
하고 사용자 스스로 즐겁고 행복한 삶을 살 수 있도록 감정의 변화
를 유도함.
Progress
• CHI’15
• “Face Recognition via Person-Specific Facial Expression Acquired By Intended Inconvenient Interface”
• 정보과학회 동계학술발표회
• “자기 정량화와 자기성찰적 피드백을 위한 표정활용 얼굴인식 기법 및 시
스템 디자인”
• 특허
신청
• “모바일 표정인식 장치로 축적된 웃음 정보를 활용한 추천 시스템”
CHI’15 Review 1
• Primary reviewer’s comment:
• “A very interesting idea with good potential”
• HOWEVER, has too severe flaws: “both user studies do not seem to be strong enough” and “(algorithm)
does not present a strong contribution”
• “I recommend the authors to re-do the user studies with a stronger design” and “make the paper much
stronger, and re-submit. I do not believe a rebuttal can change the reviewers opinions in this case.”
CHI’15 Review 2
• External reviewer’s comment:
• “a novel interesting idea to face recognition, cause most of previous works target on learning expression
invariant face representations for face recognition”
• “However, the technical approach, is not new and a generic one not specific to smiling faces.”
• “The variation of facial expressions is a relatively easy factor in face recognition. Aging, face pose, and the
uneven lighting variations (e.g., shadow), are considered more challenging issues.”
• “Overall, the paper presents an interesting idea to face recognition, however, the technical approach is not
new and a generic one”
CHI’15 Review 3
• External reviewer’s comment:
• “none of the three parts are conclusive”
• “the idea is really innovative and it would be super interesting to know what might happen if you need to
smile towards your computer (phone, etc.) everyday.
• “However as the paper is written it contains many flaws and weaknesses”
• “the experiment is to small to draw any major conclusions”
• “break this work into different parts and focus on the key contributions for each parts”
CHI’15 Review 4
• External reviewer’s comment:
• “novelty of the work is very low”
• “work was tested on 6 subjects only”
• “One of the fundamental questions that should be answered is concerned with the persistence and
individuality of facial identity patterns in either static domain or dynamic domain. The paper failed to
address this issue and the technique used for FR is not well justified as to why it is the best choice for
person identification with smiley face images. The usability study of the system needs be expanded”
정보과학회 동계학술발표회
• 제목
• “자기 정량화와 자기성찰적 피드백을 위한 표정활용 얼굴인식 기법 및 시
스템 디자인”
• 수정사항
• “얼굴 인식 등의 실험 결과에 대한 분석의 보완이 필요합니다.”
Ongoing progress
• Undergraduate Research Program (URP)
•
•
•
•
http://kchannel.kaist.ac.kr/CH751-000
창의과제, 연구과제
2-3 people per team
2014년 11월 26일 (수)까지 하드카피 및 온라인 지원
• Mobile HCI’15
• http://mobilehci.acm.org/2015/
• Aug 24-27th, 2015
• Submission for paper: Feb 6th, 2015
Mobile HCI
• 가설:
• Mobile device에서 얼굴/표정인식을 통해 1) personal data를 축적하고 2) self-reflection
할 수 있는 피드백을 주면 사용자에게 매우 유익할 것이다
• 실험/입증방법:
• 언급된 functionality들을 가지고 있는 잠금 해제 application 개발 후 teste들에게 배
포 및 실험 진행 후 qualitative survey 진행.
• 예상결과:
• 사용자의 웃음패턴/습관에 (긍정적인) 영향을 미치는 것
• 비침습적으로 감정 변동 그래프와 personal data를 획득하여 quantified self/디지털
자아 형성에 도움
Mobile HCI’14
• Suggested Topics:
• Novel user interfaces and interaction techniques / Mobile social networks / Context-aware systems/ Multimodal
interaction (including audio and speech) / User-centred design tools and methods for mobile systems / Ethnographical and field
studies with mobile technology / Group interaction and mobility communities / Services for mobile devices / The design of
location-based services for mobile devices / The design; evaluation and case studies-of-use of application development
environments / Wearable computing, smart clothes, new devices and sensors / Mobile entertainment, storytelling and location
based gaming / Aesthetic interaction and experience design / Affective Computing and modelling of the environment /
Personal assistance with mobile devices / Mobile art / Mobility and work environments / Evaluation and
usability of mobile devices and services / Mobile accessibility / Model-based design of interactive mobile systems / Visualization
techniques for the mobile context (including 3D graphics on mobile devices) / Safety issues e.g., in-car user interfaces, payments
/ Trust, privacy, content protection, legal aspects & issues in mobile application & services
Mobile HCI’14
• Proceedings
•
•
•
•
•
•
•
•
•
•
Social Network & Input and Interaction
Devices and Interaction Design
Context Awareness
3D
E-Learning
Gesture Interaction
User-Centered Design
Input and Interaction
Gesture & Text-Entry
Recommender Systems and CSCW
Slide to X: Unlocking the Potential of Smartphone Unlocking
• CHI’14
• University of Toronto
• Session: Crowdsourcing
Abstract
• “In this paper, we explore how replacing the regular unlock screen with one that asks the user to
perform a simple, optional task, can benefit a wealth of application domains, including data
collection, personal-health metrics collection, and human intelligence tasks”
• “we show that people are willing to perform microtasks presented through this interface and
continue to do so throughout the day while they visit different locations as part of their daily
routines.”
• “We then discuss how to implement this concept and demonstrate three applications.”
Video
Contributions
1. The use of micro-tasks as a replacement for traditional phone unlocking.
2. A study of regular unlocking patterns and of users’ willingness to perform micro-tasks in that
context.
3. A set of three example applications which demonstrate the Slide to X concept.
Experiment
• 10 participants, age 19-23
• Android 2.3 or higher
• Functionalities
• Position data collection: records user’s geospatial location every 10 min
• Usage data collection: records time and location whenever the screen is unlocked
• Slide to X functionality: presents one of several unlock screens
• 1)traditional android unlock screen
• 2)simple mathematical question
• 3)introspective questions about health
Slide to X interface
Application 1
Application 2
Application 3
Figure 2
Result
• In total,
• 16,398 screens unlocks occured
• 8,118 questions posed
• 8,280 simple unlock screen displayed
• 8 of 10 participants - frequency of questions reasonable
• 7 of 10 participants – wanted more variant questions with less frequency
• SlideToCure application can raise approximately $6,864,000 per day
Things learned
• “Slide To QuantifySelf” can be improved
• Using (unobtrusive) face recognition and facial expression detection
• Stress management is equivalent to smile inducement
• User’s willingness (to use the service) is critical
Questions?
https://sites.google.com/site/uvrlab2014/
mania@kaist.ac.kr