Mimo: Designing Multimodal Experience to Empower Older Adults

Overview —

The population of older adults in America is increasing faster than ever, and over 90% of them are choosing to live alone. Getting older and living alone contribute to creating significant health and technical barriers. A lot of them are living alone at home without help, emotional or wellness support -- and that is a problem. We created Mimo to foster an environment of companionship that empowers older adults to do more with their voice, like understanding their health, being informed about their schedule, and harnessing key skills to enrich their life.

My Role —

We are a team of three (designer + PM + developer) and I designed the entire Mimo interface design for both screen and voice. I also generated the research plan, led and conducted all user interviews, including analyzing generative and evaluative research results and identifying key use cases and interaction flows.

AWARDS

UW iSchool Capstone Innovation/Design Award

ROLE

VUI Design + Research Lead

TIMELINE

Jan 2018- June 2018

Our Goal —

To create an environment of companionship and empower older adults to do more

Technology often takes innovation and features at the price of adoption, which hinders the ability to accomplish even simple tasks for senior users. We deeply believe that technology should be accessible to everyone. We hope to create an environment of companionship that empowers older adults to do more with technology, and be more informed of their life and surroundings.

The Solution —

Mimo: An ambient personal voice assistant that aimed towards older adults who live alone

We created Mimo, using a RaspberryPi, Amazon Alexa, and a mirror display that enhances voice first experience. By removing the traditional mobile or web interface, and pushing voice as our primary interface, we enable the interaction in a most natural way.

Users will not only be able to do everything Amazon Alexa can, but also do things important to their well-being, like understanding their blood pressure, being informed about their schedule, and explore more skills that will help them get things done.

Check the test demo ⬇️

Demo 1
Mimo Blood Pressure

Demo 2
Mimo Help

Impact—

The Capstone event is the iSchool's most popular and prestigious event of the year. All graduating students get the opportunity to present their projects to faculty, friends, family, colleagues, and sponsors. Out of 123 UW iSchool Capstone teams, we were awarded the Innovation/Design Award by judges from Amazon, West Monroe Partners, and UW iSchool.

View Mimo Website ↗️

View Mimo Source Code ↗️

The
Approach —

01
RESEARCH

02
PROTOTYPING & TESTING

03
DESIGN ITERATIONS

04
FINAL DESIGN

Secondary Research
Stakeholder Interviews
User Interviews

Secondary Research —

Technologies for older adults are not always suitable for them

We started with online research to have more knowledge of our target group and explore related technologies for older adults. We found that despite the fact that there are many applications aiming to help older adults, they are still in the form of phone apps, which is difficult for older adults to use because of their vision impairment and shaking hands.

Additionally, to avoid the mistake that design and evaluate for accessibility is often an afterthought, I did literature reviews to expand my knowledge of accessibility in digital designs. I was inspired by Rachel Franz's work on proposing senior-centered approaches to conduct usability evaluations with older adults. I met her in person and I was luckily invited to conduct a research with her about the perception and adoption of mobile accessibility settings by older adults.

Stakeholders Interviews —

Toward deeper understanding of our target group by talking to experts

One of the biggest challenges of our research was participant recruitment. As a team of students, we had limited access to older adults. To solve this problem, I reached out to Seattle Government and local Senior Community Centers and asked if they could spread the word for us. Luckily, we were able to connect with Age Friendly Seattle to learn more about the current situation of age-friendly communities and technology access and adoption in Seattle.

User Persona —

Identify needs of our target group

By consolidating both findings from secondary research and interviews, we generated user personas to help us identify potential user needs and their end goals.

User Interviews —

Design based on the fact, not assumptions

The primary goal of our user interview was to identify the real needs of older adults. We wanted to make sure that we design based on our actual observations. Thanks to Age Friendly Seattle, we received 40+ responses after posting our research recruitment message on its monthly newsletter. We selected 6 participants in a wide range of ages, tech literacy, and state of health. All participants are over 65, living either by themselves or with their significant other.

There were two hypotheses I made to be confirmed or disproved before conducting the interview :

Key Insights —

01. What Users Want != What They Think != The Reality

By talking with our participants about their daily life routine and asking what they thought their voice assistant is capable of doing to help them, we identified gaps among what they want, what they thought the system is able to do, and what the system is actual capable of doing.

  • Health is the most important thing for older adults
    "Personally I don't care much for privacy. I would like to ensure that I am well."
  • Being able to stay in touch with primary caregiver is extremely important
  • Being able to see calendar events is highly desirable
  • There is no way to learn what their voice assistant is capable of doing
    "All I know is that you can play music and ask date and time."
    "My son bought this (Echo) for me but he's too busy to teach me"

Key Insights —

02. Older adults are way more open to new technology than we expected

Surprisingly, when we conducted the interview at participant's home, we discovered that almost all of them have voice assistant device at home. Despite their proficiency in using the technology, our participants expressed strong interest in wanting to learn more. Our hypotheses were disproved.

Key Insights —

03. However, the current voice assistant is not friendly to them

During the second half of the interview session, we asked participants to talk to their voice assistant like how they normally do. We discovered many breakdowns during the interaction:

  • Remembering name of Alexa is a challenge
  • The speech rate is too fast for older adults
    "Echo, you speak really fast, can you speak slowly?" (0:46)
  • There is no confirmation step to help user avoid mistakes
  • There is no action step to help user recover from mistakes
    "I am not quiet sure what I am gonna do right now." (1:31)

Here is a short audio clip from our interview. In just 1 minute, feel how frustrated this experience was for our participant.

Key Insights —

04. Multimodal Interaction: voice needs to be assisted with a visual component for aging users

By observing how our participants interacted with their digital device and voice assistant, we realized that voice is a good input medium, but a screen is a good output medium. It is difficult for older adults to solely rely on one medium. Not replacing or repeating, but the visual component could add extra value to voice interaction.

The
Approach —

01
RESEARCH

02
PROTOTYPING & TESTING

03
DESIGN ITERATIONS

04
FINAL DESIGN

System Persona
User Scenarios
‍Key Features

System Persona —

Why Mimo?

When we first thought of our voice assistant, we wanted our users to think of it amiable as a friend. Three of us are coming from different countries and we hope to give our voice system a name that could be easily pronounced in all three languages: Chinese, English & Hindi. Mimo ("mee-mo") came to our mind as a cute word and also easy to remember.

We defined the personality of Mimo as Patient, Amiable, Warm and Helpful. And the voice of Mimo is designed to speak naturally in slow speed.

User Scenarios & Testing —

Turning user research into design focus

Based on our user interview, we have identified three key features to focus on: Health, Social Connection, and Calendar. I created three sample dialogs as user scenarios to describe how a user might interact with Mimo.

In order to create the most immersive testing environment for our potential users, we used Sayspring for our voice prototype and showed the low-fi prototype as screen interface at the same time when doing user testing. I also included developer in the design process by testing the dialog prototype with him to reduce development cost at early stage.

Sayspring screenshot: Preview feature allows user to talk directly to the laptop

Low-fi screen prototypes

User Feedback —

User needs and use cases are not aligned

User testing for VUI was tricky. Even though we encouraged users to speak as they normally would do, it was still difficult to discover various fail cases. Instead, we received feedback concerning the medication reminder use case. User expressed their desire of tracking daily daily schedule, but could not resonate with tracking medication intake.

Developer Feedback —

Mind the gap between design and implementation

I was glad that I made the decision on including developer at this iteration phase. Understanding what is feasible from a technical perspective helped me to think of alternative solutions. However, because our voice system was built with the Alexa Skills kit, we had to follow their rules. I had to prioritize features for greater usability.

Sample Dialog ➡️ Feedback Overview

Redefine Key Features —

Health, Skill Exploration, Calendar

As a result of user testing, we decided to get rid of the "Social Connection" and "Scheduling" feature and replaced them with "Skill Exploration" and "Calendar".

The
Approach —

01
RESEARCH

02
PROTOTYPING & TESTING

03
DESIGN ITERATIONS

04
FINAL DESIGN

Voice Interface Design
UI Design

Voice Interface Design Decisions —

Understand the anatomy of voice command


I created a voice script to analyze potential user voice commands from three different factors: intent, utterance, and slot. Specifically, I used this sheet to help me to consider every variation of utterance.

Voice script for Mimo skills

Welcome Message —

Differentiate between first time users and return users

DO
Provide clear description of what Mimo could do for first time users and simplify the welcome message for return users.

First Time:

Welcome to Mimo. I can help you with several things today. You can ask about your blood pressure, view your calendar, or explore more Alexa skills. What would you like to do?

Return:
Welcome back to Mimo.

DON'T
Avoid overwhelming user by repeating long paragraph every time they use Mimo and also don't assume that user knows what to say.

First Time:

Welcome to Mimo. What would you like to do?

Return:
Welcome back to Mimo. What would you like to do?

Fail Cases —

De-ambiguity is the key

DO
Prompt users with suggested voice command can help them to recover from conversation breakdown.

When Mimo fails to understand:

Sorry, I am not sure what your blood pressure is. You can tell me your blood pressure by saying 'my blood pressure is 120 over 80'.

When user provides no response:
Hi, I am listening. What is your blood pressure today?

DON'T
Avoid ending conversations without re-prompting users.

When Mimo fails to understand:

Sorry I don't understand.

When user provides no response:
(end session without any action)

Conversation Flow —

UI Design Decisions —

Usability vs. Beauty

I generated about 20 iterations on the "home page" interface design. The design goal was to make the screen look simple and avoid cognitive overload for older users.

The
Approach —

01
RESEARCH

02
PROTOTYPING & TESTING

03
DESIGN ITERATIONS

04
FINAL DESIGN

Final Design
Tech Stack Diagram

Tackle your daily events with confidence

Get control of your surroundings simply by using your voice. We understand that people comprehend information at different speeds. With Mimo, you can adjust Alexa's rate of speech to reflect your pace. We've also built in a standard double-confirmation speech to make your interactions more accurate.

Make sense of your
well-being

Mimo comes with a variety of skills that enhance what Alexa can do. You can be more informed by asking Alexa things like what your blood pressure means, and be reminded to stand up every so often.

Forgetting is now a thing
of the past

By using both your voice and sight, Mimo can actively inform you about the events in your calendar. The mirror keeps you more informed and acquainted about your daily activities. You can now see what’s coming up even before you ask.

Enrich your life by adding more skills

With Mimo, you can now learn to add more natural and intuitive skills that fit into several aspects your life. Simply ask Alexa for help, and see the top skills that people are using.

Technology Stack Diagram —