A multi-modal Voice User Interface on the Google Assistant for Azheimer's caregivers
In response to the mounting pressures on Alzheimer's informal caregivers worldwide, I researched and designed a Multimodal Voice User Interface (VUI) prototype for the Google Home Assistant as a final UX project at the General Assembly. This innovative solution aimed to diminish pain points, combat isolation, and prepare caregivers for unexpected scenarios.
The prototype
My role and
competencies:
My role in this product design project, which was a part of my UX certificate program at General Assembly, consisted of executing an ultra-fast end-to-end design process. I delved into multimodal user interfaces (VUI and GUI) as a key aspect of the final product, for the novelty and possibilities of the technology. I also established a cross-functional collaboration with peers in the GA program, and pro-bono contributors from Alzheimer’s Research UK, and Visyon 360, with their technical expertise.
Chapter 1:
Problem space
Recent studies point that the number of dementia and Alzheimer’s cases will reach 155 million globally by 2050. Caring for those individuals often falls on family caregivers, leading to fatigue, financial strain, and emotional stress. Existing services are fragmented and lack interoperability, leaving caregivers underserved. Innovative, user-centric solutions are urgently needed to support caregivers in this challenging domain.
Report by The Economist Intelligence Unit on the impact of Alzheimer's on the UK economy
Chapter 2:
Methodolgy
Human-Centred Design and Design Thinking
I fully embraced Human-Centred Design and Design Thinking principles during the process in order to develop a deeply empathetic solution, prioritising users' specific needs and goals. This method required effective collaboration and co-creation, ensuring the use of technology in a way that supports well-being, empowerment, independence, and dignity in the context of caregiving.
Chapter 3:
Topic deep dive
1 starting with desk research
I conducted extensive desk research on the topic, reviewing reports from Alzheimer's Society UK, Alzheimer's Research UK, and The Economist Intelligence Unit. I also went through academic journals, healthcare publications, and caregiver forums on social platforms to gain comprehensive insights into the challenges and existing solutions in this field.
2 initial user interviews
I followed up with six phone interviews with family caregivers to gain insights into their experiences and challenges in a variety of contexts and the condition stages. I also interviewed two senior care home managers to understand the caregiving processes and operations from a professional perspective.
3 quantitative survey
I concluded the research step with a quantitative survey in the UK, Germany, and Spain to validate previous findings and gather insights into technology usage by caregivers. Respondents were also asked to provide summarised diaries of their daily activities. That added valuable information about their pain points and challenges.
Chapter 4:
Building a profile
The primary persona
The caregiving scenarios for dementia patients are diverse, with adult offspring often caring for their elderly parents, and spouses looking after their partners. While the group and gender distribution varied, a significant number of female caregivers were observed during the research. Therefore, for the primary persona, a middle-aged female caregiver taking care of her husband was selected to represent this prevalent group.
Family caregivers need a way to get continuous support throughout their caring journey for their loved ones because the demands are very real and can lead to self-neglect, isolation, and physical and mental exhaustion.
Readiness
As a caregiver I need relevant information about what the progression and impact of Alzheimer's on my loved one just so I can plan things in advance.
Support
As a caregiver I need all the support and resources I can get from family, friends, and even technology. It would be impossible to do all of this on my own.
Connection
As a caregiver I need social interaction especially with people in the same position, just so I can share, learn, feel less lonely and more supported.
The hypothesis
"I believe that by building a multi-modal assistive system centred on voice interaction that provides guidance for family caregivers of loved ones with Alzheimer's I will reduce friction during multi-tasking, give valuable information, and ease their sense of isolation and loneliness."
Chapter 5:
Solution space
Visualising use case scenarios
Using insights from interviews and user diaries, I crafted journey storyboards to pinpoint moments when caregivers encountered challenges in their daily routine. Envisioning a hands-free, conversational interface solution, I illustrated three primary use cases: accessing real-time information, making hands-free calls, and seeking support – all seamlessly integrated into caregivers' ongoing tasks.
Storyboards
Prototype core features
Instant access to expert information
Hands-free calls to family and friends
Task organiser and notification system
Hand-off between the VUI and GUI
Dialogue sample: VUI
The next step consisted of prototyping the voice agent, beginning with a well-defined dialogue flow for the ideal user interaction. I decided to develop it from scratch to have full control over its critical attributes: voice character, language, and fulfilment thresholds.
Graphical UI
In addition to voice interaction a graphical user interface (GUI) was available for situations where a conversational interaction might not be convenient. Since these interactions consist of numerous micro-moments, with most being voice-based, the multimodal system was necessary to handle different scenarios.
GUI wireframes
The wireframes shows a happy-path whereby first-time users get on-boarded, and go through the registration process to provide further personal details in order to get a tailored experience from the voice-agent surface.
Chapter 6:
Guerrilla testing
Uncovering UX issues, very fast
I used smartphone-based Lo-Fi paper prototypes to validate the core functions of the voice agent through guerrilla testing. Although the volunteers didn't fit the persona profile, their feedback was valuable in revealing initial design problems. I conducted six test rounds and recorded audio for later review. This approach was chosen for its efficient and immediate feedback gathering.
Chapter 7:
Development
Using a pre-trained model with supervised learning
To build NoraCare, the ML engineers used Google’s Dialogflow, an open-source platform with a pre-trained model. They then customised a scraper on top of it to fulfil specific tasks as per my designs, what we called a 'Custom Relevance Model' webhook. Supervised learning techniques, including overriding tools and manual content ingestion, were used to train the agent and improve accuracy. This approach allowed for quicker development, and avoided upfront investment for the proof-of-concept.
Chapter 10
Conclusions
Social Proof
Obtaining further social proof regarding cultural contexts and use cases for voice interfaces is crucial, especially in caregiving. Further user research and continued collaboration with dementia-oriented organisations will be key to provide caregivers with the support solutions they most need.
Resources
Building a complete product/service in this domain requires significant resources. Extensive collaboration with experts in machine learning, UX design, and domain experts is needed to deliver a fully validated, market-ready MVP.
In summary, gathering social proof, establishing a sustainable business model, and allocating necessary resources are critical for the success of a voice-user interface product in the caregiving domain.
Next steps
-
Conduct further user research with informal caregivers to uncover additional insights and unmet needs.
-
Partner with dementia-oriented organizations to access necessary databases for training and improving the voice agent.
-
Refine and train the voice agent based on insights and caregiver feedback.
-
Conduct iterative testing and continuously improve the voice agent to meet evolving caregiver needs.
-
Develop a targeted marketing strategy and execute a coordinated launch to create awareness among caregivers