eWALL for Active Long Living

Discovering the User needs and patterns and shaping eWALL

First, I set to explore the State of the Art, namely top-notch industrial and research projects providing a 'home assistant' for older adults. I noted the best practices and organised the projects according to their focus in the following domains: (1) physical training (especially COPD training); (2) cognitive training;  (3) in-home monitoring and feedback to the primary and secondary users; and (4) home automation.

Meanwhile, my colleagues at Roessingh Research and Development Institute (NL) delivered a set of ten personas of primary and secondary users (COPD, MCI, relatively healthy and caregivers). I studied this material, read forums and watched videos with patients and caregivers telling their stories and I performed additional interviews with doctors (as patients were hard to reach) in Vienna to validate the personas. From all this input, I identified a common mechanism among the primary target users. Whether they suffer from a physical or cognitive impairment, they tend to lose trust in their abilities and thus, gradually abandon their former activities. As a result, their confidence is affected and they often succumb in isolation. Moreover, rehabilitation therapies have a specific timeframe; even if they are effective, patients often lack the motivation to continue the training on their own.

Developing empathy for the end users, and better understanding the available technologies led me to brainstorm a number of possible design directions for eWALL and raised a number of questions that I presented to the consortium. For example, which user behaviours were going to be monitored for better diagnosis and prevention? There were many signs that could indicate exacerbation in users' condition, such as coughing for COPD patients and waking up in the middle of the night for MCI patients. After the analysis of the user research and the validation of personas and while discussing these issues, we collaboratively developed the user scenarios, together with researchers from the Roessignh, the developers and the management of eWALL. Finally, I created a series of interaction concepts, showcasing some design directions and collaborated in the development of use cases and scenarios. The sketches I created for the interaction concepts helped the consortium envision the system which was not presented visually in any previous document.

The User Requirements phase presented communication challenges. The project was highly compartmentalised as most of the partners were responsible for a technical subtask and few were invested in the bigger picture. I aimed to draw their attention to the user aspects early on, so I distilled all the ideas, the open issues and the knowledge we accumulated in concise tables and graphics, which found great use in official and informal meetings and deliverables and strengthened my commitment to a visual way of working.

Design, development and feedback cycles

The technical partners got to work and we had multiple teleconferences every week, where technical issues were discussed in parallel with the eWALL applications. Early on, we defined milestones for the first and the second year prototypes. From the feedback we gathered from early user testing and expert reviews, I proposed a prioritization of features and applications to be discussed and updated by the partners. Whenever required, I developed UI wireframes, provided expert reviews and run small user studies to gather feedback on the features and the UIs. Moreover, I chose to move to Enschede (NL) for a month to collaborate closely with my colleagues at Roessingh Research and Development Institute and there I came up with the concept of the Well Being Ads, which was highly regarded and also adopted in the commercial version of eWALL.

The Well Being Ads are a series of advertisements promoting the well being of the users and appear when the screen is in passive mode. The ads address important activities for the primary users: water drinking, healthy eating, showering, cognitive training, going outside, meeting with friends/family, opening the window to improve air quality, etc. Based on the data collected from the user, the Well Being Ads Reasoner decides which ads are displayed and when. For the Well Being Ads, we run separate small scale evaluations to fine-tune various parameters, such as duration, graphic style, wording and user acceptance. The video below is an prototype for the ads I made compiling illustrations (the illustrations below are not mine) and quotes.

During my stay in the Netherlands, I also collaborated with my colleagues to design the eWALL notification system. As eWALL gathers data in order to provide feedback to the primary user and to the caregivers and family, we needed a very structured approach to define the different kinds of feedback that is communicated to each user in the system, the wording, sound and communication style and the different alarms that occur when the user is (or seems to be) in danger.

Evaluation planning

The development of eWALL was an organic process that couldn't fit in the tight schedules initially planned. Thus we planned the user evaluations in an agile manner to synchronise with the developers' speed of delivering features and fixing bugs. 

This was not an easy feat, first due to the fact that four countries were involved in the evaluation. Each evaluation site had a different lab setting and different laws applied in the testing of medical equipment with frail adults. Second, at the time of the planned evaluations, there would often be technical glitches in the system that would render user evaluations obsolete. Thus, I proposed a draft protocol where we would perform expert reviews and technical evaluations in the system before it has any contact with the end users, so as to ensure technical maturity and avoid frustration.

Being responsible for the evaluation planning, I compiled a manual with goals and methods to be used according to the evaluation case at hand. The evaluation toolbox included the evaluation goals; the questions for the semi-structured interviews following the test; questionnaires with validated translations that allowed us to track our progress after each update and well-established HCI methods, such as think-aloud protocol, paper prototyping testing, true intent testing, etc. The combination of methods along with the scenarios that we wanted to test for each application were agreed upon with the colleagues at the spot. The 'evaluation toolbox' I created was shared in my institute as a good practice, because it solved an internal problem: we often had to write evaluation deliverables for the EU, before we knew the exact state of the prototype at the time of the evaluation, so compiling a collection of methods and strategies instead of describing detailed evaluation protocols was considered a quick fix for this kind of tasks.

A detailed description of the system, the evaluation protocol and the CE marks of the medical devices were submitted to the Ethical Committees of the countries and shared with the clinical professionals who helped in recruiting the patients. The recruitment of COPD and MCI patients and of formal caregivers presented many challenges, as the patients were often frail and uninterested and the medical personnel highly sought after.

In most evaluation sessions, we had a living room setup and asked participants to perform a set of defined tasks, fill in user experience questionnaires and answer to open questions in order to evaluate usability of the system. Overall, participants were keen on trying out eWALL. They were particularly motivated by the health and physical training applications and appreciated the well-being ads related to physical movement, yet they didn't feel comfortable with 'intimate' ads (e.g shower recommendations). Nevertheless, participants were sceptical of the 24/7 home monitoring required by eWALL. In addition, some of the healthy older participants resisted the idea of "a system designed for older people", as they did not feel old themselves.

Upon completion of the evaluation studies and of the majority of technical developments, a small-scale validation study was conducted to prepare the ground for the final, large-scale validation. The validation aimed to assess clinical factors such as overall physical and psychological performance of the patient from the use of eWALL before and after the  intervention. I left the project before the validation took place, but the outcome was very promising, as commercialization of eWALL is on the way under the name cloudcare2u.