Introduction: Electronic Health Record (EHR) systems often have technical and human-factors-related issues that may hinder productivity, especially right upon deployment of a new system. Those issues and sub-optimal configurations that are not caught in testing but that present themselves in production may cause a loss in working time, decreased staff morale, and compromised safety. One of the previous survey studies found that physicians are highly critical of the healthcare information systems and they would be willing to contribute to development activities, but no viable methods for feedback and improvement suggestions have been found. There has been an exploration of a user experience monitoring method and a screen recorder for that purpose, but, following recent developments on the issue, we have been looking for more information. Instead of a survey, the focus of this project is on conducting user experience monitoring in real context-of-use.
Objectives: The objective of the project is to acquire more knowledge on the impact of the method and how healthcare professionals feel upon using it. In addition to this, the aim is to acquire actionable insights on the usability of the new Epic deployment in the Finnish Capital region. The goal is to reach an understanding of how easy the tool and the method are for users. The understanding of this topic is considered and investigated among the participants by asking them to evaluate the prospective benefits of the method and their willingness to participate in such user experience monitoring in the future. The participating healthcare and social service provider organizations receive information about the gathered insights on the target systems and their usage, which can be utilized for development and training purposes.
Methods: The study is instrumented via a UX monitoring platform and an associated screen capture recorder that collects information about computer usage in real context-of-use. The application captures everything which is shown on screen, along with mouse clicks and keystrokes. In addition to this, the user can leave a comment via either speech or writing to explain an emergent observation. There are dedicated feedback buttons for tagging any good or bad or neutral comments about user experiences. These tags are timestamped for more detailed subsequent analysis as the videos can be examined by concentrating only on what takes place just before a tag and immediately after. The analysis includes coding and classifying the events with different criteria, for instance, based on severity and cumulative existence of an identified problem.
Logic of Use
- "I need help on using this." or "I learned a better way."
Configuration of User Interface Layout
- "This could be better." or "This works well/looks good to me."
- The system is giving an error message or the user notices that it is evidently not working correctly.
Slow Response Time
- Technical infrastructure causes delays and slow down the tasks intended to be done with the system in practice.
Workflow Interrupted Permanently
- The user is not able to continue working (no visible response to the user) due to technical reasons.
Results: This study resulted in participants from different units of a hospital district which were designated as participating organizations for the project. Monitored systems included specialized workflows in different organizational units. The results contain practices for improvement and resolving technical issues, positive notes on new system features, and considerations for end-user training.
Conclusion: The method applied for user experience monitoring has the potential for shortening the time from system deployment to reaching an acceptable productivity level in clinical work. This would mean quicker utilization of advantages of the next-generation EHR systems, and, even more importantly, improved patient safety upon introduction of new systems in healthcare organizations.
Objective of lecture
Verktyg för implementering
Level of knowledge
Chef/Beslutsfattare, Tekniker/IT/Utvecklare, Forskare (även studerande), Omsorgspersonal, Vårdpersonal, Patientorganisationer/Brukarorganisationer
Nytta/effekt, Styrning/Förvaltning, Innovativ/forskning, Uppföljning/Nulägesbeskrivning, Test/validering, Patientsäkerhet, Informationssäkerhet, Användbarhet
Janne Pitkänen Lecturer
CEO & Founder
Janne Pitkänen is a User Experience Specialist, the CEO & Founder at Adusso Ltd. and a Doctoral Candidate at Aalto University School of Science. He has been implementing usability testing solutions for Apotti during the procurement phase of the EHR project. Janne is doing his studies on usability research and developing User Experience (UX) monitoring solution for helping to implement and deploy new EHR systems successfully. Janne was chosen among one of the Champions of Health at HIMSS 2019 in Orlando
Petri Mannonen Lecturer
Oy Apotti AB
Petri Mannonen is a Usability Specialist at the EHR project Apotti in the Finnish Capital Region and a PhD student at Helsinki University of Technology. His major is usability and user-centered design and minor new media content production. He is interested in relationships between technologies and people. Petri's goal is to develop design methods to support designing products and services that in addition of being usable (answer to user needs and fit to physical context of use) fit to the technological context of the users.