Introduction: Electronic Health Record (EHR) systems often have technical and human-factors-related issues that may hinder productivity, especially right upon deployment of a new system. Those issues and sub-optimal configurations that are not caught in testing but that present themselves in production may cause a loss in working time, decreased staff morale, and compromised safety. One of the previous survey studies found that physicians are highly critical of the healthcare information systems and they would be willing to contribute to development activities, but no viable methods for feedback and improvement suggestions have been found. There has been an exploration of a user experience monitoring method and a screen recorder for that purpose, but, following recent developments on the issue, we have been looking for more information. Instead of a survey, the focus of this project is on conducting user experience monitoring in real context-of-use.

Objectives: The objective of the project is to acquire more knowledge on the impact of the method and how healthcare professionals feel upon using it. In addition to this, the aim is to acquire actionable insights on the usability of the new Epic deployment in the Finnish Capital region. The goal is to reach an understanding of how easy the tool and the method are for users. The understanding of this topic is considered and investigated among the participants by asking them to evaluate the prospective benefits of the method and their willingness to participate in such user experience monitoring in the future. The participating healthcare and social service provider organizations receive information about the gathered insights on the target systems and their usage, which can be utilized for development and training purposes.

Methods: The study is instrumented via a UX monitoring platform and an associated screen capture recorder that collects information about computer usage in real context-of-use. The application captures everything which is shown on screen, along with mouse clicks and keystrokes. In addition to this, the user can leave a comment via either speech or writing to explain an emergent observation. There are dedicated feedback buttons for tagging any good or bad or neutral comments about user experiences. These tags are timestamped for more detailed subsequent analysis as the videos can be examined by concentrating only on what takes place just before a tag and immediately after. The analysis includes coding and classifying the events with different criteria, for instance, based on severity and cumulative existence of an identified problem.

Issue categorization:

Logic of Use

- "I need help on using this." or "I learned a better way."

Configuration of User Interface Layout

- "This could be better." or "This works well/looks good to me."

System Error

- The system is giving an error message or the user notices that it is evidently not working correctly.

Slow Response Time

- Technical infrastructure causes delays and slow down the tasks intended to be done with the system in practice.

Workflow Interrupted Permanently

- The user is not able to continue working (no visible response to the user) due to technical reasons.

Results: This study resulted in participants from different units of a hospital district which were designated as participating organizations for the project. Monitored systems included specialized workflows in different organizational units. The results contain practices for improvement and resolving technical issues, positive notes on new system features, and considerations for end-user training.

Conclusion: The method applied for user experience monitoring has the potential for shortening the time from system deployment to reaching an acceptable productivity level in clinical work. This would mean quicker utilization of advantages of the next-generation EHR systems, and, even more importantly, improved patient safety upon introduction of new systems in healthcare organizations.