Diabetes Management Platform

Project Background

As a consultant at Bold Insight, I led a formative human factors (HF) study for our client, a medical device manufacturer, who was updating their diabetes management platform. These updates included user-facing features that impacted usability for critical tasks and therefore required HF validation.

This product was not new for Bold Insight: my company had conducted similar studies for this product before, and my manager advocated to onboard me as the project lead for the upcoming formative study so I could improve my project management skills in a more controlled environment. I led the project and moderated the sessions, and my manager served as my senior oversight and notetaker.

Device Overview

The subject of the research is a diabetes management platform for use by healthcare providers (HCPs) who care for people with diabetes. This can include registered nurses, nurse practitioners, physicians' assistants, and physicians.

The platform collects and displays data from blood glucose monitors (BGMs) and continuous glucose monitors (CGMs) and allows for HCPs and their workplaces to manage connectivity with their patients' devices.

Objectives

The following objectives guided the study:

1. Assess the safety and effectiveness of the platform for patients to receive the best care possible.

2. Understand what HCPs need when using diabetes management platforms and better align the user interface to their expectations.

3. Identify opportunities for improvement prior to validation.

4. Get lessons learned for the next formative study, which would be pre-validation testing by our UK-based partners, Bold Insight UK (see Working with Bold Insight UK)

Challenges

• Accommodating a newer client. Our EU-based point of contact was themselves onboarding to the product line and unfamiliar with the changes being made to the project. As a result, all of us learned about the product and its upcoming changes collaboratively, which brings me to...

• Designing a study without access to what we are testing. The only thing we knew for sure, including our client, was how many features required HF testing, a brief description of the feature, and 1-2 screenshots per feature. Our client worked hard to get us remote access to the features we would be testing, but internal delays led this to happening just one week before our first session. This required designing our study and creating our research materials to provide clarity and specificity to our client while leaving enough room to add details when we could obtain access to the product.

• Choreographing testing with multiple prototypes. Some features of the product were already public-facing; others were live in an internal R&D platform; still others were active only as Figma prototypes. Conducting sessions required some behind-the-scenes, Wizard-of-Oz planning to ensure that sessions participants wouldn't be mired in prototype details so they could focus on tasks.

  
  

Research Methods

Our research for this project involved usability testing with HCPs who regularly use diabetes management platforms to review patient data.

Ten tasks were assessed in total, including the following representative tasks:

1. Upload patient data directly from a patient's monitoring device (a dummy device was used for the study).

2. Determine and troubleshoot connection status for patients’ monitoring devices.

3. Interact with electronic medical records (EMR) directly in the platform.

4. Locate and interpret various displays of patient data and interpret new warning messages.

Recruitment

Recruitment was conducted by Fieldwork in Seattle, WA starting 3 weeks prior to the first session. I reviewed recruitment grids every 2-3 days for the first 2 weeks and daily for the week leading up to our first session.

Outcomes

• Client received final design recommendations for product of interest before proceeding to validation.

• Subjective feedback was collected to identify future opportunities for product development and improvement.

Deliverables

• Written report detailing study methodology, findings, and recommendations created in Microsoft Word.

• Root cause tables describing observed use events, participants' own descriptions of what happened, the real-life risk of harm if the use event were to have occurred in real life, and a root cause analysis statement.

• Study data for all participants and tasks in a Microsoft Excel datasheet and as de-identified session videos in MP4 format.

Working with Bold Insight UK

The next phase of research was to be conducted in Europe by Bold Insight UK. The research was intended as a pre-validation study to evaluate changes made based on the findings from the US-based formative study and confirm the product was ready to continue to validation testing.

To ensure a smooth transition between the US and UK teams, I prepared a presentation for our UK team to orient them to the product and the client and to review the design and deliverables from our study.

The UK team requested a review of root cause analysis so we could align on root cause reporting conventions, so I also prepared a training on conducting root cause probing and analysis. As part of the training, I reviewed the fundamentals of root cause analysis, deconstructed a root cause table that we delivered to a client, and collaborated to practice writing root cause tables from actual session footage.

What I Learned

• You don't know everything, and neither does your team, but you can learn more together. The experiences of both me and our client were very similar: both of us were taking on new roles and responsibilities under the oversight of more senior human factors engineers. Admitting what we didn't know helped us to get where we needed to be because we were able to fill in each other's knowledge gaps. There is no shame in not knowing something, and it's important to establish the psychological safety with your team to admit when you don't.

• Details are everything. When I took on this project, one of the first things I did was build a project planning dashboard in Monday.com, which Bold Insight used for their internal dashboards. I knew that some of my colleagues were using it as a collaboration tool, so I did some further investigating and came out with the following:

  • A project landing page. Some project leads created dashboards that included basic project information, such as important dates, links, and documents, to provide a common location for study materials and quickly onboard newer team members. I loved this idea and incorporated it into this project.

  • A task list. I looked at the milestones and deliverables dictated by the project plan and formatted them into a task list that included the deadline, task owner, priority level, and status for each task. I also automated the list so that tasks due within the next week showed at the top of the page, and tasks marked as "Ready for Review" would automatically send an email to my senior oversight notifying her that a document needed her eyes.