Agentic AI in Healthcare
With any health insurance, you expect a clear understanding of your medical and drug coverage, out of pocket expenses and your in-network provider availability.
Unfortunately, that is not the experience for MVP members.
- Members have to either call in, wait on hold and rely on verbal confirmations and quotes.
16% of all member calls related to benefit questions.
- Members search through large PDF documents, decode complex industry and healthcare acronyms, and then assume the risk of misunderstanding their coverage.
To address this, I worked with developers and a PM to build the first in-house, AI Chatbot that would allow members to self-service their benefit coverage.
Role
Team
- 1 Product Manager
- AI Council
- 2 Tech Teams
- 1 Designer
TimeLine
skills
Impact
- Researched 100+ member pain points, established UX x AI principles, helped build evaluation guide, and designed final prototype
- Launched a pilot AI to the 1500+ MVP employees for testing and feedback
Discover
Objective: Understand what are the main pain points and challenges for members in interpreting benefits, finding care and reading the formulary. I worked directly with 5 call center staff to document pain points, pulled analytics on frequent customer contact reasons, and defined UX problem scope for AI solution development
Result: Co-synthesized problem space, opportunities and risk considerations into a presentation for leadership. Idea was pushed to AI council, receiving green-light
.png)
Challenges with understanding benefits
2025 Data showed that the number one customer service call regarded customers not being able to find or understand their insurance benefits. Leadership wanted to address this pain point and understand what technical solution could be implemented. The current experience was a single, online spreadsheet.
Research
I conducted five 1:1 interviews with Customer Care reps to uncover key member pain points. I focused on recurring questions, sources of confusion, and how reps find and validate answers. From this research, I synthesized the insights into major opportunity areas: benefit coverage, prescription drug information (formulary), and provider network search.
Define
Objective: Pull data and APIs, tag and normalize data. For UX, set boundaries for the LLM and assist in defining testing cases. I developed a set of 50+ scenarios and questions, defined conversational principles and helped validate a grounding glossary of 280+ healthcare terms
Result: Engineers were able to successfully parse PDFs, spreadsheets and pull API data. Integrate UX considerations like tone, escalation path, actionability into model parameters
Defining the AI Capabilities
Based on past member data, I used Copilot AI to develop a set of scenarios that covered various member edge. This would help both define the AI capabilities in a clear way, but also provide some testing and training content. I worked with the product team to understand the scope of the AI tool: use cases we would be able to target, risks and available data to support those scenarios.
.png)
Design
Objective: Begin testing prototype in Claude Opus, ranking responses and validate outputs are in line with problem statement
Contribution: Worked to build a UX success measurement plan. Collaborated with customer care to pull a sample of ~500 questions from member chats for evaluation.
Co-prepared scoring guide and walkthrough with 5 agents
Result: Call center agents provided AI response feedback, allowing engineers to fine tune the model
.png)
From this research, leveraging a Claude, engineers prototyped a chat bot by uploading the document. While tedious, manually scoring the AI responses, in addition to a separate “AI Judge”, helped ensure the AI was parsing information correctly and meeting the criteria we had set out
Synthesize
Objective: Create an entry point for the AI assistant within MVP’s existing member portal experience
Contribution: Collaborated with other designer to define visual annotations eg. “presentation rules” for displaying coverage answers, citations, and escalation options. Partnered cross-functionally to create a backlog of features
Result: Team provided engineers with a ready-to-implement design system with UI rules to add to Google Vertex Agent (text, list, CTA, alert, table, link out, etc.) to match content
.png)
Building an Adaptive Chat Design
As the LLM produces a response, having tagged components for certain type of content helps the agent present information in certain formats. Much like CSS classes, I worked to create a set of “presentation rules” for various types of responses
Deliver
Objective: Helped build and present deck for VP level leadership audience
Contribution: Co-presented problem statement, research, UX guidelines and sample responses.
Result: Launched on 1/26 a prototype AI to the 1500+ MVP employees for testing and feedback. This was launched in as a standalone URL for testing.
.png)
Presenting the Experience
I worked with PMs to create a storytelling deck that showcased how the AI solution would interface throughout the member journey and the technical backend that supported the functionality. On the left, we can see the AI both identifying and matching the intent of the member to the right benefit, and explaining their coverage and then suggesting in-network providers for a given specialty by zip code and plan type.
Right: Escalation is triggered, whereby the AI’s confidence level is too low or risk flags are triggered (urgency, pain, etc), and pushes the member to emergency resources while assuring of coverage.
.png)
Data Orchestration
Left: Scenario is related to answering ambiguous questions and understanding user intent, through clarification and validation. As an example, the member is asking about a medication based on the action of “Check formulary” but mis-spells the drug name. Since the semantic match is not strong enough, the AI clarifies the intent.
Right: The AI pulling data from two different documents, — the member's benefits summary and their claims/cost history — to answer a single question about therapist visit costs. When the member asks "how much do I pay?", the AI recognizes this requires cross-referencing their plan's cost-sharing structure (deductible, coinsurance tiers) with their real-time deductible usage to surface a personalized, accurate answer rather than a generic range.
.svg)