U.S. Internal Revenue Service (IRS) – Strategic Chatbot & Conversational UI Research
Leading inclusive research with taxpayers with disabilities and limited English proficiency to modernize conversational AI, reduce accessibility barriers, and drive a 25% increase in customer satisfaction.
| Key Metadata | Details |
|---|---|
| My Role | Lead Inclusive Researcher & Accessibility Strategist |
| Primary Impact | 25% increase in IRS customer satisfaction (partially credited to this research and design system modernization); significant reduction in chatbot accessibility defects; authored formal federal research playbooks and guidance |
| Federal Context | U.S. Internal Revenue Service (IRS) Office of Online Services / MITRE. |
| Core Research Methods | Indi Young Deep Listening Sessions, Accessibility & Heuristic Audits, Service Blueprints, Journey Mapping, Intersectional Demographic Analysis, Power BI Data Dashboards |
| Core Skills | Conversational UI (CUI), Inclusive Research Methodology, Section 508 & WCAG Auditing, Equity & Intersectionality, Design System Expansion, Federal Stakeholder Briefings |
1. Context & The Challenge
As part of its digital modernization push, the IRS launched pilot chatbots on its highest-traffic self-service portals: the Pay and Refund pages. While the pilot aimed to deflect call center volume and assist taxpayers with routine tasks, IRS leadership recognized the need to ensure conversational AI tools met the highest standards of accessibility and equity before scaling further.
To achieve this, the IRS engaged MITRE to conduct foundational, inclusive research.
- The Problem: Early-stage conversational tools frequently fail users with disabilities or non-native English speakers due to inaccessible chat widgets, rigid intent models, and jargon-heavy conversational scripts.
- The Mandate: Evaluate the existing pilot chatbot, understand the lived experiences of diverse taxpayers, reduce accessibility defects, and establish reusable federal guidance for accessible AI and conversational UI.
2. Research Strategy & Inclusive Methodology
Rather than relying solely on traditional, highly scripted usability sessions, I designed and led a qualitative research strategy built on Deep Listening Sessions (pioneered by Indi Young)—a semi-structured remote interview method that opens space for participants to guide the conversation based on their lived experiences.
Intersectional demographic tracking and journey mapping framework used to synthesize qualitative listening sessions with diverse taxpayers
Prioritizing Underrepresented Taxpayers
- Targeted Recruiting: Recruited participants explicitly identifying as people with disabilities (motor, cognitive, visual, auditory) and individuals with limited English proficiency (LEP).
- Intersectional Data Analysis: Recorded and analyzed overlapping demographic factors—including Veterans, varied income levels, education, age, gender, and regional access—to identify compounding barriers in tax administration workflows.
- Team Mentorship: Trained research colleagues across MITRE on conducting deep listening sessions and capturing nuanced qualitative insights without introducing interviewer bias.
Technical Accessibility & Heuristic Audits
- Conducted rigorous manual and automated accessibility audits alongside expert heuristic evaluations of the live Pay and Refund pilot chatbots.
- Evaluated screen reader interaction patterns, focus management, color contrast, and conversational text readability against WCAG and Section 508 standards.
3. Evaluative Prototyping & Public Sector Playbooks
Evaluative Testing & Feedback Loops
- Guided research participants through live tasks on the pilot chatbot to observe real-time friction points, screen reader glitches, and comprehension gaps.
- Translated qualitative insights into static design mockups in Figma, conducting follow-up evaluative reviews with taxpayers to refine conversational layouts and UI patterns.
Endorsed Federal Guidance & Conference Leadership
This research directly informed several major published guidance documents and national conference presentations:
- Co-authored Inclusive Research Guidance: A Focus on Research with Participants with Disabilities & Differing Abilities, delivered directly to the IRS Office of Online Services (2022).
- Co-authored the Chatbot Accessibility Playbook (2021), officially endorsed and published by Section508.gov.
- Co-authored and presented A Playbook for Accessible Chatbots and AI Agents presented at numerous conferences including Grace Hopper Celebration (2025), CSUN.
- Co-authored and presented Compounding Taxes: Impacts of Disability & Intersectionality, presented at Knowbility’s AccessU (2024)
More details in Writing/Speaking
4. Stakeholder Alignment & Institutional Outcomes
To communicate complex qualitative findings and accessibility data effectively to IRS executive leadership and product teams, we delivered comprehensive briefing packages utilizing multi-modal artifacts:
- Interactive Data & Storytelling: Briefed IRS leadership using executive PowerPoint decks integrated with Power BI dashboards, journey maps, and service blueprints illustrating where taxpayers experienced conversational dead-ends.
- Design System Expansion: Delivered production-coded HTML/CSS/JS components to directly expand the IRS Design System with accessible, pre-tested conversational UI patterns.
5. Key Results & Impact
- 25% Increase in Customer Satisfaction: IRS online customer satisfaction scores increased by 25%—a major success partially attributed to the inclusive research and accessibility enhancements delivered through this chatbot and design system work.
- Measurable Defect Reduction: Significantly reduced the volume of WCAG / Section 508 accessibility bugs across the IRS chatbot ecosystem.
- Federal Standard Setting: Established institutionalized inclusive research methods and chatbot accessibility standards now adopted across federal agencies via Section508.gov.
