Designing Mongolia’s first-ever accessibility-focused AI application: A real-time pocket translator for sign language that empowers connection and inclusion.

Project Overview

With over 466 million deaf and hard-of-hearing people worldwide, communication barriers continue to isolate millions from education, employment, and social connections.

HandPro was created to challenge that reality. As Mongolia’s first-ever accessibility-focused AI application, it combines real-time sign-language translation and inclusive interface design to bridge the gap between the deaf community, caregivers, and the hearing population. 

HandPro set out to make AI-powered accessibility mainstream by transforming mobile devices into “pocket interpreters” that foster understanding and independence.

CLIENT

HandPro

COMPLETION

November 2022

DURATION

10 weeks

MY ROLE

Design Lead

UX Designer

UI Designer 

CONTRIBUTORS

Junior Designer

Design Sprint Lead

HandPro (Development Team) 

THE VISION ✨

HandPro redefines accessibility for Mongolia’s digital era using AI. The goal wasn’t just to create another translation app, but to design a technological bridge that lets everyone, regardless of hearing ability, communicate confidently and belong.

The Challenge

Design an experience that balances cutting-edge AI with human empathy with a tool that feels approachable, intuitive, and trustworthy to users of all abilities.

Core Design Needs:

1. Translate text ↔ audio ↔ sign language seamlessly through AI.
2. Build a visually accessible UI optimized for clarity and readability.
3. Offer multi-use functionality (learning, interpreting, job-finding, emergency support).

THE GOAL

We centered our sprint around three guiding questions:

1. How might we craft an AI-driven yet human-centered experience through a personable 3D avatar that translates sign language in real time?
2. How might we help individuals with disabilities integrate into society through meaningful, everyday interactions?
3. How might we support caregivers and allies with tools that simplify learning and deepen connection?

STEP 1

Discover

Research

Since there were no prior frameworks for AI sign language apps in Mongolia, our team began with deep exploratory research by studying accessibility standards (WCAG 2.1), AI translation models, and global precedents such as SignAll and Ava.

KEY FINDINGS

1. We needed an inclusive design system built from the ground up that optimized for low literacy users.
2. All critical actions (like SOS calls and interpreter access) had to be reachable within one tap on the home screen.
3. Every button or message required sign language video support, as many deaf users struggle with written literacy. 

These insights became the foundation of our accessibility-first approach.

Design Sprint brainstorming and sketching with clients

Usability Testing

We conducted two rounds of usability testing with deaf and hard-of-hearing participants, facilitated by professional interpreters.

USABILITY TESTING FINDINGS

1.  Early versions included too many secondary features, distracting from the main translation purpose.
2. Users valued simplicity and speed over customization, often preferring clear, minimal layouts.
3. Adding contextual help cues (like icon explanations and preview animations) improved comprehension.

These findings guided our refinement toward a focused, intuitive, and emotionally intelligent experience.

STEP 2

Ideate

Low-fidelity prototypes

We focused on rapid iteration using UX principles and weekly feedback loops with HandPro’s founders and sign language interpreters. Our early prototypes emphasized the translator page, refining how users trigger, view, and replay AI-generated sign language.

We created multiple layout variations, testing for legibility, button size, and gesture comfort. The process ensured that every motion, word, and symbol supported universal access, not just functionality.

STEP 3

Design

UI & Accessibility

After validating the user flows, I led the final UI design by translating feedback into a polished, empathetic visual language.

Key Design Decisions:

1. Adopted a lighter color palette and rounded corners to create a friendlier, non-intimidating tone.
2. Integrated persistent accessibility buttons and SOS shortcuts on the home screen for instant action.
3. Designed the 3D interpreter “Anu” - a warm, relatable avatar embodying inclusion and personality.

 The design system prioritized contrast, touch targets, and clarity, ensuring users could navigate confidently regardless of ability or device type.

OUTCOME AND IMPACT

In November 2022, the HandPro team soft-launched the app at Mongolia’s first-ever Disability Expo, in partnership with Microsoft, UNICEF, and other key organizations.

At the event, we introduced “Anu” who is Mongolia’s first and only AI sign language interpreter. The demo received national recognition for its innovation in accessibility and inclusive design.

FUTURE VISION

HandPro is evolving beyond translation. Future versions aim to support job listings, emergency assistance, and learning modules for individuals with disabilities by shaping Mongolia’s first AI-powered accessibility ecosystem.

Disability Expo 2022 Moments

REFLECTION

HandPro showed me how AI and design can work together for social good, and it pushed me to think beyond usability, into dignity, representation, and equity, principles I now bring into every project I lead.

Thanks for stopping by,

Let's chat!

LinkedIn

Dribbble

Email

Resume