SignAI ProcessSpeechHear
UVoice Translation

UVoice – Sign to Urdu Speech Translator

Intelligent, real-time sign language translation system that converts sign gestures into spoken Urdu using Inflated 3D ConvNet (I3D) model for precise gesture recognition.

ReactPythonI3D ModelOpenCVgTTSFlask

Project Overview

UVoice is an intelligent, real-time sign language translation system that converts sign gestures into spoken Urdu. Developed to empower the Urdu-speaking hearing-impaired community, UVoice leverages the Inflated 3D ConvNet (I3D) model for precise gesture recognition and uses a React-based front-end to deliver a smooth, interactive experience. UVoice bridges the communication gap between sign language users and non-signers by providing accessible voice output in Urdu.

How It Works

Video Recording
User records a short video or streams a live gesture via webcam.
API Transmission
Video is sent to the backend via a REST API.
Frame Extraction
OpenCV extracts relevant frames from the video stream.
I3D Processing
The frame sequence is passed to the I3D model.
Gesture Recognition
I3D recognizes the gesture by analyzing temporal patterns.
Phrase Mapping
Recognized gesture is mapped to a pre-defined Urdu phrase.
Text-to-Speech
Text-to-speech engine converts the phrase to spoken Urdu.
Audio Playback
The generated audio is returned to the frontend and played back.

Key Features

Real-time Recognition
Real-time sign language recognition with minimal latency.
Urdu Speech Output
Spoken Urdu output for improved communication.
React Interface
User-friendly React interface for smooth interaction.
3D Temporal Modeling
High accuracy using 3D temporal modeling for gesture recognition.
Expandable Library
Expandable gesture library for broader coverage.
Offline Compatibility
Offline-compatible with local AI and TTS models.

Impacts & Results

  • Bridged the communication gap between hearing-impaired individuals and Urdu speakers
  • Improved accessibility in public and private services for the deaf community
  • Empowered users to communicate independently without human interpreters
  • Promoted digital inclusivity using local language voice technology
  • Supported communication in offline or low-connectivity environments
  • Demonstrated the real-world impact of AI/ML in solving social challenges
  • Paved the way for future multilingual support in regional languages
Back to Home