A Project By "Last Branch"
About the Project
Sozia is a real-time communication assistance tool designed to reduce barriers between individuals with hearing and/or speaking impairments and people who do not know sign language. The system aims to generate low-latency live subtitles by combining multiple modalities: AI-based speech recognition supported by lip reading and Turkish Sign Language recognition.
By leveraging both visual and auditory cues, Sozia aims to improve transcription robustness in challenging conditions such as noisy environments or situations where audio is not fully available. The planned architecture uses MediaPipe for face, hand and pose landmark tracking in both lip-reading and sign language recognition scenarios, Whisper for speech-to-text conversion, and custom trained deep learning models for Turkish Sign Language recognition, with a modular design that can be extended to additional languages and sign languages in future iterations.
