SignVerse: bridging communication through a bi-directional sign language translation system

Abstract

This study introduces SignVerse, a novel bi-directional sign language translation (SLT) system, to enhance communication between the hearing-impaired community and the general public. SignVerse makes real-time, two-way conversations easy for Indian Sign Language (ISL) users—no special hardware needed. The system uses smart artificial intelligence (AI) tech: computer vision, deep learning, and natural language processing (NLP). When someone types or speaks, the text/speech-to-sign module runs the input through NLP-based syntactic reordering and shows the ISL translation using a lively 3D avatar. On the flip side, the sign-to-text/speech module leverages MediaPipe to spot hand landmarks in real time, and the convolutional neural network-long short-term memory (CNN-LSTM) model accurately recognizes each gesture. Everything works together to help ISL users connect smoothly with others 94.8% recognition accuracy, less than 1.8-second translation latency, and more than 90% gesture clarity in user studies are all demonstrated by experimental evaluations. The lightweight model, which is optimized through knowledge distillation, guarantees excellent performance even on common consumer devices. With significant potential for societal impact, SignVerse is a significant step toward real-time, AI-driven ISL translation. When everything is taken into account, it is a dependable, scalable, and reasonably priced choice for inclusive communication.