Sony Unveils Patent for In-Game Sign Language Tool

Sony has filed a groundbreaking patent aimed at enhancing accessibility for deaf gamers by introducing a real-time in-game sign language translator. The patent, titled "TRANSLATION OF SIGN LANGUAGE IN A VIRTUAL ENVIRONMENT," outlines a system that can translate American Sign Language (ASL) into Japanese Sign Language (JSL) within video games.

Sony Patents ASL to JSL Translator for Video Games

Proposed to Use VR Devices and Work Over Cloud Gaming

Sony's innovative patent introduces a system designed to assist deaf gamers through real-time translations of sign languages during in-game conversations. This technology would enable virtual indicators or avatars to communicate sign language seamlessly on-screen. The process involves translating the sign gestures of one language into text, converting the text to another specified language, and finally rendering the data into the sign gestures of the target language.

"Implementations of the present disclosure relate to methods and systems for capturing sign language of one user (e.g., Japanese), and translating the sign language to another user (e.g., English)," Sony stated in the patent. "Because sign languages vary depending on geographical origins, sign language is not universal. This provides a need for appropriately capturing the sign language of one user, understanding the native language, and generating new sign language as output for another user in their native sign language."

The implementation of this system could involve the use of VR-type devices or head-mounted displays (HMD). "In some implementations, the HMD connects via a wired or wireless connection to a user device, such as a personal computer, game console, or other computing device," Sony detailed. "In some implementations, the user device renders graphics for display through the HMD that provide immersive viewing of the virtual environment for the user."

Sony further suggests that the system could facilitate seamless communication between user devices over a network with a game server. "In some implementations, the game server executes a shared session of a video game, maintaining the canonical state of the video game and its virtual environment," Sony said, "and to which the user devices are synchronized regarding the state of the virtual environment."

This setup allows users to share and interact within the same virtual environment, or game, over a shared network or server. Additionally, Sony mentioned that in some implementations, the game server could be integrated into a cloud gaming system, which "renders and streams video" between each user device, enhancing the gaming experience and accessibility for deaf gamers.