DOI: 10.17587/prin.9.375-384

The Program Complex for Training Russian Sign Language Based on Unity3D

M. G. Grif, grifmg@mail.ru, A. V. Lukoyanychev, dizzystyle@yandex.com, Novosibirsk State technical University, Novosibirsk, 630073, Russian Federation

Corresponding author: Grif Michael G., Doctor of Department "Automated Control System", Novosibirsk State Technical University, Novosibirsk, 630073, Russian Federation E-mail: dizzystyle@yandex.com

The article presents the results of the development of a software package for teaching sign language. Computer systems for training sign languages are built on the basis of notation records. The perspective grounds of Dimskis notation use for the computer implementation of Russian sign language are presented. The peculiarities of the notation record use for the control of a computer animated character that demonstrates dactilemas and words of sign language are considered. The graphics package Unity3D is used to implement the 3D model of the animated character (avatar). Dimskis notation is not completely computer-oriented, so the notation records are supplemented with required signs for the full descriptions of the word. For the convenience of working with Unity, the software complex has been developed that allows one to translate an animation clip created in Unity3D into a script text format and vice versa — the text format of a description in the animation movement. This approach allows us to increase the visibility of animation files and simplify the management of the avatar. The service symbols and words that allow you to organize the synchronization and relativity of avatar movements were introduced to the developed script language. The features of the realization of various animation ways for the demonstration of words and word combinations are considered. To realize this concept the software package with two modes: "Developer" and "User" has been developed. The article describes the hierarchical structure and functional possibilities of the software complex for the creating of sign language guide and lessons, and the prospects for its further use.

Keywords: Russian Sign Language, a multimedia dictionary, notation Dimskis, Unity3D, animated character, avatar

pp. 375–384

For citation:
Grif M. G., Lukoyanychev A. V. The Program Complex for Training Russian Sign Language Based on Unity3D, Programmnaya Ingeneria, 2018, vol. 9, no. 8, pp. 375—384.