AN AUTOMATED APPROACH FOR TRANSLATING ENGLISH TEXT TO PAKISTAN SIGN LANGUAGE

Abstract

In Pakistan, the population of hearing-disabled children is on the rise. Unfortunately, the literacy rate of this population is far less than their counterpart peers due to a lack of knowledge and scarcity of resources for Pakistan Sign Language (PSL). However, the availability of an electronic repository of PSL (www.psl.org.pk) has opened new horizons for the well-being of the hearing-impaired population of Pakistan. In this work, we aim to propose a standardized system that focuses on converting written English sentences into PSL to enhance the readability and understanding of an international language. Currently, the system is focused on providing an interface for preschool children to enhance their confidence in understanding the English language. Our system has two input options: typed English text or images of English text. The English Input Text is Translated into PSL lexical after performing NLP and machine translation techniques such as tokenization, lemmatization, etc., this process also deals with out of vocabulary words, and the resulting system is developed in three phases that are provided in the methodology section. For translating alphabets and words, our system achieved an accuracy of 89.58% and 100% respectively in a 10-fold cross-validation setup. For the evaluation of English sentence translation, we selected 5 storybooks with 104 sentences from the Oxford Reading Tree for ages 3-4 years and converted them to PSL. Later, with the help of a domain expert, we obtained an accuracy of 81.25% and a Level of Understanding of 75.96% which was recognized as comparable and rated as user-friendly by the domain experts. The availability of an automated application to convert the international English language to PSL will eradicate the need for a third person for translation while enhancing the social inclusion of deprived children.