Smart pictorial dictionary via mobile augmented reality

ABSTRACT


INTRODUCTION
Children think in many different ways, distinct from how adults learn. As young brains can digest information more quickly and efficiently than adults, they can think considerably more quickly than adults. Children are so encouraged to receive early literacy instruction. According to Furió et al. [1], a few learning strategies can be applied and viewed from various perspectives, especially those related to learning outcomes. Using a mobile application to learn is one of the methods.
Research by Noorhidawati et al. [2], three key factors influence children's learning engagement when they use mobile applications: verbal expression, collective sensory skill and emotional expression. Combining these factors has created a fresh chance to enhance the nature of teaching and learning experiences [3]. In addition, children should be exposed to the environment more, using cutting-edge educational technologies that could increase their enjoyment, such as games [4], [5] and the use of augmented reality (AR). The most recent technology has been applied to schooling. Using AR, users can experience situations or natural environments in more realistic ways. Due to lower costs and improved usability, AR in education has been increasingly popular since it was integrated into mobile devices [6]. In this modern era, advanced technology in education is very needed, especially AR technology. It allows interaction with the real-time environment. Thus, it is an interesting technology that permits manipulation and visualization with a new teaching method for learning [7]. In addition, AR based technologies can offer effective solutions to support variable, human-related production processes [8]. This mediated reality permeates our daily lives. Work, education, exercise, downtime, and travel are just a few examples of the times made possible by AR apps [9]. Pedagogy that supports visual content encourages children to develop integrated learning environments. One example that uses visual content for educationbased is the pictorial dictionary. A pictorial dictionary is a dictionary that is represented in visuals to guide people, especially children, based on the graphic. Research by Kamil [10], the word reference that occurs with sentence development and vocabulary learning is a part of the learning exercises that involve using the picture. Students have also been researching and discovering new information utilizing pictures as sources.
Teaching English or other foreign languages to children who speak a different native language can be difficult. An effective technique minimizes boredom and detachment from learning activities [11]. Learning languages through traditional teaching and learning methods proves to be very challenging, as children have different levels of learning skills. Research reveals that children with a low grasp of language tend to score fewer marks in examinations and have less self-confidence. In learning a language, the learner should be able to spell correctly and pronounce the object's name. However, spelling and pronunciation issues are common among young learners of English as a second language. For kids with speech problems, spelling was a particular problem area [12]. They could not pronounce or even spell the word correctly.
Research by Hosni [13] indicate that verbal language development has largely been neglected in the classroom. Most of the time, teachers use verbal communication in the classroom more than students. Although there are English lessons in every school, some children still cannot grasp the lessons themselves even though the teachers have provided them with many exercises. This issue might also be due to the inability of some children to connect an image to its referent [14]. Securing a language requires a person to sort out the world through an arrangement of images and referents. Based on these factors, AR is considered one of the best possible solutions to overcome problems in learning the English language, especially among those using English as a second language. Research by Afnan et al. [15] also proved the supportive impact of learning English with AR, revealing a positive attitude and behaviour toward the AR learning method.
AR has enabled users to experience the actual circumstances or environment in various areas such as manufacturing industries, gamification, visual training, education, and application to find directions or information about places [16], [17] defined that the term AR is derived from the combination of technology that allows the interaction with real-time environments of computer-generated displays throughout the screen that immerses the user into the environment. According to Mekni and Lemieux [17], AR is a technology that should be interactive and could display in 3D to allow immersion in a real-time environment. The importance of 3D in AR was emphasized by [18], which stated that 3D graphics provide attractive and interesting opportunities that could enhance interaction and influence capabilities that reflect how we perceive our natural environment.
The benefit of AR in education is the capability of AR to enable users to gain tacit knowledge. Tacit knowledge is acquired based on personal experience and environment. It has been proven that AR greatly impacts comprehension of new lessons. In education areas, AR has a great impact which provides a new method of learning [3] and gives new opportunities for designing engaging learning environments [19]. By merging technology and pedagogical methodologies, it provides a fresh perspective for learning, enhancing learning outcomes through engagement and involvement and allowing learners to visualize complicated spatial relationships and abstract concepts [20]. When the spatial features of 3D AR models are compared to physical and computer models, they may be more advantageous for learning than standard computer models [21]. Research by Kesim and Ozarslan [22], emphasized that the existence of AR technology in education could make new kinds of automated applications and enhance the effectiveness and attractiveness of learning in real-life situations. This technology will engage students to learn as it offers a learning experience that allows them to interact between the real and virtual worlds.

METHOD
The analyze, design, develop, implement, and evaluate (ADDIE) method is used in this study to ensure that the application is created methodically. It includes the site analysis, design, development, implementation, and evaluation. The method is described in depth at each level below.

Analysis
The problem statement, purpose, and scope of this project are decided during the analysis phase. During this phase, several relevant publications and journals are studied to learn more about augmented After analyzing the data, it was shown that the children have difficulty learning English. Thus, the project aimed to create a graphical dictionary application using mobile AR to assist youngsters with fundamental spelling and listening to the word through the picture. The dictionary is divided into four sections: animals, flowers, fruits and vegetables. It was created using a 3D model that combines multimedia features like audio, text, images and animation.

Design
In this phase, a flowchart and storyboard were developed to represent the flow of the application. The flowchart demonstrates a connection between significant pieces of the application frameworks from the start to the end process. Meanwhile, the storyboard illustrates a roadmap of the application graphically. The detail of both designs explains below. Figure 1 shows the flowchart of the smart pictorial dictionary via the mobile AR application. The flowchart shows the flow of the user's process when they use this application from the beginning to the end of the scene. It starts with a loading scene showing the progress bar before it continues to the main menu. The main menu scene has five option buttons which e a start, book, quiz, setting, and exit. Before starting, a user must ensure that they have downloaded the dictionary from the given link in the book option button. There is a quiz section with three types of quizzes: image guess, word guess, and sound guess. It allows users to train their thinking skills and engage them to learn more. Once the user clicks the start button, there is a pop-up box that will ask the user if they need help by providing instructions to guide the user before starting to play. option, the camera scene will appear where the user needs to scan the image according to the right object they choose from the dictionary. In the AR scene, there is a words button with audio of how the word is pronounced in a dual language: English and Malay language. There is also an information button that displays information about the object the user chooses. For the animal category, there is an audio button that will play the sound of the animal chosen. All objects are displayed in 3D animation. Users can interact with the object by rotating, enlarging and reducing the object's size.

Storyboard
The storyboard is developed to design the application's look and guide the developer in developing the application. Figure 2 shows the main part of the application. Figure 2(a) is a user's main menu to make an option. The start button will start the application and appear in the next scene. The setting button is to adjust the application setting. The quit button is to exit the application. Meanwhile, Figure 2(b) is the scene for categories where animals, flowers, fruits and vegetables reside.

Develpoment
Using the analysis and design phase as a guide, create the application titled "smart pictorial dictionary using mobile AR" in this phase. The development of this application, which communicates the project's goals, was made possible by the data gathered. After that, construct the application using the previously made storyboard. Then create a prototype for this project that resembles the planned interface from the design phase.
The project development process includes integrating multiple platforms and stages into one component, as stated in Figure 3. The development process of this application starts with downloading elements and objects from the internet, such as background images, audio, and the GUI. The making of 3D models to create the object is done by using 3DS MAX software. The 3D object is modeled based on the 2D image as a reference and animation is added to make the object looks realistic and interesting. The next step is importing the 3D objects into the unity application development software. Arrange the elements and the 3D objects of the application by referring to the storyboard. The C# scripts were added to make the application work, link to one another, and combine all the scenes.
The image target is the place to the right object in the preferred scene as it will link to the Vuforia engine software to make the AR function well on the application. The 3D object is displayed on the right image on a printed dictionary that will appear when the user scans the image. The application will be exported to android studio software to finalize the application by setting the android version and the preference to be displayed using an android device.  Figure 3. The process of designing and developing the application using selected software

Implementation
For this application, the ARCS model created by John Keller was implemented. This model consists of four main components, which are attention (A), relevance (R), confidence (C), and satisfaction (S) [14]. This application has a positive influence on changing the attitude and behaviour of the user by performing the ARCS model. Besides that, it was believed to encourage and sustain learners motivation. Various methods could be employed for attention, such as active participation, humour, conflict, variety, and real-world examples. For this research, real-world examples have been applied as models have been developed by using 3D objects which simulate real things. Using real-life stories or examples will pique their interest and make them want to learn more. Meanwhile, some strategies could be implemented for relevance, such as a link to previous experience, perceived present worth, perceived future usefulness, modeling, and choice. In this research, the strategies chosen were linked to previous experience as they used examples that were familiar to the students. The third factor is confidence. According to Keller, few learning activities can be implemented in these elements, such as facilitating self-growth, communicating objectives and prerequisites, providing feedback, and giving learners control. Providing feedback has been chosen, it is because once students answer the quiz, they will get immediate results. Finally is the satisfaction factor with strategies such as praise or rewards and immediate application. Figure 4 shows the factor of ARCS for application in this research.

Evaluation
Once the application reaches the testing phase, an evaluation is conducted which involves students from standard one until standard three primary schools to review the application in terms of usability. The evaluation is conducted based on usability testing to measure the children's learning level. The questionnaire is created in an online form. It is modified into simple words that could be understood and easier for them to respond by using the linkers type scale, which 1 represents strongly disagree, 2 for disagree, 3 for moderate agree, 4 for agree and 5 for strongly agree.
According to Nielsen et al. [23], "usability is a quality attribute that assesses how easy user interfaces are to use, making it possible for the customers to develop tasks in a clear, transparent, agile and useful way". Nielsen considers that five quality components are included in usability testing: learnability,  [24]. The questionnaire consists of five sections: demography, attention, relevance, confidence and satisfaction. All the questions were adjusted to match the 'smart pictorial dictionary' to make them relevant.

RESULTS AND DISCUSSION
After the evaluation, an analysis is made to determine whether the application achieves the objective. There are 17 respondents from Kampung Mendapat, Merlimau Melaka, while 19 more are from Permatang Pasir, Kedah, with a total of overall respondents, are 36. The question is divided into five sections: demography, attention, relevance, confidence, and satisfaction. The total overall item in the instrument is 24.
Kun and Keller [25] stated that the ARCS model defines four components: attention, relevance, confidence, and satisfaction. Attention refers to the element of motivation that could sustain user attention while using the application. Relevance is the application's content that conveys the impression that the information is worth knowing. Confidence is the motivation element that helps to build user confidence levels when learning. Satisfaction is a category that incorporates research and practices that help users feel good about their accomplishments.
Based on Table 1, 44.4% of the respondents are male, and 55.6% are female. All the respondents are from the Malay race. 47.2% of the children are exposed to the AR application, while 19 (52.8%) have no experience with the AR application. Some children did not know what AR is and how it works.

Attention
This section is labeled part A. The items are about how the user gives attention and experience after using the application. Before the questionnaire is distributed, the respondents are given a chance to play with the application. Table 2 indicates the result received after they tried the application. Table 2 shows that 52.8% of respondents strongly agree that the application's instructions are simple to grasp. However, some of them are a little unclear due to a lack of experience with mobile phones. 75.0% of respondents strongly agree that the application's graphic design is appealing. The respondent (77.8%) feels that the music used is attractive and suitable for the application. Besides, 50.0% of respondents easily adapt to English, and some might get it difficult as they cannot read well, especially seven-year-old children. Most respondents (75.0%) feel that the application is easy to understand. This scenario means the flow of the application is simple. Most children (66.7%) strongly agree that they give full attention while using the application. They feel amazing with the usage of AR with the combination of mobile learning.

Relevance
This section is labeled part B, focusing more on the application's user interface. Table 3 shows the result receive for the user interface. All the design for the interface is created based on user interest for children's level and using the suitable fancy font for the application, with 83.3% of the respondents agreeing that the font is easy to read. About 69.4% strongly agree that the theme is suitable for the application. A user-friendly interface is important as it helps the user navigate the application easily. About 75% of the children strongly agree that the application is easy to navigate from each page, and 83.3% strongly agree that the button is functioning well and is easy to click.

Confidence
This section is labeled part C, which states how respondents feel about the application. It consists of five questions. Table 4 shows the result received for the application confidence. About 72.2% of respondents feel application helps to identify the word with the image, but some still confuse the image and the word. This scenario is one of the main purposes of conducting this application. Most children feel that this application could save the theme of learning English words. Most respondents (77.8%) strongly agree that this application could help them pronounce words better. They tried to say it louder while using the application. They both agree that this application will help them improve their academic performance. Approximately 86.1% of respondents would use this application as a reference for learning purposes because they believe it is useful and simple to use.

Satisfaction
This section is labeled as part D, where the question stated about user satisfaction after using the application. There are five questions and generally about application usability. Table 5 shows the result of user satisfaction. This application is represented in AR to display the object when the user scans the printed image on the book. Respondents feel satisfied with the use of the 3D model for AR. About 75.0% of the respondents think this application could replicate the pictorial dictionary book. There is a pronunciation audio part where the user can click the button to listen to the word's pronunciation. All the audio works well, and 77.8% feel satisfied with the audio output. However, there are still some issues with displaying the 3D model after they scan the image. This scenario is because the area is dark. Most of them are satisfied with the output after the 3D model appears with some extra animation. Most of the respondents (91.7%) are satisfied with the application and frequently use it.

CONCLUSION
The findings prove that the application successfully functions and fulfills the user's needs. Furthermore, usability testing was done to test users' perceptions of the application and their experience after using it. According to the feedback, most respondents gave positive feedback, indicating that the application met its goal of developing a 'smart pictorial dictionary via mobile AR' based on user preference. This technology creates an experience and interest for the learners as they can enjoy and, at the same time, gain knowledge. Combining entertainment with education could ease the student's remembering and recognition of the image and the words. Some requirements were followed during the testing phase to maintain students' attention during the pedagogical, develop a relevance to the student's requirements, create a positive expectation for success, and help to have satisfaction by reinforcing success.