ASSAS: An automatic smart students attendance system based on normalized cross-correlation

ABSTRACT


INTRODUCTION
Nowadays, technological development has entered every part of our life. The use of technology and electronic devices to serve people replaced humans in many cases. Universities consider one of the most important institutions involved in this development in terms of contributing to the development or use of technology. The use of technology in the right way will result in saving time (which is a very important factor in many cases) as well as effort. Moreover, it will reduce mistakes made as a result of human nature. The student attendance system that is used in both universities and schools can be enhanced through using the technological invention and design it such that it works automatically without the teacher's intervention. In this paper, we design a smart student attendance system that uses one of the image processing methods, which is face matching. Smart student attendance system (SSAS) tracks and monitors the attendance of the students on the basis of their presence in class. Universities make sure that students attend classes on a regular basis. With a very low absence rate, students are more likely to succeed in their academic studies. Taking and delivering the attendance report by lecturers and professors through a manual process will consume time and effort. This effort will increase as long as the numbers of existing students are large and the number of frequently absent students is also large. In general, the token-based attendance system is more effective for handling student attendance current innovative technology immersed in education through the delivery of effective learning that utilized smart devices to develop a smart academic system. This paper adopts the current innovative technology towards designing a smart academic system. The proposed system has been compared with the traditional ways (roll call and sign-in sheet), so the results are saving time and effort with high accuracy in the performance Students attendance system. This paper is organized as follows. Section 2 discusses related and previous work. Section 3 describes the experimental procedures of the proposed system (SSAS). Section 4 illustrates the results and analysis, and section 5 illustrates the conclusion.

RELATED WORK
In this section, the study provides a brief overview of a sample of related work. Joshi et al. [1] developed an Android-based mobile application that manages the attendance of the students in a classroom. In this system, the student has to login into the application software through a username and password. If the combination is matched, then the application will automatically register the student in the database model as a present. Whereas if the student is absent, the application will inform the parent and faculty of the student absence. Here, a malicious person can use a student username and password and login to the system causing illegitimate access to the whole system. Agrawal and Bansal [2] designed an online attendance system that used an RFID chip and a student ID to count the number of students attending the class. Here, the student can deliver its ID to one of his colleagues and will be counted as attendance in the classroom. On the other hand, in case the student forgets his ID he will be considered absent while he is actually present. Some developers used voiceprint and fingerprint recognition to identify student attendance in the classroom as illustrated in [3,4]. In the case of the voiceprint, the system will consume more time especially when the number of students is large. Moreover, when the student mentions his identity during the lecture, he will influence the concentration of other students.
Saravanan and Surender [5] and Islam et al. [6] build a mobile-based application for the student attendance and mark management system. Budi et al. [7] used an SQL server to count and track the attendance of a student and save a list of attendance in the server database. We based our system on face recognition and detection techniques. There are many techniques for face recognition and detection, for example, local binary patterns (LBP) [8,9], principal component analysis (PCA) [10,11], a combination of PCA, wavelet, and support vector machines (SVM) [12], local binary pattern histogram (LBPH) [13], independent component analysis (ICA) [14,15], eigenfaces [16], and linear discriminant analysis (LDA) [17,18], SVM [19,20], combining fast discrete curvelet transform (FDCvT) and invariant moments with SVM and deep learning technology [21,22]. Dharpure et al. [23] proposed a system that utilized counting objects techniques for a fast template matching process based on the normalized cross-correlation (NCC) algorithm. The template matching algorithm utilized to identify a similar template present in the image. The author used a data set of all the students' images and applied a deep learning approach that uses CNN for the face detection process in the designed attendance system [22].
Our system is similar to cited work in [1] and [23]. To illustrate, cited work [1] used Arduino, and cited work [23] used counting object with CNN. Our system used Arduino and counting objects. Thus, the similarities between cited work [1] and [23] are the Arduino and counting object techniques. Basically, the mechanism of the developed system in this paper is explained through the following scenario: When the student is present inside the classroom, the system takes an image from Arduino's camera that is located inside the classroom. The detecting of the exact student is done by matching the image list of students with a new capture image. The detection process is based on face detection and cross-correlation algorithms. Next, the system will record the attendance of the student in the data templates of that classroom. A simple comparison of the proposed system and previous works are presented in Table 1. We based our comparison in terms of cost, dataset categories, and limitations. While in Table 2 similar works are presented related to earlier utilized face detection algorithms and the cross-correlation algorithm utilized in this work. To sum up, our system is designed as a low-cost solution to record student attendance by utilizing the simplest techniques based on machine vision applications.  [25] Complex, multiple hidden layers for face detection CNN and Viola Jones Framework Algorithm [26] Complex, when faces rotate by angles the result not accuracy principal component analysis (PCA) [22] Complex and expensive, a lot of images in same face's person Using deep learning technique require a high performance workstation (processor, RAM) Present system Simple and cheap, the faces of the students should be clear on the source image, one image for each students Normalize cross correlation

EXPERIMENTS PROCEDURES
This section gives an account of how we carried out our experiments; the required simple operations steps for our designed automatic smart student attendance system (ASSAS) are illustrated in Figure 1. As introduced in Figure 1, the proposed system has two main Phases. The hardware phase includes the OV7670 image sensor and the Arduino UNO board. The software phase encounters prepossessing and faces recognition processes. The following subsection explains both phases in detail.

Hardware phase
Hardware phase consist of the following hardware components:

OV7670 image sensor
This sensor is small in volume and has low operating voltage. The OV7670 sensor is able to provide all functions of a single VGA camera chip of image processor with a resolution 8 bits of data and a maximum of 30 frames per second. The user can reconfigure image quality, data format and transmission mode. For a full information and details please reference to cite report in [24].

Arduino UNO board
The Arduino UNO board is built in with power pins, analog pins, ATmegs328, ICSP header, reset button, power LED, digital pins, test led 13, TX/RX pins, USB interface, an external power supply. The description of Arduino UNO board is detailed in cited work [24]. The camera was connected with Arduino UNO as illustrated in Figure 2.

Software Phase
Software phase is divided into two parts:

Pre-processing
The algorithms which used in this system are described in the following steps: − Template matching The fundamental notion of the proposed system is the template matching. This technique is widely used to classify objects that are associated with machine vision applications. The template matching algorithm is utilized to identify the exact template which is present in the image. The basic idea behind the technique is to compare part of the images with the source image as presented in Figure 3, which will reveal a sample image (template) of similar objects in the source image. This technique works effectively when the standard deviation of the template image is small enough in a comparison with the source image. The matching process moves the template image to all possible positions in a larger source image and computes a numerical index that indicates how well the template matches the image in that position. The match is done on a pixel-by-pixel basis [27]. − Normalize cross-correlation (NCC) It is the best and faster techniques to detect faces. NCC depended on a coefficient values which measures the degree of similarity between two images. The largest value will appear when the exact face matches template [28]. NCC technique needs two variables (source image and template image). In our proposed system (the template image) represent database of student faces's images as illustrated in Table 3 and (source image) represent the classroom's image. The NCC algorithm uses the following general procedure: i) Find the cross-correlation value; ii) Calculate total local sums between template and source image; iii) To get correlation coefficients use local sums to normalize the cross-correlation.
The peak cross-correlation coefficient values represent the perfect matching of extracted exact student with the target image. This technique depends on the following equation as calculated in [27] where: t: template gray level image, ̅ : average grey level in the template image, f : source image section ̅ : Average grey level in the source image, x, y: number of pixels in the section image (template image size = columns*rows).
The value correlation is between -1 and +1, the larger values of correlation represent a stronger relationship between the two images. Thus, a match is found and the exact detection template was done. Seek the exact student in above stage is depended on the following procedure, requirement of executing crosscorrelation algorithm are: a) Source image which is taken from class; b) Templates (dataset of all students in the class). These templates are most often used to identify exactly student by select sub image of all students as a template which is the person we are trying to find him and essential in cross correlation step. Template matching is theoretically a simple process. This need is to match a template by scanning a source image for detecting a position of template depends on the cross-correlation equations between template and source image. Normalize cross correlation technique is utilized to detect the right position for each template which is calculated depending on (1). The normalize cross correlation algorithm does not work with the RGB image therefore we should convert the image to gray scale. By taken the pixel values (red, green and blue) and applied (2) with a different coefficients depend on the Rec (ITU-R BT.6017) formula as calculated in [28]: Gray= 0.299 * R + 0.587 * G + 0.114 * B (2) − Threshold To select the exact template, the threshold value must be changed. The accuracy became 100% with a value of threshold range between 0.63 and 0.9 as illustrated in Table 4 as well as Figure 4. The accuracy is calculated by (3) and its value depends on finding the correct student as well as the total number of students in the classroom.
Accuracy%= ((Total no. of students in classroom -No. of result)/total -1) *100 (3) The procedure is repeated for the entire image with all student templates. As illustrated in the step above, the maximum values of the cross-correlation coefficients indicate the perfect match for exact student detection. By matching the template of student it is set to be the point where the exact position of the student (given by the template) inside the image. The binary component method is used to find the white point (maximum value) in the target image. If the white point appears, which means that the student is in class and his attendance is recorded. If a white dot is not feasible in its actual location, this means that the student is not in class and is registered as absent.

Practical procedures
This section is presenting the algorithm and the procedure followed in this work as illustrated: − Pseudocode Start Initialize and save data set of all students (lables) Read source image (S) (for example: Scenario 1, Scenario 2 and Scenario 3) from camera Convert Source Image from RGB channles to Gray channel Label =1 For lablel (template) <=16 Begin Apply NCC algorithm //Change threshold of binary components (count number of students) For threshold (0:0.01:0.99) // best threshold from 0.63 till 0.8) Accuracy%= ((Total no. of students in classroom -No. of template matching)/total -1) *100 If (template matching) =1 {student appear} Remove this label's student from absent list If (template matching) =0 Add this label's student from absent list End Repeat until label=16 − Procedure Whenever the attendance of student should be taken, the system takes a picture of the classroom as illustrated in Figure 3 and the following steps will be done: a. Convert the source image from RGB color to gray scale image b. Take the gray scale image and apply the NCC algorithm on it with all students' templates in order to find the exact template among all the templates which represent all the students of the class in the list of student profiles as illustrated in Table 3. When the student is found, the system removes that student from the absence list. For example: If we take one of the students whose name is (x.c) as shown in Figure 3(b), we will notice that the system locates it exactly in the source image as illustrated in Figure 3(c). (Noting that the yellow square presented in Figure 3(c), represents the exact position of the (x.c) student) c. In order to find the exact student the threshold values are changed many times until the best result obtained (optimum accuracy) as the appropriate values start from 0.63 till 0.9. All that done by using the binary related components technique as shown in Table 4 and Figure 4 d. When the student is found/ not found in the source image, the information will be sent as a report to the person concerned about the student's presence/absence in the attendance list e. Students' seating locations were changed in the classroom as illustrated in Figure 5 and the same results were obtained. The number of students was increased to 40, and we got quick and correct result.   Table 4. Accuracy between the best result and change threshold values

RESULTS AND ANALYSIS
In most universities, there is a traditional attendance registration system for students in the classroom. The traditional process of taking students attendance takes the time to read the names of all students. Also, it creates noise in the classroom. In exam times, traditional process can distract the focus of the student. In an alternative process (signature process), the student has to sign on list that has all the students' names. This process take time since the student has to find his name on the list, and also cause confusion among students as well as lack of focus in exams which result in a loss of time which is one of the most important resources for young youth. This paper presents a simple, cheap, and reliable solution to these problems. Our solution overcomes these problems by using a method of taking attendance automatically. Our solution relays on artificial intelligence and simple image processing that are worked out in this research. After completing the system design to record the attendance of students automatically, it was applied to a group of students of the fourth stage, Department of Computer Technology Engineering, College of Technical Engineering, Northern Technical University, Iraq.

Test setup
The steps which are taken and its result are explained in the following points: − The scenarios of attendance students in the classroom are illustrated in Figures 6-8 for half, full and partial attendance's students respectively, which are taken from Arduino's Arduino (VGA camera). − Initialize templates (labels) dataset of students' faces to represent them in the attendance recording system. As shown in the following Table 3. The results obtained by MatLab 2019b program. − Scenario #1 (Half attendance (HA-ASSAS)), there are 8 students, they changed their seat locations and we order these students to change their emotion's face. This scenario was detected all student depended on the data set for all students as illustrated in Figure 6. − Scenario #2 (Full attendance (FA-ASSAS)), there are 16 students, they changed their seat locations. In this scenario, our system detect all student depended on the dataset for all students as illustrated in Figure 7. − Scenario #3 (Partial attendance (PA-ASSAS)), there are 12 students as illustrated in Figure 8, they changed their seat locations, and we order these students to change their emotion's face as illustrated in Figure 9, and their faces is confronting at the camera. In this scenario, our system detected all current students depending on the dataset for all students.

CONCLUSION
This simple system has opened a portal for us to improve and enhance academic student attendance system. The system helped the students to save their time and at the same time. During the lecture it was easy for the teacher to record the attendance of all the students without losing time or effort. Sometimes the professor forgets to take the audience. In this system, we can retrieve information at any time and determine the presence of students in the class at the moment of giving the lecture. Ensuring student rights if the teacher by mistake does not register the attendance of a student. All of these things have been investigated in the proposed system which is inexpensive and highly efficient.