An android application and speech recognition-based IoT-enabled deployment using NodeMCU for elderly individuals

ABSTRACT


INTRODUCTION
In computing advancements, the modern control/automation system can be combined with computer networks and software applications so that human interaction with electrical objects like fans, lights, speakers, refrigerators, and air conditioners. is minimal. Software engineers and IT professionals are inspired to create new concepts for automation control systems [1]. In the addition of the Android operating system in software technology, it opens a new door of portable multitasking technique to solve several types of problem by using a smartphone. According to online statistics and previous research, more than 70% of people worldwide use the Android operating system [2], [3] so an efficient internet of things (IoT) automation system can be compatible with Android smartphone applications [4]. The aim of this proposed system is to help elderly individuals. The justification behind this is that the world's populations are aging as a result of declining birth rates and rising life expectancy. People over 60 are now present in greater numbers and proportions as a result Table 1 depicts the hardware components and software tools applied to create a workable prototype. The NodeMCU, sensors and other components are used for in the hardware section. We applied a CAD tool for designing circuit diagrams. In the software section, an API testing tool is used to manipulate and test HTTP services. Additionally, two types of IDEs are used: one for creating Android applications and another for writing microcontroller instructions. As well as the programming languages and software architectural style. Node MCU IoT platform device, the processing unit of the proposed prototype. 2 DHT-11 Sensing temperature and relative humidity of weather.

L298N
A dual H-bridge circuit controls the speed and direction of the motor. 4 Electrical appliances Light emitting diodes, 20w (Watt) LED bulb, Atom Berg and ordinary ceiling fan. 5 Relay device A programmable switch to control ON/OFF for high-voltage electrical devices. 6 Android SDK Software development kit used to develop and debug Android applications integrated with set of libraries, debugger and emulator. 7 Android studio Integrated development environment tool, used for developing Android applications. 8 Arduino IDE A software that used to write and upload code to the Arduino boards. 9 Android smartphone A smartphone that runs Android operating system (OS). 10 Fritzing Computer aided design (CAD) tool for designing electronic circuit. 11 RESTful API Representational state transfer and application programming interface are used in HTTP requests to access and manipulate data between web services and Android applications. 12 Embedded C A programming language for writing instructions in electronic gadgets or embedded systems. 13 Java Native compiled language for building Android application for the proposed system. 14 Postman API platform tool, that allows developers to easily built, share, test, and document APIs.

Circuit assembling
Home appliances like light, fan (for safety measures, we used a DC fan for prototyping). are connected with relay device. Relay device is directly connected with NodeMCU's GPIO Pin header. Figure 1(a) is the fritzing diagram of the circuit, depicting the blue block as a weather observation and the red block as an automation unit. A DHT-11 sensor is directly connected to the NodeMCU through the D5 (GPIO14) pin. The power source can be provided by an external 3-5V battery through NodeMCU's 3V pin header.
Two LED lights are connected to the NodeMCU's D1 (GPIO5) and D2 (GPIO4) pins at the home appliance control unit. The L298N motor driver's speed and rotation signal inputs are connected to D3 (GPIO0), D4 (GPIO2) pins, respectively and the fan is connected to the L298N motor driver's output pin. The L298N motor driver requires 5 volts external power source [23]. After the completion of the circuit assembly process, we obtained the actual prototype depicted in Figure 1

METHOD
This proposed system is consisting of 2 vital unit. The tasks of automation are accomplished by the first unit besides weather monitoring are done by second unit. We applied some methodologies and techniques to make this proposed system more efficient. At the beginning, we design a suitable framework so that we can observe how the data are flown by the entire system secondly the algorithm design. After the algorithm design, there are various methods and techniques that are used such as switching grid technique, sensors data manipulation with microcontroller, IoT automation, software development for prototype and Android application.

Architecture building methodology for the proposed system
There are several types of facilities that exist in the proposed system. The data communication, infrastructure and the flow of data under the system can be depicted through a framework/architecture, which is shown in Figure 2. The development of architecture can be expressed mathematically from (1), End to End communication is equipped for data layers used for IoT gateway and the cloud instant database can be expressed from (2) and (3).
Where, ∑ =1 (∝ ) denoting the summations of functionalities used in the facilities of IoT's system architecture; ∑ =1 ∑ =1 ( ) denoting the various components like DHT-11 sensors, relay device, light, fan, AC and co-Microcontrollers; ∑ =1 ∑ =1 ( ) denoting the various network technologies like Wi-Fi, Bluetooth, radio transmission, GSM/CDMA, long term evaluation (LTE), and 5G; ∑ =1 ∑ =1 ( ) denoting the programming paradigm and software architectural design patterns used for data communications between human to machine interactions (such as embedded C++, Java, HTTP, Restful API, Android studio, Arduino IDE's serial monitor, and serial plotter. Figure 2 depicting a framework that represents the data communication and integral mechanism between NodeMCU's IoT cloud platform and Android device.

Relay switching technique
A relay device works as a switch for high voltage electrical appliances. The methodology of a relay device is electromagnetism; a switch operated by an electromagnet. Therefore, a low voltage is sufficient to activate an electromagnet. This small voltage is supplied to the relay by the NodeMCU or with an intermediate driver if needed. In 5 V relay device, it has 6 pins out, 3 are used to drive the relay (low voltage side). Figure 3(a) depicts how an appliance can be connected through a relay module and the serial communication between NodeMCU and relay module. We can say that the switch is OFF by default and when the relay is energized it turns on. Normally closed (NC) it is a NC terminal. When the relay is de-energized (not ON), this pin is closed. The switch is activated by default and when the relay is energized. GND is ground pin. Signal is data pin to apply instructions on the relay. 5 V VCC is operating voltage of relay device [24].

DHT-11 real-time weather data computation through NodeMCU
In the weather observation unit, a DHT-11 sensor connected to the NodeMCU's GPIO pin header, and this sensor is actually made with ceramic. It holds a capacitive humidity sensor element and a thermistor. For sensing humidity, it uses a capacitor, which has 2 electrodes and a moisturizing substrate [25]. The levels of humidity are changes based on the changes of capacitance value. To measure temperature, the sensor uses a negative temperature coefficient thermistor, which causes its resistance value to decrease with increasing temperature. Higher resistance values are achieved even with small temperature changes. The NodeMCU's one wire communication protocol must be utilized to read data from the DHT-11. Figure 4 depicting the diagram of interfacing of the DHT-11, often available on its datasheet, and also the same diagram is provided below for reference. This diagram explains thoroughly the way to perform the aforementioned task. In order for the communication to start, DHT-11 must receive a start signal from the NodeMCU. Therefore, any time the NodeMCU wants to send a start signal to request the DHT-11 sensor to send temperature and humidity values, after the successful completion of the start-signal, the DHT-11 will transmit a response signal that contains information regarding the temperature and humidity. During this case, data communication is performed using an odd-bus data communication protocol. The full length of the data is 40 bits, and also the sensor transmits the upper data bits first. Reason for the pull-up resistor, the data or information line in idle mode always remains at the VCC level. The NodeMCU must reduce this voltage at intervals of at least 18 milliseconds to reduce overvoltage. During this period, the DHT-11 sensor detects a start signal, and the NodeMCU increments the data line by 20-40µs. Waiting time for a response from the DHT-11 is 20-40 µs. After waiting, DHT11 sends data to the microcontroller. The numbers in the data are both decimal and integral numbers. The sensor uses this data format.
"8bit integral RH data +8bit decimal RH data +8bit integral T data +8bit decimal T data +8bit checksum". Verify the checksum value with the received data. After successful data transmission, the checksum should be the sum of "8bit integral RH data +8bit decimal RH data +8bit integral T data+8bit decimal T data" [26].

Approaches to Android application development
We used several types of software development tools, the design pattern, and the Java programming language for building the "ARM Robolox" Android application. Model view controller (MVC), model view presenter (MVP), model view-view model (MVVM) are the core software architecture methodology or design patterns for Android application development. With these patterns, the project is structured in a way that all the codes can be covered in the unit testing, software code maintainability, addition and removal of features, and tracking of different important logic components. We used the MVP design pattern, because this pattern is broadly accepted and the programmer friendly to apply their own business logic. Our Android application development lifecycle can be illustrated with MVP. Figure 5 depicts the architecture of MVP [27], [28], there are: a. Model: a layer, that is responsible for handling the real-life business logic as well as communicating with the database and network layers. Here we created model classes for storing the object of DHT-11's sensor data, HTTP Post request objects for automation unit and SQlite database objects for storing defense info. b. View: it represents the Android application's user interface (UI) in its extensible markup language (XML) blueprint. This layer visualizes the data and keeps track of the user's actions so the presenter is notified. c. Presenter: this layer is used to display the UI logic based on the data from the model. For example, retrieving or reading DHT-11 sensor data as a JSON object from NodeMCU's Web server, sending data to NodeMCU's web server via Android application's web service and executes the Google RecognizerIntent. Figure 5. Architecture of MVP design pattern

Working procedure of the proposed system
NodeMCU has a useful library called WiFiManger, that allows users to connect ESP8266 to different access points (AP) without defining the Wi-Fi SSID and password in code. Also, it can store variables (custom parameters) or static internet protocol (IP) and maintains multiple SSID connections through WiFiManager library. The ESP8266 automatically connects to a known network or sets up an access point that the user can use to configure their network credentials. AutoConnectAP is the function of WiFiManager, that manages to establish a new connection with NodeMCU's Web server. This AutoConnectAP function is accessible from 192.168.4.1 default IP address. After booting NodeMCU, we will access AutoConnectAP from "ARM Robolox" Android application, then input the SSID and password of the Wi-Fi network as well as set a Static IP address.
192.168.1.152 is the static IP we set for test purposes that can be utilized by a URL; this IP address is used for data communications between the Android application and home appliance control unit. The Android application uses shared preference to store the static IP for later use. The proposed system employs two NodeMCUs, one for home appliance control unit and another for the weather observation unit. The procedure of data communication in the home appliance control unit is that when the Android application (client) sends a status via HTTP POST request through static IP/URL then NodeMCU decides to take a decision as to whether appliances should be in an ON or OFF state. For an example, the static IP/URL of NodeMCU's web server is http://192.168.1.152 and the Endpoint is /room_light. When the user switches ON/OFF or says "room light on" or "room light off" by Google RecognizerIntent from Android application, it will POST "1" or "0" as a Status in the NodeMCU's webserver. The NodeMCU will read this status. If the status is "1", then the light (appliance) will be turned on and if it is "0", the light (appliance) will be turned off. The HTTP Response from NodeMCU's web server will indicate the status being sent from the Android application. This procedure can be applied to any appliance which has been connected to NodeMCU. The logic can be described mathematically. The equation of the data transaction between Android application and home appliance control unit is, 1 + 1 =  0 + 0 =   = 1  = 0 here, 1 and 0 are denoted as ON and OFF states of home appliances. The procedure for the Wi-Fi network configuration in weather observation unit will be the same as in the home appliance control unit. Although this system is using a separate NodeMCU and a separate static IP. The static IP will be different from the previous unit. 192.168.55.78 is the accessible static IP address configured by AutoConnectAP. The DHT-11 sensor measures the temperature and humidity at the home and sends real-time sensory data to NodeMCU. The DHT-11 sensory data is generated in JSON format and uploaded to the webserver. The JSON skeleton is looks like {"temperature": "29", "kelvin": "302", "fahranheit": "84", "heatindex": "37", "humidity": "90"}. The pattern of the URL looks like http://192.168.55.78 and the Endpoint is "/data". The Android application will read those JSON objects from the http://192.168.55.78/data URL and displays them in its UI (user interface), the thread (background process) of the "ARM Robolox" Android application will update the user interface (real-time temperature, humidity, heat-index data) every 1 second. A user can set a threshold value in the Android application. If the threshold value is lower than the current temperature, it will give a potential alarm to the user.

Develop algorithms for system programming
The algorithm of proposed system consists of two units connected to the same network and configured by AutoConfigAP, as shown in Figure 6 flowchart. The instructions of NodeMCU are written in embedded C language. Algorithms followed the functional programming paradigm that contains "preparation block" and "execution block". The "preparation block" describes the type of NodeMCU-connected hardwires or components and their pinouts. On the other hand, the "execution block" contains the logic or instructions of NodeMCU. Arduino IDE compiles and uploads instructions in NodeMCU. Source code is available in the GitHub repository attached in "source code availability" section. In Algorithm 1, step 1 and 2 describes the preparation block and step 3 to step 12 describe execution block of home appliance control unit. Algorithm 2, step 2 describes the preparation block and step 3 to step 9 describe the execution block.

Prototype testing 3.7.1. Controlling home appliances through the "ARM Robolox" Android application
Before prototype testing it's necessary to configure network connection. We can access AutoConnectAp from browser through 192.168.4.1 URL, then input Wi-Fi SSID and password. From Figure 7(a), open home appliance control activity from "ARM Robolox" Android app then press (+) button to input static IP. Press the "Start" button located in the left corner of the (+) button to activate the Android web service. Turn on all switches in the app's user interface. Figure 7

Displaying DHT-11 weather data through the "ARM Robolox" Android application
Check whether the DHT-11 sensor is properly connected to the NodeMCU, then open the "ARM Robolox" Android app and the postman API platform tool. In Figure 8(a), postman makes HTTP POST request in 192. 16.0.103/data URL. Verify that the HTTP response is a valid JSON array that contains the DHT-11 weather information like temperature, kelvin, Fahrenheit, humidity, and heat-index. From Figure 8(b), press the "+" button to input static IP and threshold value to start the Android Web service. Check threshold value is less than the current temperature. The temperature and humidity values are displayed in the user interface of the "ARM Robolox" Android application, as well as the alarming activity if the threshold value is less than the current temperature.

Testing speech recognition by RecoconizerIntent service
Check whether the Android device has a supported microphone or not. In Figure 9(a), the "voice instructions" UI displays the predefined commands for the appliance control. Figure 9(b) depicts the appearance of a prompt dialog bar on the RecognizerIntent activity, instructing the user to speak predefined command. Figure 9(c) depicting the Logcat from Android studio. The text that was found to be converted from the user's speech through RecognizerIntent is seen in the Logcat Console [29].

RESULTS AND DISCUSSION
Results can be obtained by analyzing two factors. The first one is the data transaction analysis secondly the DHT-11 real-time weather data observation. In the data transaction analysis, the web service of the "ARM Robolox" Android application will post data to NodeMCU's web server. It is our goal to measure the size, type, and value of the data; furthermore, how much time each piece of data needs to complete the transaction. In contrast, we will simulate, compare, and create the datasets based on the real-time data from the DHT-11 sensor.

Data transaction analysis between the home appliance control unit and the Android application
The data transactions of the home appliance control unit are analyzed by the postman API platform tool. If a user wants to control an appliance, the web service of "ARM Robolox" Android application will send a HTTP POST request to the web server, and it will return "1" or "0" as a HTTP response. Each value is used to decide the action that will be applied to the appliance.
In Figure 10(a), HTTP Response 1 indicates that the appliance is turned on, whereas Figure 10(b) depicts HTTP response 0, which implies the appliance is turned off. The HTTP socket time and packet size can be analyzed from PostMan API platform tool. The socket time represents how long it takes an HTTP response to complete the socket as well as the size of the data packet received by the "ARM Robolox" Android application from an HTTP response. The packet size and socket size can be measured during data transactions, those are already depicting in Figures 10(a) and (b) respectively. Table 2 represents the data transactions between the home appliance control unit and the "ARM Robolox" Android application.

DHT-11 real-time sensory data observation
The DHT-11 real-time sensory data observation testbench or result has three criteria. The first criterion is to visualize the DHT-11 data from the Arduino IDE's serial plotter. The second one is to collect the dataset from the Arduino IDE's serial monitor to check accuracy and the third one is to compare the realtime temperature value of the DHT-11 sensor with the user-defined threshold value of the Android application. Serial plotter generates plots based on the DHT-11 real-time sensory data. Figure 11 depicts the differences in the rising and falling actions of humidity and temperature values based on the weather conditions at home; those are collected from several dates. The "blue plot" and the "red plot" visualize humidity and temperature, respectively. The range of temperature in DHT-11 is, 0 to 50 degrees Celsius, and the accuracy is + -2 degrees; the range of humidity is 20% to 80% with a 5% accuracy [30]. In Figure 12, serial monitor displays the different datasets of temperature and humidity from the DHT-11 sensor at several timestamps, and the red blocks depict the data we extracted for accuracy level analysis. At the beginning, we input 31.00 as a threshold value in the "ARM Robolox" Android application to compare it with the current value of temperature of the DHT-11 sensor.
If the threshold value of the Android application is<(less) or==(logical equal) to the current temperature, the "ARM Robolox" Android application will give an alarm to the user. Table 3 depicts the comparison rules between the current temperature and the threshold value. This test is based on the dataset of DHT-11 sensor during the summer weather in Bangladesh. The maximum temperature in Bangladesh during summer is approximately 32 to 34 degrees Celsius, and the falling temperature is approximately 27 to 28 degrees Celsius [31] so for the prototype testing purpose, we set the threshold value at 31.00 based on the peak temperature in the datasets. This threshold value can be varied depending on the usability of different users.

CONCLUSION
Based on our research findings, we say that this proposed system is a low-cost IoT and Wi-Fi communication-based project built with locally available prototyping components. Although this research is still in the prototype stage, it has some limitations. The first finding is that the network of this system is only configured for the first time through AutoConnectAP. If we want to connect this system with another new network, it's necessary to re-flash memory and re-program the NodeMCU. In the future, we can use the Raspberry Pi to remove this limitation for network configuration. The DHT-11 sensor can measure temperatures up to 50 degrees Celsius, it is only suitable for moderate weather. In the future, we can use the DHT-22 instead of DHT-11 sensor for better weather data measurement. Limited predefined values are used for voice command. We can enhance it by using machine learning and natural language processing (NLP) methods to store predictive values for voice command operations. This system provides an effective automation mechanism, where anyone especially elderly person can control or monitor their home electronic stuff from any distance. It also provides a safety measure to detecting the unstable home weather activity. Everything is maintained by a small Android application with efficient voice control feature. A portable, lightweight system, install anywhere in the room. The network connectivity of this system is a typical Wi-Fi network or Internet connection. The aim of this project is to help people make their lives a little bit easier with the touch of smart automation.