COGNITIVE INTERNET OF THINGS USING MQTT PROTOCOL FOR SMART DIAGNOSIS SYSTEM

- Cognitive Internet of Things (CIoT) is the next leap towards enhancing the accuracy and effectiveness of Internet of Things (IoT) technology used in combination with cognitive computing that has a signiﬁcant role in healthcare and diseases diagnosis. The work in this paper proposes a system to diagnose the sensitivity toward sound and light by developing a mechanism based on facial emotions recognition system that integrate with IoT protocols and cloud computing. The proposed system has been achieved by establishing a cognitive IoT environment composed of hardware and software components that practically implemented in a laboratory to identify the behavior of people suffering from sensitivity toward sound and light. This behavior has been observed by monitoring human face emotions through a live video capturing using camera and image processing using facial emotion recognition software. Emotions values obtained has been examined and collected in a cloud using Message Queuing Telemetry Transport (MQTT) protocol which is an IoT application layer protocol. These emotions were classiﬁed as normal and abnormal. Normal state represented by happy or natural emotions give the impression that the surrounding environment is appropriate for people senses and they do not suffer from any discomfort toward this environment, therefore the system works on a mechanism using a microcontroller to increase the intensities of sound or light in this environment. While in case of detection hypersensitivity, the system automatically decreases these intensities. The project aims to be an important platform in the diagnosis for medical and social cases that suffer from a disturbance towards the surrounding environment.


I. INTRODUCTION
The information and communication industry has a great stride in recent years by integration with the Internet of Things (IoT).IoT market is expected to witness a noticeable increase reach 25 billion " Things" in 2020 (According to Gartner).This increase may lead to manufacturing complicated processes with heterogeneous data, these data must satisfy the demands and capable the systems communicate with each other.The technologies easily collect data by sensing the physical world, but it is difficult to organize and manage it.The complexity of the systems can be handled by the automation of IoT systems management.The integration of IoT systems should be flexible and intelligent, to continuously receive and process data in a timely manner.These IoT-based systems implemented in real-time, requiring a high level of coordination management processes and the interaction that humans learn from their intelligence and supply very exact analysis [1].Lately, a modern type of Internet things has been announced called " Cognitive Internet of Things" (CIoT).
CIoT referred as an IoT system that integrates aspects of human cognitive feature processes in system style to attain 'varying levels of autonomy in perception, analysis, and action supporting self-x capabilities as well as learning, re-configuration, resilience, optimization, and management with adequate human-machine context awareness [2].The underlying cognitive processes may have many cognitive cycle abstractions such as Sense, Understand, Decide, and Act.The process of Sense is concerned with sensing and collecting data about the world's self.Understand, involves extracting information by analyzing, interpreting and representing sensor data in a structured way.This is followed by a process of decision-making to arrive at answers and actions to be taken based on the information collected.Finally, the decisions taken are applied to the system as part of the Act process by means of suitable actuation logic.The system can learn from experiences at each of these stages that can subsequently be used to adapt.Such smart control loops can provide autonomous capabilities or self-x properties for a system such as self-healing, self-configuring, self-optimizing, and self-management [3].As part of the smart community paradigm, advances in the sector of cognitive IoT -based cloud techniques and protocols have already led in enormous supply for actual-time, smart and separated service facilities.One of the most common protocols used in this field is MQTT protocol [4].MQTT is a shortened form of (Message Queuing Telemetry Transport), this protocol running on top of the TCP / IP stack.It is low weight, simple, data-centric and intended specifically for resource-restricted systems such as IoT devices [5].MQTT messages are classified mainly on publish and subscribe messages controlled by a broker.MQTT client can concurrently publish and subscribe messages to the broker with particular topics.It could be a device connected to resources such as a sensor, a backend server or a graphical client capable of communicating with MQTT.It is the broker's responsibility to receive all published and subscribed messages, filter those messages and decide on the appropriate subscribers to send the message.All that is sent by the publisher and received by the subscriber have to go through the broker [6].

II. RELATED WORK
Few published research have been proposed for inclusion in the industry of smart environment based cloud and MQTT protocol to control the effect of sound and light and this research limited only for cases of autism and for the purpose of observation and monitoring.It can be summarized as follows: Sula and et al, in 2013 [7] built a SmartBox with vibrator, light, smell and sound control to provide a relaxing and inspiring place for autism kids when participating in various learning activities.This SmartBox controls the light of the room to be altered to suit the smell control to maintain a pleasant smell in the room; vibrates the chair or bed to maintain the kid comfortable and quiet.Kids were requested to use wireless object detectors to still locate body and hand motion.
Diego and Sergio, in 2013 [8] proposed an evaluation for the performance between a multi-face identification system on a nearby device using Emgu CV and a version interacting with Windows Azure, measuring the amount of processed face identification and ranking frames in real time.In terms of reliability, the higher number of processed faces is better.Krupal and et al, in 2015 [9] described the home automation system architecture and implementation.In order to minimize system development costs, the system utilizes fundamental electronic panels for development.In addition to this low cost, the automation system's smartness can be justified with the automation scripts that the user can customize, even at runtime.Using the MQTT protocol and TCP protocol, the complete system efficiently uses the existing network infrastructure.Cheol-Hong in 2017 [10] suggested system consisting of a variety of wearable sensors based on Bluetooth method to monitor the behavioral motion patterns of autistic children.It also used microphone cameras to record videos and images for the operations of this cases.Tiffany and Pinata, in 2018 [11] report the premature effort to construct an IoT-based interactive atmosphere specially designed to explore autistic kid's emotions.This environment aims to provide wealthy data for an emotion-learning algorithm, utilized as a spatial detecting region, for cognitive information, including kid bodily motions and hand motions.Tamara and Ali, in 2018 [12] implemented a remote monitoring system for children with autism to assist Eman K. Jassim, Emad H. AL-Hemiray experts track the progress of their patients and analyze data collection using MQTT protocol.This system observes the responses of the cases to various emotions such as sadness, happiness, nervousness, starvation, relaxation and trepidation, and then records this data.Kiavash and et al, in 2018 [13] developed software for facial expression recognition of emotions.
In addition to enabling the recognition of emotions from webcam image documents and documented video recordings are used to provide facial expressions in actual-time and continuously.It utilizes the FURIA algorithm to provide prompt and appropriate feedback depending on the facial emotions of the cases for unsorted fuzzy rule testing.The primary aim of this research was first to verify the use of webcam data in learning situations for an actual-time and accurate evaluation of facial expressions.Second, transform these facial emotions into emotional regions that the FURIA algorithm has recognized.

III. SYSTEM MODEL
System mechanism can be created based on the diagnostic process that performs at several sensitivity levels at specific times.As shown in diagnosis system diagram in Fig. 1, emotions detection system can interpret captured data x i which represents video frames and produce emotions e i to describe person's reaction to environment.Several techniques such as measurement of electrical activity in the brain (Electroencephalography), galvanic skin response (GSR), and observing facial expressions using real-time image processing can used in practice.Emotions collected from the previous stage are processed in the cloud to decide whether the person under observation is in normal or abnormal state.The detection system is integrated with Cloud using IoT application protocols encapsulated in TCP datagram.At each sensitivity level, the control system sends measured sound and light intensities to Cloud that work on a mechanism to check the normality of emotions values.Normal emotions, represented by happy or natural emotions, offer the perception that the surrounding environment is suitable for people's senses at that sensitivity level, therefore sound and light intensities can be increased while abnormal emotions represented by discomfort emotions is an indication to decrease these intensities to be commensurate with the normal emotions of the individual.On the other hand, the control system calculates the sound intensity value s i and light intensity value l i that increases or decreases The practical system was implemented in a laboratory (Networks research laboratory in College of Information Engineering / Al-Nahrain University).This laboratory consists of hardware and software material that integrated using MQTT protocol to provide the data that recorded in cloud database as shown in Fig. 2 The system was performed with 35 cases, their ages were in range of 19 to 44 years.was prepared based on EmguCV algorithm to detect the Face ID of the case being examined.Emotions detection software was prepared based on FURIA algorithm to detect the value of seven types of emotions (happy, sad, surprise, fear, disgust, anger and neutral).System sends the data (Face ID and emotions values) to cloud server using MQTT protocol.The cloud server (https://iot.coie-nahrain.edu.iq) has been used as a subscriber to collect data (Face ID and emotions values) and as a publisher to control systems (Sound and Light control systems) through the internet.
• Sound control system consists of Arduino, stepper motor, breadboard, and wires that integrated to control the rotation of sound volume control knob of sound generator.Sound generator has sound intensity values ranged from 10 −12 W/m 2 up to 10 −4 W/m 2 .
• Light control system consists of Arduino, stepper motor, breadboard, and wires that integrated to control the rotation of the dimmer of AC regulator.Dimmer can control the electric current that passing towards the lamp with a power in range from 45W to 500W.These control systems shown in Fig. 3.

B. Experiments
At the beginning of examination, the case was examined in face detection software for purpose of forming a record of case information (Name and ID) to document the data for each case and to easily detect it in future examinations.The emotion detection software was prepared to calculate the values of emotions as shown in Fig. 4. The high values of both happy emotion and neutral emotion than other emotions (according to FURIA algorithm) is an indication for the normal state of the human and vice versa.Emotion detection software and control systems were connected to an account in MQTT broker (https://iot.coie-nahrain.edu.iq) that created a dashboard called (Cognitive IoT Using MQTT Protocol for Smart Diagnosis System) for observing the data streaming.These data streaming consist of emotions values that published from emotion detection software with sound and light values that published from sound and light control systems.These cases were examined to detect sensitivity to sound and light in two separate control systems.Each control system consists of stepper motor connect to central Arduino that programmed using Matlab (Arduino support package) to control the rotation of sound volume control knob and AC regulator in clockwise (Increasing) or counter-clockwise (decreasing).Each case was examined to determine its sensitivity to sound and light for intensities ranging from 1 to 5 levels.Sound control system work to increase these intensities within certain periods ranging from 106 sec to 636 sec while light control system needs from 70 sec to 420 sec depending on the patient sensitivity and examination situation.At each intensity, the case was examined to detect his/ her emotions to find out if he/ she is in normal condition or suffers from sensitivity toward this sound or light intensity.This intensity was automatically increased when the case was confirmed to be normal and stable and vice versa if it turns out to be disturbed.

V. RESULTS
By using a smartphone or laptop we can observe the data streaming in MQTT broker.In Fig. 5

A. Sound System Examination Results
Each case was examined to determine his/her sensitivity to sound using Nostalgia music for Yanni (It is adjusted in terms of duration to be appropriate for examination) for sensitivity ranging from 1 to 5 levels each level needs 106 sec to complete the examination.For example, some case showed in Fig. 6 hypersensitivity toward this sound as his comfortable hearing level didn't exceed level 2 with 4 × 10 −5 (W/m 2 ) sound intensity.

B. Light System Examination Results
The light examination system showed different sensitivity response levels for the emitting light from a halogen lamp.It performed the examination with 5 sensitivity levels each level needs 70 sec to complete the examination.For example, some case showed a normal emotional response for light up to the level 4 with 409W light intensity and showed hypersensitivity for the higher sensitivity level as shown in the chart of light sensitivity examination in Fig.This paper presented a smart cognitive IoT system using MQTT protocol to control the sound and light intensities in the environment to diagnose people that suffer from a sensitivity toward these parameters.Several hardware components were used with emotions detection software and control software systems for practical implementation in a laboratory dedicated for examination the cases.The MQTT cloud server was used to collect data and published it to sound and light control systems through the internet.Charts of sound and light intensities for some case were prepared.

VII. FUTURE WORK
The proposed system can be implemented as a playing environment at home, school, or medical centers to diagnosis autism children and embedding some sensors such as heart rate and pressure sensors into toys for easy handling with them.

Figure 1 :
Figure 1: Diagnosis system diagram of sound and light sensitivity

Figure 2 :
Figure 2: Real photo of implemented system , the dashboard (Cognitive IoT Using MQTT Protocol for Smart Diagnosis System) shows the streamed data of case ID, emotions values that ranged in the sequence of (Happy, Sad, Surprise, Fear, Disgust, Anger and Neutral), sound intensity, light intensity and examination time.

Figure 3 :
Figure 3: Sound and light control systems

Figure 6 :
Figure 6: The sound sensitivity examination chart for some case

Figure 7 :
Figure 7: The light sensitivity examination chart for some case