MOBILE CONTROLLED WIRELESS BASED APPROCH

FOR HUMAN ROBOT INTERACTION

CHAPTER 1
INTRODUCTION

Robots are increasingly being integrated into working tasks to replace humans especially to
perform the repetitive task. In general, robotics can be divided into two areas, industrial and
service robotics. International Federation of Robotics (IFR) defines a service robot as a robot
which operates semi- or fully autonomously to perform services useful to the well-being of
humans and equipment, excluding manufacturing operations. On the other hand, IOT nowadays
is becoming the center for everything. People tend to get online rather than doing household
chores. IOT is now everywhere, compare to the last decades where internet is only wired, and
people needs to be in front of the computer to access the internet but nowadays, internet is just at
the tip of your finger. This is an advantage to introduce robot to household. The robot body was
prepared mechanically and electrical components were chosen to be suitable to be used as a
robotic arm. The robot is controlled using NodeMCU as the brain of the robot, connected to
Mobile App through internet via inbuilt wifi. NodeMCU is an open-source electronics
prototyping platform based on flexible, easy-to-use hardware and software. The movement of the
robot is controlled by inputting the desired direction of movement of the robotic arm and then the
robotic arm will move to the desired movement that has been inputted. In this project, webserver
developed using HTML is used to make the user interfaced and will access the robotic arm via
the internet to control it.

Department of E&TC DPCOE PUNE Page |1

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

CHAPTER 2
LITERATURE SURVEY

2.1 HISTORY

In the past, there are some researchers that used internet to control their robot, reference [1]
suggest to use SOAP protocol and used a PC server with ASP.NET framework and also a PC
client that have been develop with ASP.NET framework. The suggested protocol takes about 30
seconds to respond due to the overhead in the Web service, since the Web service works as
request-response base, it spoils the computation efficiency in the simulator. Meanwhile
references [2] suggest using an industrial standard PC architecture running LINUX, and a
network interface which supports a wireless TCP/IP Ethernet link. The robot is the NOMAD-200
that has a server workstation that runs on UNIX, and an industrial standard PC that runs on
LINUX. Because of the similarity of the operation systems of the server and robot, the
communication between the two computers can be implemented by applying UNIX socket. Two
transmission ports are used to create two virtual channels in their system. [3]Object tracking
robots, if can be controlled smartly through voice, can be of tremendous help for the physically
handicapped people. In this paper, we try to implement a voice-controlled object tracking robot. A
speech recognition system is used to recognize a set of predefined commands such as forward,
backward, left, right and rotation at a particular turning angle. The robot navigates its way as per the
voice-command signal, while also tracking the desired object. The voice-command signal processing
is carried out in real-time, using an on-time cloud server that converts it to text form. The command
signal text is then transferred to the robot via Bluetooth network to control its differential drive.
These papers presents a simple internet based controlled robotic arm using Arduino Uno
embedded system as the core of this robot and also Arduino Ethernet Shield to interface the robot
with the internet. The robot does not require training because the robotic arm is fully controlled
by the user

Department of E&TC DPCOE PUNE Page |2

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

2.2 REVIEW PAPERS

1. Keitaro Naruse and Makoto Oya, “Muscle-like Control of Entertainment Robot over
Internet”, International Conference on Advanced Intelligent Mechatronics, Monterey,
California, USA, 24-28 July, 2005.

The robot technology is extending to social activities such as community work, home-use and
entertainment. By combining the robot technology with the popularized public Internet, it is
anticipated a new application field will come out. However, Internet, as its nature, does not
permit real-time communication among robots and other systems including human operators and
computers. A different approach is necessary regarding communication protocols and robot
control schemes. This paper proposes the combination of the Web services technology and the
autonomous robot motion generation is the proper approach. For the motion generation, we
introduce the muscle-like control method, which can be applied to any kinds of robots easily
with proving a variety of creature-like motions. The method is applied to gait generation of a
simulated four-legged robot, and the results show that it can transit from a motion to another one
smoothly. In addition, the simulated robot is controlled by the Web services over Internet for
investigating the feasibility of the proposed control scheme in a real network circumstance. The
result shows that the proposed method provides us smooth control of the robot, even we have
overhead of Web service processes and delayed communication.

2. Tse Min Chen and Ren C. Luo,” Multisensor Based Autonomous Mobile Robot Through
Internet Control”, Department of Electrical Engineering, National Chung Cheng
University.

A remote supervisory control system that combines a computer network and an autonomous
mobile robot is constructed. The main feature of this system is that users just need a general-
purpose computer and a World Wide Web browser, they can command the mobile robot in a
remote location through the Internet and Home Page. Hardware configuration of this architecture
includes an autonomous mobile robot, server workstation, and other computers, each of them is
able to communicate with others via a computer network. Software design includes assignment
of the server, communication between the server and robot, and the user interface. The system

Department of E&TC DPCOE PUNE Page |3

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

has extensive capabilities for teleoperation, telerobotics operations, and other multimedia
services. Based on this architecture we have implemented multi-level operations for the user on
various remote applications. The system allows users to directly control our mobile robot via the
Internet and a friendly user interface (via our Home Page), and they can receive real-time images
captured by a CCD camera that is mounted on the robot and other data from sensor
measurement. At present, one can select a place in our laboratory by clicking a point on the grid
map shown in the Home Page WWW. After receiving the high level command, the robot will
deliver the sequential live images during the movement to the remote user site.

3. Kishan Kumar; Shyam Nandan; Ashutosh Mishra; Kanv Ku-mar and V. K. Mittal; Voice-
Controlled Object Tracking Smart Robot; in Proc. IEEE 3rd International Conference on Signal
Processing; Computing and Control 2015 (ISPCC 2015); JUlT; Waknaghat; India; 24-26 Sep 2015.

Object tracking robots, if can be controlled smartly through voice, can be of tremendous help
for the physically handicapped people. In this paper, we try to implement a voice-controlled
object tracking robot. A speech recognition system is used to recognize a set of predefined
commands such as forward, backward, left, right and rotation at a particular turning angle. The
robot navigates its way as per the voice-command signal, while also tracking the desired object.
The voice-command signal processing is carried out in real-time, using an on-time cloud server
that converts it to text form. The command signal text is then transferred to the robot via
Bluetooth network to control its differential drive. The prototype smart robot consists of three
sub-systems: speech recognition system, object tracking system and the differential-drive based
movement's control system. The accuracy and efficiency of the speech recognition system is
examined through a set of experiments. The effect of factors such as noise and distance etc. are
examined, with encouraging results. The prototype robot can recognize the voice-commands
within a Bluetooth range, i.e, 10 m. Possible extensions are also discussed that could lead to a
wide range of further applications.

4. B. Raghavendhar Reddy; E. Mahender; Speech to Text Con-version using Android Platform; I

nternational Journal of Engi-neering Research and Applications (IJERA) ISSN: 2248- 9622
www.ijera.com Vol. 3; Issue I; January -February 2013; pp.253-258.

Department of E&TC DPCOE PUNE Page |4

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

For the past several decades, designers have processed speech for a wide variety of applications
ranging from mobile communications to automatic reading machines. Speech recognition
reduces the overhead caused by alternate communication methods. Speech has not been used
much in the field of electronics and computers due to the complexity and variety of speech
signals and sounds. However, with modern processes, algorithms, and methods we can process
speech signals easily and recognize the text. In this project, we are going to develop an on-line
speech-to-text engine. The system acquires speech at run time through a microphone and
processes the sampled speech to recognize the uttered text. The recognized text can be stored in a
file. We are developing this on android platform using eclipse workbench. Our speech-to-text
system directly acquires and converts speech to text. It can supplement other larger systems,
giving users a different choice for data entry. A speech-to-text system can also improve system
accessibility by providing data entry options for blind, deaf, or physically handicapped users.
Voice SMS is an application developed in this work that allows a user to record and convert
spoken messages into SMS text message. User can send messages to the entered phone number.
Speech recognition is done via the Internet, connecting to Google's server. The application is
adapted to input messages in English. Speech recognition for Voice uses a technique based on
hidden Markov models (HMM - Hidden Markov Model). It is currently the most successful and
most flexible approach to speech recognition.

5. Kiran Kumar Lekkala and Vinay Kumar Mittal; Artificial In- telligence for Precision
movement Robot; in Proc. IEEE 2nd International Conference on Signal Processing and I
ntegrated Networks (SPIN-20 15); Amity University; Noida; India; 19-20 Feb 2015.

Proportional Integral Differential (PID) feedback systems are known for their robustness,
accuracy and stability. These systems are used in a wide variety of applications. In this paper, we
explore the possibility of using a PID architecture in robotic 3D navigation systems. The system
developed can be implemented for robotic applications that require high precision of movements
along the three dimensions. The precision of movements may be required with reference to the
user controls provided, for example in unmanned or autonomous driving systems, which in turn
require Artificial Intelligence methodologies to give the outputs such that it attains precision. An
experimental 3D precision robot is developed, in which the PID algorithm is implemented. The

Department of E&TC DPCOE PUNE Page |5

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

results of experiments conducted, confirm the effectiveness of PID controller in achieving the
high precision of robotic movements in three dimensions.

6. Lin. H. C; Lee. S. T; Wu. C. T; Lee. W. Y and Lin. C. C; Robotic Arm drilling surgical
navigation system; in Proc. International conference on Advanced Robotics and Intelligent
Systems (ARIS); Taipei; pp. 144-147; June 2014

Nowadays, there are countless surgery going on every day. The bone drilling surgical is also
quite frequent. It requires precise positioning of the drill holes in the bone to fix the screws, or it
still can be treated by surgery drilled holes. However, the drilling surgical mostly uses handheld
medical devices by surgeon. The handheld devices may cause some problems like slight
trembling of the hands and deviation of position in surgery. It would cause bleeding problems
and neurological complications. The accuracy of the drilling position is the need for long
experience and good spatial concepts. Once the drill through will hurt internal tissue and cause
serious damage. So the key to success in this type of surgery depend on the surgeon's technical
maturity. Senior surgeons also consider such surgery alone, but there is a high risk of artificially.
Whether there is a surgical navigation device that include machinery and equipment with
precision. It can stable grip drilling surgical instruments and surgical path planning capabilities
to enhance the safety and accuracy of surgery. We hope the system can assist doctors to surgery,
reduce surgical complications, and contribute minimally invasive wound healing. Let patients to
resume their daily lives early. This study mainly surgical navigation and positioning spatial
coordinate transformation in medical-assisted surgery system with remote control. It include
force feedback device coordinate space, three-dimensional medical imaging model coordinate
space and the geodetic coordinate space of the robotic arm. There are three coordinate spaces
superimposed. The system provide surgeons in preoperative surgical planning, select the surgery
route, and observe the bone structure of the site and adjacent areas. It precise control robotic arm
to reach the affected area specified, and then assist surgeons to do the drilling operations

Department of E&TC DPCOE PUNE Page |6

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

CHAPTER 3
PROBLEM STATEMENT AND DEFINATION

3.1 PROBLEM STATEMENT

Mobile controlled wireless based approach for Robotic Arm.

3.2 PROBLEM DEFINITION

Small application robotic arms are available with wired controlling section, it is important
to establish wireless communication between human and robot. For wired robotic applications
mobility and range of access is very less. As we are introducing mobile interface with different
types of inputs through IOT application, it increases accessibility of project.

Department of E&TC DPCOE PUNE Page |7

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

CHAPTER 4

4.1 BLOCK DIAGRAM

Figure 4.1: Block Diagram of Proposed System

4.1.1 POWER SUPPLY

A power supply is an electrical device that supplies electric power to an electrical load.
The primary function of a power supply is to convert electric current from a source to the correct
voltage, current, and frequency to power the load. As a result, power supplies are sometimes
referred to as electric power converters. Some power supplies are separate standalone pieces of
equipment, while others are built into the load appliances that they power. This block consists of
Transformer. For converting 230V AC to 9V AC, 9-0-9V Transformer is used. It steps down the
voltage from 230V to 9V AC.

4.1.2 NODEMC

ESP8266EX has been designed for mobile, wearable electronics and Internet of Things
applications with the aim of achieving the lowest power consumption with a combination of
several proprietary techniques. The power saving architecture operates mainly in 3 modes: active
mode, sleep mode and deep sleep mode. By using advance power management techniques and

Department of E&TC DPCOE PUNE Page |8

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

logic to power-down functions not required and to control switching between sleep and active
modes, ESP8266EX consumes about than 60uA in deep sleep mode (with RTC clock still
running) and less than 1.0mA (DTIM=3) or less than 0.5mA (DTIM=10) to stay connected to the
access point.

Figure. 4.1.2 Node MCU

Features
 Processor: L106 32-bit RISC microprocessor core based on the Tensilica Xtensa
Diamond Standard 106Micro running at 80 MHz
 Memory:
 32 KiB instruction RAM
 80 KiB user-data RAM
 External QSPI flash: up to 16 MiB
 IEEE 802.11 b/g/n Wi-Fi
 Integrated TR switch, balun, LNA, power amplifier and matching network
 WEP or WPA/WPA2 authentication, or open networks
 16 GPIO pins
 SPI
 I²C (software implementation)
 I²S interfaces with DMA (sharing pins with GPIO)
 UART on dedicated pins, plus a transmit-only UART can be enabled on GPIO2
 10-bit ADC (successive approximation ADC)

Department of E&TC DPCOE PUNE Page |9

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

4.1.3 MOTOR DRIVER

L293D is a typical Motor driver or Motor Driver IC which allows DC motor to drive on
either direction. L293D is a 16-pin IC which can control a set of two DC motors simultaneously
in any direction. It means that you can control two DC motor with a single L293D IC. Dual H-
bridge Motor Driver integrated circuit (IC). It works on the concept of H-bridge. H-bridge is a
circuit which allows the voltage to be flown in either direction. As you know voltage need to
change its direction for being able to rotate the motor in clockwise or anticlockwise direction,
Hence H-bridge IC are ideal for driving a DC motor. In a single L293D chip there are two h-
Bridge circuit inside the IC which can rotate two dc motor independently. Due its size it is very
much used in robotic application for controlling DC motors. Given below is the pin diagram of a
L293D motor controller. There are two Enable pins on l293d. Pin 1 and pin 9, for being able to
drive the motor, the pin 1 and 9 need to be high. Fordriving the motor with left H-bridge you
need to enable pin 1 to high. And for right H-Bridge you need to make the pin 9 to high. If
anyone of the either pin1 or pin9 goes low then the motor in the corresponding section will
suspend working. It’s like a switch.

Figure 4.1.3: Motor driver pin description

Department of E&TC DPCOE PUNE Page | 10

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

4.1.4 SERVO MOTOR

A servo motor is a rotary actuator or motor that allows for a precise control in terms of
angular position, acceleration and velocity, capabilities that a regular motor does not have. It
makes use of a regular motor and pairs it with a sensor for position feedback. The controller is
the most sophisticated part of the servo motor, as it is specifically designed for the purpose. The
servo motor is a closed-loop servomechanism that uses position feedback in or-der to control its
rotational speed and position. The control signal is the input, either analog or digital, which
represents the final position command for the shaft. A type of encoder serves as a sensor,
providing speed and position feedback. In most cases, only the position is reported. The final
position is reported to the controller and this is compared to the initial position input, and then if
there is a discrepancy, the motor is moved in order to get to the correct position. The simplest
servo motors use DC motors and position sensing through a potentiometer and also use big-bang
control, which means that the motor moves at maxi-mum speed until it stops at the designated
position or is stopped. This is not widely used in industrial motion control as it can be quite
inaccurate, but these kinds of servo motors are popular in radio-controlled devices such as model
aircraft and toy cars. Sophisticated servo motors for industrial use have both position and speed
sensing as well as implement proportional-integral-derivative control algorithms, allowing the
motor to be brought to its position quickly and precisely without overshooting, as the speed of
the shaft can also be controlled.

Figure 4.1.4: Servo motor

Department of E&TC DPCOE PUNE Page | 11

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

4.1.5 WIFI MODULE- ESP8266

Figure 4.1.5 ESP8266 CKT

The ESP8266 WiFi Module is a self contained SOC with integrated TCP/IP protocol stack
that can give any microcontroller access to your WiFi network. The ESP8266 is capable of either
hosting an application or offloading all Wi-Fi networking functions from another application
processor. Each ESP8266 module comes pre-programmed with an AT command set firmware,
meaning, you can simply hook this up to your Arduino device and get about as much WiFi-
ability as a WiFi Shield offers (and that’s just out of the box)! The ESP8266 module is an
extremely cost effective board with a huge, and ever growing, community.

This module has a powerful enough on-board processing and storage capability that allows it to
be integrated with the sensors and other application specific devices through its GPIOs with
minimal development up-front and minimal loading during runtime. Its high degree of on-chip
integration allows for minimal external circuitry, including the front-end module, is designed to
occupy minimal PCB area. The ESP8266 supports APSD for VoIP applications and Bluetooth
co-existance interfaces, it contains a self-calibrated RF allowing it to work under all operating
conditions, and requires no external RF parts.

Department of E&TC DPCOE PUNE Page | 12

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

There is an almost limitless fountain of information available for the ESP8266, all of which has
been provided by amazing community support. In the Documents section below you will find
many resources to aid you in using the ESP8266, even instructions on how to transforming this
module into an IoT (Internet of Things) solution.

Features
 Processor: L106 32-bit RISC microprocessor core based on the Tensilica Xtensa
Diamond Standard 106Micro running at 80 MHz
 Memory:
 32 KiB instruction RAM
 32 KiB instruction cache RAM
 80 KiB user-data RAM
 16 KiB ETS system-data RAM
 External QSPI flash: up to 16 MiB is supported (512 KiB to 4 MiB typically included)
 IEEE 802.11 b/g/n Wi-Fi
 Integrated TR switch, balun, LNA, power amplifier and matching network
 WEP or WPA/WPA2 authentication, or open networks
 16 GPIO pins
 SPI
 I²C (software implementation)

4.1.6 ANDROID APP

An Android app is a software application running on the Android platform. Because the Android
platform is built for mobile devices, a typical Android app is designed for a smart phone or tablet
PC running on the Android OS. Although an Android app can be made available by developers
through their websites, most Android apps are uploaded and published on the Android Market,
an online store dedicated to these applications. The Android Market features both free and priced
apps.

Department of E&TC DPCOE PUNE Page | 13

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

Fig.4.1.6 Android app

Android apps are written in the Java programming language and use Java core libraries. They are
first compiled to Dalvik executables to run on the Dalvik virtual machine which is a virtual
machine specially designed for mobile devices. Developers may download the Android software
development kit (SDK) from the Android website. The SDK includes tools, sample code and
relevant documents for creating Android apps. Novice developers who simply want to play
around with Android programming can make use of the App Inventor. Using this online
application, a user can construct an Android app as if putting together pieces of a puzzle.

Applications
These are the basics of Android applications:
• Android applications are composed of one or more application components
(activities, services, content providers, and broadcast receivers)
• Each component performs a different role in the overall application behavior, and
each one can be activated individually (even by other applications)
• The manifest file must declare all components in the application and should also
declare all application requirements, such as the minimum version of Android
required and any hardware configurations required

Department of E&TC DPCOE PUNE Page | 14

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

• Non-code application resources (images, strings, layout files, etc.) should include
alternatives for different device configurations (such as different strings for
different languages)

Department of E&TC DPCOE PUNE Page | 15

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

CHAPTER 5
METHODOLOGY

Robotic arm communicated with user via WiFi to the WiFi module installed on the control
system of robotic arm. When a user sends a command on the application, input signal or
controlling signal is given from the paired Android smart phone and then sent via IOT to the
robotic arm. The data from the smart phone is sent in the form of ASCII format in order to be
received by the WiFi module and passed on to NodeMCU for further operation. The NodeMCU
detected the input of signal and converted them into command, the robot then checked the
command with its previously defined commands and the servo motors acts accordingly
depending on the command received from the microcontroller by moving forward, backward,
left, right or to stop. Thus, a smart phone Android operated robot is created to perform lifting
tasks.

In order to validate the robot arm and its component, few tests were carried out which included
testing both components and the overall robotic arm system. By varying the position of the
objects which need to be lifted by the robotic arm, the servo motor movement range is tested.
Different direct impulses to each servo motor are sent by giving a command through smart
phone. This can helps to verify the response of the servo motor whether it can moves to the right
position according to the command given by the user. This process occurred when servo motors
interpreted the signal from microcontroller through encoder which resulted in the rotation to the
desired position. The initial and final position are marked to rate the accuracy of the actuator. For
overall system performance, maximum load which able to be lifted by the robotic arm is
determined using different weights. During the test, the robotic arm picked up the weight and
moved it to a particular position.

Department of E&TC DPCOE PUNE Page | 16

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

Figure 5.1: Robotic Arm 3D view

5.1 SYSTEM MODEL

5.1.1 ROBOTIC ARM:

A robotic arm is a type of mechanical arm, usually programmable, with similar functions to a
human arm the arm may be the sum total of the mechanism or may be part of a more
complex robot. The links of such a manipulator are connected by joints allowing either rotational
motion (such as in an articulated robot or translational (linear) displacement. The links of the
manipulator can be considered to form a kinematic chain. The terminus of the kinematic chain
of the manipulator is called the end effector and it is analogous to the human hand. The end
effector, or robotic hand, can be designed to perform any desired task such as welding, gripping,
spinning etc., depending on the application. For example, robot arms in automotive assembly
lines perform a variety of tasks such as welding and parts rotation and placement during
assembly. In some circumstances, close emulation of the human hand is desired, as in robots
designed to conduct bomb disarmament and disposal.

Department of E&TC DPCOE PUNE Page | 17

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

Figure 5.1.1: Robotic Arm

5.1.2 IOT:
A thing in the internet of things can be a person with a heart monitor implant, a farm animal with
a biochip transponder, an automobile that has built-in sensors to alert the

driver when tire pressure is low or any other natural or man-made object that can be assigned an
IP address and is able to transfer data over a network. Increasingly, organizations in a variety of
industries are using IoT to operate more efficiently, better understand customers to deliver
enhanced customer service, improve decision-making and increase the value of the business.

History of IoT
Kevin Ashton, co-founder of the Auto-ID Center at MIT, first mentioned the internet of things in
a presentation he made to Procter & Gamble (P&G) in 1999. Wanting to bring radio frequency
ID (RFID) to the attention of P&G's senior management, Ashton called his presentation "Internet
Department of E&TC DPCOE PUNE Page | 18
 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

of Things" to incorporate the cool new trend of 1999: the internet. MIT professor Neil
Gershenfeld's book, When Things Start to think also appearing in 1999, didn't use the exact term
but provided a clear vision of where IoT was headed. IoT has evolved from the convergence of
wireless technologies, micro electromechanical systems (MEMS), micro services and the
internet. The convergence has helped tear down the silos between operational technologies (OT)
and information technology (IT), enabling unstructured machine-generated data to be analyzed
for insights to drive improvements. The first internet appliance, for example, was a Coke
machine at Carnegie Mellon University in the early 1980s. Using the web, programmers could
check the status of the machine and determine whether there would be a cold drink awaiting
them, should they decide to make the trip to the machine. IoT evolved from machine-to-machine
(M2M) communication, i.e., machines connecting to each other via a network without human
interaction. M2M refers to connecting a device to the cloud, managing it and collecting data.
Taking M2M to the next level, IoT is a sensor network of billions of smart devices that connect
people, systems and other applications to collect and share data. As its foundation, M2M offers
the connectivity that enables IoT.

Figure 5.1.2: IOT

Department of E&TC DPCOE PUNE Page | 19

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

5.2 FLOW CHART

Figure 5.2: Flow Chart

Department of E&TC DPCOE PUNE Page | 20

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

CHAPTER 6
IMPLEMENTATION DETAILS

6.1 HARDWARE DETAILS

1. NodeMCU.
2. Servo Motors
3.Power Supply
4.Robotic Arm 4 DOM

6.2 SOFTWARE DETAILS

1. Arduino IDE 1.8.5.
2. MySQL database.
3.Android studio.

6.2.1 Arduino IDE

Figure 6.2.1: Arduino IDE

Department of E&TC DPCOE PUNE Page | 21

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

Arduino IDE is a lightweight, cross-platform application that introduces programming to

novices. It has both an online editor and an on-premise application, for users to have the option
whether they want to save their sketches on the cloud or locally on their own computers.While
Arduino IDE is highly-rated by users according to ease of use, it is also capable of performing
complex processes without taxing computing resources.With Arduino IDE, users can easily
access contributed libraries and receive up-to-date support for the latest Arduino boards, so they
can create sketches that are backed by the newest version of the IDE.

Multi-Platform Application

Arduino IDE works on the three most popular operating systems: Windows, Mac OS, and Linux.
Aside from that, the application is also accessible from the cloud. These options provide
programmers with the choice of creating and saving their sketches on the cloud or building their
programs locally and upload it directly to the board.

Board Management

Arduino IDE comes with a board management module, where users can select the board they
want to work with at the moment. If they wish to change it, they can do so easily from the
dropdown menu. Modifying their selection also automatically updates the PORT infos with the
data they need in relation to the new board.

Straightforward Sketching

With Arduino IDE, users can create programs called sketches that are built with a text editor.
The process is a straightforward one though it has several bells and whistles that make the
experience more interactive.

Project Documentation

Arduino IDE offers programmers the option to document their projects. This function allows
them to keep track of their advancements and any changes they make every time. Apart from
that, documentations allow other people to easily employ the sketches to their own boards.

Simple Sketch Sharing

Department of E&TC DPCOE PUNE Page | 22

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

Aside from saving and archiving sketches and uploading them to the board, Arduino IDE is also
capable of sharing sketches (available only on the cloud version). Each sketch is given its own
unique URL that users can share with their colleagues and fellow Arduino hobbyists. The
recipient then has access to the code; they can save it in the cloud sketchbook or download it for
their own use.

Vast Library

Arduino IDE has more than 700 libraries integrated. These were written and shared by members
of the Arduino community that other users can utilize for their own projects without having to
install anything. This enables programmers to add a different dimension to their sketches.

Third-Party Hardware Support

While Arduino IDE is designed specifically for Arduino boards, it also supports connections
with third-party hardware. This makes the use of the application more extensive rather than
limited to proprietary boards.

6.2.2 MYSQL DATABASE

MySQL is an Oracle-backed open source relational database management system (RDBMS)
based on Structured Query Language (SQL). MySQL runs on virtually all platforms,
including Linux, UNIX and Windows. Although it can be used in a wide range of applications,
MySQL is most often associated with web applications and online publishing. MySQL is an
important component of an open source enterprise stack called LAMP. LAMP is a web
development platform that uses Linux as the operating system, Apache as the web server,
MySQL as the relational database management system and PHP as the object-oriented scripting
language. (Sometimes Perl or Python is used instead of PHP. Originally conceived by the
Swedish company MySQL AB, MySQL was acquired by Sun Microsystems in 2008 and then by
Oracle when it bought Sun in 2010. Developers can use MySQL under the GNU General Public
License (GPL), but enterprises must obtain a commercial license from Oracle.

Department of E&TC DPCOE PUNE Page | 23

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

Figure 6.2.2: MySQL

HOW MYSQL WORKS

MySQL is based on a client-server model. The core of MySQL is MySQL server, which handles
all of the database instructions (or commands). MySQL server is available as a separate program
for use in a client-server networked environment and as a library that can be embedded (or
linked) into seperate applications. MySQL operates along with several utility programs which
support the administration of MySQL databases. Commands are sent to MySQLServer via the
MySQL client, which is installed on a computer. MySQL was originally developed to handle
large databases quickly. Although MySQL is typically installed on only one machine, it is able to
send the database to multiple locations, as users are able to access it via different MySQL client
interfaces. These interfaces send SQL statements to the server and then display the results.

CORE MYSQL FEATURES

MySQL is written in C and C++ and accessible and available across over 20 platforms, including
Mac, Windows, Linux and Unix. The RDBMS supports large databases with millions records
and supports many data types including signed or unsigned integers 1, 2, 3, 4, and 8 bytes long;
Department of E&TC DPCOE PUNE Page | 24
 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

FLOAT; DOUBLE; CHAR; VARCHAR; BINARY; VARBINARY; TEXT; BLOB; DATE;

TIME; DATETIME; TIMESTAMP; YEAR; SET; ENUM; and OpenGIS spatial types. Fixed-
and variable-length string types are also supported.

6.3 ANDROID STUDIO

Figure 6.3: Android Studio

Android Studio is the official integrated development environment (IDE) for Android
application development. It is based on the IntelliJ IDEA, a Java integrated development
environment for software, and incorporates its code editing and developer tools. To support
application development within the Android operating system, Android Studio uses a Gradle-
based build system, emulator, code templates, and Github integration. Every project in Android
Studio has one or more modalities with source code and resource files. These modalities include
Android app modules, Library modules, and Google App Engine modules.

Android Studio uses an Instant Push feature to push code and resource changes to a running
application. A code editor assists the developer with writing code and offering code completion,

Department of E&TC DPCOE PUNE Page | 25

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

refraction, and analysis. Applications built in Android Studio are then compiled into the APK
format for submission to the Google Play Store. The software was first announced at Google I/O
in May 2013, and the first stable build was released in December 2014. Android Studio is
available for Mac, Windows, and Linux desktop platforms. It replaced Eclipse Android
Development Tools (ADT) as the primary IDE for Android application development. Android
Studio and the Software Development Kit can be downloaded directly from Google.

The following features are provided in the current stable version:

 Gradle-based build support

 Android-specific refactoring and quick fixes
 Lint tools to catch performance, usability, version compatibility and other problems
 ProGuard integration and app-signing capabilities
 Template-based wizards to create common Android designs and components
 A rich layout editor that allows users to drag-and-drop UI components, option to preview
layouts on multiple screen configurations
 Support for building Android Wear apps
 Built-in support for Google Cloud Platform, enabling integration with Firebase Cloud
Messaging (Earlier 'Google Cloud Messaging') and Google App Engine
 Android Virtual Device (Emulator) to run and debug apps in the Android studio.

Android is a mobile operating system developed by Google. It is based on a modified version

of Linux kernel and other open source developed software, and is designed primarily for touch
screen mobile devices such as smart phone and tablets. In addition, Google has further developed
Android TV for television, android Auto for Cars and Wear OS for Wrist Watches, each with a
specialized user interface. Variants of Android are also used on games console, digital cameras,
PCs and Other electronics.

Initially developed by Android Inc., which Google bought in 2005, Android was unveiled in
2007, with the first commercial android devices launched in September 2008. The operating
system has since gone through multiple major releases, with the current version being 9 "Pie"
Department of E&TC DPCOE PUNE Page | 26
 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

released in August 2018. The core Android source code is known as Android Open Source
Project (AOSP), and is primarily licensed under the Apache License.

6.3.1 System Requirements

Department of E&TC DPCOE PUNE Page | 27

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

CHAPTER 7
APPLICATIONS& ADVANTAGES

7.1 APPLICATIONS
1. Home application based controlled robot.
2. Pick and place robotic arm applications.
3. Semi-automated industrial robotic arms.

7.2 ADVANTAGES
1. It reduces human efforts.
2. Accuracy and security is increased due to the use of IOT.
3. Maximum mobile devices can be interface to control single ARM.
4. Due to number of input modes its is easy to code so many commands for same robotic
arm.

Department of E&TC DPCOE PUNE Page | 28

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

CONCLUSION
In this proposed system we will control 4 DOM Robotic Arm from android mobile app with
three different types of inputs like voice input, hand gesture input, phone tilt input.

Department of E&TC DPCOE PUNE Page | 29

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

FUTURE SCOPE
Concept of human-robot interaction can further be implemented in Artificial Intelligence in
which different tasks are performed by robotic arm will based on deep learning. The robotic arm
will perform more functions than just pick and place task.

Department of E&TC DPCOE PUNE Page | 30

 MOBILE CONTROLLED WIRELESS BASED APPROCH
FOR HUMAN ROBOT INTERACTION

REFERENCES
[1] Keitaro Naruse and Makoto Oya, “Muscle-like Control of Entertainment Robot over
Internet”, International Conference on Advanced Intelligent Mechatronics, Monterey, California,
USA, 24-28 July, 2005.

[2] Tse Min Chen and Ren C. Luo,” Multisensor Based Autonomous Mobile Robot Through
Internet Control”, Department of Electrical Engineering, National Chung Cheng University.

[3] Kishan Kumar; Shyam Nandan; Ashutosh Mishra; Kanv Ku-mar and V. K. Mittal; Voice-Controlled
Object Tracking Smart Robot; in Proc. IEEE 3rd International Conference on Signal Processing;
Computing and Control 2015 (ISPCC 2015); JUlT; Waknaghat; India; 24-26 Sep 2015.

[4] B. Raghavendhar Reddy; E. Mahender; Speech to Text Con-version using Android Platform; I
nternational Journal of Engi-neering Research and Applications (IJERA) ISSN: 2248- 9622
www.ijera.com Vol. 3; Issue I; January -February 2013; pp.253-258.

[5] Wong Guan Hao; Y. Y. Leck and Lim Chot Hun; 6-DOF PC-Based Robotic Arm PC-ROBOARM
with efficient trajectory planning and speed control; in Proc. 4th International Confer-ence Mechatronics
(ICOM); Kuala Lumpur; pp. 1-7; April 2011; ISBN:978-1-61284-435-0

[6] Lin. H. C; Lee. S. T; Wu. C. T; Lee. W. Y and Lin. C. C; Robotic Arm drilling surgical navigation
system; in Proc. International conference on Advanced Robotics and Intelligent Systems (ARIS); Taipei;
pp. 144-147; June 2014

[7] Kiran Kumar Lekkala and Vinay Kumar Mittal; Artificial In- telligence for Precision movement
Robot; in Proc. IEEE 2nd International Conference on Signal Processing and I ntegrated Networks
(SPIN-20 15); Amity University; Noida; India; 19-20 Feb 2015.

Department of E&TC DPCOE PUNE Page | 31