SINERGI Vol.
No.
June 2018: 77-82 DOAJ:doaj.
org/toc/2460-1217 DOI:doi.
org/10.
22441/sinergi.
HUMAN FOLLOWING ON ROS FRAMEWORK
A MOBILE ROBOT
Gigih Priyandoko1.
Choi Kah Wei2.
Muhammad Sobirin Hendriyawan Achmad3 1Department of Electrical Engineering.
Faculty of Engineering.
Universitas Widyagama Jl.
Borobudur.
No.
Malang.
Jawa Timur 65142 2Faculty of Mechanical Engineering.
University Malaysia Pahang (UMP).
Pekan 26600 Pahang.
Malaysia 3Faculty of Engineering.
Universitas Teknologi Yogyakarta, 55285 Sleman.
Yogyakarta.
Indonesia Email: gigih@widyagama.
id, kahwei93@gmail.
com, hendriyawanachmad@gmail.
Abstract -- Service mobile robot is playing a more critical role in today's society as more people such as a disabled person or the elderly are in need of mobile robot assistance.
An autonomous person following ability shows great importance to the overall role of service mobile robot in assisting human.
The objective of this paper focuses on developing a robot follow a person.
The robot is equipped with the necessary sensors such as a Microsoft Kinect sensor and a Hokuyo laser sensor.
Four suitable tracking methods are introduced in this project which is implemented and tested on the person following algorithm.
The tracking methods implemented are face detection, leg detection, color detection and person blob detection.
All of the algorithms implementations in this project is performed using Robot Operating System (ROS).
The result showed that the mobile robot could track and follow the target person based on the person movement.
Keywords: ROS.
Mobile Robot.
Autonomous Received: March 4, 2018
INTRODUCTION
The autonomous mobile robot is a relatively young field, and there is numerous potential and space for development and improvement (Linder and Arras, 2016.
Reddy and Kumar, 2016.
Specian et al.
, 2.
In recent years, mobile robot has gained great success in autonomy as researchers and companies all around the world have put great emphasis and attention on achieving a high level of autonomy.
The improve in robotics capabilities have place robots in a new environment such as replacing humans in unpleasant environments, such as doing repetitive works or operating in highly radioactive zones.
These are great examples of how mobile robots have started to coexist with us and assist human effectively (Tomoya et al.
Wunderlich et al.
, 2017.
Xiao et al.
, 2.
Service robots are also becoming more popular than ever as they play an important role in assisting and helping with our daily tasks.
The ability of a mobile robot to follow the user is an important function since robots must interact with The potential applications of following autonomous system such as autonomous autonomous luggage, etc.
An autonomous shopping cart will be able to follow a shopper during shopping period and returning to starting position autonomously after the shopper left (Shimizu et al.
, 2.
An autonomous wheelchair can be utilized in hospitals where the Revised: April 27, 2018 Accepted: April 27, 2018 patients who seat in the wheelchair can follow the This wheelchair is especially crucial during busy periods when medical assistants are not enough.
Autonomous luggage can be developed for many air-travelers and can be especially useful for a person who carries much baggage, which reduce the burden when moving around.
Autonomous following system is a huge potential and plays a huge role in the service industry where human and robot interaction is numerous.
The system developed must be able to environment to work in unusual condition.
The mobile robot can detect and track people with minimum instruction is an achievement of human-robot interaction.
For example, people with disabilities can benefit enormously from such computer autonomy if the system is mature and highly independent.
The autonomous mobile robot can assist and service blind people in mobility (Gunardi and Wibowo, 2015.
Cai and Matsumaru, 2014.
Lee et al.
, 2.
Older adults who have impaired memory or cognitive problems can make use of such companion when moving around outside (Tomoya et al.
, 2017.
Wunderlich et al.
, 2017.
Oliver et al.
, 2.
There are various other useful applications of such system besides assisting the needy such as guiding for visitors, campus mobility, and item Thus, a robot that can recognize and follow a human autonomously is a key challenge Priyandoko, et al.
Human Following on ROS Framework for a Mobile Robot SINERGI Vol.
No.
June 2018: 77-82 to help the robot to improve further and enhance human-robot interaction effectively.
The objective of this paper, to develop a mobile robot to follow and track a human effectively.
METHODOLOGY
The implementation of this research requires both hardware and software setup.
The software setup involves the installing and configuring the operating system and ROS The hardware setup comprised of setting up the sensors and base controller, starting up the system, configuring the hardware drivers and establishing remote networking.
For software setup (Cousins, 2.
ROS must first be installed on top of Ubuntu Linux.
this project, the version of ROS used is Indigo which is installed in Ubuntu 14.
Ubuntu 04 is chosen because it has long-term support (LTS) and is proven to work well with ROS Indigo.
After installing ROS, a catkin workspace is set up so that ROS packages can be built at this personal workspace and is easier to manage.
The data being processed can be visualized using a tool in ROS called Rviz, which is in the form of graphical user interface (GUI).
Rviz can display several features that are essential to person detection and trackings, such as RGB image, mono image, depth information and point cloud.
The project will be based on a mobile robot platform that is attached with the required sensors for this project.
Other components such as the base controller for the motor and the base motor are setup correctly.
The base controller is an Arduino Mega that can receive velocity commands from ROS through the rosserial driver which will be discussed later.
The Arduino will then send signals of pulse width modulation (PWM) of linear and angular velocity to the differential drive motor.
For the sensor part.
Microsoft Kinect and Hokuyo LRF are used in this The Kinect is connected to the 8000 mAh rechargeable battery source.
The sensors on the robot all can be connected via USB cable to send and receive ROS message data.
Once USB connection of the sensors and base controller to the laptop is attached and secured in place, the power of the robot can be turned on via the emergency switch behind.
Two computers are needed for remote operation and control.
Remote operation can also perform visualization across the wireless network which is one of the advantages of using ROS.
ROS made the connection of multiple fleets of robots possible where the computer at the remote station is the master computer .
r Host in ROS terminolog.
, and the robot computer is the slave computer .
r Master in ROS terminolog.
Robot to follow a person, two important components need to take place first, which is person detection and person tracking.
The system starts detecting the interesting point by scanning the surrounding.
The detected interest points are passed on to the tracking algorithm for tracking to reduce overload on the computation that is required in detection stage.
The following algorithm receives the interest points or region of interest and establishes it as the goal to follow.
The person following system continues to follow the tracked person until the target is lost, which goes back to the initialization stage to detect the interest points again.
The loop continues until the system is terminated.
Mobile Robot In this research, the mobile robot is utilized, which commonly used by researchers in addition to the Ackermann model.
The mobile robot has two independent wheel drive system that can be controlled independently, and it has a turning radius more varied than the Ackermann, even capable of rotating exactly on the center of gravity .
of the robot body.
So the mobile robot has the maneuverability to avoid obstacles better than the Ackermann model in term of narrow space.
The real robot is shown in Fig.
which accommodates two input, linear velocity and angular velocity toward the target position.
The base controller of the robot can translate the twisted message from the main controller contains linear and angular velocity as set points into the speed of wheels for both of the right and the left side as shown in Fig.
Fig.
describes the diagram of base controller system as ROS node.
The base controller node subscribes linear and angular velocities in the form of geometry_msgs/Twist as ROS standard message from the other ROS node as remote operator .
Simultaneously, nav_msgs/ Odometry is published by the base controller, which is consist of x,y,z (=.
obtained from the The odometry message from the base controller can be used for pose estimation together with laser and visual odometry for a better result.
ROS geometry_msgs/Twist is translated into right and left velocities of the wheels as set points for two PID controllers.
Afterward, the PID controllers maintain linear and angular velocities of the mobile robot toward the target position as desired values.
Robot Operating System (ROS) A ROS is an open source software framework primarily based on UNIX platform for Priyandoko, et al.
Human Following on ROS Framework for a Mobile Robot
ISSN: 1410-2331
operating robots.
Many ROS robotics types of research have been presented (Goldhoorn et al.
Jafari et al.
, 2014.
Leigh et al.
, 2015.
Linder and Arras, 2016.
Munaro and Menegatti.
Ocando et al.
, 2017.
Tomoya et al.
, 2017.
Wunderlich et al.
, 2017.
Xiao et al.
, 2017.
Cousins, 2.
ROS has three levels of concepts: the file system level, the computation graph level, and the community level.
ROS
provides the service user would expect from an operating system, including hardware layer low-level implementation of commonly-used functionality, message-passing between processes, and package management.
It also provides tools and libraries for obtaining, building, writing, and running code across multiple platforms.
The primary goal of ROS is to support code reuse in robotics research and development.
ROS is a distributed framework of processes .
that enables executables to be individually designed and flexible at runtime.
These processes can be divided into stacks and packages, which can be easily shared and distributed.
ROS also supports a federated system of code Repositories that enable collaboration to be distributed as well.
Fig.
3 shows the basic concept of ROS with distributed system capabilities.
ROS Master served as a major control the entire distribution of messages within the Each node will register the identity to the ROS Master include the IP address, and the name of the topic will be published or subscribed.
When ROS nodes subscribe to a particular topic.
ROS nodes will contact ROS master to get the IP address of other ROS nodes which publish the associated topics.
The name of the topic is unique, so ROS nodes should not publish the topic with the same name in the same ROS Relationships between ROS nodes not merely publish and subscribe, but also clientserver relationship.
ROS node can ask the service .
to the other nodes to do something and then get the results .
without processing it themselves.
It indicates the advantages of ROS system associated with the concept of distribution of information, to facilitate users to build a robotic system that has a complicated mission with a simple solution.
RESULTS AND DISCUSSION
Face Detection The results of the detection ability are visualized using the Rviz tool as shown in Fig.
The detection results are relatively good due to the OpenCV that provides three levels of cascade for detecting front faces and side faces.
When the first cascade fails, the algorithm will move to the second cascade until the face is The image processing and face detection continue to run in a loop when no face is detected.
When detection is positive, the interest points projected on the face is processed by the tracking algorithm to track the movement of the points and publish the processed data as a region of interest.
However, there are few limitations of the face detector if it is to be applied in a real application.
Figure 1.
The mobile robot model used in this study Priyandoko, et al.
Human Following on ROS Framework for a Mobile Robot SINERGI Vol.
No.
June 2018: 77-82
VR setpoint
PID
Motor Driver
,O) subscribe Encoder
Wheel
Encoder
right actual speed
ROS
BASE NODE
Wheel
left actual speed
VL setpoint
PID
Motor Driver .
Figure 2.
The mobile robot base controller system connected to the ROS environment ROS
MASTER
Node.
Node.
ROS Node.
/topic Node.
Node.
ROS Node.
Figure 3.
ROS basic concept Figure 4.
Face detection of the mobile robot Figure 5.
Person following performance evaluation Table 1.
Results of Following Performance Evaluation Person Following Method Face Detection Leg Detection Colour Detection Max Range Sharp Turn Test Average CPU Load (%) Following Efficiency (%) 5m, 57o Pass 6m, 240o Yes Failed Pass 5m, 57 One of the limitations is that the face detection requires high computational power to detect a face effectively due to RGB-D image processing that contains high pixels need to be processed through OpenCV and provide the detection results.
There is usually a lag time of 3 to 5 seconds for a face to be detected based on mid-level 60GHz Occlusion Test Another limitation of face detection is that the camera level of the robot must be tilted upwards so that it can perceive the face.
If this detector is used on lower platforms, the detection results may not be as robust and consistent.
This is especially true as most of the mobile robots were not designed up to human height.
Another shortcoming of this method to apply in person following is that the person must always face the robot to be tracked and follow.
Priyandoko, et al.
Human Following on ROS Framework for a Mobile Robot
ISSN: 1410-2331
Leg Detection The leg detection is the only method that utilizes the Hokuyo LRF.
2D laser depth data have a more precise measurement of the depth data compared to the KinectAos infrared sensor.
But in the implementation of the leg detection to follow, the detection is highly inconsistent during movement as compared to static detection.
Color Detection From the implementation of the color detection algorithm, it has the advantage of the low computational requirement for detection and tracking due to its low level of data extraction.
The color detector uses a filter to track that requires low processing compare to detection of The color detection approach can be used to detect the color of the clothing of the person to follow.
The detection is started by selecting the color region of the clothes where the filter will process according to the color histogram in the figure.
The results of the color detection and tracking are showing effectively.
Person Following Outcome The performance evaluation is conducted in the control system laboratory of Faculty of Mechanical Engineering.
The route is AoSAo shape around two workbenches for a distance of 32 The evaluation is conducted indoor due to the limitation of the Kinect sensor that will cause an error in infrared sensing due to direct sunlight interference.
The sharp-turn test is conducted on the four approaches to evaluate the tracking algorithm and sensors performance on a sharp turn.
During turning, the rotational speed of the wheel is important.
Therefore optimum parameters for angular velocity is set for the test.
The test is conducted by the same person for few times to confirm the results.
The ability to follow and turn for four consecutive 90degree turns can prove the following robustness instead of just following a straight line.
The following efficiency is another important factor since the following algorithm is not guaranteed to follow hundred percent of the Certain factors that disturb the following performance includes occlusion, and sharp turn is not included in the following efficiency evaluation where the target person will take the highest consideration in making sure the tracking of the person is optimum.
For example, when taking a turn, the target person will slow down and take a smooth curve instead of a quick sharp The following robot efficiency can be computed as the distance followed divided by the planned distance.
Based on the evaluation metrics that is conducted, person blob detection is better at tracking and following a person, where the following efficiency is 100 percent.
However, it failed at occlusion test, due to the inability of the mobile robot to recognize the target person as it follows any person-like blob that is within the boundary box of detection.
However, this is not a major concern given that shorten the goal distance between the mobile robot and the target person can reduce such issue.
For other tracking methods, such as face detection and colour detection does not make it for the sharp turn test at few consecutive repetitions due to the limitation of the tracking algorithm that cannot handle high scaling factor in each successive The points of interest may not be tracked effectively during turning sessions where initialization of redetection consumes time to detect the target again.
For leg detection which uses the laser range scanner, the main failing factor is occlusion test where the detector can be confused by multiple legs, and the following control system cannot predict the centroid of the legs of the targeted person.
Therefore, these tests able to prove the effectiveness of the different methods of following a person.
The person following performance can be conducted by running a series of tests repetitively to evaluate each tracking method on the person following as shown in Fig.
Among the tests conducted are sharp turn test, occlusion test, and performance test.
With the results of these test, the tracking method can then be evaluated and decided upon leveraging the pros and cons.
Another deciding factor is the computational performance as it is an important consideration for real-life application where the cost of breakeven point with the features that it provides.
The cost of computation cannot be too high.
Therefore, we also take into consideration the runtime performance when conducting the person, the following test by using the system monitor tool in Linux.
Results of following performance evaluation is shown in Table 1.
CONCLUSION
The person was tracking, and the successfully, with works involved in determining the most suitable detection and tracking The person was tracking and following into real application, the cost of the system is not desired to be too high.
Since this project uses Kinect sensor and Hokuyo LRF, it can be concluded that Kinect sensor, while cheap, can provide good detection and tracking.
After Priyandoko, et al.
Human Following on ROS Framework for a Mobile Robot SINERGI Vol.
No.
June 2018: 77-82 evaluating the performance of a person following, the person blob following have the best ability to follow a person compared to the other methods.
Having free access to the open source platform for computer vision such as OpenCV and PCL is also another important component for this project to be conducted ACKNOWLEDGEMENTS The authors would like to thank the Faculty of Mechanical Engineering.
University Malaysia Pahang for providing laboratory facilities and financial support.
This project is supported by Universiti Malaysia Pahang under Research Grant Scheme (RDU140.
REFERENCES