INSTITUTE OF INDUSTRIAL ELECTRONICS ENGINEERING PROJECT NAME Localization and 3D Mapping for Indoor Environment FINAL YEAR PROJECT PRESENTATION Muhammad Hammad khan (18021) Syed Mazhar Hussain Rizvi (18043) Syed Murtaza Hussain (18045) Arsal Abbasi (17008) PROJECT SUPERVISOR Engr Asif Ahmed Memon PROJECT CO-SUPERVISOR Engr Agha Mohammad Shakir 1 Presentation Overview Introduction Problem Statement Possible Solutions Proposed Solution Objectives Literature Review Methodology 1. System Model 2. Detail of the Hardware and Software 3. System Analysis and Specifications Results and Discussion 1. Simulation Results 2. Experimental Results 3. Comparison between Simulation and Experimental Results Conclusion and Recommendations References 2 Introduction The importance of robotics is rapidly increasing as reflected by the number of available robotic systems present in industrial, commercial, educational and research facilities. During the last 50 years, robotics research has been aimed at finding solutions to the technical issues raised in multiple domains of human society. The evolution of sophisticated robotic solutions has been witnessed all around the global communities. Robot in Industry [www.azorobotics.com] Robot in warehouse [www.ehstoday.com] Robot in Healthcare [www.nobel-project.eu/] Robot in Education [www.slashgear.com]3 Introduction The earlier developments of robotic systems have been solely emerged for industrial robotic solutions. A research article “A Brief History of Industrial Robotics in the 20th Century”, published in August 2019 states that the word robot would be assigned to a system that performs physical tasks automatically with some intelligence. The article described three generations of industrial robots till the year 1999 when this field got saturated. The First Generation of Industrial Robots (1950-1967). The Second Generation of Industrial Robots (1968-1977). The Third Generation of Industrial Robots (1978-1999). 4 Introduction to Surveying and Mapping Robots Exploring unknown environment and to build its 3D representation is one of the most focused multi-disciplinary research field which has been evolved through the usage of Simultaneous Localization and Mapping (SLAM) technique. Surveying and mapping the environment is one of the recent topic for the robotic researchers and started to evolve since last decade. There are different varieties of mapping robots available which can do the surveying job in indoor and outdoor environments. An indoor surveying robot may be developed by using a laser scanner and odometry setup supported by a single board computer. But on the other hand, an outdoor surveying robot becomes a complex setup of sensors supported by a high-speed computer. 5 Surveying and Mapping Robots 6 Problem Statement In recent years researchers developed several solutions for simultaneous localization and mapping (SLAM), however, 3D real-time map development and scanning of a large area is still a big challenge and requires autonomy of the system by involving multiple robotic units. Increasing the number of robotic units for large area mapping creates several challenges, like the power consumption, data transfer and the determination of the exact lidars transformation and exact joint together them for real-time 3D mapping. Therefore over all system exceeds the acceptable cost with high power consumption solution for 3D surveying and real-time mapping. 7 POSSIBLE SOLUTIONS We can 3d laser scanners like velodyne .Velodyne is approx 2,200$. We can also use SICK-TIM510 but the issue is they are very high cost. SICK-TIM510 is approx. 1,800$. The other one solution is the usage of 2D RP-Lidar A1 for low cost 3D mapping. 8 POSSIBLE SOLUTIONS 9 POSSIBLE SOLUTIONS The one another solution is to make the servo motor stand which rotates the 2D lidar on x-y and z axis through this we can capture 3D map of the whole environment. 10 PROPOSED SOLUTION Our proposed solution is meant to design a “SLAM BASED 3D MAP BUILDING USING 2 LOW COST 2D LASER SCANNERS (RPLIDAR A1)”and perform motion in real-time from the initial position to the final destination (surveyed region). Main theme of “SLAM BASED 3D MAP BUILDING USING 2 LOW COST 2D LASER SCANNERS (RPLIDAR A1)” is an orthogonal combination of two RP-Lidar 2D laser scanners which we will used on slam’s vehicle to build real-time 3D point cloud map of the explored region. The interfacing of scanners, transformation joints, zero degree transformation and the processing of simultaneous localization and mapping (SLAM) technique will achieve using Robot Operating System (ROS). 11 PROPOSED SOLUTION The developed 3D point cloud map will further utilize to establish Building Information Model (BIM) to found validity for the surveyed region. Simulated model Real model 12 Objectives The main objective is to make the real-time 3D Map development of the corridor using ROS. The existing solution are costly and not economically feasible to the local market of the country. Therefore, the main motivation of our project is to develop an economical autonomous mapping solution acceptable to the local market. 13 Objectives To accomplish the desired project goals the overall work has been divided into multiple objectives as follows: Fabrication of small mapping rover. Characterizing and selection of 2D scanners. Simulated and Real robot transformation Lidar’s joints, plugin and it’s transformations on real-time bases. Real-Time Map development of the environment by rover. 14 A Rover Team Based 3D Map Building Using Low Cost 2D Laser Scanners This paper presents a solution to develop 3D map of indoor place using a team of rovers equipped with low cost 2D laser scanners. They used uwb 1000 for sensing 2nd rover another thing they used identity marks on both rover and read them of 1st rover to 2nd and 2nd rover to 1st . Both rovers make the map and stop at meeting point by reading identity mark. Researchers get the lidars data of both rovers wirelessly. Then the researchers process data offline in MATLAB for 3d map merging using some equations. The rover equipped with the on board computer(raspberry pi) ,2 lidars, uwb1000. The results of research is given in below slide. 15 A Rover Team Based 3D Map Building Using Low Cost 2D Laser Scanners 16 Ground Robot Mapping Infrastructure Ground Robot Mapping Infrastructure GRoMI is composed of two major parts, a hybrid laser scanning system and an autonomous mobile robot platform The upper laser scanning system part, which is for collecting 3D mapping information, consists of Five SICK 2D laser scanners A built-in digital single-lens reflex (DSLR) camera The lower mobile robot part, which is for collecting localization data, has four wheels with encoders, object avoidance sensors, an Inertial Measurement Unit (IMU), and a navigation camera 17 (a) raw scan data from LiDAR (b) incrementally-built 2D map of the environment (c) final recovered trajectory with position and orientation of robot 18 • Once the robot finds the best static scan position, • it calculates the coordinate transformation from the previous static scan position to the current position using SLAM localization data • and then moves to a next static scan position. • Once the scanning process is completed at the last scan position, • the final registered point cloud is simultaneously generated. • Figure shows the registered RGB-mapped point clouds for the test environment 19 Methodology Designing and Development of Rover Based 3D Scanning System. Simulated and Real robot transformation Lidar’s joints, plugin and it’s transformations on real-time bases. Check visualization in Rviz (the debugging tool of ROS). Check tf-tree broadcasting to check data is publishing and subscribing correctly. Lidar zero-degree transformation. Implementation of customize real-time 3d map development package on ROS. Its name is IRA laser tools. Rover Instrumentation and control. Real-Time 3D Map Development using Rover. 20 System Model Designing and Development of Rover Based 3D Scanning System: The motive of our project is to scan the environment using two 2D scanners and develop a 3D model of the scanned vicinity. The experimental setup is consisting of A rover, equipped with an appropriate control system and instrumentations as discussed later A communication system. Computer. We are using a vehicle its fabrication as discussed later. This vehicle were installed with a control system and sensors with support of additional mechanical attachments. The communication system we are using is simple a Wifi network between our rover and pc. Our pc which is a laptop is installed with Robot Operation System (ROS), and some packages we required for data accusation and control. 21 The CAD model The CAD model of the experimental setup is shown in Figure. The model is developed on Autodesk Fusion 360, an online CAD model development software. The model developed was further used for simulation in the ROS environment. The model is indicating the desired connections of two orthogonal RPLIDARs . Rover model of Sim & Exp setup 22 Selection of Rover The rover we selected is customize 4WD mobile platform model. There are two sections where we can mount our additional hardware. These sections may be called as lower section and upper section 23 Modifications in Rover Additionally, for the support of LIDARs, an acrylic stand in orthogonal shape has been designed and fabricated. Two surfaces have been developed on the stand Rover with support for RPLIDARs 24 Modifications in Rover We use the 1000 Rpm encoder motors to get the feedback from encoder for correct odometry of the rover in the environment. Odometry from encoders 1000 Rpm 12V Dc gear motor with encoder. 25 Real Rover Real Rover with using acrylic sheet 26 Software Details and Control Simulated and Real Rover Transformation: To visualize the movement of the robot in Rviz we must have to do the robot parts transformation for broadcasting and make joints for visualize the movement of the rover. 27 Software Details and Control TF-Tree and Topics Publishing by Frames: To check the transformation that broadcasting of the frames is perfect or not we make TFTree of our rover. 28 Software Details and Control Worlds Modeling Models for Indoors Environments: Occupancy Grid-based Map Feature Based Maps Topological Maps Point-Features Based Map 29 Software Details and Control Occupancy Grid-based Map: In 1980, Moravec and Elfes introduced this type of map. They used a sonar sensor for data collection from the environment, with a measuring range of 0.9 to 35 feet The advantage of occupancy grid maps lies in the fact that they do not rely on any predefined features One of the major drawbacks of the occupancy grid map is the consumption of memory 30 Software Details and Control Two-dimensional Occupancy Grid used: (a) 2D laser scanner for mapping. (b) Fused sensor data for mapping. 31 Software Details and Control Implementation of IRA Laser Tools (The Real Time) Package: IRA Laser Tools is a customize package which we use here to merge both (vertical and horizontal) Lidar readings and publish into a single point cloud. It is used to hold the real sample or reading which comes from both Lidars to make 3D Real time map of the whole environment. 32 Software Details and Control Lidar’s Pose Update: The working of Ira-laser tools is that is merge the both lidar (vertical and horizontal) and produce single point cloud another is important think is that its transformation use to update the the pose of lidar w.r.t odometry. Map to odom transformation is used to update the map using odometry. If we don’t do this issue in visualization in Rviz. 33 Hardware Details and Control Rover Instrumentation and control: 34 Hardware Details and Control Rover Connectivity and control: 35 SYSTEM ANALYSIS AND SPECIFICATIONS Lidar Zero-Degree Transformation: The objective of lidar’s zero degree transformation is to reduce length of lidar’s frame (reduction in the lidar’s frame, compact in size). Another objective is to increase the flexibility in lidar. 36 SYSTEM ANALYSIS AND SPECIFICATIONS Performance Evaluation and Selection of 2D laser scanners: There are various 2D laser scanners available that have been designed to operate in numerous environments and provide the benefit of non-contact sensing of the surveyed region. Based on general features of scanners such as compactness interfacing cost Following three 2D laser scanners specifications are as follows: Hokuyo URG-04LX Hokuyo UTM-30LX RPLidar A1 37 SPECIFICATION OF 2D LASER SCANNERS Parameter URG-04LX RPLIDAR A1 UTM-30LX Maximum measurement 4 6 30 Measurement error ±10mm up to 1m range; ±50 mm ±30mm for 0.1 to 10m (mm) 1% of measurement range, ±50mm for 10 to value for 1-4m range 30m range range (m) Scanning angle (deg) 240 360 270 Angular resolution (deg) 0.36 ≤1 0.25 Scanning time 100 180 25 (ms/cycle) 38 Results Rover Based Localization and Mapping: we are using two RPLIDARs. One is for horizontal scanning and the other is for vertical scanning One LIDAR scanning horizontally is being used for 2D SLAM The 2D map has been developed using hector SLAM. There is no requirement for odometer data from wheels in Hector SLAM The package matches the current scan data with previous data and estimates the value for u As a result, an occupancy grid map of the environment is generated, along with the related pose of the rover 39 Results Real-Time Point Cloud Generation: For taking the data and generation of real-time point cloud an experiment has been performed with a rover at the 3rd-floor corridor in the IIEE. It is clear that the environment has different pillars and walls, and the path is E-shape. The rover was controlled through a wifi network using teleop twist keyboard package. 40 Simulated Results Simulated Map Generation: In Ros there are 2 tools which are widely use. Gazebo (a simulation plateform). Rviz (the debugging tool). 41 Simulated Results 2D Grid Map Generation: The data of both scanners, were also transmitted using the same network. The SLAM package used a Horizontally mounted scanner and estimated the 2D map and pose in the simulated corridor. The result of these two values is shown in Figure Red arrows shows the odometry of the simulated rover at every movement. 42 Simulated Results Real-Time 3D PointCloud Generation: Real-Time point cloud generation of simulated corridor as shown in given video. 43 Simulated Results Real-Time 3D PointCloud Generation: 44 Hardware Results 2D Grid Map Generation: Similarly in hardware data of both scanners, were also transmitted using the same network. The SLAM package used a Horizontally mounted scanner and estimated the 2D map and pose in the real corridor. In the given 2D grid map of the corridor the is little bar of extra scan shows that the room is open. Green line shows the trajectory of the real rover which comes from odometry. 45 Hardware Results Real-Time 3D PointCloud Generation: Real-Time point cloud generation of real corridor as shown in given figures. 46 Hardware Results Real-Time 3D PointCloud Generation: 47 Hardware Results Comparison Between Simulation an Hardware Results: Simulation : The total volume of the simulated corridor developed by the rover is 27.7 x 2 x 3.2 in meters. in simulated 2D map there is no bad scanning present. Simulated rover weight is about 1.5 KG so the linear and angular motion of the motors are perfect due to less load. The total operation completed by the simulated rover is about 5 min. Experimental: The total volume of the real corridor developed by the rover is 27.37 x 1.92 x 3.04 in meters. in real 2D map there is some bad scanning present at one place. Real rover weight is about 4.5 KG so the linear and angular motion of the motors are good but not perfect due to enough load. The total operation completed by the simulated rover is about 10 min. 48 Conclusion This project has presented the efficient working of a scanning and mapping system comprising of a low cost rover. The working of the system provides ease of operation at affordable rates. The successful Real-Time map development technique has been utilized to show the use of rover for the surveying application of comparatively large vicinities. 49 References Asif Memon, Syed Riaz un Nabi Jafri, Syed Muhammad Usman Ali,” A Rover Team based 3D Map Building using Low Cost 2D Laser Scanners”. https://www.researchgate.net/publication/357363038_A_Rover_Team_based_3D_Ma p_Building_using_Low_Cost_2D_Laser_Scanners PileunKim,JingdaoChen ,Yong K.Cho,“SLAM-driven robotic mapping and registration of 3D point clouds”. https://www.sciencedirect.com/science/article/abs/pii/S0926580517303990 Zheng Fang , Shibo Zhao, Shiguang Wen, and Yu Zhang, “A Real-Time 3D Perception and Reconstruction System Based on a 2D Laser Scanner”. https://www.hindawi.com/journals/js/2018/2937694/ David M. Cole and Paul M. Newman,“Using Laser Range Data for 3D SLAM in Outdoor Environments”. https://www.robots.ox.ac.uk/~mobile/Papers/3DScanMacthingCole_ICRA2006.pdf 50 Refrences P. Lepej, M. Lakota and J. Rakun,”Robotic real-time 3D object reconstruction using multiple laser range finders”. https://www.cambridge.org/core/journals/advances-in-animalbiosciences/article/abs/robotic-realtime-3d-object-reconstruction-usingmultiple-laser-range-finders/B7168728D5D093B9492BCEBA4F49C038 Kyeong-Hwan Lee, Reza Ehsani,“Comparison of two 2D laser scanners for sensing object distances, shapes, and surface patterns”. https://www.sciencedirect.com/science/article/abs/pii/S0168169907001834 Jan Weingarten and Roland Siegwart,“EKF-based 3D SLAM for structured environment reconstruction”. https://ieeexplore.ieee.org/document/1545285 Peter Zhang, Evangelous Millos & Jason Gu,“General Concept of 3D SLAM”. https://www.researchgate.net/publication/221786534_General_Concept_of_3 D_SLAM#:~:text=Simultaneous%20localization%20and%20mapping%20(SL AM,map%20that%20includes%20feature%20locations. 51 52 53