ALIKE Creates 61km-long HD Map of Gangnam, Seoul

Seoul is a global megacity and Gangnam its most complicated area, heavily congested with cars. The city of Seoul and NAVER LABS collaborated on a demonstration project to create HD maps of this area.

Figure 1. An HD map of Gangnam, Seoul

HD maps are called the brains of autonomous vehicles as it is core data for urban autonomous driving. To process 3D high-precision data of a wide area, we need technology that is both efficient and accurate. ALIKE, which can create city-level digital twin data is such a technology. Utilizing this solution, we completed HD mapping of Gangnam, which is a total of 61km. This is a vivid report of how we developed this map.

 

0. Brief introduction of ALIKE

ALIKE is a digital twin solution that has integrated NAVER LABS’ digital twin data creation technologies (for instance, hybrid HD mapping presented at CES2019). The solution creates three outputs, ALIKE 3D (3D model), ALIKE RD (road layout), and ALIKE HD (HD map). Instead of producing each output separately, ALIKE is a solution that scans the entire city from the sky and produces each data in succession. 

Figure 2. ALIKE solution

>> Find out more about ALIKE

We previously produced a 3D model and road layout of the entire Seoul area using ALIKE 3D/RD and subsequently selected Gangnam, the most challenging area, this year to create/demonstrate HD maps using ALIKE HD. In comparison to the commonly MMS (mobile mapping system)-based HD mapping, ALIKE has the advantage of maximizing cost efficiency when mapping complex, intertwined urban roads.

 

1. Scanning an entire city

NAVER LABS scans an entire city with an airplane and creates a large-scale 3D model. This city 3D model guarantees a very high location accuracy and thus serves as a foundation for the production of road layouts and HD maps. Using a reference point we actually measured, we correct the pose and generate diverse models. Then, we validate the location accuracy by comparing the output’s location and the national reference point’s location. In this demonstration, the average of horizontal/vertical error was 2.5cm and 8.0cm respectively in comparison to 21 national reference points spread across the eastern area below the Han River.

Figure 3. 61km-long road layout of Gangnam, Seoul

Moreover, NAVER LABS can extract precise road layouts from the city 3D model. We are gradually automating road marker and lane information extraction by utilizing AI technologies such as deep learning, computer vision, etc. The completed road layout is precise enough to provide lane-level guidance.

 

2. Integration with MMS data

Figure 4. Underground roads, road markers obstructed by trees and vehicles, etc., that are difficult to detected from aerial images

However, it is difficult to extract overpasses/roads below tunnels or lanes/markers obstructed by tress/vehicles using aerial images alone. Besides, we need to consider how often we will update the latest information. Figure 5 is one of the roads that went through drastic road marker changes—this road near Gangnam Renaissance Hotel was under construction at the time of the aerial images, and was changed significantly following completion (changes in crosswalk and marker location). The road expanded from six lanes to seven lanes as well.

Figure 5. Comparison of aerial images and latest road markers

NAVER LABS uses R1, an internally-developed MMS vehicle to overcome these challenges.

Figure 6. R1, NAVER LABS’ internally-developed MMS

However, if we only use MMS, we cannot guarantee location accuracy with inertial navigation systems (INS) alone. Thus, in ALIKE, we integrated various information to maintain data accuracy. This approach also mitigates boundary issues that could occur in data boundary regions when mapping very large areas over multiple days.

 

3. The problem of excessive cars and buildings in Gangnam

As mentioned before, Gangnam was the most difficult challenge for us. First, there are too many vehicles. Road markers were often obstructed in aerial images and moving vehicles interfered with data collection when logging with MMS. During the day, it was especially difficult to acquire pocket lane road data as many vehicles were waiting on the lane to turn left. Figure 7 compares LiDAR point cloud data collected during the day and night for the same area.

Figure 7. Top: daytime logging, Bottom: nighttime logging

For buildings, their density has a significant impact. For example, the area below Nambusunhwan-ro, shown in figure 8, is relatively less dense. There is no section that experiences GNSS satellite obstruction for prolonged periods of time. Thus, we were able to acquire satisfactory results just from the driving trajectory automatically generated by INS location-based mapping pipeline alone.

Figure 8. The area below Nambusunhwan-ro


On the other hand, GNSS location accuracy decreases rapidly when an MMS vehicle enters an area with high building density. The top image in figure 9 shows raw GNSS data obtained while passing through multiple glass buildings. We can correct the MMS vehicle’s precise movements to some extent by integrating accelerometer, gyroscope, and wheel encoder (which counts the number of revolutions) values. However, if the GNSS position, which serves as the standard for the entire area’s location, shows a large error over a long time as in the top image of figure 9, correction is not possible. The error in the final location increases due to incorrect prediction that the vehicle is further away than in reality, due to satellites being obscured by skyscrapers and receiving signals reflected off glass walls or scattered due to collision with tree leaves.

Figure 9. Top : driving through glass buildings, Bottom : driving below Nambusunhwan-ro

We are solving this problem by internally-developing an annotation tool within ALIKE, that corrects GNSS errors that occur when driving through areas with a high building density. Based on the recent demonstration, we have recently began research on technologies that can automatically detect GNSS errors.  

Figure 10. Image of area with high building density (glass buildings)
 

4. Processing point cloud data and vertical facilities

Through the aforementioned process, we have prepared for point cloud data processing which was obtained by the MMS vehicle. Point cloud data contains scanned information of everything that existed around the vehicle at the time it was driven. It also includes data from moving objects such as vehicles or pedestrians. These data need to be deleted, as they should not be included in maps. Who should work on this? The size of the 3D data and UI would be too challenging for humans to manually carry out deletion. Therefore, we use semantic segmentation algorithms to semantically categorize all individual points. For instance, this point represents a car, that point represents the road surface, etc. These algorithms automatically categorize and delete moving objects.

Figure 11. Identifying and deleting moving objects using semantic segmentation algorithms

We also have to delete noise in the data. As LiDAR sensors measures the reflection of a laser, it can incorrectly detect illusions appearing in midair, created by glass or mirror reflections. We again use algorithms to delete this noise.

However, the amount of noise we must process is immense. The size of the point cloud data is proportional to the size of the area mapped. Thus, we need to optimize codes to deal with the massive data. Otherwise, size difference of the data causes various problems. We need a code that could more efficiently use memory and computational resources. Validation of this was one of the major accomplishments of this demonstration project.

Furthermore, vertical facilities (street lights, traffic signs) need to be located appropriately and categorized. Because it is not easy to detect vertical facilities from the point cloud data, we extract information from images. As tree branches and other objects often obscure traffic signs, we refine the final blind spots from the point cloud data.

 

5. Epilogue

Until now, we briefly introduced the HD mapping process of Gangnam, Seoul using NAVER LABS’ solution ALIKE. With our original solution, we created highly efficient and highly precise HD maps. 

Although roads in Gangnam only amount to 1/33 of all the roads in Seoul, we faced an abundance of new issues and updates as it was the most challenging environment for us to map. Considerations on climate variables like the monsoon season, efficient driving trajectory planning for R1 (MMS vehicle) accordingly, sensor design upgrades, etc. could only be obtained from work in the field.

Currently, many countries and cities around the world are planning to create digital twin data such as HD maps. NAVER LABS has developed the most original digital twin solution and is quickly advancing technology through collaborations, such as with the city of Seoul. We will strive continuously to maintain this technological predominance.

 

 

Related Articles