Hololabo Nishimatsu Construction Company Uses XR Technology to Develop Drone Support and 3D Model Superimposition Technology on Aerial Footage – IoT News

In recent years, the use of 3D models (CIM models) has increased during the construction phase on construction sites, allowing sharing of construction condition images, early identification of problems, suggesting improvement measures, and preventing construction rework. Augmented reality technology is attracting attention.

However, current AR technology mainly uses smartphones, tablets, head-mounted displays, etc., and the viewpoint is limited to the ground. Additionally, in order to check multiple sites, it was necessary to move around the site and align the augmented reality at each site.

Therefore, Hololab Co., Ltd., in joint research with Nihimatsu Construction Co., Ltd., has developed a technology that superimposes a 3D model on aerial footage captured by a drone in real time, in conjunction with a display mounted on The head, visualizes the construction work and supports the drone operation.

This technology uses drones capable of high-speed movement to overlay 3D models. This eliminates the need to align the AR at each site. In addition, by using the drone's position and camera information, markerless augmented reality is achieved.

In addition, the design's latitude and longitude information is used to place the 3D model, and it is possible to place it directly on the registered coordinates selected from the web browser.

Furthermore, we have developed a system that uses Trimble's transparent head-mounted XR display

A spatial head-up display (HUD) is displayed on the XR10's displays, as well as overlaid AR aerial footage, a 3D model control UI that can be operated with manual tracking, shooting functions, and various states of the drone in flight to check out.

In addition, the drone's flight location can be displayed on a map displayed using augmented reality based on location information, eliminating the need to switch camera directions for verification.

Moreover, in order to avoid the burden of processing the AR display affecting the control of UAVs and XR devices, these calculations are processed on high-performance computers within the field network. By sharing images from a computer over the on-site network, we achieved real-time high-resolution viewing and connection speed while reducing the burden on each device.

In addition, by linking to a teleconferencing system, it is now possible to share aerial footage with remote locations over the Internet.

As a result, pilots can now check multiple pieces of information at the same time while operating the drone, such as aircraft status, flight position, and 3D model composition.

In addition, when Nishimatsu Construction Company conducted a field test at the construction site of an under-construction dam, it was confirmed that it was possible to understand the situation over a wide area and reduce the time.

However, problems are also found, such as the difference in overlay accuracy due to the difference in transmission speed and display speed while the drone is moving.

In the future, in order to solve these problems, we plan to continue development aimed at improving accuracy, improving the system and improving comfort.