[مشروع خاص]Gunpla and oversized characters appear! Take augmented reality photos with Object Capture on iPhone – HOBBY Watch

The iOS operating system installed on iPhone is updated year after year, and various new features are added every year. “iOS 17” was just released on September 19 this year, and the new feature “Object Capture” included in iOS 17 has quietly attracted attention.

Object Capture is a function that allows you to scan objects with your iPhone and convert them into data as 3D models. Originally a job for developers, using Object Capture to create 3D models from plastic models and figures, you can make 3D models appear anywhere and capture AR images.

In this article, we will present how to use the Object Capture feature in iOS 17 to take augmented reality photos. We’ll show you the appeal of Object Capture, which allows you to easily create 3D models of your plastic models and figures and take life-sized photos.

Requires iPhone 12 Pro or later Pro Series. Easy scanning by simply turning around the target 3 times!

First, before using Object Capture, let’s check if your iPhone is compatible with Object Capture. Object Capture can be used on an iPhone or iPad equipped with a sensor called a LiDAR Scanner. Specifically, the compatible devices are the iPhone Pro series starting from the iPhone 12 Pro released in 2020, and the iPad Pro released in 2021 onwards.

Although it has the Pro name, the “iPhone 11 Pro” released in 2019 does not have a LiDAR Scanner, and the “iPad Pro (A12Z equipped model)” released in 2020 does not have a LiDAR Scanner. . Please note that this is not supported by the chip it is equipped with. Additionally, this verification uses the iPhone 14 Pro Max released in 2022.

From the specifications page for the latest “iPhone 15 Pro” model. The sensor called “LiDAR Scanner” is the core of “Object Capture”.

Specifications page for the latest “iPhone 15” model. Non-Pro iPhones don’t have a LiDAR scanner, so they don’t support object capture.

This time I used my iPhone 14 Pro Max.

It’s a bit hard to tell in the photo, but the black circle opposite the lens when looking from the flash is the ‘LiDAR Scanner’.

If you have a compatible device, please ensure that the iOS installed is the latest version of iOS 17 (iOS 17.0.2 as of September 28). If you have not updated it yet, please consider updating it according to your use case.

If you own an iPhone Pro series from 2020 or later and need the latest iOS 17, the hurdles are a bit high, but if you meet those conditions, move on to the next step. Install an “Object Capture” compatible app from the App Store. Object Capture is installed in iOS 17 as an interface called API. However, Apple’s native Camera app does not support Object Capture, so you need to download a compatible app from the App Store.

Several apps that support Object Capture have already been released on the App Store. This time, we used Pictorial Analogy, which supports Japanese and has a simple user interface. It’s a decent app that’s free to download from the App Store, and although it shows a small ad, there are no in-app purchases.

You can check the installed iOS from “General” → “Info” → “iOS Version” in the Settings app.

This time we used Photogrammetry as a compatible app with Object Capture.

Simple menu screen. Only a small ad is displayed, and there are no additional charges.

I immediately started using Photogrammetry to create 3D models of plastic models and figures. You can create a new project by pressing the “+” button at the top of the list, and a field for entering the name will appear. This time, we will create a 3D model to the specifications of the YOASOBI collaborative poster of the “HG 1/144 Gundam Aerial” plastic model from the anime “Mobile Suit Gundam: Witch of Mercury”.

Select the name and press Create to launch the camera. When I edited the item I wanted to scan in the displayed window and pressed Continue, the Gundam Aerial was surrounded by a square frame. This is the frame that defines the area to be scanned, and it is necessary to adjust the length, width and height of this frame so that the Gundam Aerial can be fitted inside. It’s very easy to understand, it’s displayed in 3D and you can see if it’s out of frame by moving your iPhone.

After setting the frame, press “Start Capture” to start scanning. A bulleted list-like 3D model appears immediately, but I don’t care and move my iPhone slowly, focusing on the “Gundam Aerial.” At this time, it will warn you if the iPhone is too close to the body or if the brightness is insufficient, so follow the on-screen instructions and scan.

Moreover, when scanning, it is necessary to “hold the iPhone itself” and walk around the model or plastic form. In addition to making sure you have as much space as possible, you also have to be careful because if your iPhone is fixed, you won’t be able to scan it even if you rotate the plastic form.

The first scan ends when you wander into the “Gundam Aerial”. Next, a message will appear saying “This object is suitable for shooting in a different direction,” and this time the object will be rotated and scanned once while lying on its back. After the second scan, the robot will scan again while lying on its stomach, and after three turns, the scan will be complete.

The Gundam Aerial was able to change its orientation, such as lying on its back or lying on its stomach, but with plastic models and figures that have sensitive parts, you may want to avoid turning it sideways. In this case, you can select “Capture more at different heights” to scan from a lower or higher position the first time without changing the direction. Damage to plastic models and figures is something owners want to avoid, so this is a relief.

Also, this time I used a photography box while scanning, but normally a photography box or green screen wouldn’t be necessary. Even if other objects are included in the scan, they will be cropped automatically, creating a high-resolution 3D model. Of course, if you unify the background with one color, a better 3D model will be created.

The first scan is completed. Perform the second and third scans by changing the object orientation or scanning height.

In the case of “Gundam Aerial”, the direction was changed and scanning was performed.

The point cloud displayed on the screen is interesting

Once the scan is complete, your iPhone will take care of the rest. When you click “Create Model” after scanning, the process will automatically proceed in the following order: Preprocessing → Image processing → Point cloud creation → Mesh generation → Texture mapping → Finishing. The entire process, from three scans to creating the 3D model, took about 15 minutes.

However, creating a 3D model from a scan is cumbersome, which greatly reduces remaining battery power and causes the iPhone itself to overheat. We recommend avoiding scanning on the go and creating a 3D model in advance at home, as it is easier to charge and cool.

The created 3D model is saved in a folder called “Model” from “This iPhone → Photogrammetry” in the native iOS “File” app. Let’s take a look at the 3D model created for the “HG 1/144 Gundam Aerial”.

The “Gundam Aerial” scan has been completed! When you click Create Form…

Your iPhone will handle the rest automatically.

The 3D model created from File can be checked in the native iOS app.

Makes you want to go out on the town and take “augmented reality photos”! Easy 3D design for your iPhone hobby

Next, we’ll take photos using augmented reality while checking out the 3D model we’ve created. The 3D model of the previously mentioned “Gundam Aerial” is saved with the “.usdz” extension, and by simply clicking on the saved “Gundam Aerial.usdz” in the “File” app, it will be automatically downloaded to iOS “App called” AR Quick Look will open “.

With AR Quick Look, you can choose to view the created 3D model in virtual space or use AR (augmented reality) to make it appear in the real world through a display screen. It’s fun just looking at the 3D model of the “Gundam Aerial” from different angles, but it’s the “AR” that shows its true value.

A native “File” app for iOS.The 3D model of the previously mentioned “Gundam Aerial” is saved with the extension .usdz.

This is the 3D model of the “Gundam Aerial” that I scanned earlier.

Although it’s a bit difficult, I’m amazed that this 3D model can be created in about 15 minutes.

Of course, since it’s a 3D model, you can rotate it.

Through augmented reality, the “Second Gundam Aerial” that appears in the real world is so realistic that it can be compared to the real thing. Of course, there are differences depending on the light source and environment when scanning, so it is possible to tell the difference to some extent. However, in my case, I had several “accidents” while shooting where I tried to touch the 3D model through the screen to change the angle of the antenna, only to discover that it wasn’t actually there.

All you have to do is press the shutter button and take a photo. Moreover, the displayed 3D model can be enlarged with augmented reality. Although there is a limit of 1000% (= 10x), the height of the “HG 1/144 Gundam Aerial” is about 1 meter and 40 cm, so it seems possible to take pictures of the “Gundam Aerial” that blend with the real world. . .

From here, I will not only create a 3D model of the previously mentioned “HG 1/144 Gundam Aerial,” but I will also create 3D models of various figures in my house using “Object Capture,” and then create captured AR images using “Preview Quick AR” while walking around. Introduction with pictures. Using this function, you can take photos that look like characters from different cartoons and games that appear in your daily life, which is very fun.

However, unlike regular camera apps, AR Quick Look has a bit rough image quality, and the color tone of the scanned 3D model cannot be easily corrected, so the 3D model may float depending on the scene. Additionally, since you can’t place two or more 3D models at the same time, it’s a little difficult to recreate the battle scenes. I look forward to future updates regarding this area.

On the other hand, if a 3D model is placed in an area where the LiDAR scanner cannot measure distance, it is sometimes possible to capture an AR image at a larger than 1000% scale, which increases the sense of realism. The images presented here should be viewed with the above points in mind. Please note that the settings in action may not match the actual size, so please forgive us.

The ‘Object Capture’ app lets you create 3D models of plastic models and shapes by just downloading one app, and then capture the augmented reality images as they are. In the past, 3D scanning required large hardware, but now it’s easy to be able to do it all with a single iPhone.

Many people have probably dreamed of making this plastic figure or model larger than life size. Why not create a 3D model of your hobby using iOS 17 x iPhone and take photos of your favorite people going about their daily lives?

A 3D model being photographed. If you looked at it from the side, you might think you were photographing nothingness.