Classic Photogrammetry with an iPhone

Photogrammetric 3D scanning can turn a bunch of regular photos into a 3D model or a scene. So, we only need a camera and some knowledge of how to take photos. In the previous article, we tested the latest iPhone camera with a few different apps. Now that we have some experience and control of our camera, we can dive into 3D photogrammetry, which essentially is just taking a lot of photos.

Use Cases and Limitations

Photogrammetry is limited by what you can reliably capture. The first and most obvious factor is accessibility. If you can not reach it with your camera and capture it, we can not scan it. A slightly less obvious factor is subject texture, or rather, the lack thereof. Photogrammetry works by matching the same visual features between all of the photos and calculating depth.

If there are no features to match or those features change between shots, photogrammetry will not work. In practice, this means that featureless, clean, single-colored, reflective, or transparent objects are impossible to scan without some kind of treatment.

If what you are taking photos of does not fall into these categories, you can get a digital copy of it. There are many reasons for getting a 3D model of some real-world object or scene, from serious preservation, monitoring, reproduction, or measurement use cases to just making 3D models for the memes. Whichever category you might fall into - the fundamentals of photo capture are the same.

Technique

This being a photogrammetry blog, we have discussed the technique of capturing numerous times. Some of the articles best suited for phone photography - 3D scanning cases are these:

Smartphone Photogrammetry for Stockpile Scanning
3D Asset Creation With Photogrammetry
Manual Referencing in Pixpro - Scale Without GPS

But if you want an abridged version, here are the bullet points:

• You need to take photos of the same thing from different perspectives for photogrammetry. So, when scanning something, you can go around and take a picture of every step. Scale that for small objects to less than a step, and a few steps around large objects like houses. Your goal is to make a lot of overlapping photos of the same object, with the technical overlap percentage being 80 percent. In practice, that means moving 1 fifth of the view between each image, but we do not need to be that precise.

• This reiterates the first point: do not take photos without physically moving. Pointing a camera in a different direction from the same spot DOES NOT COUNT. Again, there are strictly no photos from the same place.

• Ensure good image quality. This means no motion blur, no defocus blur, and not too dark or too bright. Photos for photogrammetry have to be technically good.

• Ensure that what you want to scan is covered with photos. So, keeping in mind the first three points, you need to cover your subject with those three points in mind. Every little spot you need to scan has to be clearly visible in multiple photos; I would say at least 10. We can use a simple rule: if a thing is not visible clearly in at least 10 photos, it may not be reconstructible in 3D.

To summarize, photogrammetry is about taking many photos of the same subject from different heavily overlapping perspectives, while those photos are technically sound and cover your subject or area plentifully. Once you get the hang of it, it will become just a matter of going around things and shooting hundreds of photos instinctively.

Iphone Specific Quirks

In the previous article, we discussed how an iPhone works for someone who has picked up one for the first time after using an Android device for years and with no intention of going to Apple for desktop needs. It was quite an experience with some mid-boggling setbacks, like using a special tool to copy the photos to a PC. Since then, I have discovered a few more quirks that may need attention. This may get a bit deeper and more advanced at some point:

1. While third-party apps allow for more control, the default camera app is best optimized. This intuitively encourages taking more photos and working faster. The difference is not extreme, but it is noticeable, and when making comparisons, it may skew the results a little.

2. Depending on the settings, the phone can produce RAW files, which can be interpreted extremely differently by a raw converter; in my case, I use Adobe Lightroom Classic. The phone has a compression setting that allows you to choose JPEG XL compression, which applies to the raw files. The lossless one saves you space at no cost, which is a nice feature. However, different resolution options (48 vs 12 megapixels) will produce drastically different-looking photos in Lightroom even without any adjustments being applied. So, my advice is to use the same app and file format settings for each scan. This way, it is easy to adjust all relevant photos at once if you shoot raw for maximum quality and consistency.

3. Not all apps allow all kinds of format and resolution options. More specifically, some apps do not have the option to produce jpegs at 48-megapixel resolution. Lumina, the app I recommended in the previous article, does have this option, and it is pretty good, in my opinion.

4. When viewing the examples from the last article on the web, with all the compressions applied, the difference between different resolutions and raw files is less noticeable than when looking at the photo natively on a computer. I still maintain that choosing 48 megapixels or raw formats does give a noticeable but slight increase in image quality. However, in photogrammetric scanning, this slight increase in quality is not that important and relevant only when all other factors are perfectly executed, and we are maximizing the output's quality. I would not dwell on resolutions and formats until you can call yourself a 3D scanning enthusiast. 

Examples

I have included a few examples, which you have already seen above. We processed the photos using Pixpro photogrammetry software with minimal manual intervention, like using a clipping box and cleaning up point clouds.

These are pretty low-effort, fun, and quick scans, obtained by just strolling around town, looking for more interesting objects during record-breaking warmth for a January in Lithuania.

You can do this yourself right now, no matter what phone you might have. The iPhone has great camera hardware. Many Android phones have as good or even better cameras without the bad aftertaste of Apple's default camera app or money-grubbing app store. And even entry-level smartphones these days have cameras that are perfectly usable for outdoor scans like these.

Conclusion

However, an iPhone has another card up its sleeve, which we conveniently left out for this article - a lidar sensor. Next, we will test that and compare it to classic photogrammetry. As you can see, photogrammetric 3D scanning is easy and accessible to everyone. Still, that lidar module may lower the bar of entry even more. I highly doubt that, but we will see. For now, go out and take a bunch of photos.