NeRF: The Future of Reality Mapping?

It’s impossible to talk about the future of construction project monitoring without considering the impact artificial intelligence will continue to have on the rapidly-changing construction, engineering, and inspection industries. In this article about the future of neural radiance fields (NeRF), we’ll zero in on this form of AI, what’s so promising about it, and why—at least for now—photogrammetry remains the best option for construction stakeholders who conduct remote visual progress monitoring. 

Key Takeaways:

• NeRF is a form of AI that produces a 3D representation of a scene using limited 2D images.
• In the future, NeRF may become a significant component of a “holy grail” of AI technologies that help automate many aspects of construction project monitoring. 
• For now, photogrammetry (reality mapping) remains the gold standard for creating a 3D twin of a construction site for remote project monitoring. 

What is NeRF?

NeRF technology produces 3D (and sometimes 4D) representations of objects, structures, or scenes using only 2D images. 

NeRF vs. Photogrammetry

While both photogrammetry and NeRF generate 3D representations of real-world objects or structures, they achieve this in different ways. Photogrammetry consumes 2D footage (or footage from a 360 camera) and “reconstructs” those images into a 3D model by directly solving for 3D points that correspond to matching features across the images. The most advanced photogrammetry engines offer reality mapping features that make these 3D models (also known as online as-builts or digital twins) measurable down to a fraction of an inch and do not require users to upload any floorplans or similar data to get started. 

NeRF is a newer technology that uses cutting-edge machine learning models to encode color and opacity at every point and orientation in the 3D scene, such that the input images can be perfectly reproduced by the model. Photogrammetric 3D models can produce accurate geometry for well-textured matte surfaces but often have holes or bad geometry in textureless or glossy surfaces. NeRF models can produce photorealistic images, even for textureless and reflective surfaces, but model the geometry more loosely.

Related: Reality Capture & Reality Mapping for Construction Job Sites

Top features and benefits of NeRF

NeRF has promising features and benefits that could eventually impact how construction stakeholders perform various tasks and to-dos, including: 

• Interpolation: NeRF produces very nice, high-quality visuals—so long as you do not drift too far from the camera's original “capture position” when “walking through” the NeRF model.

• Stabilization: NeRF also generates excellent, smooth results from once-wobbly footage captured via a mounted camera (e.g., a smartphone mounted to a hard hat on shaky terrain.)
• Video Compression: NeRF can compress a video into a tiny model, often achieving a 50x compression without losing realism.

Still, NeRF technology is extremely new—and when it comes to construction progress monitoring, nearly all of NeRF’s most promising benefits remain “potential” benefits, not ones that could be reaped today. More exciting future benefits of NeRF for progress monitoring may include:

• Viewing the exact same scene from the exact same viewpoint at different times. Because NeRF is so good at interpolation, it could eventually allow stakeholders to perfectly compare—from the exact same viewpoint—how one scene looked at different times, even if you had no timelapse camera or if you had captured footage from a slightly different viewpoint every time you walked a job site.

• Smarter, semantic maps of job sites. Potentially, NeRF can encode 3D representations of many scene properties beyond color, such as object and surface identities and material properties. That way, you’ll be able to search for things in a scene using text, see exactly where that object is in 3D, and then count and measure the objects. 

Ultimately, once NeRF is working alongside other AI technologies like large language models (LLM; i.e., ChatGPT) and recognition, the trio could become a holy grail of automated project monitoring that provides a full assessment of construction completion. 

Related: The Future of AI Project Monitoring in Construction

Current limitations of NeRF

As previously mentioned, NeRF is a relatively new technology, and while research and interest are vast, the majority of NeRF’s most promising benefits aren’t ready for their formal rollout. For now, photogrammetry remains the gold standard for producing 3D digital twins for remote construction monitoring. 

Some of the biggest limitations of NeRF are:

• Poor measurability. For now, NeRF models are not precisely measurable—whereas those produced by top-notch reality mapping engines are. 
• Poor scalability. Today, creating a good NeRF model from several hundred images could take hours. And if you wanted to create a larger model of a city or even the floor of a building, you’d have to break it up into multiple different processes because the processing would take too much memory and compute. Photogrammetry engines can process large scenes rapidly and with ease. 

While NeRF does have some practical use cases today, we’re not seeing it in construction monitoring because the technology simply isn’t ready. But when it is—and it almost certainly will be soon—NeRF will be everywhere.