Neuralangelo by Nvidia

Neuralangelo is an AI model created by NVIDIA Research for 3D reconstruction using neural networks. This new tool can transform 2D video clips into detailed 3D structures, generating virtual replicas of various real-world objects, such as buildings, sculptures, and more. The generated 3D objects have intricate details and textures, and creative professionals can import them into design applications to further edit them for use in a wide range of industries, from video game development to industrial digital twins. Neuralangelo's ability to translate textures of complex materials, including roof shingles, panes of glass, and smooth marble, from 2D videos to 3D assets surpasses previous methods.The model's high fidelity simplifies the task of rapidly creating usable virtual objects for projects using footage captured by smartphones. Neuralangelo enables developers to import highly detailed objects, from small statues to massive buildings, into virtual environments for video games or industrial digital twins, helping recreate the real world in the digital world.Neuralangelo uses instant neural graphics primitives, the same technology behind NVIDIA Instant NeRF, to capture finer details such as repetitive texture patterns, homogenous colors, and strong color variations. The model selects several frames from a 2D video, captured from different viewpoints, to create a rough 3D representation of the scene, then optimizes it to sharpen the details, similar to a sculptor refining the texture of a statue. The resulting 3D object or large-scale scene can be used in virtual reality applications, digital twins, or robotics development.