NVIDIA researchers present a new AI model – Neuralangelo. Its ability to translate the textures of complex materials from 2D videos to 3D assets is mind-blowing!
NVIDIA has unveiled an AI model that harnesses the power of neural networks to reconstruct 3D structures from 2D videos. This groundbreaking technology opens up exciting possibilities for creative professionals in various fields.
Just like Michelangelo chiseled masterpieces from blocks of marble, Neuralangelo breathes life into virtual replicas of buildings, sculptures, and real-world objects, complete with intricate textures. Neuralangelo empowers creative professionals to import and manipulate 3D objects in design applications, allowing them to further refine and incorporate these virtual assets into their projects.
Digital Revolution with Neuralangelo
With lifelike precision and attention to detail, Neuralangelo surpasses previous methods of 3D reconstruction. It effortlessly translates the textures of complex materials found in 2D videos, such as roof shingles, panes of glass, and smooth marble, into stunning 3D assets. This advancement enables developers and artists to quickly create usable virtual objects, using footage captured by smartphones, for their artistic and practical endeavors.
“The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world,” said Ming-Yu Liu, senior director of research and co-author of the paper. “This tool will eventually enable developers to import detailed objects — whether small statues or massive buildings — into virtual environments for video games or industrial digital twins.”
NVIDIA researchers showcased Neuralangelo’s prowess in reconstructing iconic objects like Michelangelo’s David and everyday items like a flatbed truck. The model’s versatility extends beyond individual objects, as it can faithfully recreate both the interiors and exteriors of buildings. To demonstrate this, a detailed 3D model of the park at NVIDIA’s Bay Area campus was presented, highlighting the potential for immersive experiences and architectural visualizations.
Neural Rendering Model Sees in 3D
Traditional AI models struggle with accurately depicting repetitive textures, uniform colors, and intense color variations when reconstructing 3D scenes. Neuralangelo overcomes this challenge by leveraging instant neural graphics primitives, inspired by NVIDIA Instant NeRF.
The model carefully selects frames that provide diverse viewpoints by analyzing a 2D video of an object or scene from different angles. This approach, similar to an artist examining a subject from multiple perspectives, enables Neuralangelo to perceive depth, size, and shape.
Once the camera positions are determined for each frame, Neuralangelo’s AI constructs an initial 3D representation of the scene. Next, the model refines the render to enhance details, much like a sculptor meticulously chisels stone to replicate fabric textures or human forms. The outcome is a high-quality 3D object or expansive scene suitable for virtual reality applications, digital twins, or robotics development.
SOURCE: NVIDIA
Join our community & Get inspired by TOP artists:
Most popular RAW A-POSE 3D SCANS
See more on our website:
