Now Reading
Modifying 3D Scenes with Directions

Modifying 3D Scenes with Directions

2023-03-22 23:49:03

We suggest a way for modifying NeRF scenes with text-instructions. Given a NeRF of a scene and the
of photographs used to reconstruct it, our technique makes use of an image-conditioned diffusion mannequin (InstructPix2Pix) to
iteratively edit the enter photographs whereas optimizing the underlying scene, leading to an optimized 3D
scene that
respects the edit instruction. We reveal that our proposed technique is ready to edit large-scale,
scenes, and is ready to accomplish extra lifelike, focused edits than prior work.

Our technique regularly updates a reconstructed NeRF scene by iteratively updating the dataset photographs whereas
coaching the NeRF:

  1. A picture is rendered from the scene at a coaching viewpoint.
  2. It’s edited by InstructPix2Pix given a worldwide textual content instruction.
  3. The coaching dataset picture is changed with the edited picture.
  4. The NeRF continues coaching as ordinary.

Instance updates to the coaching dataset photographs over time. Discover that the edits are regularly turning into extra

“Make it appear to be autumn”

“Make it appear to be autumn”

See Also


For those who use this work or discover it useful, please take into account citing: (bibtex)

         writer = {Haque, Ayaan and Tancik, Matthew and Efros, Alexei and Holynski, Aleksander and Kanazawa, Angjoo},
         title = {Instruct-NeRF2NeRF: Modifying 3D Scenes with Directions},
         journal = {arXiv preprint arXiv:XXXXXX},
         yr = {2023},

We thank our colleagues for his or her insightful suggestions useful discussions, particularly Ethan Weber, Frederik
Warburg, Ben Poole, Richard Szeliski, Jon Barron, Alexander Kristoffersen, Rohan Mathur, Alejandro Escontrela,
and the Nerfstudio group.

Source Link

What's Your Reaction?
In Love
Not Sure
View Comments (0)

Leave a Reply

Your email address will not be published.

2022 Blinking Robots.
WordPress by Doejo

Scroll To Top