PhysDreamer: Developed in collaboration with multiple universities, including MIT, Stanford University, Colombia University and Cornell University.
Real object interactions are critical to creating immersive virtual experiences, but synthesizing real 3D object dynamics in response to novel interactions remains a major challenge. Unlike unconditional or textual conditional dynamics generation, conditional dynamics of action requires perceiving the physical material properties of the object and basing 3D motion predictions on these properties, such as object stiffness.
However, due to the lack of real data on materials, estimating physical material properties is an open issue because measuring these properties of real objects is very difficult. We propose PhysDreamer, a physics-based approach to give interactive dynamics to static 3D objects by using object dynamics priors learned from video generation models.
By refining this prior knowledge, PhysDreamer is able to synthesize the responses of real-life objects to novel interactions, such as external forces or proxy operations. We demonstrated our method on different examples of elastic objects and evaluated the authenticity of the synthetic interaction through user research.
PhysDreamer is a step towards a more engaging and realistic virtual experience by enabling static 3D objects to dynamically respond to interactive stimuli in a physically reasonable way.
If you want to learn more, you can click on the link below the video.
Thank you for watching this video. If you like it, please subscribe and like it. thank
Project page:https://physdreamer.github.io/
Huggingface：https://huggingface.co/papers/2404.13026
Github：https://github.com/a1600012888/PhysDreamer
Video: