2 code implementations • ICML 2020 • Jie Xu, Yunsheng Tian, Pingchuan Ma, Daniela Rus, Shinjiro Sueda, Wojciech Matusik
Many real-world control problems involve conflicting objectives where we desire a dense and high-quality set of control policies that are optimal for different objective preferences (called Pareto-optimal).
Multi-Objective Reinforcement Learning
reinforcement-learning
+1
no code implementations • 27 Jan 2025 • Beichen Li, Rundi Wu, Armando Solar-Lezama, Changxi Zheng, Liang Shi, Bernd Bickel, Wojciech Matusik
Procedural materials, represented as functional node graphs, are ubiquitous in computer graphics for photorealistic material appearance design.
no code implementations • 5 Nov 2024 • Kevin Tirta Wijaya, Minghao Guo, Michael Sun, Hans-Peter Seidel, Wojciech Matusik, Vahid Babaei
In this work, we introduce MoleVers, a versatile pretrained model designed for various types of molecular property prediction in the wild, i. e., where experimentally-validated molecular property labels are scarce.
no code implementations • 26 Oct 2024 • Wenqiang Chen, Jiaxuan Cheng, Leyao Wang, Wei Zhao, Wojciech Matusik
This paper presents Sensor2Text, a model proficient in tracking daily activities and engaging in conversations using wearable sensors.
1 code implementation • 5 Oct 2024 • Gang Liu, Michael Sun, Wojciech Matusik, Meng Jiang, Jie Chen
While large language models (LLMs) have integrated images, adapting them to graphs remains challenging, limiting their applications in materials and drug design.
no code implementations • 4 Oct 2024 • Peter Yichen Chen, Chao Liu, Pingchuan Ma, John Eastman, Daniela Rus, Dylan Randle, Yuri Ivanov, Wojciech Matusik
Leveraging this insight, we develop differentiable simulations of robot-object interactions to inversely identify the properties of the manipulated objects.
no code implementations • 24 Aug 2024 • Michael Sun, Alston Lo, Wenhao Gao, Minghao Guo, Veronika Thost, Jie Chen, Connor Coley, Wojciech Matusik
Drawing inspiration from syntax-guided synthesis approaches, we decouple the syntactic skeleton from the semantics of a synthetic tree to create a bilevel framework for reasoning about the combinatorial space of synthesis pathways.
1 code implementation • 19 Aug 2024 • Ziming Liu, Pingchuan Ma, YiXuan Wang, Wojciech Matusik, Max Tegmark
The synergy is bidirectional: science to KAN (incorporating scientific knowledge into KANs), and KAN to science (extracting scientific insights from KANs).
no code implementations • 15 Jul 2024 • Pramod Rao, Gereon Fox, Abhimitra Meka, Mallikarjun B R, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, Christian Theobalt
Achieving photorealistic 3D view synthesis and relighting of human portraits is pivotal for advancing AR/VR applications.
no code implementations • 30 May 2024 • Minghao Guo, Bohan Wang, Kaiming He, Wojciech Matusik
We introduce TetSphere Splatting, a Lagrangian geometry representation designed for high-quality 3D shape modeling.
no code implementations • 30 May 2024 • Minghao Guo, Bohan Wang, Pingchuan Ma, Tianyuan Zhang, Crystal Elaine Owens, Chuang Gan, Joshua B. Tenenbaum, Kaiming He, Wojciech Matusik
We present a computational framework that transforms single images into 3D physical objects.
no code implementations • 22 May 2024 • Yifei Li, Yuchen Sun, Pingchuan Ma, Eftychios Sifakis, Tao Du, Bo Zhu, Wojciech Matusik
We present a novel framework to explore neural control and design of complex fluidic systems with dynamic solid boundaries.
1 code implementation • 16 May 2024 • Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo, Zhiqing Sun, Joshua B. Tenenbaum, Daniela Rus, Chuang Gan, Wojciech Matusik
Large Language Models have recently gained significant attention in scientific discovery for their extensive knowledge and advanced reasoning capabilities.
no code implementations • 24 Mar 2024 • Yicheng Zhan, Liang Shi, Wojciech Matusik, Qi Sun, Kaan Akşit
Although the emerging learned approaches have enabled rapid and high-quality hologram generation, any alteration in display hardware still requires a retraining of the model.
no code implementations • 13 Mar 2024 • Michael Sun, Minghao Guo, Weize Yuan, Veronika Thost, Crystal Elaine Owens, Aristotle Franklin Grosz, Sharvaa Selvan, Katelyn Zhou, Hassan Mohiuddin, Benjamin J Pedretti, Zachary P Smith, Jie Chen, Wojciech Matusik
Recent research in molecular discovery has primarily been devoted to small, drug-like molecules, leaving many similarly important applications in material design without adequate technology.
1 code implementation • 12 Feb 2024 • Yunsheng Tian, Ane Zuniga, Xinwei Zhang, Johannes P. Dürholt, Payel Das, Jie Chen, Wojciech Matusik, Mina Konaković Luković
In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima.
no code implementations • CVPR 2024 • Yifei Li, Hsiao-yu Chen, Egor Larionov, Nikolaos Sarafianos, Wojciech Matusik, Tuur Stuyck
By integrating physical simulation into the optimization loop and accounting for the complex nonlinear behavior of cloth and its intricate interaction with the body, our framework recovers body and garment geometry and extracts important material parameters in a physically plausible way.
no code implementations • 26 Oct 2023 • Zeshun Zong, Xuan Li, Minchen Li, Maurizio M. Chiaramonte, Wojciech Matusik, Eitan Grinspun, Kevin Carlberg, Chenfanfu Jiang, Peter Yichen Chen
We propose a hybrid neural network and physics framework for reduced-order modeling of elastoplasticity and fracture.
no code implementations • 29 Sep 2023 • Yunsheng Tian, Karl D. D. Willis, Bassel Al Omari, Jieliang Luo, Pingchuan Ma, Yichen Li, Farhad Javid, Edward Gu, Joshua Jacob, Shinjiro Sueda, Hui Li, Sachin Chitta, Wojciech Matusik
The automated assembly of complex products requires a system that can automatically plan a physically feasible sequence of actions for assembling many parts together.
1 code implementation • 4 Sep 2023 • Minghao Guo, Veronika Thost, Samuel W Song, Adithya Balachandran, Payel Das, Jie Chen, Wojciech Matusik
Still, these techniques are faced with a common challenge in practice: Labeled data are limited by the cost of manual extraction from literature and laborious experimentation.
no code implementations • 25 Jul 2023 • Liane Makatura, Michael Foshey, Bohan Wang, Felix HähnLein, Pingchuan Ma, Bolei Deng, Megan Tjandrasuwita, Andrew Spielberg, Crystal Elaine Owens, Peter Yichen Chen, Allan Zhao, Amy Zhu, Wil J Norton, Edward Gu, Joshua Jacob, Yifei Li, Adriana Schulz, Wojciech Matusik
The advancement of Large Language Models (LLMs), including GPT-4, provides exciting new opportunities for generative design.
no code implementations • 27 Apr 2023 • Pingchuan Ma, Peter Yichen Chen, Bolei Deng, Joshua B. Tenenbaum, Tao Du, Chuang Gan, Wojciech Matusik
Many NN approaches learn an end-to-end model that implicitly models both the governing PDE and constitutive models (or material models).
no code implementations • CVPR 2024 • Yichen Li, Kaichun Mo, Yueqi Duan, He Wang, Jiequan Zhang, Lin Shao, Wojciech Matusik, Leonidas Guibas
A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment.
no code implementations • 1 Feb 2023 • Beichen Li, Bolei Deng, Wan Shou, Tae-Hyun Oh, Yuanming Hu, Yiyue Luo, Liang Shi, Wojciech Matusik
The conflict between stiffness and toughness is a fundamental problem in engineering materials design.
no code implementations • ICLR 2022 • Pingchuan Ma, Tao Du, Joshua B. Tenenbaum, Wojciech Matusik, Chuang Gan
To train this predictor, we formulate a new loss on rendering variances using gradients from differentiable rendering.
1 code implementation • ICLR 2022 • Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojciech Matusik, Animesh Garg, Miles Macklin
In this work we present a high-performance differentiable simulator and a new policy learning algorithm (SHAC) that can effectively leverage simulation gradients, even in the presence of non-smoothness.
no code implementations • 30 Mar 2022 • Elvis Nava, John Z. Zhang, Mike Y. Michelis, Tao Du, Pingchuan Ma, Benjamin F. Grewe, Wojciech Matusik, Robert K. Katzschmann
For the deformable solid simulation of the swimmer's body, we use state-of-the-art techniques from the field of computer graphics to speed up the finite-element method (FEM).
1 code implementation • ICLR 2022 • Minghao Guo, Veronika Thost, Beichen Li, Payel Das, Jie Chen, Wojciech Matusik
This is a non-trivial task for neural network-based generative models since the relevant chemical knowledge can only be extracted and generalized from the limited training data.
no code implementations • NeurIPS 2021 • Jagdeep Singh Bhatia, Holly Jackson, Yunsheng Tian, Jie Xu, Wojciech Matusik
In this paper, we propose Evolution Gym, the first large-scale benchmark for co-optimizing the design and control of soft robots.
2 code implementations • CVPR 2022 • Karl D. D. Willis, Pradeep Kumar Jayaraman, Hang Chu, Yunsheng Tian, Yifei Li, Daniele Grandi, Aditya Sanghi, Linh Tran, Joseph G. Lambourne, Armando Solar-Lezama, Wojciech Matusik
Physical products are often complex assemblies combining a multitude of 3D parts modeled in computer-aided design (CAD) software.
no code implementations • 7 Oct 2021 • Aldair E. Gongora, Siddharth Mysore, Beichen Li, Wan Shou, Wojciech Matusik, Elise F. Morgan, Keith A. Brown, Emily Whiting
Advancements in additive manufacturing have enabled design and fabrication of materials and structures not previously realizable.
no code implementations • 30 Sep 2021 • John Z. Zhang, Yu Zhang, Pingchuan Ma, Elvis Nava, Tao Du, Philip Arm, Wojciech Matusik, Robert K. Katzschmann
Accurate simulation of soft mechanisms under dynamic actuation is critical for the design of soft robots.
no code implementations • 29 Sep 2021 • Michal Piovarci, Michael Foshey, Timothy Erps, Jie Xu, Vahid Babaei, Piotr Didyk, Wojciech Matusik, Szymon Rusinkiewicz, Bernd Bickel
We further show that in combination with reinforcement learning, our model can be used to discover control policies that outperform state-of-the-art controllers.
no code implementations • 29 Sep 2021 • Yunsheng Tian, Mina Konakovic Lukovic, Michael Foshey, Timothy Erps, Beichen Li, Wojciech Matusik
We present AutoOED, an Automated Optimal Experimental Design platform powered by machine learning to accelerate discovering solutions with optimal objective trade-offs.
no code implementations • International Conference on Intelligent Robots and Systems (IROS) 2021 • Qiang Zhang, Yunzhu Li, Yiyue Luo, Wan Shou, Michael Foshey, Junchi Yan, Joshua B. Tenenbaum, Wojciech Matusik, Antonio Torralba
This work takes a step on dynamics modeling in hand-object interactions from dense tactile sensing, which opens the door for future applications in activity learning, human-computer interactions, and imitation learning for robotics.
no code implementations • 9 Sep 2021 • Qiang Zhang, Yunzhu Li, Yiyue Luo, Wan Shou, Michael Foshey, Junchi Yan, Joshua B. Tenenbaum, Wojciech Matusik, Antonio Torralba
This work takes a step on dynamics modeling in hand-object interactions from dense tactile sensing, which opens the door for future applications in activity learning, human-computer interactions, and imitation learning for robotics.
1 code implementation • 15 Jul 2021 • Jie Xu, Tao Chen, Lara Zlokapa, Michael Foshey, Wojciech Matusik, Shinjiro Sueda, Pulkit Agrawal
Existing methods for co-optimization are limited and fail to explore a rich space of designs.
no code implementations • CVPR 2021 • Yiyue Luo, Yunzhu Li, Michael Foshey, Wan Shou, Pratyusha Sharma, Tomas Palacios, Antonio Torralba, Wojciech Matusik
In this work, leveraging such tactile interactions, we propose a 3D human pose estimation approach using the pressure maps recorded by a tactile carpet as input.
1 code implementation • 9 Jun 2021 • Yifei Li, Tao Du, Kui Wu, Jie Xu, Wojciech Matusik
This work presents a differentiable cloth simulator whose additional gradient information facilitates cloth-related applications.
no code implementations • 5 May 2021 • Minghao Guo, Wan Shou, Liane Makatura, Timothy Erps, Michael Foshey, Wojciech Matusik
Here, we present a parametric, context-sensitive grammar designed specifically for the representation and generation of polymers.
1 code implementation • 13 Apr 2021 • Yunsheng Tian, Mina Konaković Luković, Timothy Erps, Michael Foshey, Wojciech Matusik
We present AutoOED, an Optimal Experiment Design platform powered with automated machine learning to accelerate the discovery of optimal solutions.
no code implementations • 2 Apr 2021 • Pingchuan Ma, Tao Du, John Z. Zhang, Kui Wu, Andrew Spielberg, Robert K. Katzschmann, Wojciech Matusik
The computational design of soft underwater swimmers is challenging because of the high degrees of freedom in soft-body modeling.
no code implementations • 13 Mar 2021 • Mallikarjun B R, Ayush Tewari, Abdallah Dib, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Louis Chevallier, Mohamed Elgharib, Christian Theobalt
We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image.
no code implementations • 15 Jan 2021 • Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, Wojciech Matusik
Inspired by Projective Dynamics (PD), we present Differentiable Projective Dynamics (DiffPD), an efficient differentiable soft-body simulator based on PD with implicit time integration.
1 code implementation • NeurIPS 2020 • Mina Konakovic Lukovic, Yunsheng Tian, Wojciech Matusik
To further reduce the evaluation time in the optimization process, testing of several samples in parallel can be deployed.
1 code implementation • 5 Oct 2020 • Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar-Lezama, Wojciech Matusik
Parametric computer-aided design (CAD) is a standard paradigm used to design manufactured objects, where a 3D shape is represented as a program supported by the CAD software.
no code implementations • 28 Sep 2020 • Karl Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph Lambourne, Armando Solar-Lezama, Wojciech Matusik
We provide a dataset of 8, 625 designs, comprising sequential sketch and extrude modeling operations, together with a complementary environment called the Fusion 360 Gym, to assist with performing CAD reconstruction.
no code implementations • CVPR 2021 • Mallikarjun B R., Ayush Tewari, Tae-Hyun Oh, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, Christian Theobalt
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.
1 code implementation • ICML 2020 • Pingchuan Ma, Tao Du, Wojciech Matusik
We present a novel, efficient method that generates locally continuous Pareto sets and Pareto fronts, which opens up the possibility of continuous analysis of Pareto optimal solutions in machine learning problems.
no code implementations • NeurIPS 2019 • Andrew Spielberg, Allan Zhao, Yuanming Hu, Tao Du, Wojciech Matusik, Daniela Rus
We validate the behavior of our algorithm with visualizations of the learned representation.
2 code implementations • ICCV 2019 • Petr Kellnhofer, Adria Recasens, Simon Stent, Wojciech Matusik, Antonio Torralba
Finally, we demonstrate an application of our model for estimating customer attention in a supermarket setting.
Ranked #4 on
Gaze Estimation
on Gaze360
no code implementations • 25 Sep 2019 • Tao Du, Yunfei Li, Jie Xu, Andrew Spielberg, Kui Wu, Daniela Rus, Wojciech Matusik
Over the last decade, two competing control strategies have emerged for solving complex control tasks with high efficacy.
3 code implementations • CVPR 2019 • Tae-Hyun Oh, Tali Dekel, Changil Kim, Inbar Mosseri, William T. Freeman, Michael Rubinstein, Wojciech Matusik
How much can we infer about a person's looks from the way they speak?
2 code implementations • 11 Apr 2019 • Alexandre Kaspar, Liane Makatura, Wojciech Matusik
This work presents a novel interactive system for simple garment composition and surface patterning.
Human-Computer Interaction
1 code implementation • 7 Feb 2019 • Alexandre Kaspar, Tae-Hyun Oh, Liane Makatura, Petr Kellnhofer, Jacqueline Aslarus, Wojciech Matusik
Motivated by the recent potential of mass customization brought by whole-garment knitting machines, we introduce the new problem of automatic machine instruction generation using a single image of the desired physical product, which we apply to machine knitting.
no code implementations • 2 Oct 2018 • Yuanming Hu, Jian-Cheng Liu, Andrew Spielberg, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu, Daniela Rus, Wojciech Matusik
The underlying physical laws of deformable objects are more complex, and the resulting systems have orders of magnitude more degrees of freedom and therefore they are significantly more computationally expensive to simulate.
1 code implementation • ECCV 2018 • Adrià Recasens, Petr Kellnhofer, Simon Stent, Wojciech Matusik, Antonio Torralba
We introduce a saliency-based distortion layer for convolutional neural networks that helps to improve the spatial sampling of input data for a given task.
no code implementations • ECCV 2018 • Yagiz Aksoy, Changil Kim, Petr Kellnhofer, Sylvain Paris, Mohamed Elgharib, Marc Pollefeys, Wojciech Matusik
We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations.
1 code implementation • 15 May 2018 • Changil Kim, Hijung Valentina Shin, Tae-Hyun Oh, Alexandre Kaspar, Mohamed Elgharib, Wojciech Matusik
We computationally model the overlapping information between faces and voices and show that the learned cross-modal representation contains enough information to identify matching faces and voices with performance similar to that of humans.
2 code implementations • ECCV 2018 • Tae-Hyun Oh, Ronnachai Jaroensri, Changil Kim, Mohamed Elgharib, Frédo Durand, William T. Freeman, Wojciech Matusik
We show that the learned filters achieve high-quality results on real videos, with less ringing artifacts and better noise characteristics than previous methods.
no code implementations • ICCV 2017 • Ajay Nandoriya, Mohamed Elgharib, Changil Kim, Mohamed Hefeeda, Wojciech Matusik
The novelty of our work is in our optimization formulation as well as the motion initialization strategy.
no code implementations • 10 May 2017 • Sung-Ho Bae, Mohamed Elgharib, Mohamed Hefeeda, Wojciech Matusik
We present two FCN architectures for SIVG.
4 code implementations • 9 May 2017 • Ahmed Hassanien, Mohamed Elgharib, Ahmed Selim, Sung-Ho Bae, Mohamed Hefeeda, Wojciech Matusik
Since current datasets are not large enough to train an accurate SBD CNN, we present a new dataset containing more than 3. 5 million frames of sharp and gradual transitions.
2 code implementations • CVPR 2016 • Kyle Krafka, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchendra Bhandarkar, Wojciech Matusik, Antonio Torralba
We believe that we can put the power of eye tracking in everyone's palm by building eye tracking software that works on commodity hardware such as mobile phones and tablets, without the need for additional sensors or devices.