2 code implementations • CVPR 2016 • Kyle Krafka, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchendra Bhandarkar, Wojciech Matusik, Antonio Torralba
We believe that we can put the power of eye tracking in everyone's palm by building eye tracking software that works on commodity hardware such as mobile phones and tablets, without the need for additional sensors or devices.
4 code implementations • 9 May 2017 • Ahmed Hassanien, Mohamed Elgharib, Ahmed Selim, Sung-Ho Bae, Mohamed Hefeeda, Wojciech Matusik
Since current datasets are not large enough to train an accurate SBD CNN, we present a new dataset containing more than 3. 5 million frames of sharp and gradual transitions.
no code implementations • 10 May 2017 • Sung-Ho Bae, Mohamed Elgharib, Mohamed Hefeeda, Wojciech Matusik
We present two FCN architectures for SIVG.
no code implementations • ICCV 2017 • Ajay Nandoriya, Mohamed Elgharib, Changil Kim, Mohamed Hefeeda, Wojciech Matusik
The novelty of our work is in our optimization formulation as well as the motion initialization strategy.
2 code implementations • ECCV 2018 • Tae-Hyun Oh, Ronnachai Jaroensri, Changil Kim, Mohamed Elgharib, Frédo Durand, William T. Freeman, Wojciech Matusik
We show that the learned filters achieve high-quality results on real videos, with less ringing artifacts and better noise characteristics than previous methods.
1 code implementation • 15 May 2018 • Changil Kim, Hijung Valentina Shin, Tae-Hyun Oh, Alexandre Kaspar, Mohamed Elgharib, Wojciech Matusik
We computationally model the overlapping information between faces and voices and show that the learned cross-modal representation contains enough information to identify matching faces and voices with performance similar to that of humans.
no code implementations • ECCV 2018 • Yagiz Aksoy, Changil Kim, Petr Kellnhofer, Sylvain Paris, Mohamed Elgharib, Marc Pollefeys, Wojciech Matusik
We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations.
1 code implementation • ECCV 2018 • Adrià Recasens, Petr Kellnhofer, Simon Stent, Wojciech Matusik, Antonio Torralba
We introduce a saliency-based distortion layer for convolutional neural networks that helps to improve the spatial sampling of input data for a given task.
no code implementations • 2 Oct 2018 • Yuanming Hu, Jian-Cheng Liu, Andrew Spielberg, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu, Daniela Rus, Wojciech Matusik
The underlying physical laws of deformable objects are more complex, and the resulting systems have orders of magnitude more degrees of freedom and therefore they are significantly more computationally expensive to simulate.
1 code implementation • 7 Feb 2019 • Alexandre Kaspar, Tae-Hyun Oh, Liane Makatura, Petr Kellnhofer, Jacqueline Aslarus, Wojciech Matusik
Motivated by the recent potential of mass customization brought by whole-garment knitting machines, we introduce the new problem of automatic machine instruction generation using a single image of the desired physical product, which we apply to machine knitting.
2 code implementations • 11 Apr 2019 • Alexandre Kaspar, Liane Makatura, Wojciech Matusik
This work presents a novel interactive system for simple garment composition and surface patterning.
Human-Computer Interaction
3 code implementations • CVPR 2019 • Tae-Hyun Oh, Tali Dekel, Changil Kim, Inbar Mosseri, William T. Freeman, Michael Rubinstein, Wojciech Matusik
How much can we infer about a person's looks from the way they speak?
no code implementations • 25 Sep 2019 • Tao Du, Yunfei Li, Jie Xu, Andrew Spielberg, Kui Wu, Daniela Rus, Wojciech Matusik
Over the last decade, two competing control strategies have emerged for solving complex control tasks with high efficacy.
1 code implementation • ICCV 2019 • Petr Kellnhofer, Adria Recasens, Simon Stent, Wojciech Matusik, Antonio Torralba
Finally, we demonstrate an application of our model for estimating customer attention in a supermarket setting.
Ranked #4 on Gaze Estimation on Gaze360
no code implementations • NeurIPS 2019 • Andrew Spielberg, Allan Zhao, Yuanming Hu, Tao Du, Wojciech Matusik, Daniela Rus
We validate the behavior of our algorithm with visualizations of the learned representation.
1 code implementation • ICML 2020 • Pingchuan Ma, Tao Du, Wojciech Matusik
We present a novel, efficient method that generates locally continuous Pareto sets and Pareto fronts, which opens up the possibility of continuous analysis of Pareto optimal solutions in machine learning problems.
no code implementations • CVPR 2021 • Mallikarjun B R., Ayush Tewari, Tae-Hyun Oh, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, Christian Theobalt
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.
no code implementations • 28 Sep 2020 • Karl Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph Lambourne, Armando Solar-Lezama, Wojciech Matusik
We provide a dataset of 8, 625 designs, comprising sequential sketch and extrude modeling operations, together with a complementary environment called the Fusion 360 Gym, to assist with performing CAD reconstruction.
1 code implementation • 5 Oct 2020 • Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar-Lezama, Wojciech Matusik
Parametric computer-aided design (CAD) is a standard paradigm used to design manufactured objects, where a 3D shape is represented as a program supported by the CAD software.
1 code implementation • NeurIPS 2020 • Mina Konakovic Lukovic, Yunsheng Tian, Wojciech Matusik
To further reduce the evaluation time in the optimization process, testing of several samples in parallel can be deployed.
no code implementations • 15 Jan 2021 • Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, Wojciech Matusik
Inspired by Projective Dynamics (PD), we present Differentiable Projective Dynamics (DiffPD), an efficient differentiable soft-body simulator based on PD with implicit time integration.
1 code implementation • 13 Mar 2021 • Mallikarjun B R, Ayush Tewari, Abdallah Dib, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Louis Chevallier, Mohamed Elgharib, Christian Theobalt
We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image.
no code implementations • 2 Apr 2021 • Pingchuan Ma, Tao Du, John Z. Zhang, Kui Wu, Andrew Spielberg, Robert K. Katzschmann, Wojciech Matusik
The computational design of soft underwater swimmers is challenging because of the high degrees of freedom in soft-body modeling.
1 code implementation • 13 Apr 2021 • Yunsheng Tian, Mina Konaković Luković, Timothy Erps, Michael Foshey, Wojciech Matusik
We present AutoOED, an Optimal Experiment Design platform powered with automated machine learning to accelerate the discovery of optimal solutions.
no code implementations • 5 May 2021 • Minghao Guo, Wan Shou, Liane Makatura, Timothy Erps, Michael Foshey, Wojciech Matusik
Here, we present a parametric, context-sensitive grammar designed specifically for the representation and generation of polymers.
1 code implementation • 9 Jun 2021 • Yifei Li, Tao Du, Kui Wu, Jie Xu, Wojciech Matusik
This work presents a differentiable cloth simulator whose additional gradient information facilitates cloth-related applications.
no code implementations • CVPR 2021 • Yiyue Luo, Yunzhu Li, Michael Foshey, Wan Shou, Pratyusha Sharma, Tomas Palacios, Antonio Torralba, Wojciech Matusik
In this work, leveraging such tactile interactions, we propose a 3D human pose estimation approach using the pressure maps recorded by a tactile carpet as input.
1 code implementation • 15 Jul 2021 • Jie Xu, Tao Chen, Lara Zlokapa, Michael Foshey, Wojciech Matusik, Shinjiro Sueda, Pulkit Agrawal
Existing methods for co-optimization are limited and fail to explore a rich space of designs.
no code implementations • 9 Sep 2021 • Qiang Zhang, Yunzhu Li, Yiyue Luo, Wan Shou, Michael Foshey, Junchi Yan, Joshua B. Tenenbaum, Wojciech Matusik, Antonio Torralba
This work takes a step on dynamics modeling in hand-object interactions from dense tactile sensing, which opens the door for future applications in activity learning, human-computer interactions, and imitation learning for robotics.
no code implementations • 29 Sep 2021 • Yunsheng Tian, Mina Konakovic Lukovic, Michael Foshey, Timothy Erps, Beichen Li, Wojciech Matusik
We present AutoOED, an Automated Optimal Experimental Design platform powered by machine learning to accelerate discovering solutions with optimal objective trade-offs.
no code implementations • 29 Sep 2021 • Michal Piovarci, Michael Foshey, Timothy Erps, Jie Xu, Vahid Babaei, Piotr Didyk, Wojciech Matusik, Szymon Rusinkiewicz, Bernd Bickel
We further show that in combination with reinforcement learning, our model can be used to discover control policies that outperform state-of-the-art controllers.
no code implementations • 30 Sep 2021 • John Z. Zhang, Yu Zhang, Pingchuan Ma, Elvis Nava, Tao Du, Philip Arm, Wojciech Matusik, Robert K. Katzschmann
Accurate simulation of soft mechanisms under dynamic actuation is critical for the design of soft robots.
no code implementations • 7 Oct 2021 • Aldair E. Gongora, Siddharth Mysore, Beichen Li, Wan Shou, Wojciech Matusik, Elise F. Morgan, Keith A. Brown, Emily Whiting
Advancements in additive manufacturing have enabled design and fabrication of materials and structures not previously realizable.
2 code implementations • CVPR 2022 • Karl D. D. Willis, Pradeep Kumar Jayaraman, Hang Chu, Yunsheng Tian, Yifei Li, Daniele Grandi, Aditya Sanghi, Linh Tran, Joseph G. Lambourne, Armando Solar-Lezama, Wojciech Matusik
Physical products are often complex assemblies combining a multitude of 3D parts modeled in computer-aided design (CAD) software.
no code implementations • NeurIPS 2021 • Jagdeep Singh Bhatia, Holly Jackson, Yunsheng Tian, Jie Xu, Wojciech Matusik
In this paper, we propose Evolution Gym, the first large-scale benchmark for co-optimizing the design and control of soft robots.
1 code implementation • ICLR 2022 • Minghao Guo, Veronika Thost, Beichen Li, Payel Das, Jie Chen, Wojciech Matusik
This is a non-trivial task for neural network-based generative models since the relevant chemical knowledge can only be extracted and generalized from the limited training data.
no code implementations • 30 Mar 2022 • Elvis Nava, John Z. Zhang, Mike Y. Michelis, Tao Du, Pingchuan Ma, Benjamin F. Grewe, Wojciech Matusik, Robert K. Katzschmann
For the deformable solid simulation of the swimmer's body, we use state-of-the-art techniques from the field of computer graphics to speed up the finite-element method (FEM).
no code implementations • ICLR 2022 • Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojciech Matusik, Animesh Garg, Miles Macklin
In this work we present a high-performance differentiable simulator and a new policy learning algorithm (SHAC) that can effectively leverage simulation gradients, even in the presence of non-smoothness.
no code implementations • ICLR 2022 • Pingchuan Ma, Tao Du, Joshua B. Tenenbaum, Wojciech Matusik, Chuang Gan
To train this predictor, we formulate a new loss on rendering variances using gradients from differentiable rendering.
no code implementations • 1 Feb 2023 • Beichen Li, Bolei Deng, Wan Shou, Tae-Hyun Oh, Yuanming Hu, Yiyue Luo, Liang Shi, Wojciech Matusik
The conflict between stiffness and toughness is a fundamental problem in engineering materials design.
no code implementations • 10 Mar 2023 • Yichen Li, Kaichun Mo, Yueqi Duan, He Wang, Jiequan Zhang, Lin Shao, Wojciech Matusik, Leonidas Guibas
A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment.
no code implementations • 27 Apr 2023 • Pingchuan Ma, Peter Yichen Chen, Bolei Deng, Joshua B. Tenenbaum, Tao Du, Chuang Gan, Wojciech Matusik
Many NN approaches learn an end-to-end model that implicitly models both the governing PDE and constitutive models (or material models).
no code implementations • 25 Jul 2023 • Liane Makatura, Michael Foshey, Bohan Wang, Felix HähnLein, Pingchuan Ma, Bolei Deng, Megan Tjandrasuwita, Andrew Spielberg, Crystal Elaine Owens, Peter Yichen Chen, Allan Zhao, Amy Zhu, Wil J Norton, Edward Gu, Joshua Jacob, Yifei Li, Adriana Schulz, Wojciech Matusik
The advancement of Large Language Models (LLMs), including GPT-4, provides exciting new opportunities for generative design.
1 code implementation • 4 Sep 2023 • Minghao Guo, Veronika Thost, Samuel W Song, Adithya Balachandran, Payel Das, Jie Chen, Wojciech Matusik
Still, these techniques are faced with a common challenge in practice: Labeled data are limited by the cost of manual extraction from literature and laborious experimentation.
no code implementations • 29 Sep 2023 • Yunsheng Tian, Karl D. D. Willis, Bassel Al Omari, Jieliang Luo, Pingchuan Ma, Yichen Li, Farhad Javid, Edward Gu, Joshua Jacob, Shinjiro Sueda, Hui Li, Sachin Chitta, Wojciech Matusik
The automated assembly of complex products requires a system that can automatically plan a physically feasible sequence of actions for assembling many parts together.
no code implementations • 26 Oct 2023 • Zeshun Zong, Xuan Li, Minchen Li, Maurizio M. Chiaramonte, Wojciech Matusik, Eitan Grinspun, Kevin Carlberg, Chenfanfu Jiang, Peter Yichen Chen
We propose a hybrid neural network and physics framework for reduced-order modeling of elastoplasticity and fracture.
no code implementations • 20 Nov 2023 • Yifei Li, Hsiao-yu Chen, Egor Larionov, Nikolaos Sarafianos, Wojciech Matusik, Tuur Stuyck
By integrating physical simulation into the optimization loop and accounting for the complex nonlinear behavior of cloth and its intricate interaction with the body, our framework recovers body and garment geometry and extracts important material parameters in a physically plausible way.
no code implementations • 12 Feb 2024 • Yunsheng Tian, Ane Zuniga, Xinwei Zhang, Johannes P. Dürholt, Payel Das, Jie Chen, Wojciech Matusik, Mina Konaković Luković
In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima.
no code implementations • 13 Mar 2024 • Michael Sun, Minghao Guo, Weize Yuan, Veronika Thost, Crystal Elaine Owens, Aristotle Franklin Grosz, Sharvaa Selvan, Katelyn Zhou, Hassan Mohiuddin, Benjamin J Pedretti, Zachary P Smith, Jie Chen, Wojciech Matusik
Recent research in molecular discovery has primarily been devoted to small, drug-like molecules, leaving many similarly important applications in material design without adequate technology.
2 code implementations • ICML 2020 • Jie Xu, Yunsheng Tian, Pingchuan Ma, Daniela Rus, Shinjiro Sueda, Wojciech Matusik
Many real-world control problems involve conflicting objectives where we desire a dense and high-quality set of control policies that are optimal for different objective preferences (called Pareto-optimal).
Multi-Objective Reinforcement Learning reinforcement-learning