Search Results for author:
Found 50 papers, 20 papers with code
Eye Tracking for Everyone
We believe that we can put the power of eye tracking in everyone's palm by building eye tracking software that works on commodity hardware such as mobile phones and tablets, without the need for additional sensors or devices.
Learning-based Video Motion Magnification
We show that the learned filters achieve high-quality results on real videos, with less ringing artifacts and better noise characteristics than previous methods.
Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Construction from Human Design Sequences
Parametric computer-aided design (CAD) is a standard paradigm used to design manufactured objects, where a 3D shape is represented as a program supported by the CAD software.
JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints
Physical products are often complex assemblies combining a multitude of 3D parts modeled in computer-aided design (CAD) software.
Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks
Since current datasets are not large enough to train an accurate SBD CNN, we present a new dataset containing more than 3. 5 million frames of sharp and gradual transitions.
DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact
This work presents a differentiable cloth simulator whose additional gradient information facilitates cloth-related applications.
Gaze360: Physically Unconstrained Gaze Estimation in the Wild
Finally, we demonstrate an application of our model for estimating customer attention in a supermarket setting.
Ranked #4 on
Gaze Estimation
on Gaze360
Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control
Many real-world control problems involve conflicting objectives where we desire a dense and high-quality set of control policies that are optimal for different objective preferences (called Pareto-optimal).
Multi-Objective Reinforcement Learning
reinforcement-learning
Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks
We introduce a saliency-based distortion layer for convolutional neural networks that helps to improve the spatial sampling of input data for a given task.
Speech2Face: Learning the Face Behind a Voice
How much can we infer about a person's looks from the way they speak?
Efficient Continuous Pareto Exploration in Multi-Task Learning
We present a novel, efficient method that generates locally continuous Pareto sets and Pareto fronts, which opens up the possibility of continuous analysis of Pareto optimal solutions in machine learning problems.
AutoOED: Automated Optimal Experiment Design Platform
We present AutoOED, an Optimal Experiment Design platform powered with automated machine learning to accelerate the discovery of optimal solutions.
An End-to-End Differentiable Framework for Contact-Aware Robot Design
Existing methods for co-optimization are limited and fail to explore a rich space of designs.
Diversity-Guided Multi-Objective Bayesian Optimization With Batch Evaluations
To further reduce the evaluation time in the optimization process, testing of several samples in parallel can be deployed.
Data-Efficient Graph Grammar Learning for Molecular Generation
This is a non-trivial task for neural network-based generative models since the relevant chemical knowledge can only be extracted and generalized from the limited training data.
Neural Inverse Knitting: From Images to Manufacturing Instructions
Motivated by the recent potential of mass customization brought by whole-garment knitting machines, we introduce the new problem of automatic machine instruction generation using a single image of the desired physical product, which we apply to machine knitting.
PhotoApp: Photorealistic Appearance Editing of Head Portraits
We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image.
Hierarchical Grammar-Induced Geometry for Data-Efficient Molecular Property Prediction
Still, these techniques are faced with a common challenge in practice: Labeled data are limited by the cost of manual extraction from literature and laborious experimentation.
Knitting Skeletons: A Computer-Aided Design Tool for Shaping and Patterning of Knitted Garments
This work presents a novel interactive system for simple garment composition and surface patterning.
Human-Computer Interaction
On Learning Associations of Faces and Voices
We computationally model the overlapping information between faces and voices and show that the learned cross-modal representation contains enough information to identify matching faces and voices with performance similar to that of humans.
Efficient and Scalable View Generation from a Single Image using Fully Convolutional Networks
We present two FCN architectures for SIVG.
ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics
The underlying physical laws of deformable objects are more complex, and the resulting systems have orders of magnitude more degrees of freedom and therefore they are significantly more computationally expensive to simulate.
A Dataset of Flash and Ambient Illumination Pairs from the Crowd
We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations.
Video Reflection Removal Through Spatio-Temporal Optimization
The novelty of our work is in our optimization formulation as well as the motion initialization strategy.
Learning-In-The-Loop Optimization: End-To-End Control And Co-Design Of Soft Robots Through Learned Deep Latent Representations
We validate the behavior of our algorithm with visualizations of the learned representation.
Monocular Reconstruction of Neural Face Reflectance Fields
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.
DiffPD: Differentiable Projective Dynamics
Inspired by Projective Dynamics (PD), we present Differentiable Projective Dynamics (DiffPD), an efficient differentiable soft-body simulator based on PD with implicit time integration.
DiffAqua: A Differentiable Computational Design Pipeline for Soft Underwater Swimmers with Shape Interpolation
The computational design of soft underwater swimmers is challenging because of the high degrees of freedom in soft-body modeling.
Polygrammar: Grammar for Digital Polymer Representation and Generation
Here, we present a parametric, context-sensitive grammar designed specifically for the representation and generation of polymers.
Intelligent Carpet: Inferring 3D Human Pose From Tactile Signals
In this work, leveraging such tactile interactions, we propose a 3D human pose estimation approach using the pressure maps recorded by a tactile carpet as input.
Dynamic Modeling of Hand-Object Interactions via Tactile Sensing
This work takes a step on dynamics modeling in hand-object interactions from dense tactile sensing, which opens the door for future applications in activity learning, human-computer interactions, and imitation learning for robotics.
Sim2Real for Soft Robotic Fish via Differentiable Simulation
Accurate simulation of soft mechanisms under dynamic actuation is critical for the design of soft robots.
Closed-Loop Control of Additive Manufacturing via Reinforcement Learning
We further show that in combination with reinforcement learning, our model can be used to discover control policies that outperform state-of-the-art controllers.
AutoOED: Automated Optimal Experimental Design Platform with Data- and Time-Efficient Multi-Objective Optimization
We present AutoOED, an Automated Optimal Experimental Design platform powered by machine learning to accelerate discovering solutions with optimal objective trade-offs.
Designing Composites with Target Effective Young's Modulus using Reinforcement Learning
Advancements in additive manufacturing have enabled design and fabrication of materials and structures not previously realizable.
D3PG: Deep Differentiable Deterministic Policy Gradients
Over the last decade, two competing control strategies have emerged for solving complex control tasks with high efficacy.
Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Reconstruction
We provide a dataset of 8, 625 designs, comprising sequential sketch and extrude modeling operations, together with a complementary environment called the Fusion 360 Gym, to assist with performing CAD reconstruction.
Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots
In this paper, we propose Evolution Gym, the first large-scale benchmark for co-optimizing the design and control of soft robots.
Accelerated Policy Learning with Parallel Differentiable Simulation
In this work we present a high-performance differentiable simulator and a new policy learning algorithm (SHAC) that can effectively leverage simulation gradients, even in the presence of non-smoothness.
Fast Aquatic Swimmer Optimization with Differentiable Projective Dynamics and Neural Network Hydrodynamic Models
For the deformable solid simulation of the swimmer's body, we use state-of-the-art techniques from the field of computer graphics to speed up the finite-element method (FEM).
RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation
To train this predictor, we formulate a new loss on rendering variances using gradients from differentiable rendering.
Computational Discovery of Microstructured Composites with Optimal Stiffness-Toughness Trade-Offs
The conflict between stiffness and toughness is a fundamental problem in engineering materials design.
Category-Level Multi-Part Multi-Joint 3D Shape Assembly
A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment.
Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics
Many NN approaches learn an end-to-end model that implicitly models both the governing PDE and constitutive models (or material models).
How Can Large Language Models Help Humans in Design and Manufacturing?
The advancement of Large Language Models (LLMs), including GPT-4, provides exciting new opportunities for generative design.
ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibility
The automated assembly of complex products requires a system that can automatically plan a physically feasible sequence of actions for assembling many parts together.
Neural Stress Fields for Reduced-order Elastoplasticity and Fracture
We propose a hybrid neural network and physics framework for reduced-order modeling of elastoplasticity and fracture.
DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation
By integrating physical simulation into the optimization loop and accounting for the complex nonlinear behavior of cloth and its intricate interaction with the body, our framework recovers body and garment geometry and extracts important material parameters in a physically plausible way.
Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints
In this paper, we observe that in such scenarios optimal solution typically lies on the boundary between feasible and infeasible regions of the design space, making it considerably more difficult than that with interior optima.
Representing Molecules as Random Walks Over Interpretable Grammars
Recent research in molecular discovery has primarily been devoted to small, drug-like molecules, leaving many similarly important applications in material design without adequate technology.
