Search Results for author:
Found 33 papers, 15 papers with code
Rethinking Software Engineering in the Foundation Model Era: From Task-Driven AI Copilots to Goal-Driven AI Pair Programmers
We envision AI pair programmers that are goal-driven, human partners, SE-aware, and self-learning.
PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environments
Robotic manipulation of ungraspable objects with two-finger grippers presents significant challenges due to the paucity of graspable features, while traditional pre-grasping techniques, which rely on repositioning objects and leveraging external aids like table edges, lack the adaptability across object categories and scenes.
Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware
Finally, we (i) show how the unique properties of FMArts enabled us to design and develop a complex FMware for a large customer in a timely manner and (ii) discuss the lessons that we learned in doing so.
Aligner: Achieving Efficient Alignment through Weak-to-Strong Correction
Here we introduce Aligner, a new efficient alignment paradigm that bypasses the whole RLHF process by learning the correctional residuals between the aligned and the unaligned answers.
DittoGym: Learning to Control Soft Shape-Shifting Robots
Inspired by nature and recent novel robot designs, we propose to go a step further and explore the novel reconfigurable robots, defined as robots that can change their morphology within their lifetime.
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
By training a VLM on such data, we significantly enhance its ability on both qualitative and quantitative spatial VQA.
Towards End-to-End Structure Solutions from Information-Compromised Diffraction Data via Generative Deep Learning
The revolution in materials in the past century was built on a knowledge of the atomic arrangements and the structure-property relationship.
AI Alignment: A Comprehensive Survey
The former aims to make AI systems aligned via alignment training, while the latter aims to gain evidence about the systems' alignment and govern them appropriately to avoid exacerbating misalignment risks.
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills.
Understanding Why ViT Trains Badly on Small Datasets: An Intuitive Perspective
Vision transformer (ViT) is an attention neural network architecture that is shown to be effective for computer vision tasks.
Extraneousness-Aware Imitation Learning
To tackle this problem, we introduce Extraneousness-Aware Imitation Learning (EIL), a self-supervised approach that learns visuomotor policies from third-person demonstrations with extraneous subsequences.
Force-Aware Interface via Electromyography for Natural VR/AR Interaction
Closing the gap between real-world physicality and immersive virtual experience requires a closed interaction loop: applying user-exerted physical forces to the virtual environment and generating haptic sensations back to the users.
Open-vocabulary Queryable Scene Representations for Real World Planning
Large language models (LLMs) have unlocked new capabilities of task planning from human instructions.
On the Origins of Self-Modeling
Self-Modeling is the process by which an agent, such as an animal or machine, learns to create a predictive model of its own dynamics.
Towards Training Reproducible Deep Learning Models
However, DL models are challenging to be reproduced due to issues like randomness in the software (e. g., DL algorithms) and non-determinism in the hardware (e. g., GPU).
Towards a consistent interpretation of AIOps models
Artificial Intelligence for IT Operations (AIOps) has been adopted in organizations in various tasks, including interpreting models to identify indicators of service failures.
Discovering State Variables Hidden in Experimental Data
All physical laws are described as relationships between state variables that give a complete and non-redundant description of the relevant system dynamics.
Full-Body Visual Self-Modeling of Robot Morphologies
Internal computational models of physical bodies are fundamental to the ability of robots and animals alike to plan and control their actions.
Can I use this publicly available dataset to build commercial AI software? -- A Case Study on Publicly Available Image Datasets
Publicly available datasets are one of the key drivers for commercial AI software.
Unsupervised Learning of Visual 3D Keypoints for Control
Prior works show that structured latent space such as visual keypoints often outperforms unstructured representations for robotic control.
Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models
At present, progress in this field is hindered by the fact that each facial expression needs to be programmed by humans.
The Boombox: Visual Reconstruction from Acoustic Vibrations
Due to the many ways that robots use containers, we believe the box will have a number of applications in robotics.
Visual Perspective Taking for Opponent Behavior Modeling
We suggest that visual behavior modeling and perspective taking skills will play a critical role in the ability of physical robots to fully integrate into real-world multi-agent activities.
Beyond Categorical Label Representations for Image Classification
Our experiments reveal that high dimensional, high entropy labels achieve comparable accuracy to text (categorical) labels on the standard image classification task, but features learned through our label representations exhibit more robustness under various adversarial attacks and better effectiveness with a limited amount of training data.
Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization
We propose a simple, general and effective technique, Reward Randomization for discovering diverse strategic policies in complex multi-agent games.
SSCNav: Confidence-Aware Semantic Scene Completion for Visual Semantic Navigation
We introduce SSCNav, an algorithm that explicitly models scene priors using a confidence-aware semantic scene completion module to complete the scene and guide the agent's navigation planning.
Oops! Predicting Unintentional Action in Video
We train a supervised neural network as a baseline and analyze its performance compared to human consistency on the tasks.
Zero-shot Policy Learning with Spatial Temporal RewardDecomposition on Contingency-aware Observation
The agent is first presented with previous experiences in the training environment, along with task description in the form of trajectory-level sparse rewards.
Visual Hide and Seek
We train embodied agents to play Visual Hide and Seek where a prey must navigate in a simulated environment in order to avoid capture from a predator.
Scoring-Aggregating-Planning: Learning task-agnostic priors from interactions and sparse rewards for zero-shot generalization
In this paper, we propose Scoring-Aggregating-Planning (SAP), a framework that can learn task-agnostic semantics and dynamics priors from arbitrary quality interactions as well as the corresponding sparse rewards and then plan on unseen tasks in zero-shot condition.
Workspace Aware Online Grasp Planning
This framework greatly improves the performance of standard online grasp planning algorithms by incorporating a notion of reachability into the online grasp planning process.
Robotics
Autostacker: A Compositional Evolutionary Learning System
We introduce an automatic machine learning (AutoML) modeling architecture called Autostacker, which combines an innovative hierarchical stacking architecture and an Evolutionary Algorithm (EA) to perform efficient parameter search.
Autostacker: an Automatic Evolutionary Hierarchical Machine Learning System
We significantly reduce the time of AutoML with a naturally inspired algorithm - Parallel Hill Climbing (PHC).
