Search Results for author:
Found 135 papers, 59 papers with code
CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulation
Furthermore, we propose the first manual-based manipulation planning model ManualPlan to set up a group of baselines for the CheckManual benchmark.
BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assembly
Shape assembly, the process of combining parts into a complete whole, is a crucial robotic skill with broad real-world applications.
SR3D: Unleashing Single-view 3D Reconstruction for Transparent and Specular Object Grasping
To address this, leveraging the power of single view 3D object reconstruction approaches, we propose a training free framework SR3D that enables robotic grasping of transparent and specular objects from a single view observation.
To Trust Or Not To Trust Your Vision-Language Model's Prediction
Vision-Language Models (VLMs) have demonstrated strong capabilities in aligning visual and textual modalities, enabling a wide range of applications in multimodal understanding and generation.
From Strangers to Assistants: Fast Desire Alignment for Embodied Agent-User Adaptation
The ego agent must interact with this proxy user to infer and adapt to the user's latent desires.
SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams
To address this gap, we propose SpikeStereoNet, a brain-inspired framework and the first to estimate stereo depth directly from raw spike streams.
GCAL: Adapting Graph Models to Evolving Domain Shifts
This paper addresses the challenge of graph domain adaptation on evolving, multiple out-of-distribution (OOD) graphs.
Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation
Out-of-distribution (OOD) detection and segmentation are crucial for deploying machine learning models in safety-critical applications such as autonomous driving and robot-assisted surgery.
Disentangled Multi-span Evolutionary Network against Temporal Knowledge Graph Reasoning
However, they still have limitations: 1) In modeling subgraph semantic evolution, they usually neglect the internal structural interactions between subgraphs, which are actually crucial for encoding TKGs.
DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy
To improve generalization across diverse garment shapes and deformations, we propose a Hierarchical gArment-manipuLation pOlicy (HALO).
Improving Unsupervised Task-driven Models of Ventral Visual Stream via Relative Position Predictivity
Based on the concept that ventral visual stream (VVS) mainly functions for object recognition, current unsupervised task-driven methods model VVS by contrastive learning, and have achieved good brain similarity.
3DWG: 3D Weakly Supervised Visual Grounding via Category and Instance-Level Alignment
The 3D weakly-supervised visual grounding task aims to localize oriented 3D boxes in point clouds based on natural language descriptions without requiring annotations to guide model learning.
Boosting Universal LLM Reward Design through the Heuristic Reward Observation Space Evolution
Large Language Models (LLMs) are emerging as promising tools for automated reinforcement learning (RL) reward design, owing to their robust capabilities in commonsense reasoning and code generation.
PartRM: Modeling Part-Level Dynamics with Large Cross-State Reconstruction Model
As interest grows in world models that predict future states from current observations and actions, accurately modeling part-level dynamics has become increasingly relevant for various applications.
Foundation Feature-Driven Online End-Effector Pose Estimation: A Marker-Free and Learning-Free Approach
Recent learning-based robot pose estimation methods, while advancing online calibration, struggle with cross-robot generalization and require the robot to be fully visible.
GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulation
Cluttered garments manipulation poses significant challenges due to the complex, deformable nature of garments and intricate garment relations.
AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learning
To enhance the diversity and complexity of adaptive manipulation mechanisms, we build a novel articulated object manipulation environment and equip it with 9 categories of objects.
CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-World
However, obtaining high-quality 3D representations presents two key problems: (1) the quality of point clouds captured by a single-view camera is significantly affected by factors such as camera resolution, positioning, and occlusions caused by the dexterous hand; (2) the global point clouds lack crucial contact information and spatial correspondences, which are necessary for fine-grained dexterous manipulation tasks.
Interactive Symbolic Regression through Offline Reinforcement Learning: A Co-Design Framework
Symbolic Regression (SR) holds great potential for uncovering underlying mathematical and physical relationships from observed data.
Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models
Besides, the recent advent of large-scale pre-trained multimodal foundation models, such as CLIP, has inspired works leveraging these models to enhance adaptation and generalization performances or adapting them to downstream tasks.
An Automatic Sound and Complete Abstraction Method for Generalized Planning with Baggable Types
We define a class of domains called proper baggable domains, and show that for such domains, the BQNP problem got by our automatic method is a sound and complete abstraction for a generalized planning problem whose instances share the same bags with the given instance but the sizes of the bags might be different.
Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization
This task becomes more challenging when multiple modalities are involved.
Unseen Visual Anomaly Generation
Visual anomaly detection (AD) presents significant challenges due to the scarcity of anomalous data samples.
Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulation
In robotic manipulation, task goals can be conveyed through various modalities, such as language, goal images, and goal videos.
ManipGPT: Is Affordance Segmentation by Large Vision Models Enough for Articulated Object Manipulation?
Visual actionable affordance has emerged as a transformative approach in robotics, focusing on perceiving interaction areas prior to manipulation.
PGSO: Prompt-based Generative Sequence Optimization Network for Aspect-based Sentiment Analysis
The rule-based approach relies on handcraft priority of dependency relation to reorder the context, while the score-based algorithm dynamically regulates the contextual sequence by calculating word position scores using neural network.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
Recall and Refine: A Simple but Effective Source-free Open-set Domain Adaptation Framework
Open-set Domain Adaptation (OSDA) aims to adapt a model from a labeled source domain to an unlabeled target domain, where novel classes - also referred to as target-private unknown classes - are present.
Is Precise Recovery Necessary? A Task-Oriented Imputation Approach for Time Series Forecasting on Variable Subset
Variable Subset Forecasting (VSF) refers to a unique scenario in multivariate time series forecasting, where available variables in the inference phase are only a subset of the variables in the training phase.
DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection
Extensive experiments on two tasks, five datasets, and nine base OOD algorithms demonstrate that DPU significantly improves OOD detection performance, setting a new state-of-the-art in multimodal OOD detection, with improvements of up to 80 percent in Far-OOD detection.
ET-SEED: Efficient Trajectory-Level SE(3) Equivariant Diffusion Policy
We theoretically extend equivariant Markov kernels and simplify the condition of equivariant diffusion process, thereby significantly improving training efficiency for trajectory-level SE(3) equivariant diffusion policy in an end-to-end manner.
GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation
Recent successes in reinforcement learning and vision-based methods offer promising avenues for learning garment manipulation.
RAD: A Dataset and Benchmark for Real-Life Anomaly Detection with Robotic Observations
To address this gap, we introduce the Realistic Anomaly Detection (RAD) dataset, the first multi-view RGB-based anomaly detection dataset specifically collected using a real robot arm, providing unique and realistic data scenarios.
CoSEC: A Coaxial Stereo Event Camera Dataset for Autonomous Driving
In this work, we introduce hybrid coaxial event-frame devices to build the multimodal system, and propose a coaxial stereo event camera (CoSEC) dataset for autonomous driving.
EqvAfford: SE(3) Equivariance for Point-Level Affordance Learning
Humans perceive and interact with the world with the awareness of equivariance, facilitating us in manipulating different objects in diverse poses.
Local Occupancy-Enhanced Object Grasping with Multiple Triplanar Projection
This paper addresses the challenge of robotic grasping of general objects.
TARGO: Benchmarking Target-driven Object Grasping under Occlusions
3) We also generate a large-scale training dataset via a scalable pipeline, which can be used to boost the performance of grasping under occlusion and generalized to the real world.
Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision
In this work, we introduce a novel approach to address Multimodal Open-Set Domain Generalization (MM-OSDG) for the first time, utilizing self-supervision.
Human-centered In-building Embodied Delivery Benchmark
This environment also includes autonomous human characters and robots with grasping and mobility capabilities, as well as a large number of interactive items.
Make Graph Neural Networks Great Again: A Generic Integration Paradigm of Topology-Free Patterns for Traffic Speed Prediction
To tackle this challenge, we propose a generic model for enabling the current GNN-based methods to preserve topology-free patterns.
SpatialBot: Precise Spatial Understanding with Vision Language Models
Vision Language Models (VLMs) have achieved impressive performance in 2D image understanding, however they are still struggling with spatial understanding which is the foundation of Embodied AI.
Ranked #4 on
Spatial Reasoning
on 6-DoF SpatialBench
AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation
The ability to reflect on and correct failures is crucial for robotic systems to interact stably with real-life objects. Observing the generalization and reasoning capabilities of Multimodal Large Language Models (MLLMs), previous approaches have aimed to utilize these models to enhance robotic systems accordingly. However, these methods typically focus on high-level planning corrections using an additional MLLM, with limited utilization of failed samples to correct low-level contact poses which is particularly prone to occur during articulated object manipulation. To address this gap, we propose an Autonomous Interactive Correction (AIC) MLLM, which makes use of previous low-level interaction experiences to correct SE(3) pose predictions for articulated object.
GFPack++: Improving 2D Irregular Packing by Learning Gradient Field with Attention
The difficulty of learning from teacher packing is to capture the complex geometric relationships among packing examples, which include the spatial (position, orientation) relationships of objects, their geometric features, and container boundary conditions.
InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environment
To reach this goal, we introduce Dynamic Chain-of-Navigation (DCoN) to unify the planning process for different types of navigation instructions.
Omni6DPose: A Benchmark and Model for Universal 6D Object Pose Estimation and Tracking
6D Object Pose Estimation is a crucial yet challenging task in computer vision, suffering from a significant lack of large-scale datasets.
Anomaly Anything: Promptable Unseen Visual Anomaly Generation
Visual anomaly detection (AD) presents significant challenges due to the scarcity of anomalous data samples.
Learning Manipulation by Predicting Interaction
To this end, we propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction (MPI) and enhances the visual representation. Given a pair of keyframes representing the initial and final states, along with language instructions, our algorithm predicts the transition frame and detects the interaction object, respectively.
MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities
Extensive experiments on MultiOOD demonstrate that training with A2D and NP-Mix improves existing OOD detection algorithms by a large margin.
UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence
Garment manipulation (e. g., unfolding, folding and hanging clothes) is essential for future robots to accomplish home-assistant tasks, while highly challenging due to the diversity of garment configurations, geometries and deformations.
No Time to Train: Empowering Non-Parametric Networks for Few-shot 3D Scene Segmentation
To reduce the reliance on large-scale datasets, recent works in 3D segmentation resort to few-shot learning.
PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environments
Robotic manipulation with two-finger grippers is challenged by objects lacking distinct graspable features.
SCANet: Correcting LEGO Assembly Errors with Self-Correct Assembly Network
To support research in this area, we present the LEGO Error Correction Assembly Dataset (LEGO-ECA), comprising manual images for assembly steps and instances of assembly failures.
NaturalVLM: Leveraging Fine-grained Natural Language for Affordance-Guided Visual Manipulation
To address these challenges, we introduce a comprehensive benchmark, NrVLM, comprising 15 distinct manipulation tasks, containing over 4500 episodes meticulously annotated with fine-grained language instructions.
JSTR: Joint Spatio-Temporal Reasoning for Event-based Moving Object Detection
In spatial reasoning stage, we project the compensated events into the same image coordinate, discretize the timestamp of events to obtain a time image that can reflect the motion confidence, and further segment the moving object through adaptive threshold on the time image.
Sym-Q: Adaptive Symbolic Regression via Sequential Decision-Making
Our results demonstrate that Sym-Q excels not only in recovering underlying mathematical structures but also uniquely learns to efficiently refine the output expression based on reward signals, thereby discovering underlying expressions.
SplatMesh: Interactive 3D Segmentation and Editing Using Mesh-Based Gaussian Splatting
A key challenge in fine-grained 3D-based interactive editing is the absence of an efficient representation that balances diverse modifications with high-quality view synthesis under a given memory constraint.
ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation
By fine-tuning the injected adapters, we preserve the inherent common sense and reasoning ability of the MLLMs while equipping them with the ability for manipulation.
Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers
Therefore, there is a need to develop a scalable framework for geometric fracture assembly without relying on semantic information.
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
Learning Part Motion of Articulated Objects Using Spatially Continuous Neural Implicit Representations
More importantly, while many methods could only model a certain kind of joint motion (such as the revolution in the clockwise order), our proposed framework is generic to different kinds of joint motions in that transformation matrix can model diverse kinds of joint motions in the space.
NNG-Mix: Improving Semi-supervised Anomaly Detection with Pseudo-anomaly Generation
While AD is typically treated as an unsupervised learning task due to the high cost of label annotation, it is more practical to assume access to a small set of labeled anomaly samples from domain experts, as is the case for semi-supervised anomaly detection.
SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization
In real-world scenarios, achieving domain generalization (DG) presents significant challenges as models are required to generalize to unknown target distributions.
SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation
Central to SparseDFF is a feature refinement network, optimized with a contrastive loss between views and a point-pruning mechanism for feature continuity.
Learning Gradient Fields for Scalable and Generalizable Irregular Packing
In this work, we delve deeper into a novel machine learning-based approach that formulates the packing problem as conditional generative modeling.
ImageManip: Image-based Robotic Manipulation with Affordance-guided Next View Selection
This framework is designed to capture multiple perspectives of the target object and infer depth information to complement its geometry.
Improving Compositional Text-to-image Generation with Large Vision-Language Models
However, compositional text-to-image models frequently encounter difficulties in generating high-quality images that accurately align with input texts describing multiple objects, variable attributes, and intricate spatial relationships.
Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions
The performances on the representative VLN task R2R show that our method surpasses the leading zero-shot VLN model by a large margin on all metrics.
Semi-supervised Domain Adaptation in Graph Transfer Learning
To deal with the domain shift, we add adaptive shift parameters to each of the source nodes, which are trained in an adversarial manner to align the cross-domain distributions of node embedding, thus the node classifier trained on labeled source nodes can be transferred to the target nodes.
GRAPH DOMAIN ADAPTATION
Semi-supervised Domain Adaptation
+2
Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation
The task of Visual Object Navigation (VON) involves an agent's ability to locate a particular object within a given scene.
Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects
Our framework explicitly estimates the geometric similarity across different categories, identifying local areas that differ from shapes in the training categories for efficient exploration while concurrently transferring affordance knowledge to similar parts of the objects.
Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assembly
Shape assembly aims to reassemble parts (or fragments) into a complete object, which is a common task in our daily life.
GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Grasping
In this paper, we propose a novel task called human-assisting dexterous grasping that aims to train a policy for controlling a robotic hand's fingers to assist users in grasping objects.
Score-PA: Score-based 3D Part Assembly
In this paper, we formulate this task from a novel generative perspective, introducing the Score-based 3D Part Assembly framework (Score-PA) for 3D part assembly.
Temporal Inductive Path Neural Network for Temporal Knowledge Graph Reasoning
Subsequently, we utilize the defined query-aware temporal paths on a history temporal graph to model historical path information related to queries for reasoning.
Mixup-Augmented Meta-Learning for Sample-Efficient Fine-Tuning of Protein Simulators
Molecular dynamics simulations have emerged as a fundamental instrument for studying biomolecules.
Less is More: Towards Efficient Few-shot 3D Semantic Segmentation via Training-free Networks
However, the prior pre-training stage not only introduces excessive time overhead, but also incurs a significant domain gap on `unseen' classes.
3D Semantic Segmentation
Few-shot 3D semantic segmentation
+1
GenPose: Generative Category-level Object Pose Estimation via Diffusion Models
Object pose estimation plays a vital role in embodied AI and computer vision, enabling intelligent agents to comprehend and interact with their surroundings.
Ranked #1 on
6D Pose Estimation using RGBD
on REAL275
Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation
In this paper, we propose MUTR, a Multi-modal Unified Temporal transformer for Referring video object segmentation.
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model
This paper presents Instruct2Act, a framework that utilizes Large Language Models to map multi-modal instructions to sequential actions for robotic manipulation tasks.
Personalize Segment Anything Model with One Shot
Driven by large-data pre-training, Segment Anything Model (SAM) has been demonstrated as a powerful and promptable framework, revolutionizing the segmentation models.
Ranked #2 on
Personalized Segmentation
on PerSeg
Adaptive Path-Memory Network for Temporal Knowledge Graph Reasoning
Temporal knowledge graph (TKG) reasoning aims to predict the future missing facts based on historical information and has gained increasing research interest recently.
Learning a Universal Human Prior for Dexterous Manipulation from Human Preference
Generating human-like behavior on robots is a great challenge especially in dexterous manipulation tasks with robotic hands.
Skill Reinforcement Learning and Planning for Open-World Long-Horizon Tasks
Our method outperforms baselines by a large margin and is the most sample-efficient demonstration-free RL method to solve Minecraft Tech Tree tasks.
PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations
Learning a generalizable object manipulation policy is vital for an embodied agent to work in complex real-world scenes.
Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation
In this paper, we study deformable object manipulation using dense visual affordance, with generalization towards diverse states, and propose a novel kind of foresightful dense affordance, which avoids local optima by estimating states' values for long-term manipulation.
Resilient Binary Neural Network
In this paper, we introduce a Resilient Binary Neural Network (ReBNN) to mitigate the frequent oscillation for better BNNs' training.
GFPose: Learning 3D Human Pose Prior with Gradient Fields
During the denoising process, GFPose implicitly incorporates pose priors in gradients and unifies various discriminative and generative tasks in an elegant framework.
SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation
However, current works are based on raw data or network feature-level fusion and only consider short-range HD map generation, limiting their deployment to realistic autonomous driving applications.
MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library
A significant challenge facing researchers in the area of multi-agent reinforcement learning (MARL) pertains to the identification of a library that can offer fast and compatible development for multi-agent tasks and algorithm combinations, while obviating the need to consider compatibility issues.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Learning-Based Dimensionality Reduction for Computing Compact and Effective Local Feature Descriptors
A distinctive representation of image patches in form of features is a key component of many computer vision and robotics tasks, such as image matching, image retrieval, and visual localization.
End-to-End Affordance Learning for Robotic Manipulation
Such contact prediction process then leads to an end-to-end affordance learning framework that can generalize over different types of manipulation tasks.
Collaboration of Pre-trained Models Makes Better Few-shot Learner
In this paper, we propose CoMo, a Collaboration of pre-trained Models that incorporates diverse prior knowledge from various pre-training paradigms for better few-shot learning.
Hierarchical Interdisciplinary Topic Detection Model for Research Proposal Classification
Specifically, we first propose a hierarchical transformer to extract the textual semantic information of proposals.
Estimation of Average Derivatives of Latent Regressors: With an Application to Inference on Buffer-Stock Saving
Based on the proposed estimator, we construct a formal test on the sub-unity of the marginal propensity to consume out of permanent income (MPCP) under a nonparametric consumption model and a permanent-transitory model of income dynamics with nonparametric distribution.
TarGF: Learning Target Gradient Field to Rearrange Objects without Explicit Goal Specification
For object rearrangement, the TarGF can be used in two ways: 1) For model-based planning, we can cast the target gradient into a reference control and output actions with a distributed path planner; 2) For model-free reinforcement learning, the TarGF is not only used for estimating the likelihood-change as a reward but also provides suggested actions in residual policy learning.
Online Pole Segmentation on Range Images for Long-term LiDAR Localization in Urban Environments
We further use the extracted poles as pseudo labels to train a deep neural network for online range image-based pole segmentation.
Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects
Commercial depth sensors usually generate noisy and missing depths, especially on specular and transparent objects, which poses critical issues to downstream depth or point cloud-based tasks.
Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARL
The necessity for cooperation among intelligent machines has popularised cooperative multi-agent reinforcement learning (MARL) in the artificial intelligence (AI) research community.
Scalable Model-based Policy Optimization for Decentralized Networked Systems
Reinforcement learning algorithms require a large amount of samples; this often limits their real-world applications on even simple tasks.
Consecutive Pretraining: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domain
Currently, under supervised learning, a model pretrained by a large-scale nature scene dataset and then fine-tuned on a few specific task labeling data is the paradigm that has dominated the knowledge transfer learning.
DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulation
It is essential yet challenging for future home-assistant robots to understand and manipulate diverse 3D objects in daily human environments.
Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
In this study, we propose the Bimanual Dexterous Hands Benchmark (Bi-DexHands), a simulator that involves two dexterous hands with tens of bimanual manipulation tasks and thousands of target objects.
Who Should Review Your Proposal? Interdisciplinary Topic Path Detection for Research Proposals
After extracting the semantic and interdisciplinary knowledge, we design a level-wise prediction component to fuse the two types of knowledge representations and detect interdisciplinary topic paths for each proposal.
GraspARL: Dynamic Grasping via Adversarial Reinforcement Learning
we formulate the dynamic grasping problem as a 'move-and-grasp' game, where the robot is to pick up the object on the mover and the adversarial mover is to find a path to escape it.
RoboAssembly: Learning Generalizable Furniture Assembly Policy in a Novel Multi-robot Contact-rich Simulation Environment
Part assembly is a typical but challenging task in robotics, where robots assemble a set of individual parts into a complete shape.
Edge-Enhanced Dual Discriminator Generative Adversarial Network for Fast MRI with Parallel Imaging Using Multi-view Information
The dual discriminator design aims to improve the edge information in MRI reconstruction.
AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions
Perceiving and interacting with 3D articulated objects, such as cabinets, doors, and faucets, pose particular challenges for future home-assistant robots performing daily tasks in human environments.
Fully Decentralized Model-based Policy Optimization with Networked Agents
Model-based RL is an effective approach for reducing sample complexity.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
Fast and Flexible Human Pose Estimation with HyperPose
When it comes to customising these algorithms for real-world applications, none of the existing libraries can offer both the flexibility of developing custom pose estimation algorithms and the high-performance of executing these algorithms on commodity devices.
Contrastive Multimodal Fusion with TupleInfoNCE
Another approach is to concatenate all the modalities into a tuple and then contrast positive and negative tuple correspondences.
Ranked #82 on
Semantic Segmentation
on NYU Depth v2
VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects
In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware, interaction-aware, and task-aware visual action affordance and trajectory proposals.
Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learning
However, grasping distinguishable skills for some tasks with non-unique optima can be essential for further improving its learning efficiency and performance, which may lead to a multimodal policy represented as a mixture-of-experts (MOE).
Product semantics translation from brain activity via adversarial learning
To modify a design semantic of a given product from personalised brain activity via adversarial learning, in this work, we propose a deep generative transformation model to modify product semantics from the brain signal.
LightCAKE: A Lightweight Framework for Context-Aware Knowledge Graph Embedding
Knowledge graph embedding (KGE) models learn to project symbolic entities and relations into a continuous vector space based on the observed triplets.
P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding
Self-supervised representation learning is a critical problem in computer vision, as it provides a way to pretrain feature extractors on large unlabeled datasets that can be used as an initialization for more efficient and effective training on downstream tasks.
End-to-End Object Detection with Adaptive Clustering Transformer
In this paper, a novel variant of transformer named Adaptive Clustering Transformer(ACT) has been proposed to reduce the computation cost for high-resolution input.
Bilateral Asymmetry Guided Counterfactual Generating Network for Mammogram Classification
Mammogram benign or malignant classification with only image-level labels is challenging due to the absence of lesion annotations.
The DongNiao International Birds 10000 Dataset
DongNiao International Birds 10000 (DIB-10K) is a challenging image dataset which has more than 10 thousand different types of birds.
Lyapunov-Based Reinforcement Learning for Decentralized Multi-Agent Control
More importantly, the existing multi-agent reinforcement learning (MARL) algorithms cannot ensure the closed-loop stability of a multi-agent system from a control-theoretic perspective, so the learned control polices are highly possible to generate abnormal or dangerous behaviors in real applications.
Deep Reinforcement Learning
Multi-agent Reinforcement Learning
+2
Efficient Reinforcement Learning Development with RLzoo
RLzoo provides developers with (i) high-level yet flexible APIs for prototyping DRL agents, and further customising the agents for best performance, (ii) a model zoo where users can import a wide range of DRL agents and easily compare their performance, and (iii) an algorithm that can automatically construct DRL agents with custom components (which are critical to improve agent's performance in custom applications).
Generative 3D Part Assembly via Dynamic Graph Learning
Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation.
Role-Wise Data Augmentation for Knowledge Distillation
To our knowledge, existing methods overlook the fact that although the student absorbs extra knowledge from the teacher, both models share the same input data -- and this data is the only medium by which the teacher's knowledge can be demonstrated.
Unpaired Image-to-Image Translation using Adversarial Consistency Loss
Cycle-consistency loss is a widely used constraint for such problems.
DLGAN: Disentangling Label-Specific Fine-Grained Features for Image Manipulation
By mapping the discrete label-specific attribute features into a continuous prior distribution, we leverage the advantages of both discrete labels and reference images to achieve image manipulation in a hybrid fashion.
Gear Training: A new way to implement high-performance model-parallel training
The training of Deep Neural Networks usually needs tremendous computing resources.
Generative Creativity: Adversarial Learning for Bionic Design
Bionic design refers to an approach of generative creativity in which a target object (e. g. a floor lamp) is designed to contain features of biological source objects (e. g. flowers), resulting in creative biologically-inspired design.
Dropping Activation Outputs with Localized First-layer Deep Network for Enhancing User Privacy and Data Security
However, current architecture of deep networks suffers the privacy issue that users need to give out their data to the model (typically hosted in a server or a cluster on Cloud) for training or prediction.
TensorLayer: A Versatile Library for Efficient Deep Learning Development
Deep learning has enabled major advances in the fields of computer vision, natural language processing, and multimedia among many others.
Semantic Image Synthesis via Adversarial Learning
In this paper, we propose a way of synthesizing realistic images directly with natural language description, which has many useful applications, e. g. intelligent image manipulation.
Deep De-Aliasing for Fast Compressive Sensing MRI
Fast Magnetic Resonance Imaging (MRI) is highly in demand for many clinical applications in order to reduce the scanning cost and improve the patient experience.
Automatic Brain Tumor Detection and Segmentation Using U-Net Based Fully Convolutional Networks
In this context, a reliable fully automatic segmentation method for the brain tumor segmentation is necessary for an efficient measurement of the tumor extent.
I2T2I: Learning Text to Image Synthesis with Textual Data Augmentation
We demonstrate that %the capability of our method to understand the sentence descriptions, so as to I2T2I can generate better multi-categories images using MSCOCO than the state-of-the-art.
DeepSleepNet: a Model for Automatic Sleep Stage Scoring based on Raw Single-Channel EEG
This demonstrated that, without changing the model architecture and the training algorithm, our model could automatically learn features for sleep stage scoring from different raw single-channel EEGs from different datasets without utilizing any hand-engineered features.
Ranked #4 on
Sleep Stage Detection
on MASS SS3
Unsupervised Image-to-Image Translation with Generative Adversarial Networks
It's useful to automatically transform an image from its original form to some synthetic form (style, partial contents, etc.
Mixed Neural Network Approach for Temporal Sleep Stage Classification
Use of this recording configuration with neural network deconvolution promises to make clinically indicated home sleep studies practical.
DropNeuron: Simplifying the Structure of Deep Neural Networks
We proposed regularisers which support a simple mechanism of dropping neurons during a network training process.
