Search Results for author:
Found 306 papers, 104 papers with code
Rethink the Role of Deep Learning towards Large-scale Quantum Systems
Characterizing the ground state properties of quantum systems is fundamental to capturing their behavior but computationally challenging.
A Finite-Sample Analysis of Distributionally Robust Average-Reward Reinforcement Learning
Robust reinforcement learning (RL) under the average-reward criterion is crucial for long-term decision making under potential environment mismatches, yet its finite-sample complexity study remains largely unexplored.
Not All Documents Are What You Need for Extracting Instruction Tuning Data
Instruction tuning improves the performance of large language models (LLMs), but it heavily relies on high-quality training data.
KINDLE: Knowledge-Guided Distillation for Prior-Free Gene Regulatory Network Inference
Subsequent methods integrate prior knowledge to mitigate this challenge by restricting the solution space to biologically plausible interactions.
Divide, Optimize, Merge: Fine-Grained LLM Agent Optimization at Scale
LLM-based optimization has shown remarkable potential in enhancing agentic systems.
Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Manipulating Human Perceptual Variability
Human decision-making in cognitive tasks and daily life exhibits considerable variability, shaped by factors such as task difficulty, individual preferences, and personal experiences.
DianJin-R1: Evaluating and Enhancing Financial Reasoning in Large Language Models
Effective reasoning remains a core challenge for large language models (LLMs) in the financial domain, where tasks often require domain-specific knowledge, precise numerical calculations, and strict adherence to compliance rules.
Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models
The composition of pre-training datasets for large language models (LLMs) remains largely undisclosed, hindering transparency and efforts to optimize data quality, a critical driver of model performance.
Physics Informed Constrained Learning of Dynamics from Static Data
In this study, we developed a new PINN learning paradigm, namely Constrained Learning, that enables the approximation of first-order derivatives or motions using non-time course or partially observed data.
Probing and Inducing Combinational Creativity in Vision-Language Models
The ability to combine existing concepts into novel ideas stands as a fundamental hallmark of human intelligence.
Phased Array Calibration based on Rotating-Element Harmonic Electric-Field Vector with Time Modulation
While amplitude-only calibration methods offer advantages when phase measurements are impractical, conventional approaches face two key challenges: they typically require high-resolution phase shifters and remain susceptible to phase errors inherent in these components.
Metric-Solver: Sliding Anchored Metric Depth Estimation from a Single Image
Our design enables a unified and adaptive depth representation across diverse environments.
OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding
In this paper, we propose a novel framework for controllable video diffusion, OmniVDiff, aiming to synthesize and comprehend multiple video visual content in a single diffusion model.
Learning to Be A Doctor: Searching for Effective Medical Agent Architectures
Motivated by the success of automated machine learning (AutoML), this paper introduces a novel framework for the automated design of medical agent architectures.
A simulation-heuristics dual-process model for intuitive physics
The role of mental simulation in human physical reasoning is widely acknowledged, but whether it is employed across scenarios with varying simulation costs and where its boundary lies remains unclear.
HD-RAG: Retrieval-Augmented Generation for Hybrid Documents Containing Text and Hierarchical Tables
The Hybrid Document RAG task aims to integrate textual and hierarchical tabular data for more comprehensive retrieval and generation in complex scenarios.
Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
We introduce Seed1. 5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks.
VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
We present VAPO, Value-based Augmented Proximal Policy Optimization framework for reasoning models., a novel framework tailored for reasoning models within the value-based paradigm.
CADCrafter: Generating Computer-Aided Design Models from Unconstrained Images
Creating CAD digital twins from the physical world is crucial for manufacturing, design, and simulation.
Patients Speak, AI Listens: LLM-based Analysis of Online Reviews Uncovers Key Drivers for Urgent Care Satisfaction
Our results show that interpersonal factors and operational efficiency emerge as the strongest determinants of patient satisfaction in urgent care, while technical quality, finances, and facilities show no significant independent effects when adjusted for in multivariate models.
InstructVEdit: A Holistic Approach for Instructional Video Editing
Video editing according to instructions is a highly challenging task due to the difficulty in collecting large-scale, high-quality edited video pair data.
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Inference scaling empowers LLMs with unprecedented reasoning ability, with reinforcement learning as the core technique to elicit complex reasoning.
Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness
While image-text foundation models have succeeded across diverse downstream tasks, they still face challenges in the presence of spurious correlations between the input and label.
NFIG: Autoregressive Image Generation with Next-Frequency Prediction
Autoregressive models have achieved promising results in natural language processing.
AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
Introducing AgiBot World, a large-scale platform comprising over 1 million trajectories across 217 tasks in five deployment scenarios, we achieve an order-of-magnitude increase in data scale compared to existing datasets.
Towards Universal Learning-based Model for Cardiac Image Reconstruction: Summary of the CMRxRecon2024 Challenge
In addition, through a detailed analysis of the results submitted to the challenge, we have also made several findings, including: 1) adaptive prompt-learning embedding is an effective means for achieving strong generalization in reconstruction models; 2) enhanced data consistency based on physics-informed networks is also an effective pathway toward a universal model; 3) traditional evaluation metrics have limitations when assessing ground-truth references with moderate or lower image quality, highlighting the need for subjective evaluation methods.
AppAgentX: Evolving GUI Agents as Proficient Smartphone Users
This allows the agent to focus on tasks requiring more complex reasoning, while simplifying routine actions.
Distill Any Depth: Distillation Creates a Stronger Monocular Depth Estimator
Recent advances in zero-shot monocular depth estimation(MDE) have significantly improved generalization by unifying depth distributions through normalized depth representations and by leveraging large-scale unlabeled data via pseudo-label distillation.
Ranked #1 on
Depth Estimation
on ScanNetV2
External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation
Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system.
Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusion
To address these issues, we propose a monocular framework, which is the first to excel in both segmentation and depth estimation of transparent objects, with only a single-image input.
Neural Force Field: Learning Generalized Physical Representation from a Few Examples
Physical reasoning is a remarkable human ability that enables rapid learning and generalization from limited experience.
PoI: Pixel of Interest for Novel View Synthesis Assisted Scene Coordinate Regression
The task of estimating camera poses can be enhanced through novel view synthesis techniques such as NeRF and Gaussian Splatting to increase the diversity and extension of training data.
UniForm: A Unified Multi-Task Diffusion Transformer for Audio-Video Generation
With the rise of diffusion models, audio-video generation has been revolutionized.
MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agent
Specifically, the agent extracts the object movement and camera motion described in the text and converts them into object trajectories and camera extrinsics, respectively.
Learning to Plan with Personalized Preferences
Effective integration of AI agents into daily life requires them to understand and adapt to individual human preferences, particularly in collaborative roles.
Behavior Modeling Space Reconstruction for E-Commerce Search
Delivering superior search services is crucial for enhancing customer experience and driving revenue growth.
Exploring Siamese Networks in Self-Supervised Fast MRI Reconstruction
Reconstructing MR images using deep neural networks from undersampled k-space data without using fully sampled training references offers significant value in practice, which is a self-supervised regression problem calling for effective prior knowledge and supervision.
DualOpt: A Dual Divide-and-Optimize Algorithm for the Large-scale Traveling Salesman Problem
This paper proposes a dual divide-and-optimize algorithm (DualOpt) for solving the large-scale traveling salesman problem (TSP).
Molecule-dynamic-based Aging Clock and Aging Roadmap Forecast with Sundial
Addressing the unavoidable bias inherent in supervised aging clocks, we introduce Sundial, a novel framework that models molecular dynamics through a diffusion field, capturing both the population-level aging process and the individual-level relative aging order.
Training-Free Mitigation of Adversarial Attacks on Deep Learning-Based MRI Reconstruction
In this work, we propose a novel approach for mitigating adversarial attacks on MRI reconstruction models without any retraining.
VAST 1.0: A Unified Framework for Controllable and Consistent Video Generation
Generating high-quality videos from textual descriptions poses challenges in maintaining temporal coherence and control over subject motion.
Adaptable and Precise: Enterprise-Scenario LLM Function-Calling Capability Training Pipeline
In this paper, we propose a training pipeline for function-calling capabilities tailored to real-world business scenarios.
Proposing and solving olympiad geometry with guided tree search
Mathematics olympiads are prestigious competitions, with problem proposing and solving highly honored.
StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elements
Recent advancements in text-to-image models have improved the nuance of style transformations, yet significant challenges remain, particularly with overfitting to reference styles, limiting stylistic control, and misaligning with textual content.
UniScene: Unified Occupancy-centric Driving Scene Generation
UniScene employs a progressive generation process that decomposes the complex task of scene generation into two hierarchical steps: (a) first generating semantic occupancy from a customized scene layout as a meta scene representation rich in both semantic and geometric information, and then (b) conditioned on occupancy, generating video and LiDAR data, respectively, with two novel transfer strategies of Gaussian-based Joint Rendering and Prior-guided Sparse Modeling.
Human Multi-View Synthesis from a Single-View Model:Transferred Body and Face Representations
Specifically, we use a single-view model pretrained on a large-scale human dataset to develop a multi-view body representation, aiming to extend the 2D knowledge of the single-view model to a multi-view diffusion model.
Predicting Pedestrian Crossing Behavior in Germany and Japan: Insights into Model Transferability
When comparing the differences between countries, pedestrians from the study conducted in Japan are more cautious, selecting larger gaps compared to those in Germany.
LoRA of Change: Learning to Generate LoRA for the Editing Instruction from A Single Before-After Image Pair
Furthermore, generalizable models for image editing with visual instructions typically require quad data, i. e., a before-after image pair, along with query and target images.
Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry
Here, we present the outcomes from the second Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry, which engaged participants across global hybrid locations, resulting in 34 team submissions.
On the Foundation Model for Cardiac MRI Reconstruction
ML-based reconstruction, however, also requires substantial data and computational time to train the neural network, which is often optimized for a fixed acceleration rate or image contrast.
Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models
We use GPT4V to bridge the gap between the reference image and the text input for the T2I model, allowing T2I models to understand image content.
MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D
Furthermore, MVPaint employs a UVR module to improve the texture quality in the UV space, which first performs a UV-space Super-Resolution, followed by a Spatial-aware Seam-Smoothing algorithm for revising spatial texturing discontinuities caused by UV unwrapping.
LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systems
Transformer models have achieved remarkable success in sequential recommender systems (SRSs).
LLM4PR: Improving Post-Ranking in Search Engine with Large Language Models
In this study, we introduce a novel paradigm named Large Language Models for Post-Ranking in search engine (LLM4PR), which leverages the capabilities of LLMs to accomplish the post-ranking task in SE.
ROA-BEV: 2D Region-Oriented Attention for BEV-based 3D Object
Vision-based BEV (Bird-Eye-View) 3D object detection has recently become popular in autonomous driving.
Multi-Agent Collaborative Data Selection for Efficient LLM Pretraining
Efficient data selection is crucial to accelerate the pretraining of large language models (LLMs).
HybridFlow: A Flexible and Efficient RLHF Framework
Traditional RL can be modeled as a dataflow, where each node represents computation of a neural network (NN) and each edge denotes data dependencies between the NNs.
Harnessing Diversity for Important Data Selection in Pretraining Large Language Models
For each cluster, if we opt to select data from it, we take some samples to evaluate the influence to prevent processing all instances.
dnaGrinder: a lightweight and high-capacity genomic foundation model
In this context, recent advancements in large language model research have led to the development of both encoder-only and decoder-only foundation models designed to decode intricate information in DNA sequences.
Unleashing the Power of Generic Segmentation Models: A Simple Baseline for Infrared Small Target Detection
Recent advancements in deep learning have greatly advanced the field of infrared small object detection (IRSTD).
Panacea+: Panoramic and Controllable Video Generation for Autonomous Driving
The field of autonomous driving increasingly demands high-quality annotated video training data.
Divide and Conquer: Improving Multi-Camera 3D Perception with 2D Semantic-Depth Priors and Input-Dependent Queries
Additionally, the input-independent nature of initial queries also limits the learning capacity of Transformer-based models.
MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization
Meshes are the de facto 3D representation in the industry but are labor-intensive to produce.
AppAgent v2: Advanced Agent for Flexible Mobile Interactions
In the deployment phase, RAG technology enables efficient retrieval and update from this knowledge base, thereby empowering the agent to perform tasks effectively and accurately.
StreamMOTP: Streaming and Unified Framework for Joint 3D Multi-Object Tracking and Trajectory Prediction
3D multi-object tracking and trajectory prediction are two crucial modules in autonomous driving systems.
Efficient and Long-Tailed Generalization for Pre-trained Vision-Language Model
For efficient adaptation, we treat the CLIP model as a black box and leverage the extracted features to obtain visual and textual prototypes for prediction.
MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers
Recently, 3D assets created via reconstruction and generation have matched the quality of manually crafted assets, highlighting their potential for replacement.
EMMA: Your Text-to-Image Diffusion Model Can Secretly Accept Multi-Modal Prompts
Recent advancements in image generation have enabled the creation of high-quality images from text conditions.
MeshXL: Neural Coordinate Field for Generative 3D Foundation Models
The polygon mesh representation of 3D data exhibits great flexibility, fast rendering speed, and storage efficiency, which is widely preferred in various applications.
Do Finetti: On Causal Effects for Exchangeable Data
We study causal effect estimation in a setting where the data are not i. i. d.
The RoboDrive Challenge: Drive Anytime Anywhere in Any Condition
In the realm of autonomous driving, robust perception under out-of-distribution conditions is paramount for the safe deployment of vehicles.
Learning Monocular Depth from Focus with Event Focal Stack
Depth from Focus estimates depth by determining the moment of maximum focus from multiple shots at different focal distances, i. e. the Focal Stack.
Super-Resolving Blurry Images with Events
Super-resolution from motion-blurred images poses a significant challenge due to the combined effects of motion blur and low spatial resolution.
Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond
General world models represent a crucial pathway toward achieving Artificial General Intelligence (AGI), serving as the cornerstone for various applications ranging from virtual environments to decision-making systems.
Non-Uniform Exposure Imaging via Neuromorphic Shutter Control
To address this challenge, we propose a novel Neuromorphic Shutter Control (NSC) system to avoid motion blurs and alleviate instant noises, where the extremely low latency of events is leveraged to monitor the real-time motion and facilitate the scene-adaptive exposure.
SSVT: Self-Supervised Vision Transformer For Eye Disease Diagnosis Based On Fundus Images
Machine learning-based fundus image diagnosis technologies trigger worldwide interest owing to their benefits such as reducing medical resource power and providing objective evaluation results.
Diagnosis of Multiple Fundus Disorders Amidst a Scarcity of Medical Experts Via Self-supervised Machine Learning
Fundus diseases are major causes of visual impairment and blindness worldwide, especially in underdeveloped regions, where the shortage of ophthalmologists hinders timely diagnosis.
FipTR: A Simple yet Effective Transformer Framework for Future Instance Prediction in Autonomous Driving
In this paper, we propose a simple yet effective fully end-to-end framework named Future Instance Prediction Transformer(FipTR), which views the task as BEV instance segmentation and prediction for future frames.
HybriMap: Hybrid Clues Utilization for Effective Vectorized HD Map Construction
Constructing vectorized high-definition maps from surround-view cameras has garnered significant attention in recent years.
Predicting and Analyzing Pedestrian Crossing Behavior at Unsignalized Crossings
Predicting gap selection behavior and the use of zebra crossing enables driving systems to proactively respond and prevent potential conflicts.
SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Driving
End-to-End paradigms use a unified framework to implement multi-tasks in an autonomous driving system.
MotionChain: Conversational Motion Controllers via Multimodal Prompts
However, this proficiency remains largely unexplored in other multimodal generative models, particularly in human motion models.
SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Control
Autonomous driving progress relies on large-scale annotated datasets.
Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation
For metric depth estimation, we show that the key to a zero-shot single-view model lies in resolving the metric ambiguity from various camera models and large-scale data training.
Ranked #1 on
Surface Normals Estimation
on NYU Depth v2
(using extra training data)
SSDRec: Self-Augmented Sequence Denoising for Sequential Recommendation
In the second stage, we devise a self-augmentation module to augment sequences to alleviate OUPs.
Edge-based Parametric Digital Twins for Intelligent Building Indoor Climate Modeling
Digital transformation in the built environment generates vast data for developing data-driven models to optimize building operations.
MovieLLM: Enhancing Long Video Understanding with AI-Generated Movies
Development of multimodal models has marked a significant step forward in how machines understand videos.
LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language
Experiments illustrate that LangGPT significantly enhances the performance of LLMs.
Neural Networks with (Low-Precision) Polynomial Approximations: New Insights and Techniques for Accuracy Improvement
Experiments showed that combination of our solutions is very effective: at the same precision, our PANN is 10% to 50% more accurate than state-of-the-arts; and at the same accuracy, our PANN only requires a precision of 2^{-9} while state-of-the-art solution requires a precision of 2^{-12} using the ResNet-20 model on CIFAR-10 dataset.
CounterCLR: Counterfactual Contrastive Learning with Non-random Missing Data in Recommendation
Recommender systems are designed to learn user preferences from observed feedback and comprise many fundamental tasks, such as rating prediction and post-click conversion rate (pCVR) prediction.
Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detection
Multi-view 3D object detection systems often struggle with generating precise predictions due to the challenges in estimating depth from images, increasing redundant and incorrect detections.
Ranked #2 on
3D Object Detection
on nuScenes Camera Only
Integration of cognitive tasks into artificial general intelligence test for large models
During the evolution of large models, performance evaluation is necessarily performed to assess their capabilities and ensure safety before practical application.
A Survey on Data-Centric Recommender Systems
Recommender systems (RSs) have become an essential tool for mitigating information overload in a range of real-world applications.
Stream Query Denoising for Vectorized HD Map Construction
This paper introduces the Stream Query Denoising (SQD) strategy as a novel approach for temporal modeling in high-definition map (HD-map) construction.
Adaptive Hardness Negative Sampling for Collaborative Filtering
Negative sampling is essential for implicit collaborative filtering to provide proper negative training signals so as to achieve desirable performance.
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding Reasoning and Planning
Recent progress in Large Multimodal Models (LMM) has opened up great possibilities for various applications in the field of human-machine interactions.
DreamGaussian4D: Generative 4D Gaussian Splatting
Specifically, we propose an integral framework with two major modules: 1) Image-to-4D GS - we initially generate static GS with DreamGaussianHD, followed by HexPlane-based dynamic generation with Gaussian deformation; and 2) Video-to-Video Texture Refinement - we refine the generated UV-space texture maps and meanwhile enhance their temporal consistency by utilizing a pre-trained image-to-video diffusion model.
AppAgent: Multimodal Agents as Smartphone Users
Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks.
M3DBench: Let's Instruct Large Models with Multi-modal 3D Prompts
Furthermore, we establish a new benchmark for assessing the performance of large models in understanding multi-modal 3D prompts.
Lever LM: Configuring In-Context Sequence to Lever Large Vision Language Models
As Archimedes famously said, ``Give me a lever long enough and a fulcrum on which to place it, and I shall move the world'', in this study, we propose to use a tiny Language Model (LM), \eg, a Transformer with 67M parameters, to lever much larger Vision-Language Models (LVLMs) with 9B parameters.
Non-Cartesian Self-Supervised Physics-Driven Deep Learning Reconstruction for Highly-Accelerated Multi-Echo Spiral fMRI
Functional MRI (fMRI) is an important tool for non-invasive studies of brain function.
Learn to Optimize Denoising Scores for 3D Generation: A Unified and Improved Diffusion Prior on NeRF and 3D Gaussian Splatting
We propose a unified framework aimed at enhancing the diffusion priors for 3D generation tasks.
Creative Agents: Empowering Agents with Imagination for Creative Tasks
We benchmark creative tasks with the challenging open-world game Minecraft, where the agents are asked to create diverse buildings given free-form language instructions.
FaceStudio: Put Your Face Everywhere in Seconds
This study investigates identity-preserving image synthesis, an intriguing task in image generation that seeks to maintain a subject's identity while adding a personalized, stylistic touch.
I-PHYRE: Interactive Physical Reasoning
Current evaluation protocols predominantly assess physical reasoning in stationary scenes, creating a gap in evaluating agents' abilities to interact with dynamic events.
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
However, developing LMMs that can comprehend, reason, and plan in complex and diverse 3D environments remains a challenging topic, especially considering the demand for understanding permutation-invariant point cloud 3D representations of the 3D scene.
ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Model
The advent of large language models, enabling flexibility through instruction-driven approaches, has revolutionized many traditional generative tasks, but large models for 3D data, particularly in comprehensively handling 3D shapes with other modalities, are still under-explored.
Panacea: Panoramic and Controllable Video Generation for Autonomous Driving
This work notably propels the field of autonomous driving by effectively augmenting the training dataset used for advanced BEV perception techniques.
ChartLlama: A Multimodal LLM for Chart Understanding and Generation
Next, we introduce ChartLlama, a multi-modal large language model that we've trained using our created dataset.
GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting
3D editing plays a crucial role in many areas such as gaming and virtual reality.
Transfer Attacks and Defenses for Large Language Models on Coding Tasks
They improve upon previous neural network models of code, such as code2seq or seq2seq, that already demonstrated competitive results when performing tasks such as code summarization and identifying code vulnerabilities.
ADriver-I: A General World Model for Autonomous Driving
Based on the vision-action pairs, we construct a general world model based on MLLM and diffusion model for autonomous driving, termed ADriver-I.
3DGAUnet: 3D generative adversarial networks with a 3D U-Net based generator to achieve the accurate and effective synthesis of clinical tumor image data for pancreatic cancer
Furthermore, this model has the potential to be adapted to other types of solid tumors, hence making significant contributions to the field of medical imaging in terms of image processing models.
Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior
By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to generate its pseudo flow labels.
CrossZoom: Simultaneously Motion Deblurring and Event Super-Resolving
Even though the collaboration between traditional and neuromorphic event cameras brings prosperity to frame-event based vision applications, the performance is still confined by the resolution gap crossing two modalities in both spatial and temporal domains.
Learning Parallax for Stereo Event-based Motion Deblurring
To tackle this problem, we propose a novel coarse-to-fine framework, named NETwork of Event-based motion Deblurring with STereo event and intensity cameras (St-EDNet), to recover high-quality images directly from the misaligned inputs, consisting of a single blurry image and the concurrent event streams.
Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering
In this paper, we propose a learning framework that trains models to predict geometry-preserving depth without requiring extra data or annotations.
PivotNet: Vectorized Pivot Learning for End-to-end HD Map Construction
Furthermore, to supervise the position and topology of the vectorized point predictions, we propose a dynamic vectorized sequence loss.
DPF-Net: Combining Explicit Shape Priors in Deformable Primitive Field for Unsupervised Structural Reconstruction of 3D Objects
Unsupervised methods for reconstructing structures face significant challenges in capturing the geometric details with consistent structures among diverse shapes of the same category.
IT3D: Improved Text-to-3D Generation with Explicit View Synthesis
Recent strides in Text-to-3D techniques have been propelled by distilling knowledge from powerful large text-to-image diffusion models (LDMs).
X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events
Intuitive physics is pivotal for human understanding of the physical world, enabling prediction and interpretation of events even in infancy.
MacFormer: Map-Agent Coupled Transformer for Real-time and Robust Trajectory Prediction
Predicting the future behavior of agents is a fundamental task in autonomous vehicle domains.
StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data
However, these datasets often exhibit domain bias, potentially constraining the generative capabilities of the models.
Ranked #149 on
Visual Question Answering
on MM-Vet
Weakly supervised learning for pattern classification in serial femtosecond crystallography
Th is heavy dependence on labeled datasets will seriously restrict the application of networks, because it is very costly to annotate a large number of diffraction patterns.
ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution
Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation.
A Phase-Coded Time-Domain Interleaved OTFS Waveform with Improved Ambiguity Function
To enhance the sensing performance of the orthogonal time frequency space (OTFS) waveform, we propose a novel time-domain interleaved cyclic-shifted P4-coded OTFS (TICP4-OTFS) with improved ambiguity function.
Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image
State-of-the-art (SOTA) monocular metric depth estimation methods can only handle a single camera model and are unable to perform mixed-data training due to the metric ambiguity.
Ranked #29 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety alignment in large language models (LLMs).
Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models
In this perspective paper, we first comprehensively review existing evaluations of Large Language Models (LLMs) using both standardized tests and ability-oriented benchmarks.
Event Detection from Social Media Stream: Methods, Datasets and Opportunities
Social media streams contain large and diverse amount of information, ranging from daily-life stories to the latest global and local events and news.
MachMap: End-to-End Vectorized Solution for Compact HD-Map Construction
This report introduces the 1st place winning solution for the Autonomous Driving Challenge 2023 - Online HD-map Construction.
End-to-End Vectorized HD-map Construction with Piecewise Bezier Curve
Vectorized high-definition map (HD-map) construction, which focuses on the perception of centimeter-level environmental information, has attracted significant research interest in the autonomous driving community.
MEWL: Few-shot multimodal word learning with referential uncertainty
To fill in this gap, we introduce the MachinE Word Learning (MEWL) benchmark to assess how machines learn word meaning in grounded visual scenes.
StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation
The recent advancements in image-text diffusion models have stimulated research interest in large-scale 3D generative models.
An Overview of Resource Allocation in Integrated Sensing and Communication
Integrated sensing and communication (ISAC) is considered as a promising solution for improving spectrum efficiency and relieving wireless spectrum congestion.
Leveraging Deep Learning and Digital Twins to Improve Energy Performance of Buildings
Digital transformation in buildings accumulates massive operational data, which calls for smart solutions to utilize these data to improve energy performance.
You Only Need Two Detectors to Achieve Multi-Modal 3D Multi-Object Tracking
In the classical tracking-by-detection (TBD) paradigm, detection and tracking are separately and sequentially conducted, and data association must be properly performed to achieve satisfactory tracking performance.
Cross or Wait? Predicting Pedestrian Interaction Outcomes at Unsignalized Crossings
Predicting pedestrian behavior when interacting with vehicles is one of the most critical challenges in the field of automated driving.
Model-Agnostic Reachability Analysis on Deep Neural Networks
Verification plays an essential role in the formal analysis of safety-critical systems.
Skill Reinforcement Learning and Planning for Open-World Long-Horizon Tasks
Our method outperforms baselines by a large margin and is the most sample-efficient demonstration-free RL method to solve Minecraft Tech Tree tasks.
Cyclic Delay-Doppler Shift: A Simple Transmit Diversity Technique for Delay-Doppler Waveforms in Doubly Selective Channels
Delay-Doppler waveform design has been considered as a promising solution to achieve reliable communication under high-mobility channels for the space-air-ground-integrated networks (SAGIN).
Denoising and Prompt-Tuning for Multi-Behavior Recommendation
In practical recommendation scenarios, users often interact with items under multi-typed behaviors (e. g., click, add-to-cart, and purchase).
XFL: A High Performace, Lightweighted Federated Learning Framework
This paper introduces XFL, an industrial-grade federated learning project.
Two-Stage Constrained Actor-Critic for Short Video Recommendation
One the one hand, the platforms aims at optimizing the users' cumulative watch time (main goal) in long term, which can be effectively optimized by Reinforcement Learning.
Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learning
Due to the inability to interact with the environment, offline reinforcement learning (RL) methods face the challenge of estimating the Out-of-Distribution (OOD) points.
Reachability Analysis of Neural Network Control Systems
We then reveal the working principles of applying Lipschitzian optimisation on NNCS verification and illustrate it by verifying an adaptive cruise control model.
Computationally Efficient 3D MRI Reconstruction with Adaptive MLP
Recon3DMLP improves HR 3D reconstruction and outperforms several existing CNN-based models under similar GPU memory consumption, which demonstrates that Recon3DMLP is a practical solution for HR 3D MRI reconstruction.
Generative Gradient Inversion via Over-Parameterized Networks in Federated Learning
Federated learning has gained recognitions as a secure approach for safeguarding local private data in collaborative learning.
Label-Guided Knowledge Distillation for Continual Semantic Segmentation on 2D Images and 3D Point Clouds
To address this issue, we propose a new label-guided knowledge distillation (LGKD) loss, where the old model output is expanded and transplanted (with the guidance of the ground truth label) to form a semantically appropriate class correspondence with the new model output.
Ranked #1 on
Continual Semantic Segmentation
on ScanNet
Discrepant and Multi-Instance Proxies for Unsupervised Person Re-Identification
To completely and accurately represent the information contained in a cluster and learn discriminative features, we propose to maintain discrepant cluster proxies and multi-instance proxies for a cluster.
BEAR: Physics-Principled Building Environment for Control and Reinforcement Learning
Recent advancements in reinforcement learning algorithms have opened doors for researchers to operate and optimize building energy management systems autonomously.
Semantics-Preserving Sketch Embedding for Face Generation
With a semantic feature matching loss for effective semantic supervision, our sketch embedding precisely conveys the semantics in the input sketches to the synthesized images.
Hierarchical Normalization for Robust Monocular Depth Estimation
In this paper, we address monocular depth estimation with deep neural networks.
Ranked #8 on
Monocular Depth Estimation
on ETH3D
Dual Clustering Co-teaching with Consistent Sample Mining for Unsupervised Person Re-Identification
However, training two networks with a set of noisy pseudo labels reduces the complementarity of the two networks and results in label noise accumulation.
On the Learning Mechanisms in Physical Reasoning
Taken together, the results on the challenging benchmark of PHYRE show that LfI is, if not better, as good as LfD for dynamics prediction.
Infrared: A Meta Bug Detector
The recent breakthroughs in deep learning methods have sparked a wave of interest in learning-based bug detectors.
MRF-PINN: A Multi-Receptive-Field convolutional physics-informed neural network for solving partial differential equations
To fill the gaps above, we propose three initiatives in this paper: (1) A Multi-Receptive-Field PINN (MRF-PINN) model is established to solve different types of PDEs on various mesh resolutions without manual tuning; (2) The dimensional balance method is used to estimate the loss weights when solving Navier-Stokes equations; (3) The Taylor polynomial is used to pad the virtual nodes near the boundaries for implementing high-order finite difference.
Learning Practical Communication Strategies in Cooperative Multi-Agent Reinforcement Learning
In Multi-Agent Reinforcement Learning, communication is critical to encourage cooperation among agents.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
CRCNet: Few-shot Segmentation with Cross-Reference and Region-Global Conditional Networks
Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images.
Sobolev Training for Implicit Neural Representations with Approximated Image Derivatives
Recently, Implicit Neural Representations (INRs) parameterized by neural networks have emerged as a powerful and promising tool to represent different kinds of signals due to its continuous, differentiable properties, showing superiorities to classical discretized representations.
KD-MVS: Knowledge Distillation Based Self-supervised Learning for Multi-view Stereo
Supervised multi-view stereo (MVS) methods have achieved remarkable progress in terms of reconstruction quality, but suffer from the challenge of collecting large-scale ground-truth depth.
Few-shot Open-set Recognition Using Background as Unknowns
First, instead of learning the decision boundaries between seen classes, as is done in standard close-set classification, we reserve space for unseen classes, such that images located in these areas are recognized as the unseen classes.
A Synergistic Compilation Workflow for Tackling Crosstalk in Quantum Machines
Evaluations through simulation and on real IBM-Q devices show that our framework can significantly reduce the error rate by up to 6$\times$, with only $\sim$60\% circuit depth compared to state-of-the-art gate scheduling approaches.
TENET: Transformer Encoding Network for Effective Temporal Flow on Motion Prediction
This technical report presents an effective method for motion prediction in autonomous driving.
Ranked #12 on
Motion Forecasting
on Argoverse CVPR 2020
Automatic Generation of Product-Image Sequence in E-commerce
For a platform with billions of products, it is extremely time-costly and labor-expensive to manually pick and organize qualified images.
DETR++: Taming Your Multi-Scale Detection Transformer
Convolutional Neural Networks (CNN) have dominated the field of detection ever since the success of AlexNet in ImageNet classification [12].
On the Perils of Cascading Robust Classifiers
We present \emph{cascade attack} (CasA), an adversarial attack against cascading ensembles, and show that: (1) there exists an adversarial input for up to 88\% of the samples where the ensemble claims to be certifiably robust and accurate; and (2) the accuracy of a cascading ensemble under our attack is as low as 11\% when it claims to be certifiably robust and accurate on 97\% of the test set.
Multi-agent Databases via Independent Learning
Thus, the paper attempts to address the question "Is it possible to design a database consisting of various learned components that cooperatively work to improve end-to-end query latency?".
Constrained Reinforcement Learning for Short Video Recommendation
In this paper, we formulate the problem of short video recommendation as a constrained Markov Decision Process (MDP), where platforms want to optimize the main goal of user watch time in long term, with the constraint of accommodating the auxiliary responses of user interactions such as sharing/downloading videos.
Scenario-based Multi-product Advertising Copywriting Generation for E-Commerce
In this paper, we proposed an automatic Scenario-based Multi-product Advertising Copywriting Generation system (SMPACG) for E-Commerce, which has been deployed on a leading Chinese e-commerce platform.
Correction of out-of-focus microscopic images by deep learning
Results To solve the out-of-focus issue in microscopy, we developed a Cycle Generative Adversarial Network (CycleGAN) based model and a multi-component weighted loss function.
Efficient Few-Shot Object Detection via Knowledge Inheritance
Upon this baseline, we devise an initializer named knowledge inheritance (KI) to reliably initialize the novel weights for the box classifier, which effectively facilitates the knowledge transfer process and boosts the adaptation speed.
Learning the Pedestrian-Vehicle Interaction for Pedestrian Trajectory Prediction
In this paper, we study the interaction between pedestrians and vehicles and propose a novel neural network structure called the Pedestrian-Vehicle Interaction (PVI) extractor for learning the pedestrian-vehicle interaction.
RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior
In this work, we focus on scene flow learning on point clouds in a self-supervised manner.
Multi-Centroid Representation Network for Domain Adaptive Person Re-ID
First, we present a Domain-Specific Contrastive Learning (DSCL) mechanism to fully explore intradomain information by comparing samples only from the same domain.
Contrastive Learning
Domain Adaptive Person Re-Identification
+2
DSGPT: Domain-Specific Generative Pre-Training of Transformers for Text Generation in E-commerce Title and Review Summarization
We propose a novel domain-specific generative pre-training (DS-GPT) method for text generation and apply it to the product titleand review summarization problems on E-commerce mobile display. First, we adopt a decoder-only transformer architecture, which fitswell for fine-tuning tasks by combining input and output all to-gether.
Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning
Extensive experiments show that by incorporating an algebraic treatment, the ALANS learner outperforms various pure connectionist models in domains requiring systematic generalization.
Spatial Ensemble: a Novel Model Smoothing Mechanism for Student-Teacher Framework
Model smoothing is of central importance for obtaining a reliable teacher model in the student-teacher framework, where the teacher generates surrogate supervision signals to train the student.
Effectiveness of Optimization Algorithms in Deep Image Classification
Adam is applied widely to train neural networks.
Parallel Actors and Learners: A Framework for Generating Scalable RL Implementations
Current implementations exhibit poor performance due to challenges such as irregular memory accesses and thread-level synchronization overheads on CPU.
BRAC+: Improved Behavior Regularized Actor Critic for Offline Reinforcement Learning
This leads to large overestimations of the Q values and performance deterioration of the learned policy.
Degradation Attacks on Certifiably Robust Neural Networks
Certifiably robust neural networks employ provable run-time defenses against adversarial examples by checking if the model is locally robust at the input under evaluation.
Adaptive Reliability Analysis for Multi-fidelity Models using a Collective Learning Strategy
In this context, CLF provides a new direction for quantifying the impact of new training points and can be easily extended with new learning functions to adapt to different reliability problems.
Meta Navigator: Search for a Good Adaptation Policy for Few-shot Learning
Inspired by the recent success in Automated Machine Learning literature (AutoML), in this paper, we present Meta Navigator, a framework that attempts to solve the aforementioned limitation in few-shot learning by seeking a higher-level strategy and proffer to automate the selection from various few-shot learning designs.
GeneAnnotator: A Semi-automatic Annotation Tool for Visual Scene Graph
In this manuscript, we introduce a semi-automatic scene graph annotation tool for images, the GeneAnnotator.
Scalable Data Annotation Pipeline for High-Quality Large Speech Datasets Development
Our A/B testing and pilot results demonstrated the HITL pipeline can improve annotation speed and capacity by at least 80% and quality is comparable to or higher than manual double pass annotation.
Spatially and Robustly Hybrid Mixture Regression Model for Inference of Spatial Dependence
Compared with existing spatial regression models, our proposed model assumes the existence a few distinct regression models that are estimated based on observations that exhibit similar response-predictor relationships.
Calibrating Class Activation Maps for Long-Tailed Visual Recognition
First, we present a Class Activation Map Calibration (CAMC) module to improve the learning and prediction of network classifiers, by enforcing network prediction based on important image regions.
Binocular Mutual Learning for Improving Few-shot Classification
From the perspective of class space on base set, existing methods either focus on utilizing all classes under a global view by normal pretraining, or pay more attention to adopt an episodic manner to train meta-tasks within few classes in a local view.
Temporal Knowledge Consistency for Unsupervised Visual Representation Learning
The instance discrimination paradigm has become dominant in unsupervised learning.
DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection
Few-shot object detection, which aims at detecting novel objects rapidly from extremely few annotated examples of previously unseen classes, has attracted significant research interest in the community.
Ranked #5 on
Few-Shot Object Detection
on MS-COCO (1-shot)
Few-shot Segmentation with Optimal Transport Matching and Message Flow
In this work, we argue that every support pixel's information is desired to be transferred to all query pixels and propose a Correspondence Matching Network (CMNet) with an Optimal Transport Matching module to mine out the correspondence between the query and support images.
Unified Regularity Measures for Sample-wise Learning and Generalization
Fundamental machine learning theory shows that different samples contribute unequally both in learning and testing processes.
M2IOSR: Maximal Mutual Information Open Set Recognition
In this work, we aim to address the challenging task of open set recognition (OSR).
IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID
To ensure these two properties to better characterize appropriate intermediate domains, we enforce the bridge losses on intermediate domains' prediction space and feature space, and enforce a diversity loss on the two domain factors.
Principled Hyperedge Prediction with Structural Spectral Features and Neural Networks
SNALS captures the joint interactions of a hyperedge by its local environment, which is retrieved by collecting the spectrum information of their connections.
Social-IWSTCNN: A Social Interaction-Weighted Spatio-Temporal Convolutional Neural Network for Pedestrian Trajectory Prediction in Urban Traffic Scenarios
In this paper, we use the recently released large-scale Waymo Open Dataset in urban traffic scenarios, which includes 374 urban training scenes and 76 urban testing scenes to analyze the performance of our proposed algorithm in comparison to the state-of-the-art (SOTA) models.
More Separable and Easier to Segment: A Cluster Alignment Method for Cross-Domain Semantic Segmentation
Feature alignment between domains is one of the mainstream methods for Unsupervised Domain Adaptation (UDA) semantic segmentation.
Few-Shot Incremental Learning with Continually Evolved Classifiers
First, we adopt a simple but effective decoupled learning strategy of representations and classifiers that only the classifiers are updated in each incremental session, which avoids knowledge forgetting in the representations.
Ranked #9 on
Few-Shot Class-Incremental Learning
on CIFAR-100
class-incremental learning
Few-Shot Class-Incremental Learning
+1
