Search Results for author:
Found 146 papers, 56 papers with code
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
In this way, we can preserve the low-rank constraint in the optimizer while achieving full-rank training for better performance.
Harnessing Diversity for Important Data Selection in Pretraining Large Language Models
For each cluster, if we opt to select data from it, we take some samples to evaluate the influence to prevent processing all instances.
PSLF: A PID Controller-incorporated Second-order Latent Factor Analysis Model for Recommender System
A second-order-based latent factor (SLF) analysis model demonstrates superior performance in graph representation learning, particularly for high-dimensional and incomplete (HDI) interaction data, by incorporating the curvature information of the loss landscape.
COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation
Experiments on three challenging benchmarks demonstrate the effectiveness of COIN, which outperforms the state-of-the-art methods in terms of global human motion estimation and camera motion estimation.
Image Segmentation in Foundation Model Era: A Survey
Image segmentation is a long-standing challenge in computer vision, studied continuously over several decades, as evidenced by seminal algorithms such as N-Cut, FCN, and MaskFormer.
The Key of Parameter Skew in Federated Learning
Federated Learning (FL) has emerged as an excellent solution for performing deep learning on different data owners without exchanging raw data.
Macformer: Transformer with Random Maclaurin Feature Attention
Random feature attention (RFA) adopts random fourier feature (RFF) methods to approximate the softmax function, resulting in a linear time and space attention mechanism that enables the construction of an efficient Transformer.
Neighbor Overlay-Induced Graph Attention Network
Graph neural networks (GNNs) have garnered significant attention due to their ability to represent graph data.
AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
In this paper, we present AquilaMoE, a cutting-edge bilingual 8*16B Mixture of Experts (MoE) language model that has 8 experts with 16 billion parameters each and is developed using an innovative training methodology called EfficientScale.
A Differential Smoothness-based Compact-Dynamic Graph Convolutional Network for Spatiotemporal Signal Recovery
To address this issue, this paper proposes a Compact-Dynamic Graph Convolutional Network (CDGCN) for spatiotemporal signal recovery with the following two-fold ideas: a) leveraging the tensor M-product to build a unified tensor graph convolution framework, which considers both spatial and temporal patterns simultaneously; and b) constructing a differential smoothness-based objective function to reduce the noise interference in spatiotemporal signal, thereby further improve the recovery accuracy.
Generative AI as a Service in 6G Edge-Cloud: Generation Task Offloading by In-context Learning
In particular, we first introduce the communication system model, i. e., allocating radio resources and calculating link capacity to support generated content transmission, and then we present the LLM inference model to calculate the delay of content generation.
Large Language Model (LLM)-enabled In-context Learning for Wireless Network Optimization: A Case Study of Power Control
It avoids the complexity of tedious model training and hyper-parameter fine-tuning, which is a well-known bottleneck of many ML algorithms.
An Extended Kalman Filter Integrated Latent Feature Model on Dynamic Weighted Directed Graphs
A dynamic weighted directed graph (DWDG) is commonly encountered in various application scenarios.
Retrieval-Augmented Generation for Natural Language Processing: A Survey
Large language models (LLMs) have demonstrated great success in various fields, benefiting from their huge amount of parameters that store knowledge.
Vision-Braille: An End-to-End Tool for Chinese Braille Image-to-Text Translation
By incorporating the braille recognition algorithm, this project is the first publicly available braille translation system and can benefit lots of visually impaired students and families who are preparing for the Chinese College Test and help to propel their college dreams in the future.
MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation
Large Multimodal Models (LMMs) exhibit impressive cross-modal understanding and reasoning abilities, often assessed through multiple-choice questions (MCQs) that include an image, a question, and several options.
SMPLOlympics: Sports Environments for Physically Simulated Humanoids
We present SMPLOlympics, a collection of physically simulated environments that allow humanoids to compete in a variety of Olympic sports.
Beyond Raw Videos: Understanding Edited Videos with Large Multimodal Model
However, a large portion of videos in real-world applications are edited videos, \textit{e. g.}, users usually cut and add effects/modifications to the raw video before publishing it on social media platforms.
Keypoint-based Progressive Chain-of-Thought Distillation for LLMs
Chain-of-thought distillation is a powerful technique for transferring reasoning abilities from large language models (LLMs) to smaller student models.
Design Editing for Offline Model-based Optimization
Offline model-based optimization (MBO) aims to maximize a black-box objective function using only an offline dataset of designs and scores.
Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities
Then, we introduce LLM-enabled key techniques and telecom applications in terms of generation, classification, optimization, and prediction problems.
PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios
We address the challenge of content diversity and controllability in pedestrian simulation for driving scenarios.
Measuring Social Norms of Large Language Models
This enables the direct comparison of the social understanding of large language models to humans, more specifically, elementary students.
Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases
We proposed an end-to-end system design towards utilizing Retrieval Augmented Generation (RAG) to improve the factual accuracy of Large Language Models (LLMs) for domain-specific and time-sensitive queries related to private knowledge-bases.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we introduce the Gemini 1. 5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Ranked #1 on
Zero-Shot Video Question Answer
on Video-MME
On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving
However, the oversized neural networks render them impractical for deployment on resource-constrained systems, which unavoidably requires more computational time and resources during reference. To handle this, knowledge distillation offers a promising approach that compresses models by enabling a smaller student model to learn from a larger teacher model.
Measuring Vision-Language STEM Skills of Neural Models
Compared to existing datasets that often focus on examining expert-level ability, our dataset includes fundamental skills and questions designed based on the K-12 curriculum.
Learning to Extract Structured Entities Using Language Models
Recent advances in machine learning have significantly impacted the field of information extraction, with Language Models (LMs) playing a pivotal role in extracting structured information from unstructured text.
FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models
To address the limited data resources and reduce the annotation cost, we introduce FinLLMs, a method for generating financial question-answering data based on common financial formulas using Large Language Models.
Preparing Lessons for Progressive Training on Language Models
The rapid progress of Transformers in artificial intelligence has come at the cost of increased resource consumption and greenhouse gas emissions due to growing model sizes.
An ADRC-Incorporated Stochastic Gradient Descent Algorithm for Latent Factor Analysis
However, such a model commonly encounters the problem of slow convergence because a standard SGD algorithm only considers the current learning error to compute the stochastic gradient without considering the historical and future state of the learning error.
Tensor Graph Convolutional Network for Dynamic Graph Representation Learning
Dynamic graphs (DG) describe dynamic interactions between entities in many practical scenarios.
AGG: Amortized Generative 3D Gaussians for Single Image to 3D
To overcome these challenges, we introduce an Amortized Generative 3D Gaussian framework (AGG) that instantly produces 3D Gaussians from a single image, eliminating the need for per-instance optimization.
GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning
Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations.
Good Better Best: Self-Motivated Imitation Learning for noisy Demonstrations
Imitation Learning (IL) aims to discover a policy by minimizing the discrepancy between the agent's behavior and expert demonstrations.
PACE: Human and Camera Motion Estimation from in-the-wild Videos
This design combines the strengths of SLAM and motion priors, which leads to significant improvements in human and camera motion estimation.
Learning to Generate Parameters of ConvNets for Unseen Image Data
Typical Convolutional Neural Networks (ConvNets) depend heavily on large amounts of image data and resort to an iterative optimization algorithm (e. g., SGD or Adam) to learn network parameters, which makes training very time- and resource-intensive.
TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models
Automated theorem proving (ATP) has become an appealing domain for exploring the reasoning ability of the recent successful generative language models.
FIMO: A Challenge Formal Dataset for Automated Theorem Proving
We present FIMO, an innovative dataset comprising formal mathematical problem statements sourced from the International Mathematical Olympiad (IMO) Shortlisted Problems.
Semantic Graph Representation Learning for Handwritten Mathematical Expression Recognition
Then we design a semantic aware module (SAM), which projects the visual and classification feature into semantic space.
TREA: Tree-Structure Reasoning Schema for Conversational Recommendation
TREA constructs a multi-hierarchical scalable tree as the reasoning structure to clarify the causal relationships between mentioned entities, and fully utilizes historical conversations to generate more reasonable and suitable responses for recommended results.
DREAM: Domain-free Reverse Engineering Attributes of Black-box Model
There is a crucial limitation: these works assume the dataset used for training the target model to be known beforehand and leverage this dataset for model attribute attack.
Shared Growth of Graph Neural Networks via Prompted Free-direction Knowledge Distillation
Knowledge distillation (KD) has shown to be effective to boost the performance of graph neural networks (GNNs), where the typical objective is to distill knowledge from a deeper teacher GNN into a shallower student GNN.
NeRFool: Uncovering the Vulnerability of Generalizable Neural Radiance Fields against Adversarial Perturbations
Generalizable Neural Radiance Fields (GNeRF) are one of the most promising real-world solutions for novel view synthesis, thanks to their cross-scene generalization capability and thus the possibility of instant rendering on new scenes.
Robust Tickets Can Transfer Better: Drawing More Transferable Subnetworks in Transfer Learning
Transfer learning leverages feature representations of deep neural networks (DNNs) pretrained on source tasks with rich data to empower effective finetuning on downstream tasks.
Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion
We introduce a method for generating realistic pedestrian trajectories and full-body animations that can be controlled to meet user-defined goals.
EquiPocket: an E(3)-Equivariant Geometric Graph Neural Network for Ligand Binding Site Prediction
Predicting the binding sites of target proteins plays a fundamental role in drug discovery.
Almost Sure Saddle Avoidance of Stochastic Gradient Methods without the Bounded Gradient Assumption
We prove that various stochastic gradient descent methods, including the stochastic gradient descent (SGD), stochastic heavy-ball (SHB), and stochastic Nesterov's accelerated gradient (SNAG) methods, almost surely avoid any strict saddle manifold.
Learning Human Dynamics in Autonomous Driving Scenarios
In this work, we propose a holistic framework for learning physically plausible human dynamics from real driving scenarios, narrowing the gap between real and simulated human behavior in safety-critical applications.
PhysDiff: Physics-Guided Human Motion Diffusion Model
Specifically, we propose a physics-based motion projection module that uses motion imitation in a physics simulator to project the denoised motion of a diffusion step to a physically-plausible motion.
A Node-collaboration-informed Graph Convolutional Network for Precise Representation to Undirected Weighted Graphs
Based on this idea, a Node-collaboration-informed Graph Convolutional Network (NGCN) is proposed with three-fold ideas: a) Learning latent collaborative information from the interaction of node pairs via a node-collaboration module; b) Building the residual connection and weighted representation propagation to obtain high representation capacity; and c) Implementing the model optimization in an end-to-end fashion to achieve precise representation to the target UWG.
Prototype as Query for Few Shot Semantic Segmentation
In this way, the spatial details can be better captured and the semantic features of target class in the query image can be focused.
Ranked #13 on
Few-Shot Semantic Segmentation
on COCO-20i (5-shot)
1st Place Solutions for UG2+ Challenge 2022 ATMOSPHERIC TURBULENCE MITIGATION
In this technical report, we briefly introduce the solution of our team ''summer'' for Atomospheric Turbulence Mitigation in UG$^2$+ Challenge in CVPR 2022.
PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion
For instance, we outperform the fully finetuning approaches on a KG completion benchmark by tuning only 1% of the parameters.
Ranked #5 on
Link Prediction
on UMLS
Beta R-CNN: Looking into Pedestrian Detection from Another Perspective
What's more, to fully exploit Beta Representation, a novel pipeline Beta R-CNN equipped with BetaHead and BetaMask is proposed, leading to high detection performance in occluded and crowded scenes.
Ranked #11 on
Pedestrian Detection
on CityPersons
RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control
We present a system for collision-free control of a robot manipulator that uses only RGB views of the world.
Adaptive Latent Factor Analysis via Generalized Momentum-Incorporated Particle Swarm Optimization
Stochastic gradient descent (SGD) algorithm is an effective learning strategy to build a latent factor analysis (LFA) model on a high-dimensional and incomplete (HDI) matrix.
A Nonlinear PID-Enhanced Adaptive Latent Factor Analysis Model
High-dimensional and incomplete (HDI) data holds tremendous interactive information in various industrial applications.
When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition
Recently, most handwritten mathematical expression recognition (HMER) methods adopt the encoder-decoder networks, which directly predict the markup sequences from formula images with the attention mechanism.
Robust Knowledge Adaptation for Dynamic Graph Neural Networks
By this means, we can adaptively propagate knowledge to other nodes for learning robust node embedding representations.
Multi-Prior Learning via Neural Architecture Search for Blind Face Restoration
To this end, we propose a Face Restoration Searching Network (FRSNet) to adaptively search the suitable feature extraction architecture within our specified search space, which can directly contribute to the restoration quality.
SearchMorph:Multi-scale Correlation Iterative Network for Deformable Registration
Deformable image registration can obtain dynamic information about images, which is of great significance in medical image analysis.
From Universal Humanoid Control to Automatic Physically Valid Character Creation
Second, we use a design-and-control framework to optimize a humanoid's physical attributes to find body designs that can better imitate the pre-specified human motion sequence(s).
Embodied Scene-aware Human Pose Estimation
Since 2D third-person observations are coupled with the camera pose, we propose to disentangle the camera pose and use a multi-step projection gradient defined in the global coordinate frame as the movement cue for our embodied agent.
Ranked #322 on
3D Human Pose Estimation
on Human3.6M
FreeKD: Free-direction Knowledge Distillation for Graph Neural Networks
Knowledge distillation (KD) has demonstrated its effectiveness to boost the performance of graph neural networks (GNNs), where its goal is to distill knowledge from a deeper teacher GNN into a shallower student GNN.
Patcher: Patch Transformers with Mixture of Experts for Precise Medical Image Segmentation
We also propose a new mixture-of-experts (MoE) based decoder, which treats the feature maps from the encoder as experts and selects a suitable set of expert features to predict the label for each pixel.
Symbolic Expression Transformer: A Computer Vision Approach for Symbolic Regression
Symbolic Regression (SR) is a type of regression analysis to automatically find the mathematical expression that best fits the data.
PI-NLF: A Proportional-Integral Approach for Non-negative Latent Factor Analysis
A high-dimensional and incomplete (HDI) matrix frequently appears in various big-data-related applications, which demonstrates the inherently non-negative interactions among numerous nodes.
A Sampling Theorem for Exact Identification of Continuous-time Nonlinear Dynamical Systems
The necessary and sufficient condition is proposed -- which is built from Koopman operator -- to the exact identification of the CT system from sampled data.
Unified Simulation, Perception, and Generation of Human Behavior
Understanding and modeling human behavior is fundamental to almost any computer vision and robotics applications that involve humans.
Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering
Inheriting the advantages from information bottleneck, SIB-MSC can learn a latent space for each view to capture common information among the latent representations of different views by removing superfluous information from the view itself while retaining sufficient information for the latent representations of other views.
Online No-regret Model-Based Meta RL for Personalized Navigation
Our theoretical analysis shows that our method is a no-regret algorithm and we provide the convergence rate in the agnostic setting.
Adaptive Divergence-based Non-negative Latent Factor Analysis
High-Dimensional and Incomplete (HDI) data are frequently found in various industrial applications with complex interactions among numerous nodes, which are commonly non-negative for representing the inherent non-negativity of node interactions.
Syntax-Aware Network for Handwritten Mathematical Expression Recognition
In this paper, we propose a simple and efficient method for HMER, which is the first to incorporate syntax information into an encoder-decoder network.
On Almost Sure Convergence Rates of Stochastic Gradient Methods
We further provide last-iterate almost sure convergence rates analysis for stochastic gradient methods on weakly convex smooth functions, in contrast with most existing results in the literature that only provide convergence in expectation for a weighted average of the iterates.
Boosting Contrastive Learning with Relation Knowledge Distillation
We delve into this problem and find that the lightweight model is prone to collapse in semantic space when simply performing instance-wise contrast.
GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Since the joint reconstruction of human motions and camera poses is underconstrained, we propose a global trajectory predictor that generates global human trajectories based on local body movements.
Ranked #1 on
Global 3D Human Pose Estimation
on EMDB
Deep Unsupervised Active Learning on Learnable Graphs
To make the learnt graph structure more stable and effective, we take into account $k$-nearest neighbor graph as a priori, and learn a relation propagation graph structure.
FBNet: Feature Balance Network for Urban-Scene Segmentation
In this paper, we present a novel add-on module, named Feature Balance Network (FBNet), to eliminate the feature camouflage in urban-scene segmentation.
Learning Deep Representation with Energy-Based Self-Expressiveness for Subspace Clustering
In this paper, we propose a new deep subspace clustering framework, motivated by the energy-based models.
Causal Effect Estimation using Variational Information Bottleneck
A typical approach is to formulate causal inference as a supervised learning problem and so counterfactual could be predicted.
Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
Specifically, we learn a conditional policy that, in an episode, first applies a sequence of transform actions to modify an agent's skeletal structure and joint attributes, and then applies control actions under the new design.
Backdoor Attacks on Federated Learning with Lottery Ticket Hypothesis
Edge devices in federated learning usually have much more limited computation and communication resources compared to servers in a data center.
POI Alias Discovery in Delivery Addresses using User Locations
People often refer to a place of interest (POI) by an alias.
Font Completion and Manipulation by Cycling Between Multi-Modality Representations
The novel graph constructor maps a glyph's latent code to its graph representation that matches expert knowledge, which is trained to help the translation task.
Temporal Knowledge Consistency for Unsupervised Visual Representation Learning
The instance discrimination paradigm has become dominant in unsupervised learning.
Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study
Existing diversity tests of samples from GANs are usually conducted qualitatively on a small scale, and/or depends on the access to original training data as well as the trained model parameters.
DeceFL: A Principled Decentralized Federated Learning Framework
Although there has been a joint effort in tackling such a critical issue by proposing privacy-preserving machine learning frameworks, such as federated learning, most state-of-the-art frameworks are built still in a centralized way, in which a central client is needed for collecting and distributing model information (instead of data itself) from every other client, leading to high communication pressure and high vulnerability when there exists a failure at or attack on the central client.
Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
By comparing the pose instructed by the kinematic model against the pose generated by the dynamics model, we can use their misalignment to further improve the kinematic model.
Egocentric Pose Estimation
Human-Object Interaction Detection
+2
LambdaUNet: 2.5D Stroke Lesion Segmentation of Diffusion-weighted MR Images
Thus, it is not ideal to apply most existing segmentation methods as they are designed for either 2D or 3D images.
SimPoE: Simulated Character Control for 3D Human Pose Estimation
Based on this refined kinematic pose, the policy learns to compute dynamics-based control (e. g., joint torques) of the character to advance the current-frame pose estimate to the pose estimate of the next frame.
Ranked #245 on
3D Human Pose Estimation
on Human3.6M
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Instead, we would prefer a method that allows an agent's state at one time to directly affect another agent's state at a future time.
Ranked #10 on
Trajectory Prediction
on ETH/UCY
FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding
We present Few-Shot object detection via Contrastive proposals Encoding (FSCE), a simple yet effective approach to learning contrastive-aware object proposal encodings that facilitate the classification of detected objects.
Ranked #6 on
Cross-Domain Few-Shot Object Detection
on Artaxor
Contrastive Learning
Cross-Domain Few-Shot Object Detection
+4
Causal inference using deep neural networks
Causal inference from observation data is a core problem in many scientific fields.
Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation
We propose a method for incorporating object interaction and human body dynamics into the task of 3D ego-pose estimation using a head-mounted camera.
Automated data extraction of bar chart raster images
Objective: To develop software utilizing optical character recognition toward the automatic extraction of data from bar charts for meta-analysis.
Optical Character Recognition
Optical Character Recognition (OCR)
+1
Scalable Graph Neural Networks via Bidirectional Propagation
Most notably, GBP can deliver superior performance on a graph with over 60 million nodes and 1. 8 billion edges in less than half an hour on a single machine.
PCAL: A Privacy-preserving Intelligent Credit Risk Modeling Framework Based on Adversarial Learning
While machine learning is increasingly used in this field, the resulting large-scale collection of user private information has reinvigorated the privacy debate, considering dozens of data breach incidents every year caused by unauthorized hackers, and (potentially even more) information misuse/abuse by authorized parties.
End-to-End 3D Multi-Object Tracking and Trajectory Forecasting
To evaluate this hypothesis, we propose a unified solution for 3D MOT and trajectory forecasting which also incorporates two additional novel computational units.
AutoPose: Searching Multi-Scale Branch Aggregation for Pose Estimation
Such simplification limits the fusion of information at different scales and fails to maintain high-resolution representations.
Efficient Non-Line-of-Sight Imaging from Transient Sinograms
Non-line-of-sight (NLOS) imaging techniques use light that diffusely reflects off of visible surfaces (e. g., walls) to see around corners.
On Deep Unsupervised Active Learning
Unsupervised active learning has attracted increasing attention in recent years, where its goal is to select representative samples in an unsupervised setting for human annotating.
AnchorFace: An Anchor-based Facial Landmark Detector Across Large Poses
Based on the prediction of each anchor template, we propose to aggregate the results, which can reduce the landmark uncertainty due to the large poses.
Ranked #1 on
Face Alignment
on AFLW-Full
(Mean NME metric)
Self-PU: Self Boosted and Calibrated Positive-Unlabeled Training
Many real-world applications have to tackle the Positive-Unlabeled (PU) learning problem, i. e., learning binary classifiers from a large amount of unlabeled data and a few labeled positive examples.
Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis
Our approach is the first humanoid control method that successfully learns from a large-scale human motion dataset (Human3. 6M) and generates diverse long-term motions.
Curiosity-Driven Energy-Efficient Worker Scheduling in Vehicular Crowdsourcing: A Deep Reinforcement Learning Approach
Spatial crowdsourcing (SC) utilizes the potential of a crowd to accomplish certain location based tasks.
Semi-Supervised Cervical Dysplasia Classification With Learnable Graph Convolutional Network
In this paper, we propose a novel and more flexible GCN model with a feature encoder that adaptively updates the adjacency matrix during learning and demonstrate that this model design leads to improved performance.
Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation
We bring together a diverse set of technologies from NLOS imaging, human pose estimation and deep reinforcement learning to construct an end-to-end data processing pipeline that converts a raw stream of photon measurements into a full 3D human pose sequence estimate.