Search Results for author:
Found 84 papers, 29 papers with code
Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets
Work on bias in hate speech typically aims to improve classification performance while relatively overlooking the quality of the data.
Controlled Text Generation Using Dictionary Prior in Variational Autoencoders
While variational autoencoders (VAEs) have been widely applied in text generation tasks, they are troubled by two challenges: insufficient representation capacity and poor controllability.
SG-LRA: Self-Generating Automatic Scoliosis Cobb Angle Measurement with Low-Rank Approximation
Automatic Cobb angle measurement from X-ray images is crucial for scoliosis screening and diagnosis.
Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
The core idea of our framework is to conduct hallucination evaluation on (object, relation, object) triplets extracted from LVLMs' responses, and thus, could be easily generalized to different vision-language tasks.
Fourier Amplitude and Correlation Loss: Beyond Using L2 Loss for Skillful Precipitation Nowcasting
Our method is generic, parameter-free and efficient.
Selection-p: Self-Supervised Task-Agnostic Prompt Compression for Faithfulness and Transferability
Large Language Models (LLMs) have demonstrated impressive capabilities in a wide range of natural language processing tasks when leveraging in-context learning.
AnyAttack: Towards Large-scale Self-supervised Generation of Targeted Adversarial Examples for Vision-Language Models
Due to their multimodal capabilities, Vision-Language Models (VLMs) have found numerous impactful applications in real-world scenarios.
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
GPT-4o, an omni-modal model that enables vocal conversations with diverse emotions and tones, marks a milestone for omni-modal foundation models.
Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detection
The Bird's-Eye-View (BEV) representation is a critical factor that directly impacts the 3D object detection performance, but the traditional BEV grid representation induces quadratic computational cost as the spatial resolution grows.
JointDreamer: Ensuring Geometry Consistency and Text Congruence in Text-to-3D Generation via Joint Score Distillation
However, this paradigm distills view-agnostic 2D image distributions into the rendering distribution of 3D representation for each view independently, overlooking the coherence across views and yielding 3D inconsistency in generations.
Rethinking Targeted Adversarial Attacks For Neural Machine Translation
To this end, this paper presents a new setting for NMT targeted adversarial attacks that could lead to reliable attacking results.
Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases
Moreover, with our CODA-LM, we build CODA-VLM, a new driving LVLM surpassing all open-sourced counterparts on CODA-LM.
DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception
Furthermore, image syntheses from DetDiffusion can effectively augment training data, significantly enhancing downstream detection performance.
TransformMix: Learning Transformation and Mixing Strategies from Data
Sample-mixing is a popular data augmentation approach that creates additional data by combining existing samples.
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation
To construct robust MLLMs, we propose ECSO (Eyes Closed, Safety On), a novel training-free protecting approach that exploits the inherent safety awareness of MLLMs, and generates safer responses via adaptively transforming unsafe images into texts to activate the intrinsic safety mechanism of pre-aligned LLMs in MLLMs.
Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning
Instruction tuning of Large Vision-language Models (LVLMs) has revolutionized the development of versatile models with zero-shot generalization across a wide range of downstream vision-language tasks.
TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models
Despite remarkable achievements in video synthesis, achieving granular control over complex dynamics, such as nuanced movement among multiple interacting objects, still presents a significant hurdle for dynamic world modeling, compounded by the necessity to manage appearance and disappearance, drastic scale changes, and ensure consistency for instances across frames.
Gaussian Shell Maps for Efficient 3D Human Generation
Instead of rasterizing the shells directly, we sample 3D Gaussians on the shells whose attributes are encoded in the texture features.
Towards General Error Diagnosis via Behavioral Testing in Machine Translation
Behavioral testing offers a crucial means of diagnosing linguistic errors and assessing capabilities of NLP models.
Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis
The rapid development of large language models (LLMs) has not only provided numerous opportunities but also presented significant challenges.
Implicit Concept Removal of Diffusion Models
Subsequently, the model is optimized to identify and disentangle this information, which is then adopted as negative prompts during generation.
MagicDrive: Street View Generation with Diverse 3D Geometry Control
Recent advancements in diffusion models have significantly enhanced the data synthesis with 2D control.
SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation
To take full advantage of the historical frames high-efficiently, we shunt the historical points into two groups with reference to the current points.
SCAT: Robust Self-supervised Contrastive Learning via Adversarial Training for Text Classification
Specifically, SCAT modifies random augmentations of the data in a fully labelfree manner to generate adversarial examples.
GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation
Diffusion models have attracted significant attention due to the remarkable ability to create content and generate data for tasks like image classification.
Mixed Autoencoder for Self-supervised Visual Representation Learning
Specifically, our MixedAE outperforms MAE by +0. 3% accuracy, +1. 7 mIoU and +0. 9 AP on ImageNet-1K, ADE20K and COCO respectively with a standard ViT-Base.
CLIP$^2$: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data
Contrastive Language-Image Pre-training, benefiting from large-scale unlabeled text-image pairs, has demonstrated great performance in open-world vision understanding tasks.
Ranked #3 on
Zero-shot 3D Point Cloud Classification
on ScanNetV2
Deep COVID-19 Forecasting for Multiple States with Data Augmentation
As such, it has a two-fold advantage: 1) more actual observations can be used for training, and 2) the model can be validated on data which has distribution closer to the expected situation.
Learning 3D-aware Image Synthesis with Unknown Pose Distribution
Existing methods for 3D-aware image synthesis largely depend on the 3D pose distribution pre-estimated on the training set.
CLIP2: Contrastive Language-Image-Point Pretraining From Real-World Point Cloud Data
Contrastive Language-Image Pre-training, benefiting from large-scale unlabeled text-image pairs, has demonstrated great performance in open-world vision understanding tasks.
SongRewriter: A Chinese Song Rewriting System with Controllable Content and Rhyme Scheme
Although lyrics generation has achieved significant progress in recent years, it has limited practical applications because the generated lyrics cannot be performed without composing compatible melodies.
Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator
We argue that, considering the two-player game in the formulation of GANs, only making the generator 3D-aware is not enough.
Trading off Quality for Efficiency of Community Detection: An Inductive Method across Graphs
The trained model is then directly generalized to new unseen graphs for online CD without additional optimization, where a better trade-off between quality and efficiency can be achieved.
Earthformer: Exploring Space-Time Transformers for Earth System Forecasting
With the explosive growth of the spatiotemporal Earth observation data in the past decade, data-driven models that apply Deep Learning (DL) are demonstrating impressive potential for various Earth system forecasting tasks.
Ranked #1 on
Earth Surface Forecasting
on EarthNet2021 OOD Track
Detection Recovery in Online Multi-Object Tracking with Sparse Graph Tracker
The strong edge features allow SGT to track targets with tracking candidates selected by top-K scored detections with large K. As a result, even low-scored detections can be tracked, and the missed detections are also recovered.
Ranked #2 on
Multi-Object Tracking
on HiEve
CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving
One main reason that impedes the development of truly reliably self-driving systems is the lack of public datasets for evaluating the performance of object detectors on corner cases.
3D-Aware Indoor Scene Synthesis with Depth Priors
In this way, the discriminator can take the spatial arrangement into account and advise the generator to learn an appropriate depth condition.
AdaAug: Learning Class- and Instance-adaptive Data Augmentation Policies
However, the augmentation policies found are not adaptive to the dataset used, hindering the effectiveness of these AutoDA methods.
Trading Quality for Efficiency of Graph Partitioning: An Inductive Method across Graphs
IGP is also a generic framework that can capture the permutation invariant partitioning ground-truth of historical snapshots in the offline training and tackle the online GP on graphs with non-fixed number of nodes and clusters.
Aug-ILA: More Transferable Intermediate Level Attacks with Augmented References
We start by looking into the effect of common image augmentation techniques and exploring novel augmentation with the aid of adversarial perturbations.
MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving
By pre-training on SODA10M, a large-scale autonomous driving dataset, MultiSiam exceeds the ImageNet pre-trained MoCo-v2, demonstrating the potential of domain-specific pre-training.
Stereo Waterdrop Removal with Row-wise Dilated Attention
Thus, we propose a learning-based model for waterdrop removal with stereo images.
Probing Toxic Content in Large Pre-Trained Language Models
Large pre-trained language models (PTLMs) have been shown to carry biases towards different social groups which leads to the reproduction of stereotypical and toxic content by major NLP systems.
MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space
Data augmentation is an efficient way to expand a training dataset by creating additional artificial data.
Multilingual and Multi-Aspect Hate Speech Analysis
Current research on hate speech analysis is typically oriented towards monolingual and single classification tasks.
Knowledge Query Network: How Knowledge Interacts with Skills
This involves abstract concepts of students' states of knowledge and the interactions between those states and skills.
Movable-Object-Aware Visual SLAM via Weakly Supervised Semantic Segmentation
Moving objects can greatly jeopardize the performance of a visual simultaneous localization and mapping (vSLAM) system which relies on the static-world assumption.
Marginalized Average Attentional Network for Weakly-Supervised Learning
The MAAN employs a novel marginalized average aggregation (MAA) module and learns a set of latent discriminative probabilities in an end-to-end fashion.
Ranked #14 on
Weakly Supervised Action Localization
on ActivityNet-1.3
(mAP@0.5 metric)
Weakly Supervised Action Localization
Weakly-supervised Learning
+2
Fully Using Classifiers for Weakly Supervised Semantic Segmentation with Modified Cues
This paper proposes a novel weakly-supervised semantic segmentation method using image-level label only.
Knowledge Query Network for Knowledge Tracing
This involves abstract concepts of students' states of knowledge and the interactions between those states and skills.
Effective Feature Learning with Unsupervised Learning for Improving the Predictive Models in Massive Open Online Courses
A major challenge that has to be addressed when building such models is to design handcrafted features that are effective for the prediction task at hand.
Semi-Semantic Line-Cluster Assisted Monocular SLAM for Indoor Environments
In the back end, we optimize the map imposing the constraint that the line segments of the same cluster should be the same.
Point-cloud-based place recognition using CNN feature extraction
This paper proposes a novel point-cloud-based place recognition system that adopts a deep learning approach for feature extraction.
Machine Learning for Spatiotemporal Sequence Forecasting: A Survey
Forecasting the multi-step future of these spatiotemporal systems based on the past observations, or, Spatiotemporal Sequence Forecasting (STSF), is a significant and challenging problem.
Addressing Two Problems in Deep Knowledge Tracing via Prediction-Consistent Regularization
In recent years, a recurrent neural network model called deep knowledge tracing (DKT) has been proposed to handle the knowledge tracing task and literature has shown that DKT generally outperforms traditional methods.
Incorporating Features Learned by an Enhanced Deep Knowledge Tracing Model for STEM/Non-STEM Job Prediction
The 2017 ASSISTments Data Mining competition aims to use data from a longitudinal study for predicting a brand-new outcome of students which had never been studied before by the educational data mining research community.
GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
We propose a new network architecture, Gated Attention Networks (GaAN), for learning on graphs.
Ranked #1 on
Node Property Prediction
on ogbn-proteins
Robust path-based spectral clustering
In this paper, based on M-estimation from robust statistics, we develop a robust path-based spectral clustering method by defining a robust path-based similarity measure for spectral clustering under both unsupervised and semi-supervised settings.
Learning Unmanned Aerial Vehicle Control for Autonomous Target Following
While deep reinforcement learning (RL) methods have achieved unprecedented successes in a range of challenging problems, their applicability has been mainly limited to simulation or game domains due to the high sample complexity of the trial-and-error learning process.
Temporal Dynamic Graph LSTM for Action-driven Video Object Detection
A common issue, however, is that objects of interest that are not involved in human actions are often absent in global action descriptions known as "missing label".
Ranked #3 on
Weakly Supervised Object Detection
on Charades
Spatiotemporal Modeling for Crowd Counting in Videos
To exploit the otherwise very useful temporal information in video sequences, we propose a variant of a recent deep learning model called convolutional LSTM (ConvLSTM) for crowd counting.
Fine-Grained Categorization via CNN-Based Automatic Extraction and Integration of Object-Level and Part-Level Features
Fine-grained categorization can benefit from part-based features which reveal subtle visual differences between object categories.
Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model
To address these problems, we propose both a new model and a benchmark for precipitation nowcasting.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
ZM-Net: Real-time Zero-shot Image Manipulation Network
We cast this problem as manipulating an input image according to a parametric model whose key parameters can be conditionally generated from any guiding signal (even unseen ones).
Dynamic Key-Value Memory Networks for Knowledge Tracing
Knowledge Tracing (KT) is a task of tracing evolving knowledge state of students with respect to one or more concepts as they engage in a sequence of learning activities.
Natural-Parameter Networks: A Class of Probabilistic Neural Networks
Another shortcoming of NN is the lack of flexibility to customize different distributions for the weights and neurons according to the data, as is often done in probabilistic graphical models.
Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks
To address this problem, we develop a collaborative recurrent autoencoder (CRAE) which is a denoising recurrent autoencoder (DRAE) that models the generation of content sequences in the collaborative filtering (CF) setting.
Sparse Boltzmann Machines with Structure Learning as Applied to Text Analysis
We are interested in exploring the possibility and benefits of structure learning for deep models.
Towards Bayesian Deep Learning: A Framework and Some Existing Methods
While perception tasks such as visual object recognition and text understanding play an important role in human intelligence, the subsequent tasks that involve inference, reasoning and planning require an even higher level of intelligence.
A Survey on Bayesian Deep Learning
The past decade has seen major advances in many perception tasks such as visual object recognition and speech recognition using deep learning models.
Human Action Recognition using Factorized Spatio-Temporal Convolutional Networks
Human actions in video sequences are three-dimensional (3D) spatio-temporal signals characterizing both the visual appearance and motion dynamics of the involved humans and objects.
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
DevNet: A Deep Event Network for Multimedia Event Detection and Evidence Recounting
Taking key frames of videos as input, we first detect the event of interest at the video level by aggregating the CNN features of the key frames.
Bayesian Adaptive Matrix Factorization With Automatic Model Selection
Low-rank matrix factorization has long been recognized as a fundamental problem in many computer vision applications.
Understanding and Diagnosing Visual Tracking Systems
Surprisingly, our findings are discrepant with some common beliefs in the visual tracking research community.
Transferring Rich Feature Hierarchies for Robust Visual Tracking
To fit the characteristics of object tracking, we first pre-train the CNN to recognize what is an object, and then propose to generate a probability map instead of producing a simple class label.
Collaborative Deep Learning for Recommender Systems
(CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix.
Learning a Deep Compact Image Representation for Visual Tracking
In this paper, we study the challenging problem of tracking the trajectory of a moving object in a video with possibly very complex background.
Co-Regularized Hashing for Multimodal Data
Hashing-based methods provide a very promising approach to large-scale similarity search.
A Convex Formulation for Learning Task Relationships in Multi-Task Learning
In this paper, we propose a regularization formulation for learning the relationships between tasks in multi-task learning.
Probabilistic Multi-Task Feature Selection
In this paper, we unify the $l_{1, 2}$ and $l_{1,\infty}$ norms by considering a family of $l_{1, q}$ norms for $1 < q\le\infty$ and study the problem of determining the most appropriate sparsity enforcing norm to use in the context of multi-task feature selection.
Worst-Case Linear Discriminant Analysis
In this paper, we first analyze the scatter measures used in the conventional linear discriminant analysis~(LDA) model and note that the formulation is based on the average-case view.
Posterior Consistency of the Silverman g-prior in Bayesian Model Choice
The duality between regularization and prior leads to interpreting regularization methods in terms of maximum a posteriori estimation and has motivated Bayesian interpretations of kernel methods.
