Search Results for author:
Found 158 papers, 76 papers with code
Understanding Humans in Crowded Scenes: Deep Nested Adversarial Learning and A New Benchmark for Multi-Human Parsing
Despite the noticeable progress in perceptual tasks like detection, instance segmentation and human parsing, computers still perform unsatisfactorily on visually understanding humans in crowded scenes, such as group behavior analysis, person re-identification and autonomous driving, etc.
Ranked #1 on
Multi-Human Parsing
on PASCAL-Part
EnlightenGAN: Deep Light Enhancement without Paired Supervision
Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data?
UNITER: UNiversal Image-TExt Representation Learning
Different from previous work that applies joint random masking to both modalities, we use conditional masking on pre-training tasks (i. e., masked language/region modeling is conditioned on full observation of image/text).
Ranked #3 on
Visual Question Answering (VQA)
on VCR (Q-A) test
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks.
Ranked #3 on
Natural Language Inference
on ANLI test
(using extra training data)
Look Across Elapse: Disentangled Representation Learning and Photorealistic Cross-Age Face Synthesis for Age-Invariant Face Recognition
Benchmarking our model on one of the most popular unconstrained face recognition datasets IJB-C additionally verifies the promising generalizability of AIM in recognizing faces in the wild.
Ranked #1 on
Age-Invariant Face Recognition
on MORPH Album2
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond
The results show that our framework can smoothly synthesize pedestrians on background images of variations and different levels of details.
Ranked #2 on
Scene Text Recognition
on MSDA
FreeLB: Enhanced Adversarial Training for Natural Language Understanding
Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models.
StoryGAN: A Sequential Conditional GAN for Story Visualization
We therefore propose a new story-to-image-sequence generation model, StoryGAN, based on the sequential conditional GAN framework.
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
We present HERO, a novel framework for large-scale video+language omni-representation learning.
Ranked #1 on
Video Retrieval
on TVR
AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Therefore, many fine-tuning methods are proposed to learn incremental updates of pre-trained weights in a parameter efficient way, e. g., low-rank increments.
Patient Knowledge Distillation for BERT Model Compression
Pre-trained language models such as BERT have proven to be highly effective for natural language processing (NLP) tasks.
MMD GAN: Towards Deeper Understanding of Moment Matching Network
In this paper, we propose to improve both the model expressiveness of GMMN and its computational efficiency by introducing adversarial kernel learning techniques, as the replacement of a fixed Gaussian kernel in the original GMMN.
Relation-Aware Graph Attention Network for Visual Question Answering
In order to answer semantically-complicated questions about an image, a Visual Question Answering (VQA) model needs to fully understand the visual scene in the image, especially the interactive dynamics between different objects.
Discourse-Aware Neural Extractive Text Summarization
Recently BERT has been adopted for document encoding in state-of-the-art text summarization models.
VIOLIN: A Large-Scale Dataset for Video-and-Language Inference
We introduce a new task, Video-and-Language Inference, for joint multimodal understanding of video and text.
Monocular 3D Multi-Person Pose Estimation by Integrating Top-Down and Bottom-Up Networks
Besides the integration of top-down and bottom-up networks, unlike existing pose discriminators that are designed solely for single person, and consequently cannot assess natural inter-person interactions, we propose a two-person pose discriminator that enforces natural two-person interactions.
3D Multi-Person Pose Estimation (absolute)
3D Multi-Person Pose Estimation (root-relative)
+2
Dual networks based 3D Multi-Person Pose Estimation from Monocular Video
Most of the methods focus on single persons, which estimate the poses in the person-centric coordinates, i. e., the coordinates based on the center of the target person.
Ranked #1 on
3D Human Pose Estimation
on JTA
3D Multi-Person Pose Estimation (absolute)
3D Multi-Person Pose Estimation (root-relative)
+4
Graph Optimal Transport for Cross-Domain Alignment
In GOT, cross-domain alignment is formulated as a graph matching problem, by representing entities into a dynamically-constructed graph.
Distilling Knowledge Learned in BERT for Text Generation
Experiments show that the proposed approach significantly outperforms strong Transformer baselines on multiple language generation tasks such as machine translation and text summarization.
Diverse Few-Shot Text Classification with Multiple Metrics
We study few-shot learning in natural language domains.
Large-Scale Adversarial Training for Vision-and-Language Representation Learning
We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning.
Ranked #7 on
Visual Entailment
on SNLI-VE val
(using extra training data)
Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis
Diffusion models (DMs) have shown great potential for high-quality image synthesis.
Graph and Temporal Convolutional Networks for 3D Multi-person Pose Estimation in Monocular Videos
To tackle this problem, we propose a novel framework integrating graph convolutional networks (GCNs) and temporal convolutional networks (TCNs) to robustly estimate camera-centric multi-person 3D poses that do not require camera parameters.
Ranked #1 on
Root Joint Localization
on Human3.6M
3D Absolute Human Pose Estimation
3D Multi-Person Pose Estimation (absolute)
+5
A Content-Driven Micro-Video Recommendation Dataset at Scale
Micro-videos have recently gained immense popularity, sparking critical research in micro-video recommendation with significant implications for the entertainment, advertising, and e-commerce industries.
Chasing Sparsity in Vision Transformers: An End-to-End Exploration
For example, our sparsified DeiT-Small at (5%, 50%) sparsity for (data, architecture), improves 0. 28% top-1 accuracy, and meanwhile enjoys 49. 32% FLOPs and 4. 40% running time savings.
Ranked #20 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
Jointly Attentive Spatial-Temporal Pooling Networks for Video-based Person Re-Identification
Person Re-Identification (person re-id) is a crucial task as its applications in visual surveillance and human-computer interaction.
Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
We conduct extensive experiments to demonstrate that the proposed framework achieves large performance margins (eg, 3. 83% on robust accuracy and 1. 3% on standard accuracy, on the CIFAR-10 dataset), compared with the conventional end-to-end adversarial training baseline.
Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective
Training generative adversarial networks (GANs) with limited real image data generally results in deteriorated performance and collapsed models.
Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-training
Large-scale multi-modal contrastive pre-training has demonstrated great utility to learn transferable features for a range of downstream tasks by mapping multiple modalities into a shared embedding space.
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
Most existing video-and-language (VidL) research focuses on a single dataset, or multiple datasets of a single task.
An Image Dataset for Benchmarking Recommender Systems with Raw Pixels
Then, to show the effectiveness of the dataset's image features, we substitute the itemID embeddings (from IDNet) with a powerful vision encoder that represents items using their raw image pixels.
Ranked #1 on
Recommendation Systems
on PixelRec
Domain Adaptive Text Style Transfer
These data may demonstrate domain shift, which impedes the benefits of utilizing such data for training.
M$^3$ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design
However, when deploying MTL onto those real-world systems that are often resource-constrained or latency-sensitive, two prominent challenges arise: (i) during training, simultaneously optimizing all tasks is often difficult due to gradient conflicts across tasks; (ii) at inference, current MTL regimes have to activate nearly the entire model even to just execute a single task.
Dialog-based Interactive Image Retrieval
Experiments on both simulated and real-world data show that 1) our proposed learning framework achieves better accuracy than other supervised and reinforcement learning baselines and 2) user feedback based on natural language rather than pre-specified attributes leads to more effective retrieval results, and a more natural and expressive communication interface.
BachGAN: High-Resolution Image Synthesis from Salient Object Layout
We propose a new task towards more practical application for image generation - high-quality image synthesis from salient object layout.
RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL
Our model can incorporate almost all types of existing relations in the literature, and in addition, we propose introducing co-reference relations for the multi-turn scenario.
Ranked #1 on
Dialogue State Tracking
on CoSQL
Efficient Robust Training via Backward Smoothing
In this work, we develop a new understanding towards Fast Adversarial Training, by viewing random initialization as performing randomized smoothing for better optimization of the inner maximization problem.
DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face Alignment
Thanks to the proposed fusion module, our method is robust not only to occlusion and large pitch and roll view angles, which is the benefit of our image space approach, but also to noise and large yaw angles, which is the benefit of our model space method.
Ranked #1 on
3D Face Reconstruction
on AFLW2000-3D
(Mean NME metric)
Cross-Thought for Sentence Encoder Pre-training
In this paper, we propose Cross-Thought, a novel approach to pre-training sequence encoder, which is instrumental in building reusable sequence embeddings for large-scale NLP tasks such as question answering.
Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
Sparsely activated Mixture-of-Experts (SMoE) has shown promise to scale up the learning capacity of neural networks, however, they have issues like (a) High Memory Usage, due to duplication of the network layers into multiple copies as experts; and (b) Redundancy in Experts, as common learning-based routing policies suffer from representational collapse.
NineRec: A Benchmark Dataset Suite for Evaluating Transferable Recommendation
The development of TransRec has encountered multiple challenges, among which the lack of large-scale, high-quality transfer learning recommendation dataset and benchmark suites is one of the biggest obstacles.
Bayesian Cycle-Consistent Generative Adversarial Networks via Marginalizing Latent Sampling
We evaluate the proposed Bayesian CycleGAN on multiple benchmark datasets, including Cityscapes, Maps, and Monet2photo.
Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification
Multi-task learning aims to improve generalization performance of multiple prediction tasks by appropriately sharing relevant information across them.
Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning
Few-shot text classification is a fundamental NLP task in which a model aims to classify text into a large number of categories, given only a few training examples per category.
Meta Module Network for Compositional Visual Reasoning
To design a more powerful NMN architecture for practical use, we propose Meta Module Network (MMN) centered on a novel meta module, which can take in function recipes and morph into diverse instance modules dynamically.
Contrastive Distillation on Intermediate Representations for Language Model Compression
Existing language model compression methods mostly use a simple L2 loss to distill knowledge in the intermediate representations of a large BERT model to a smaller one.
A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models
Large pre-trained vision-language (VL) models can learn a new task with a handful of examples and generalize to a new task without fine-tuning.
Ranked #4 on
Image Captioning
on Flickr30k Captions test
(SPICE metric)
Measuring Patient Similarities via a Deep Architecture with Medical Concept Embedding
One major carrier for conducting patient similarity research is Electronic Health Records(EHRs), which are usually heterogeneous, longitudinal, and sparse.
INSET: Sentence Infilling with INter-SEntential Transformer
Missing sentence generation (or sentence infilling) fosters a wide range of applications in natural language generation, such as document auto-completion and meeting note expansion.
What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
Previous storytelling approaches mostly focused on optimizing traditional metrics such as BLEU, ROUGE and CIDEr.
Ranked #10 on
Visual Storytelling
on VIST
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy
However, a "head-to-toe assessment" regarding the extent of redundancy in ViTs, and how much we could gain by thoroughly mitigating such, has been absent for this field.
Context-aware Biaffine Localizing Network for Temporal Sentence Grounding
This paper addresses the problem of temporal sentence grounding (TSG), which aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query.
Adversarial Feature Augmentation and Normalization for Visual Recognition
Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models.
SemAttack: Natural Textual Attacks via Different Semantic Spaces
In particular, SemAttack optimizes the generated perturbations constrained on generic semantic spaces, including typo space, knowledge space (e. g., WordNet), contextualized semantic space (e. g., the embedding space of BERT clusterings), or the combination of these spaces.
Generative Adversarial Networks as Variational Training of Energy Based Models
We propose VGAN, which works by minimizing a variational lower bound of the negative log likelihood (NLL) of an energy based model (EBM), where the model density $p(\mathbf{x})$ is approximated by a variational distribution $q(\mathbf{x})$ that is easy to sample from.
Backdoor Attacks on Crowd Counting
In this paper, we propose two novel Density Manipulation Backdoor Attacks (DMBA$^{-}$ and DMBA$^{+}$) to attack the model to produce arbitrarily large or small density estimations.
EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
Heavily overparameterized language models such as BERT, XLNet and T5 have achieved impressive success in many NLP tasks.
S3Pool: Pooling with Stochastic Spatial Sampling
We view the pooling operation in CNNs as a two-step procedure: first, a pooling window (e. g., $2\times 2$) slides over the feature map with stride one which leaves the spatial resolution intact, and second, downsampling is performed by selecting one pixel from each non-overlapping pooling window in an often uniform and deterministic (e. g., top-left) manner.
The Elastic Lottery Ticket Hypothesis
Based on these results, we articulate the Elastic Lottery Ticket Hypothesis (E-LTH): by mindfully replicating (or dropping) and re-ordering layers for one network, its corresponding winning ticket could be stretched (or squeezed) into a subnetwork for another deeper (or shallower) network from the same family, whose performance is nearly the same competitive as the latter's winning ticket directly found by IMP.
Deep Structured Energy Based Models for Anomaly Detection
In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures.
DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models
To address these pain points, we propose a framework for resource- and parameter-efficient fine-tuning by leveraging the sparsity prior in both weight updates and the final model weights.
Sobolev GAN
We show that the Sobolev IPM compares two distributions in high dimensions based on weighted conditional Cumulative Distribution Functions (CDF) of each coordinate on a leave one out basis.
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
In this paper, we present Adversarial GLUE (AdvGLUE), a new multi-task benchmark to quantitatively and thoroughly explore and evaluate the vulnerabilities of modern large-scale language models under various types of adversarial attacks.
Ranked #1 on
Adversarial Robustness
on AdvGLUE
Deep Multimodality Model for Multi-task Multi-view Learning
However, there is no existing deep learning algorithm that jointly models task and view dual heterogeneity, particularly for a data set with multiple modalities (text and image mixed data set or text and video mixed data set, etc.).
Improving Low-resource Prompt-based Relation Representation with Multi-view Decoupling Learning
Recently, prompt-tuning with pre-trained language models (PLMs) has demonstrated the significantly enhancing ability of relation extraction (RE) tasks.
Deep Co-Attention Network for Multi-View Subspace Learning
For example, given the diagnostic result that a model provided based on the X-ray images of a patient at different poses, the doctor needs to know why the model made such a prediction.
Reinforcement Learning with Token-level Feedback for Controllable Text Generation
To meet the requirements of real-world applications, it is essential to control generations of large language models (LLMs).
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients
Adaptive gradient methods such as RMSProp and Adam use exponential moving estimate of the squared gradient to compute adaptive step sizes, achieving better convergence than SGD in face of noisy objectives.
Local Byte Fusion for Neural Machine Translation
Subword tokenization schemes are the dominant technique used in current NLP models.
Outlier-Robust Sparse Estimation via Non-Convex Optimization
We explore the connection between outlier-robust high-dimensional statistics and non-convex optimization in the presence of sparsity constraints, with a focus on the fundamental tasks of robust sparse mean estimation and robust sparse PCA.
Hiding Data Helps: On the Benefits of Masking for Sparse Coding
Furthermore, drawing from the growing body of work on self-supervised learning, we propose a novel masking objective for which recovering the ground-truth dictionary is in fact optimal as the signal increases for a large class of data-generating processes.
Non-Convex Matrix Completion Against a Semi-Random Adversary
Matrix completion is a well-studied problem with many machine learning applications.
Deep Nearest Class Mean Model for Incremental Odor Classification
That is, the odor datasets are dynamically growing while both training samples and number of classes are increasing over time.
A Survey of Model Compression and Acceleration for Deep Neural Networks
Methods of parameter pruning and quantization are described first, after that the other techniques are introduced.
On the Distortion of Voting with Multiple Representative Candidates
Our main result is a clean and tight characterization of positional voting rules that have constant expected distortion (independent of the number of candidates and the metric space).
Catching Anomalous Distributed Photovoltaics: An Edge-based Multi-modal Anomaly Detection
A significant challenge in energy system cyber security is the current inability to detect cyber-physical attacks targeting and originating from distributed grid-edge devices such as photovoltaics (PV) panels, smart flexible loads, and electric vehicles.
Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records
We use this generative model together with a convolutional neural network (CNN) based prediction model to improve the onset prediction performance.
Of the People: Voting Is More Effective with Representative Candidates
However, we show that independence alone is not enough to achieve the upper bound: even when candidates are drawn independently, if the population of candidates can be different from the voters, then an upper bound of $2$ on the approximation is tight.
Exploiting Convolutional Neural Network for Risk Prediction with Medical Feature Embedding
To account for high dimensionality, we use the embedding medical features in the CNN model which hold the natural medical concepts.
Doubly Convolutional Neural Networks
Building large models with parameter sharing accounts for most of the success of deep convolutional neural networks (CNNs).
Robust Learning of Fixed-Structure Bayesian Networks
We investigate the problem of learning Bayesian networks in a robust model where an $\epsilon$-fraction of the samples are adversarially corrupted.
Walk and Learn: Facial Attribute Representation Learning from Egocentric Video and Contextual Data
These image pairs are then fed into a deep network that preserves similarity of images connected by the same track, in order to capture identity-related attribute features, and optimizes for location and weather prediction to capture additional facial attribute features.
An exploration of parameter redundancy in deep networks with circulant projections
We explore the redundancy of parameters in deep neural networks by replacing the conventional linear projection in fully-connected layers with the circulant projection.
Spectral Sparsification of Random-Walk Matrix Polynomials
Our work is particularly motivated by the algorithmic problems for speeding up the classic Newton's method in applications such as computing the inverse square-root of the precision matrix of a Gaussian random field, as well as computing the $q$th-root transition (for $q\geq1$) in a time-reversible Markov model.
Scalable Parallel Factorizations of SDD Matrices and Efficient Sampling for Gaussian Graphical Models
random samples for $n$-dimensional Gaussian random fields with SDDM precision matrices.
Spatial-Temporal Synergic Residual Learning for Video Person Re-Identification
We present a novel method - Spatial-Temporal Synergic Residual Network (STSRN) for this problem.
High-Dimensional Robust Mean Estimation in Nearly-Linear Time
We study the fundamental problem of high-dimensional mean estimation in a robust model where a constant fraction of the samples are adversarially corrupted.
Sequential Attention GAN for Interactive Image Editing
The main challenges in this sequential and interactive image generation task are two-fold: 1) contextual consistency between a generated image and the provided textual description; 2) step-by-step region-level modification to maintain visual consistency across the generated image sequence in each session.
On the Recursive Teaching Dimension of VC Classes
This is the first upper bound for $RTD(C)$ that depends only on $VCD(C)$, independent of the size of the concept class $|C|$ and its~domain size $n$.
Towards Pose Invariant Face Recognition in the Wild
To this end, we propose a Pose Invariant Model (PIM) for face recognition in the wild, with three distinct novelties.
Temporal Sequence Modeling for Video Event Detection
Based on this idea, we represent a video by a sequence of visual words learnt from the video, and apply the Sequence Memoizer [21] to capture long-range dependencies in a temporal context in the visual sequence.
Few-shot Learning with Meta Metric Learners
Our meta metric learning approach consists of task-specific learners, that exploit metric learning to handle flexible labels, and a meta learner, that discovers good parameters and gradient decent to specify the metrics in task-specific learners.
Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog
This paper presents a new model for visual dialog, Recurrent Dual Attention Network (ReDAN), using multi-step reasoning to answer a series of questions about an image.
POP-CNN: Predicting Odor's Pleasantness with Convolutional Neural Network
Predicting odor's pleasantness simplifies the evaluation of odors and has the potential to be applied in perfumes and environmental monitoring industry.
A Hybrid Approach with Optimization and Metric-based Meta-Learner for Few-Shot Learning
The task-specific classifiers are required to be homogeneous-structured to ease the parameter prediction, so the meta-learning approaches could only handle few-shot learning problems where the tasks share a uniform number of classes.
Adversarial Category Alignment Network for Cross-domain Sentiment Classification
Cross-domain sentiment classification aims to predict sentiment polarity on a target domain utilizing a classifier learned from a source domain.
Faster Algorithms for High-Dimensional Robust Covariance Estimation
We study the problem of estimating the covariance matrix of a high-dimensional distribution when a small constant fraction of the samples can be arbitrarily corrupted.
Mixed-Supervised Dual-Network for Medical Image Segmentation
The model is trained jointly in a multi-task learning setting.
Attend To Count: Crowd Counting with Adaptive Capacity Multi-scale CNNs
ACM-CNN consists of three types of modules: a coarse network, a fine network, and a smooth network.
Contrastively Smoothed Class Alignment for Unsupervised Domain Adaptation
Recent unsupervised approaches to domain adaptation primarily focus on minimizing the gap between the source and the target domains through refining the feature generator, in order to learn a better alignment between the two domains.
Tell-the-difference: Fine-grained Visual Descriptor via a Discriminating Referee
In this paper, we investigate a novel problem of telling the difference between image pairs in natural language.
Occlusion-Aware Networks for 3D Human Pose Estimation in Video
In addition, we use this model to create a pose regularization constraint, preferring the 2D estimations of unreliable keypoints to be occluded.
Ranked #4 on
3D Human Pose Estimation
on HumanEva-I
Distinguishing Distributions When Samples Are Strategically Transformed
In other settings, the principal may not even be able to observe samples directly; instead, she must rely on signals that the agent is able to send based on the samples that he obtains, and he will choose these signals strategically.
Towards Better Understanding of Disentangled Representations via Mutual Information
Most existing works on disentangled representation learning are solely built upon an marginal independence assumption: all factors in disentangled representations should be statistically independent.
Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV based Random Access IoT Networks with NOMA
In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers.
3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training
Estimating 3D poses from a monocular video is still a challenging task, despite the significant progress that has been made in recent years.
Ranked #3 on
3D Human Pose Estimation
on HumanEva-I
Contextual Text Style Transfer
To realize high-quality style transfer with natural context preservation, we propose a Context-Aware Style Transfer (CAST) model, which uses two separate encoders for each input sentence and its surrounding context.
APo-VAE: Text Generation in Hyperbolic Space
In this paper, we investigate text generation in a hyperbolic latent space to learn continuous hierarchical representations.
High-Dimensional Robust Mean Estimation via Gradient Descent
We study the problem of high-dimensional robust mean estimation in the presence of a constant fraction of adversarial outliers.
Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models
To reveal the secrets behind the scene of these powerful models, we present VALUE (Vision-And-Language Understanding Evaluation), a set of meticulously designed probing tasks (e. g., Visual Coreference Resolution, Visual Relation Detection, Linguistic Probing Tasks) generalizable to standard pre-trained V+L models, aiming to decipher the inner workings of multimodal pre-training (e. g., the implicit knowledge garnered in individual attention heads, the inherent cross-modal alignment learned through contextualized multimodal embeddings).
Fine-grained Iterative Attention Network for TemporalLanguage Localization in Videos
In this paper, we propose a Fine-grained Iterative Attention Network (FIAN) that consists of an iterative attention module for bilateral query-video in-formation extraction.
Object Tracking using Spatio-Temporal Networks for Future Prediction Location
To facilitate the future prediction ability, we follow three key observations: 1) object motion trajectory is affected significantly by camera motion; 2) the past trajectory of an object can act as a salient cue to estimate the object motion in the spatial domain; 3) previous frames contain the surroundings and appearance of the target object, which is useful for predicting the target object’s future locations.
Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
Transformer has become ubiquitous in the deep learning field.
Ranked #1 on
Open-Domain Question Answering
on SearchQA
Cluster-Former: Clustering-based Sparse Transformer for Question Answering
Transformer has become ubiquitous in the deep learning field.
Adversarial Masking: Towards Understanding Robustness Trade-off for Generalization
By incorporating different feature maps after the masking, we can distill better features to help model generalization.
ALFA: Adversarial Feature Augmentation for Enhanced Image Recognition
Adversarial training is an effective method to combat adversarial attacks in order to create robust neural networks.
Multi-Fact Correction in Abstractive Text Summarization
Pre-trained neural abstractive summarization systems have dominated extractive strategies on news summarization performance, at least in terms of ROUGE.
Object Tracking Using Spatio-Temporal Future Prediction
Our trajectory prediction module predicts the target object's locations in the current and future frames based on the object's past trajectory.
DSAM: A Distance Shrinking with Angular Marginalizing Loss for High Performance Vehicle Re-identificatio
To obtain a high-performance vehicle ReID model, we present a novel Distance Shrinking with Angular Marginalizing (DSAM) loss function to perform hybrid learning in both the Original Feature Space (OFS) and the Feature Angular Space (FAS) using the local verification and the global identification information.
Light dark matter from dark sector decay
We find that the mass of the dark sector singlet fermion can be GeV scale or MeV scale and the interaction of the dark sector singlet fermion is very weak.
High Energy Physics - Phenomenology
Fair for All: Best-effort Fairness Guarantees for Classification
Instead, we propose a fairness notion whose guarantee, on each group $g$ in a class $\mathcal{G}$, is relative to the performance of the best classifier on $g$.
Structure Of Flavor Changing Goldstone Boson Interactions
Or it may be the Majoron in models from lepton number violation in producing seesaw Majorana neutrino masses if the symmetry breaking scale is much higher than the electroweak scale.
High Energy Physics - Phenomenology
UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
Vision-and-language pre-training has achieved impressive success in learning multimodal representations between vision and language.
CUPID: Adaptive Curation of Pre-training Data for Video-and-Language Representation Learning
This work concerns video-language pre-training and representation learning.
Automated Mechanism Design for Classification with Partial Verification
We study the problem of automated mechanism design with partial verification, where each type can (mis)report only a restricted set of types (rather than any other type), induced by the principal's limited verification power.
Playing Lottery Tickets with Vision and Language
However, we can find "relaxed" winning tickets at 50%-70% sparsity that maintain 99% of the full accuracy.
Robust Learning of Fixed-Structure Bayesian Networks in Nearly-Linear Time
We achieve this by establishing a direct connection between robust learning of Bayesian networks and robust mean estimation.
Few-Shot Object Detection via Classification Refinement and Distractor Retreatment
The failure modes of FSOD are investigated that the performance degradation is mainly due to the classification incapability (false positives), which motivates us to address it from a novel aspect of hard example mining.
MA-CLIP: Towards Modality-Agnostic Contrastive Language-Image Pre-training
Large-scale multimodal contrastive pretraining has demonstrated great utility to support high performance in a range of downstream tasks by mapping multiple modalities into a shared embedding space.
Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding
Recent work has focused on compressing pre-trained language models (PLMs) like BERT where the major focus has been to improve the in-distribution performance for downstream tasks.
UNITER: Learning UNiversal Image-TExt Representations
Joint image-text embedding is the bedrock for most Vision-and-Language (V+L) tasks, where multimodality inputs are jointly processed for visual and textual understanding.
Memory-Guided Semantic Learning Network for Temporal Sentence Grounding
To tackle this issue, we propose a memory-augmented network, called Memory-Guided Semantic Learning Network (MGSL-Net), that learns and memorizes the rarely appeared content in TSG tasks.
Unsupervised Temporal Video Grounding with Deep Semantic Clustering
Temporal video grounding (TVG) aims to localize a target segment in a video according to a given sentence query.
ZOOMER: Boosting Retrieval on Web-scale Graphs by Regions of Interest
We introduce ZOOMER, a system deployed at Taobao, the largest e-commerce platform in China, for training and serving GNN-based recommendations over web-scale graphs.
Efficient Algorithms for Planning with Participation Constraints
Our approach can also be extended to the (discounted) infinite-horizon case, for which we give an algorithm that runs in time polynomial in the size of the input and $\log(1/\varepsilon)$, and returns a policy that is optimal up to an additive error of $\varepsilon$.
Bottom-Up 2D Pose Estimation via Dual Anatomical Centers for Small-Scale Persons
In multi-person 2D pose estimation, the bottom-up methods simultaneously predict poses for all persons, and unlike the top-down methods, do not rely on human detection.
Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding
All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning.
Hypotheses Tree Building for One-Shot Temporal Sentence Localization
Given an untrimmed video, temporal sentence localization (TSL) aims to localize a specific segment according to a given sentence query.
Transform-Equivariant Consistency Learning for Temporal Sentence Grounding
This paper addresses the temporal sentence grounding (TSG).
A Theory of General Difference in Continuous and Discrete Domain
This stems from a key disconnect between the infinitesimal quantities in continuous differentiation and the finite intervals in its discrete counterpart.
You Are Catching My Attention: Are Vision Transformers Bad Learners Under Backdoor Attacks?
Vision Transformers (ViTs), which made a splash in the field of computer vision (CV), have shaken the dominance of convolutional neural networks (CNNs).
Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights
Furthermore, we compare the performance of the TCF paradigm utilizing the most powerful LMs to the currently dominant ID embedding-based paradigm and investigate the transferability of this TCF paradigm.
GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions
Generalization to unseen tasks is an important ability for few-shot learners to achieve better zero-/few-shot performance on diverse tasks.
Low-Switching Policy Gradient with Exploration via Online Sensitivity Sampling
We show that, our algorithm obtains an $\varepsilon$-optimal policy with only $\widetilde{O}(\frac{\text{poly}(d)}{\varepsilon^3})$ samples, where $\varepsilon$ is the suboptimality gap and $d$ is a complexity measure of the function class approximating the policy.
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Yet, while the literature on the trustworthiness of GPT models remains limited, practitioners have proposed employing capable GPT models for sensitive applications such as healthcare and finance -- where mistakes can be costly.
Unified Single-Stage Transformer Network for Efficient RGB-T Tracking
With this structure, the network can extract fusion features of the template and search region under the mutual interaction of modalities.
Ranked #1 on
Rgb-T Tracking
on GTOT
ORTexME: Occlusion-Robust Human Shape and Pose via Temporal Average Texture and Mesh Encoding
In 3D human shape and pose estimation from a monocular video, models trained with limited labeled data cannot generalize well to videos with occlusion, which is common in the wild videos.
ProS: Facial Omni-Representation Learning via Prototype-based Self-Distillation
This paper presents a novel approach, called Prototype-based Self-Distillation (ProS), for unsupervised face representation learning.
Applications of Tao General Difference in Discrete Domain
Tao general difference (TGD) is a novel theory and approach to difference computation for discrete sequences and arrays in multidimensional space.
Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning
Self-Supervised Contrastive Learning has proven effective in deriving high-quality representations from unlabeled data.
Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuning
Despite vision-language models' (VLMs) remarkable capabilities as versatile visual assistants, two substantial challenges persist within the existing VLM frameworks: (1) lacking task diversity in pretraining and visual instruction tuning, and (2) annotation error and bias in GPT-4 synthesized instruction tuning data.
Multimodal Instruction Tuning with Conditional Mixture of LoRA
Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in diverse tasks across different domains, with an increasing focus on improving their zero-shot generalization capabilities for unseen multimodal tasks.
Robust Second-Order Nonconvex Optimization and Its Application to Low Rank Matrix Sensing
We introduce a general framework for efficiently finding an approximate SOSP with \emph{dimension-independent} accuracy guarantees, using $\widetilde{O}({D^2}/{\epsilon})$ samples where $D$ is the ambient dimension and $\epsilon$ is the fraction of corrupted datapoints.
MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
Motivated by the empirical findings, we propose a novel LLM-based Multi-Agent framework for GitHub Issue reSolution, MAGIS, consisting of four kinds of agents customized for the software evolution: Manager, Repository Custodian, Developer, and Quality Assurance Engineer agents.
Research on Detection of Floating Objects in River and Lake Based on AI Intelligent Image Recognition
With the rapid advancement of artificial intelligence technology, AI-enabled image recognition has emerged as a potent tool for addressing challenges in traditional environmental monitoring.
