Search Results for author:
Found 64 papers, 31 papers with code
ControlRetriever: Harnessing the Power of Instructions for Controllable Retrieval
Recent studies have shown that dense retrieval models, lacking dedicated training data, struggle to perform well across diverse retrieval tasks, as different retrieval tasks often entail distinct search intents.
Empowering Vision-Language Models to Follow Interleaved Vision-Language Instructions
To address this issue, we propose a generic and lightweight controllable knowledge re-injection module, which utilizes the sophisticated reasoning ability of LLMs to control the VPG to conditionally extract instruction-specific visual information and re-inject it into the LLM.
Multi-Scale Prototypical Transformer for Whole Slide Image Classification
The PT is developed to reduce redundant instances in bags by integrating prototypical learning into the Transformer architecture.
Weakly Supervised Lesion Detection and Diagnosis for Breast Cancers with Partially Annotated Ultrasound Images
To address this issue, a novel Two-Stage Detection and Diagnosis Network (TSDDNet) is proposed based on weakly supervised learning to enhance diagnostic accuracy of the ultrasound-based CAD for breast cancers.
Multi-scale Efficient Graph-Transformer for Whole Slide Image Classification
The key idea of MEGT is to adopt two independent Efficient Graph-based Transformer (EGT) branches to process the low-resolution and high-resolution patch embeddings (i. e., tokens in a Transformer) of WSIs, respectively, and then fuse these tokens via a multi-scale feature fusion module (MFFM).
Sim-Suction: Learning a Suction Grasp Policy for Cluttered Environments Using a Synthetic Benchmark
This paper presents Sim-Suction, a robust object-aware suction grasp policy for mobile manipulation platforms with dynamic camera viewpoints, designed to pick up unknown objects from cluttered environments.
Global Structure Knowledge-Guided Relation Extraction Method for Visually-Rich Document
To alleviate such limitations, we propose a GlObal Structure knowledgeguided relation Extraction (GOSE) framework, which captures dependencies between entity pairs in an iterative manner.
Interactive Data Synthesis for Systematic Vision Adaptation via LLMs-AIGCs Collaboration
Recent text-to-image generation models have shown promising results in generating high-fidelity photo-realistic images.
InstructVid2Vid: Controllable Video Editing with Natural Language Instructions
To improve the consistency between adjacent frames of generated videos, we propose the Frame Difference Loss, which is incorporated during the training process.
Sim-MEES: Modular End-Effector System Grasping Dataset for Mobile Manipulators in Cluttered Environments
Our dataset generation process combines analytic models and dynamic simulations of the entire cluttered environment to provide accurate grasp labels.
Fast MRI Reconstruction via Edge Attention
To solve this problem, we propose a lightweight and accurate Edge Attention MRI Reconstruction Network (EAMRI) to reconstruct images with edge guidance.
EWT: Efficient Wavelet-Transformer for Single Image Denoising
Transformer-based image denoising methods have achieved encouraging results in the past year.
PFT-SSR: Parallax Fusion Transformer for Stereo Image Super-Resolution
Stereo image super-resolution aims to boost the performance of image super-resolution by exploiting the supplementary information provided by binocular systems.
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World
Based on that, we further introduce a novel Entangled cross-modal prompt approach for open-world predicate scene graph generation (Epic), where models can generalize to unseen predicates in a zero-shot manner.
Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization
Though effective, prompt tuning under few-shot settings on the one hand heavily relies on a good initialization of soft prompts.
Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models
Prompt tuning, a recently emerging paradigm, enables the powerful vision-language pre-training models to adapt to downstream tasks in a parameter -- and data -- efficient way, by learning the ``soft prompts'' to condition frozen pre-training models.
Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN
In the past decade, convolutional neural networks (CNNs) have shown prominence for semantic segmentation.
Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding
To systematically benchmark the compositional generalizability of temporal grounding models, we introduce a new Compositional Temporal Grounding task and construct two new dataset splits, i. e., Charades-CG and ActivityNet-CG.
Are Binary Annotations Sufficient? Video Moment Retrieval via Hierarchical Uncertainty-Based Active Learning
Moreover, we treat the uncertainty score of frames in a video as a whole, and estimate the difficulty of each video, which can further relieve the burden of video selection.
Efficient Image Super-Resolution with Feature Interaction Weighted Hybrid Network
Recently, great progress has been made in single-image super-resolution (SISR) based on deep learning technology.
DBA: Efficient Transformer with Dynamic Bilinear Low-Rank Attention
Furthermore, we show that the hidden state dimension can be approximated by extending the Johnson-Lindenstrauss lemma, optimizing the attention in bilinear form.
Fine-Grained Semantically Aligned Vision-Language Pre-Training
Large-scale vision-language pre-training has shown impressive advances in a wide range of downstream tasks.
Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos
In this paper, we introduce a new task, named Temporal Emotion Localization in videos~(TEL), which aims to detect human emotions and localize their corresponding temporal boundaries in untrimmed videos with aligned subtitles.
Quantized Consensus under Data-Rate Constraints and DoS Attacks: A Zooming-In and Holding Approach
This paper is concerned with the quantized consensus problem for uncertain nonlinear multi-agent systems under data-rate constraints and Denial-of-Service (DoS) attacks.
Masked Autoencoders that Listen
Following the Transformer encoder-decoder design in MAE, our Audio-MAE first encodes audio spectrogram patches with a high masking ratio, feeding only the non-masked tokens through encoder layers.
Ranked #2 on
Speaker Identification
on VoxCeleb1
(using extra training data)
Snow Mask Guided Adaptive Residual Network for Image Snow Removal
Recently, some methods have been proposed for snow removing, and most methods deal with snow images directly as the optimization object.
BOSS: Bottom-up Cross-modal Semantic Composition with Hybrid Counterfactual Training for Robust Content-based Image Retrieval
In this scenario, the input image serves as an intuitive context and background for the search, while the corresponding language expressly requests new traits on how specific characteristics of the query image should be modified in order to get the intended target image.
Cross-receptive Focused Inference Network for Lightweight Image Super-Resolution
Recently, Transformer-based methods have shown impressive performance in single image super-resolution (SISR) tasks due to the ability of global feature extraction.
Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation
Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning.
Lightweight Bimodal Network for Single-Image Super-Resolution via Symmetric CNN and Recursive Transformer
Single-image super-resolution (SISR) has achieved significant breakthroughs with the development of deep learning.
NTIRE 2022 Challenge on Stereo Image Super-Resolution: Methods and Results
In this paper, we summarize the 1st NTIRE challenge on stereo image super-resolution (restoration of rich details in a pair of low-resolution stereo images) with a focus on new solutions and results.
CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution
Then, we design an efficient Feature Refinement Module (FRM) to enhance the encoded features.
Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning
To systematically measure the compositional generalizability of temporal grounding models, we introduce a new Compositional Temporal Grounding task and construct two new dataset splits, i. e., Charades-CG and ActivityNet-CG.
Feature Distillation Interaction Weighting Network for Lightweight Image Super-Resolution
Convolutional neural networks based single-image super-resolution (SISR) has made great progress in recent years.
MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning
We propose the Multimodal relAtional Graph adversarIal inferenCe (MAGIC) framework for diverse and unpaired TextCap.
From Beginner to Master: A Survey for Deep Learning-based Single-Image Super-Resolution
In this survey, we give an overview of DL-based SISR methods and group them according to their targets, such as reconstruction efficiency, reconstruction accuracy, and perceptual accuracy.
FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation
Specifically, FBSNet employs a symmetrical encoder-decoder structure with two branches, semantic information branch and spatial detail branch.
Transformer for Single Image Super-Resolution
LTB is composed of a series of Efficient Transformers (ET), which occupies a small GPU memory occupation, thanks to the specially designed Efficient Multi-Head Attention (EMHA).
Structure-Preserving Deraining with Residue Channel Prior Guidance
Although these methods can remove part of the rain streaks, it is difficult for them to adapt to real-world scenarios and restore high-quality rain-free images with clear and accurate structures.
Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inference
Secondly, we introduce semantic coherence learning to explicitly encourage the semantic coherence of the adaptive hierarchical graph network from three hierarchies.
Feedback Network for Mutually Boosted Stereo Image Super-Resolution and Disparity Estimation
Besides the cross-view information exploitation in the low-resolution (LR) space, HR representations produced by the SR process are utilized to perform HR disparity estimation with higher accuracy, through which the HR features can be aggregated to generate a finer SR result.
Lightweight Image Super-Resolution with Multi-scale Feature Interaction Network
Recently, the single image super-resolution (SISR) approaches with deep and complex convolutional neural network structures have achieved promising performance.
Efficient and Accurate Multi-scale Topological Network for Single Image Dehazing
To achieve this, we propose a Multi-scale Topological Network (MSTN) to fully explore the features at different scales.
Scale-Aware Network with Regional and Semantic Attentions for Crowd Counting under Cluttered Background
Although the emergence of deep learning has greatly promoted the development of this field, crowd counting under cluttered background is still a serious challenge.
Robust Meta-learning with Noise via Eigen-Reptile
In particular, we first cast the meta-overfitting problem (overfitting on sampling and label noise) as a gradient noise problem since few available samples cause meta-learner to overfit on existing examples (clean or corrupted) of an individual task at every gradient step.
Revisiting Factorizing Aggregated Posterior in Learning Disentangled Representations
In this paper, we provide a theoretical explanation that low total correlation of sampled representation cannot guarantee low total correlation of the mean representation.
MDCN: Multi-scale Dense Cross Network for Image Super-Resolution
Among them, MDCB aims to detect multi-scale features and maximize the use of image features flow at different scales, HFDB focuses on adaptively recalibrate channel-wise feature responses to achieve feature distillation, and DRB attempts to reconstruct SR images with different upsampling factors in a single model.
Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling
In this paper, we focus on enhancing the generalization ability of the VIST model by considering the few-shot setting.
Disentangle Perceptual Learning through Online Contrastive Learning
Perceptual learning approaches like perceptual loss are empirically powerful for such tasks but they usually rely on the pre-trained classification network to provide features, which are not necessarily optimal in terms of visual perception of image transformation.
Towards Zero-shot Learning for Automatic Phonemic Transcription
The difficulty of this task is that phoneme inventories often differ between the training languages and the target language, making it infeasible to recognize unseen phonemes.
Universal Phone Recognition with a Multilingual Allophone System
Multilingual models can improve language processing, particularly for low resource situations, by sharing parameters across languages.
Fast Loop Closure Detection via Binary Content
Our method is compared with the state-of-the-art loop closure detection methods and the results show that it outperforms the traditional methods at both recall rate and speed.
HighEr-Resolution Network for Image Demosaicing and Enhancing
However, plenty of studies have shown that global information is crucial for image restoration tasks like image demosaicing and enhancing.
Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation
Visual navigation is a task of training an embodied agent by intelligently navigating to a target object (e. g., television) using only visual observations.
Walking with MIND: Mental Imagery eNhanceD Embodied QA
The experimental results and further analysis prove that the agent with the MIND module is superior to its counterparts not only in EQA performance but in many other aspects such as route planning, behavioral interpretation, and the ability to generalize from a few examples.
Adversarial camera stickers: A physical camera-based attack on deep learning systems
In this work, we consider an alternative question: is it possible to fool deep classifiers, over all perceived objects of a certain type, by physically manipulating the camera itself?
A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling
This paper compares five types of pooling functions both theoretically and experimentally, with special focus on their performance of localization.
Sound Audio and Speech Processing
Progressive Feature Fusion Network for Realistic Image Dehazing
Most of them follow a classic atmospheric scattering model which is an elegant simplified physical model based on the assumption of single-scattering and homogeneous atmospheric medium.
An Effective Single-Image Super-Resolution Model Using Squeeze-and-Excitation Networks
Recent works on single-image super-resolution are concentrated on improving performance through enhancing spatial encoding between convolutional layers.
Multi-scale Residual Network for Image Super-Resolution
Meanwhile, we let these features interact with each other to get the most efficacious image information, we call this structure Multi-scale Residual Block (MSRB).
Learning Joint Embedding with Multimodal Cues for Cross-Modal Video-Text Retrieval
Constructing a joint representation invariant across different modalities (e. g., video, language) is of significant importance in many multimedia applications.
Ranked #30 on
Video Retrieval
on MSR-VTT
A Comparison of deep learning methods for environmental sound
On these features, we apply five models: Gaussian Mixture Model (GMM), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Convolutional Deep Neural Net- work (CNN) and i-vector.
Learning Filter Banks Using Deep Learning For Acoustic Signals
Based on the procedure of log Mel-filter banks, we design a filter bank learning layer.
Very Deep Convolutional Neural Networks for Raw Waveforms
Our CNNs, with up to 34 weight layers, are efficient to optimize over very long sequences (e. g., vector of size 32000), necessary for processing acoustic waveforms.
