Search Results for author:
Found 22 papers, 14 papers with code
PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
Consequently, a simple combination of them cannot guarantee accomplishing both training efficiency and inference efficiency with minimal costs.
Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval
In this study, we aim to shed light on this issue by revealing that masked auto-encoder (MAE) pre-training with enhanced decoding significantly improves the term coverage of input tokens in dense representations, compared to vanilla BERT checkpoints.
RepViT-SAM: Towards Real-Time Segmenting Anything
Here, to achieve real-time segmenting anything on mobile devices, following MobileSAM, we replace the heavyweight image encoder in SAM with RepViT model, ending up with the RepViT-SAM model.
MiLe Loss: a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models
Experiments reveal that models incorporating the proposed MiLe Loss can gain consistent performance improvement on downstream benchmarks.
Ranked #89 on
Multi-task Language Understanding
on MMLU
KwaiYiiMath: Technical Report
Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on mathematical tasks requiring multi-step reasoning.
Ranked #87 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs
We further employ a consistent dynamic channel pruning (CDCP) strategy to dynamically prune unimportant channels in ViTs.
Confidence-based Visual Dispersal for Few-shot Unsupervised Domain Adaptation
To address this issue, recent works consider the Few-shot Unsupervised Domain Adaptation (FUDA) where only a few source samples are labeled, and conduct knowledge transfer via self-supervised learning methods.
Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrieval
Concretely, we leverage the capabilities of LLMs for document expansion, i. e. query generation, and effectively transfer expanded knowledge to retrievers using pre-training strategies tailored for passage retrieval.
RepViT: Revisiting Mobile CNN From ViT Perspective
Recently, lightweight Vision Transformers (ViTs) demonstrate superior performance and lower latency, compared with lightweight Convolutional Neural Networks (CNNs), on resource-constrained mobile devices.
CoT-MAE v2: Contextual Masked Auto-Encoder with Multi-view Modeling for Passage Retrieval
As an effective representation bottleneck pretraining technique, the contextual masked auto-encoder utilizes contextual embedding to assist in the reconstruction of passages.
Query-as-context Pre-training for Dense Passage Retrieval
Recently, methods have been developed to improve the performance of dense passage retrieval by using context-supervised pre-training.
RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval
Sparse sampling is also likely to miss important frames corresponding to some text portions, resulting in textual redundancy.
InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddings
Contrastive learning has been extensively studied in sentence embedding learning, which assumes that the embeddings of different views of the same sentence are closer.
ConTextual Masked Auto-Encoder for Dense Passage Retrieval
Dense passage retrieval aims to retrieve the relevant passages of a query from a large corpus based on dense representations (i. e., vectors) of the query and the passages.
UniTrans: Unifying Model Transfer and Data Transfer for Cross-Lingual Named Entity Recognition with Unlabeled Data
Prior works in cross-lingual named entity recognition (NER) with no/little labeled data fall into two primary categories: model transfer based and data transfer based methods.
Ranked #1 on
Cross-Lingual NER
on NoDaLiDa Norwegian Bokmål
Shallow Feature Based Dense Attention Network for Crowd Counting
In this paper, we propose a Shallow feature based Dense Attention Network (SDANet) for crowd counting from still images, which diminishes the impact of backgrounds via involving a shallow feature based attention model, and meanwhile, captures multi-scale information via densely connecting hierarchical image features.
Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language
However, such methods either are not applicable if the labeled data in the source languages is unavailable, or do not leverage information contained in unlabeled data in the target language.
Ranked #1 on
Cross-Lingual NER
on CoNLL German
IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval
Existing methods leverage the attention mechanism to explore such correspondence in a fine-grained manner.
Ranked #18 on
Cross-Modal Retrieval
on Flickr30k
Enhanced Meta-Learning for Cross-lingual Named Entity Recognition with Minimal Resources
For languages with no annotated resources, transferring knowledge from rich-resource languages is an effective solution for named entity recognition (NER).
Ranked #1 on
Cross-Lingual NER
on MSRA
GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition
The dominant approaches for named entity recognition (NER) mostly adopt complex recurrent neural networks (RNN), e. g., long-short-term-memory (LSTM).
Ranked #23 on
Named Entity Recognition (NER)
on Ontonotes v5 (English)
Semantics-Preserving Hashing for Cross-View Retrieval
With benefits of low storage costs and high query speeds, hashing methods are widely researched for efficiently retrieving large-scale data, which commonly contains multiple views, e. g. a news report with images, videos and texts.
Image Tag Completion via Image-Specific and Tag-Specific Linear Sparse Reconstructions
Though widely utilized for facilitating image management, user-provided image tags are usually incomplete and insufficient to describe the whole semantic content of corresponding images, resulting in performance degradations in tag-dependent applications and thus necessitating effective tag completion methods.
