Search Results for author:
Found 57 papers, 23 papers with code
面向垂直领域的阅读理解数据增强方法(Method for reading comprehension data enhancement in vertical field)
阅读理解问答系统是利用语义理解等自然语言处理技术, 根据输入问题, 对非结构化文档数据进行分析, 生成一个答案, 具有很高的研究和应用价值。在垂直领域应用过程中, 阅读理解问答数据标注成本高且用户问题表达复杂多样, 使得阅读理解问答系统准确率低、鲁棒性差。针对这一问题, 本文提出一种面向垂直领域的阅读理解问答数据的增强方法, 该方法基于真实用户问题, 构造阅读理解训练数据, 一方面降低标注成本, 另一方面增加训练数据多样性, 提升模型的准确率和鲁棒性。本文用汽车领域数据对该方法进行实验验证, 其结果表明该方法对垂直领域阅读理解模型的准确率和鲁棒性均能有效提升。
Representation Discrepancy Bridging Method for Remote Sensing Image-Text Retrieval
On the one hand, a Cross-Modal Asymmetric Adapter (CMAA) is designed to enable modality-specific optimization and improve feature alignment.
IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
We study the enlarge-and-prune pipeline as an integrated system to address two critical questions: whether it is worth pretraining an enlarged model even when the model is never deployed, and how to optimize the entire pipeline for better pruned models.
Instruction-Following Pruning for Large Language Models
In our method, the pruning mask is input-dependent and adapts dynamically based on the information described in a user instruction.
Distribution alignment based transfer fusion frameworks on quantum devices for seeking quantum advantages
Two transfer fusion frameworks are proposed in this paper to predict the labels of a target domain data by aligning its distribution to a different but related labelled source domain on quantum devices.
EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing
Diffusion transformers have been widely adopted for text-to-image synthesis.
PMR-Net: Parallel Multi-Resolution Encoder-Decoder Network Framework for Medical Image Segmentation
The multi-resolution context encoder fuses the global context semantic features of different receptive fields from different encoder branches to maintain effectively the integrity of global information.
Apple Intelligence Foundation Language Models
We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute.
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons.
Ranked #73 on
Visual Question Answering
on MM-Vet
Learning to Skip for Language Modeling
Overparameterized large-scale language models have impressive generalization performance of in-context few-shot learning.
TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation
The hybrid architecture of convolution neural networks (CNN) and Transformer has been the most popular method for medical image segmentation.
CiT-Net: Convolutional Neural Networks Hand in Hand with Vision Transformers for Medical Image Segmentation
The hybrid architecture of convolutional neural networks (CNNs) and Transformer are very popular for medical image segmentation.
Lightweight Structure-aware Transformer Network for VHR Remote Sensing Image Change Detection
Popular Transformer networks have been successfully applied to remote sensing (RS) image change detection (CD) identifications and achieve better results than most convolutional neural networks (CNNs), but they still suffer from two main problems.
Rethinking the Role of Token Retrieval in Multi-Vector Retrieval
Multi-vector retrieval models such as ColBERT [Khattab and Zaharia, 2020] allow token-level interactions between queries and documents, and hence achieve state of the art on many information retrieval benchmarks.
Ultralightweight Spatial–Spectral Feature Cooperation Network for Change Detection in Remote Sensing Images
First, the existing multiscale feature fusion methods often use redundant feature extraction and fusion strategies, which often lead to high computational costs and memory usage.
Ranked #4 on
Change Detection
on DSIFN-CD
Building change detection for remote sensing images
Change Detection
+1
CoLT5: Faster Long-Range Transformers with Conditional Computation
Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but also from applying feedforward and projection layers to every token.
Ranked #1 on
Long-range modeling
on SCROLLS
Lightweight Facial Attractiveness Prediction Using Dual Label Distribution
The data processing is simplified to a minimum for a lightweight design, and MobileNetV2 is selected as our backbone.
Multi-Vector Retrieval as Sparse Alignment
With sparsified unary saliences, we are able to prune a large number of query and document token vectors and improve the efficiency of multi-vector retrieval.
Training Language Models with Memory Augmentation
Recent work has improved language models (LMs) remarkably by equipping them with a non-parametric memory component.
Simple Recurrence Improves Masked Language Models
In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into the Transformer.
Mixture-of-Experts with Expert Choice Routing
Prior work allocates a fixed number of experts to each token using a top-k function regardless of the relative importance of different tokens.
SRU++: Pioneering Fast Recurrence with Attention for Speech Recognition
The Transformer architecture has been well adopted as a dominant architecture in most sequence transduction tasks including automatic speech recognition (ASR), since its attention mechanism excels in capturing long-range dependencies.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Channel-Temporal Attention for First-Person Video Domain Adaptation
Finally, we propose a Channel-Temporal Attention Network (CTAN) to integrate these blocks into existing architectures.
Team PyKale (xy9) Submission to the EPIC-Kitchens 2021 Unsupervised Domain Adaptation Challenge for Action Recognition
Under the team name xy9, our submission achieved 5th place in terms of top-1 accuracy for verb class and all top-5 accuracies.
Nutri-bullets Hybrid: Consensual Multi-document Summarization
We present a method for generating comparative summaries that highlight similarities and contradictions in input documents.
Nutribullets Hybrid: Multi-document Health Summarization
We present a method for generating comparative summaries that highlights similarities and contradictions in input documents.
Nutri-bullets: Summarizing Health Studies by Composing Segments
We introduce \emph{Nutri-bullets}, a multi-document summarization task for health and nutrition.
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute
In this work, we present SRU++, a highly-efficient architecture that combines fast recurrence and attention for sequence modeling.
Ranked #4 on
Language Modelling
on enwik8
Medical Image Segmentation Using Deep Learning: A Survey
Firstly, compared to traditional surveys that directly divide literatures of deep learning on medical image segmentation into many groups and introduce literatures in detail for each group, we classify currently popular literatures according to a multi-level structure from coarse to fine.
Autoregressive Knowledge Distillation through Imitation Learning
The performance of autoregressive models on natural language generation tasks has dramatically improved due to the adoption of deep, self-attentive architectures.
Ranked #22 on
Machine Translation
on IWSLT2014 German-English
Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport
Selecting input features of top relevance has become a popular method for building self-explaining models.
ASAPP-ASR: Multistream CNN and Self-Attentive SRU for SOTA Speech Recognition
In this paper we present state-of-the-art (SOTA) performance on the LibriSpeech corpus with two novel neural network architectures, a multistream CNN for acoustic modeling and a self-attentive simple recurrent unit (SRU) for language modeling.
Ranked #10 on
Speech Recognition
on LibriSpeech test-clean
(using extra training data)
Interactive Classification by Asking Informative Questions
We study the potential for interaction in natural language classification.
Metric Learning for Dynamic Text Classification
However, in many real-world applications the label set is frequently changing.
Structured Pruning of Large Language Models
Large language models have recently achieved state of the art performance across a wide variety of natural language tasks.
Building a Production Model for Retrieval-Based Chatbots
Response suggestion is an important task for building human-computer conversation systems.
AUTOHOME-ORCA at SemEval-2019 Task 8: Application of BERT for Fact-Checking in Community Forums
In order to address this task, we propose a system based on the BERT model with meta information of questions.
Adaptive Morphological Reconstruction for Seeded Image Segmentation
However, MR might mistakenly filter meaningful seeds that are required for generating accurate segmentation and it is also sensitive to the scale because a single-scale structuring element is employed.
Adversarial Domain Adaptation for Duplicate Question Detection
We address the problem of detecting duplicate questions in forums, which is an important step towards automating the process of answering new questions.
Significantly Fast and Robust Fuzzy C-MeansClustering Algorithm Based on MorphologicalReconstruction and Membership Filtering
However, the introduction oflocal spatial information often leads to a high computationalcomplexity, arising out of an iterative calculation of the distancebetween pixels within local spatial neighbors and clusteringcenters.
Training RNNs as Fast as CNNs
Common recurrent neural network architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.
Simple Recurrent Units for Highly Parallelizable Recurrence
Common recurrent neural architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.
Ranked #32 on
Question Answering
on SQuAD1.1 dev
Style Transfer from Non-Parallel Text by Cross-Alignment
We demonstrate the effectiveness of this cross-alignment method on three tasks: sentiment modification, decipherment of word substitution ciphers, and recovery of word order.
Ranked #7 on
Text Style Transfer
on Yelp Review Dataset (Small)
Deriving Neural Architectures from Sequence and Graph Kernels
The design of neural architectures for structured objects is typically guided by experimental insights rather than a formal process.
Rationalizing Neural Predictions
Our approach combines two modular components, generator and encoder, which are trained to operate well together.
Semi-supervised Question Retrieval with Gated Convolutions
Question answering forums are rapidly growing in size with no effective automated ability to refer to and reuse answers already available for previous posted questions.
Molding CNNs for text: non-linear, non-consecutive convolutions
Moreover, we extend the n-gram convolution to non-consecutive words to recognize patterns with intervening words.
Exploring Compositional Architectures and Word Vector Representations for Prepositional Phrase Attachment
In this paper, we show that word vector representations can yield significant PP attachment performance gains.
