Search Results for author:
Found 102 papers, 61 papers with code
不同类型噪声环境下言语理解的脑机制研究(Brain Mechanism of Speech Comprehension in Different Noise Conditions)
“文章使用ERP技术, 对比分析了安静、白噪声、汉语噪声、英语噪声四种听觉条件下, 听力正常的汉语母语者加工汉语句子的情况, 以探究信息掩蔽条件下语义加工的神经机制。研究发现不同噪声条件下诱发的N100、N400、LPC等ERPs成分具有不同的波形表现, 据此本文得出以下结论:首先, 在语音掩蔽条件下, 对于难度较大的语义加工, 目标语音与掩蔽噪声在知觉层面的相似性并非主要影响因素, 而掩蔽噪声语义内容上的可懂度发挥着更关键的作用。其次, 当言语噪声为听者极其熟悉或完全陌生的语言, 其对语义加工的掩蔽干扰较小, 而当掩蔽噪声为听者接触过的语言但不是母语或主要语言, 其掩蔽效应可能更强。最后, 不熟悉的言语噪声中所包含的出现频率较小但能够被听者理解的语义内容, 与听者的预期相冲突, 引发听者的注意转移, 这些语义信息被传输至听觉中枢神经, 占用了原本用于目标刺激的认知资源, 从而增强了信息掩蔽的效果。”
Think about it! Improving defeasible reasoning by first modeling the question scenario.
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
A Self-enhancement Approach for Domain-specific Chatbot Training via Knowledge Mining and Digest
This paper introduces a novel approach to enhance LLMs by effectively extracting the relevant knowledge from domain-specific textual sources, and the adaptive training of a chatbot with domain-specific inquiries.
AutoMix: Automatically Mixing Language Models
Large language models (LLMs) are now available in various sizes and configurations from cloud API providers.
SALMON: Self-Alignment with Principle-Following Reward Models
Central to our approach is a principle-following reward model.
Functional Interpolation for Relative Positions Improves Long Context Transformers
Preventing the performance decay of Transformers on inputs longer than those used for training has been an important challenge in extending the context length of these models.
Aligning Large Multimodal Models with Factually Augmented RLHF
Large Multimodal Models (LMM) are built across modalities and the misalignment between two modalities can result in "hallucination", generating textual outputs that are not grounded by the multimodal information in context.
Accelerating Diffusion-based Combinatorial Optimization Solvers by Progressive Distillation
Graph-based diffusion models have shown promising results in terms of generating high-quality solutions to NP-complete (NPC) combinatorial optimization (CO) problems.
Efficient Temporal Sentence Grounding in Videos with Multi-Teacher Knowledge Distillation
To tackle this problem, we propose a novel efficient multi-teacher model (EMTM) based on knowledge distillation to transfer diverse knowledge from both heterogeneous and isomorphic networks.
Balancing Exploration and Exploitation in Hierarchical Reinforcement Learning via Latent Landmark Graphs
However, existing works often overlook the temporal coherence in GCHRL when learning latent subgoal representations and lack an efficient subgoal selection strategy that balances exploration and exploitation.
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification
We present PESCO, a novel contrastive learning framework that substantially improves the performance of zero-shot text classification.
Policy Representation via Diffusion Probability Model for Reinforcement Learning
Popular reinforcement learning (RL) algorithms tend to produce a unimodal policy distribution, which weakens the expressiveness of complicated policy and decays the ability of exploration.
Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs
A popular approach for improving the correctness of output from large language models (LLMs) is Self-Consistency - poll the LLM multiple times and output the most frequent solution.
Active Retrieval Augmented Generation
In this work, we provide a generalized view of active retrieval augmented generation, methods that actively decide when and what to retrieve across the course of the generation.
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
Recent AI-assistant agents, such as ChatGPT, predominantly rely on supervised fine-tuning (SFT) with human annotations and reinforcement learning from human feedback (RLHF) to align the output of large language models (LLMs) with human intentions, ensuring they are helpful, ethical, and reliable.
Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-tuned GPT
Moreover, GPT-based zero-shot classification models tend to make independent predictions over test instances, which can be sub-optimal as the instance correlations and the decision boundaries in the target space are ignored.
A Wideband Reconfigurable Intelligent Surface for 5G Millimeter-Wave Applications
The measurement results demonstrate that the proposed RIS could maintain a 3 dB peak gain variation bandwidth among various array configurations within 22. 5 to 29. 5 GHz (26. 9%) and with a beam scanning capability of 50{\deg}, making this design a good candidate for 5G mm-wave applications.
Self-Refine: Iterative Refinement with Self-Feedback
Motivated by how humans refine their written text, we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement.
Ranked #6 on
Arithmetic Reasoning
on GSM8K
A Neural PDE Solver with Temporal Stencil Modeling
Numerical simulation of non-linear partial differential equations plays a crucial role in modeling physical science and engineering phenomena, such as weather, climate, and aerodynamics.
DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization
For the MIS problem, DIFUSCO outperforms the previous state-of-the-art neural solver on the challenging SATLIB benchmark.
A Via-less Fully Screen-Printed Reconfigurable Intelligent Surface for 5G Millimeter Wave Communication
In this paper, we propose a via-less fully screen-printed reconfigurable intelligent surface which can establish a second line-of-sight communication from 23. 5GHz to 29. 5GHz.
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers
For 3D-data FLTs are, to the best of our knowledge, the first Transformers architectures providing RPE-enhanced linear attention.
PAL: Program-aided Language Models
Much of this success can be attributed to prompting methods such as "chain-of-thought'', which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem.
Ranked #7 on
Arithmetic Reasoning
on GSM8K
Language Models of Code are Few-Shot Commonsense Learners
In all these natural language tasks, we show that using our approach, a code generation LM (CODEX) outperforms natural-LMs that are fine-tuned on the target task (e. g., T5) and other strong LMs such as GPT-3 in the few-shot setting.
DIMES: A Differentiable Meta Solver for Combinatorial Optimization Problems
Recently, deep reinforcement learning (DRL) models have shown promising results in solving NP-hard Combinatorial Optimization (CO) problems.
Recitation-Augmented Language Models
We propose a new paradigm to help Large Language Models (LLMs) generate more accurate factual knowledge without retrieving from an external corpus, called RECITation-augmented gEneration (RECITE).
FLOWGEN: Fast and slow graph generation
Machine learning systems typically apply the same model to both easy and tough cases.
Conditional set generation using Seq2seq models
Conditional set generation learns a mapping from an input sequence of tokens to a set.
Long-tailed Extreme Multi-label Text Classification with Generated Pseudo Label Descriptions
Extreme Multi-label Text Classification (XMTC) has been a tough challenge in machine learning research and applications due to the sheer sizes of the label spaces and the severe data scarce problem associated with the long tail of rare labels in highly skewed distributions.
Multi Label Text Classification
Multi-Label Text Classification
+3
Exploiting Local and Global Features in Transformer-based Extreme Multi-label Text Classification
Extreme multi-label text classification (XMTC) is the task of tagging each document with the relevant labels from a very large space of predefined categories.
Multi Label Text Classification
Multi-Label Text Classification
+1
Traffic4cast at NeurIPS 2021 -- Temporal and Spatial Few-Shot Transfer Learning in Gridded Geo-Spatial Processes
The IARAI Traffic4cast competitions at NeurIPS 2019 and 2020 showed that neural networks can successfully predict future traffic conditions 1 hour into the future on simply aggregated GPS probe data in time and space bins.
Memory-assisted prompt editing to improve GPT-3 after deployment
Large LMs such as GPT-3 are powerful, but can commit mistakes that are obvious to humans.
Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback
Our goal is for an LM to continue to improve after deployment, without retraining, using feedback from the user.
Interscript: A dataset for interactive learning of scripts through error feedback
We present a new dataset, Interscript, containing user feedback on a deployed model that generates complex everyday tasks.
Think about it! Improving defeasible reasoning by first modeling the question scenario
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
Dual Encoding U-Net for Spatio-Temporal Domain Shift Frame Prediction
The landscape of city-wide mobility behaviour has altered significantly over the past 18 months.
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
The recent proposed Fusion-in-Decoder (FiD), which is built on top of the pretrained generative model T5, achieves the state-of-the-art performance in the reading module.
Sparse Attention with Learning to Hash
To overcome these issues, this paper proposes a new strategy for sparse attention, namely LHA (Learning-to-Hash Attention), which directly learns separate parameterized hash functions for queries and keys, respectively.
Could you give me a hint? Generating inference graphs for defeasible reasoning
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence.
Improving Neural Model Performance through Natural Language Feedback on Their Explanations
A class of explainable NLP models for reasoning tasks support their decisions by generating free-form or structured explanations, but what happens when these supporting structures contain errors?
Improving Hyper-Relational Knowledge Graph Completion
Different from traditional knowledge graphs (KGs) where facts are represented as entity-relation-entity triplets, hyper-relational KGs (HKGs) allow triplets to be associated with additional relation-entity pairs (a. k. a qualifiers) to convey more complex information.
CURIE: An Iterative Querying Approach for Reasoning About Situations
Recently, models have been shown to predict the effects of unexpected situations, e. g., would cloudy skies help or hinder plant growth?
Meta Back-translation
Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data.
Handling Noisy Labels via One-Step Abductive Multi-Target Learning and Its Application to Helicobacter Pylori Segmentation
This situation raises two more difficult problems: 1) the methodology of approaches making corrections corresponding to potentially noisy-labeled instances has limitations due to the complex noise existing in labels; and 2) the appropriate evaluation strategy for validation/testing is unclear because of the great difficulty in collecting the noisy-free ground-truth labels.
Rethinking Transformer-based Set Prediction for Object Detection
DETR is a recently proposed Transformer-based method which views object detection as a set prediction problem and achieves state-of-the-art performance but demands extra-long training time to converge.
On the Sentence Embeddings from Pre-trained Language Models
Pre-trained contextual representations like BERT have achieved great success in natural language processing.
Ranked #16 on
Semantic Textual Similarity
on STS16
EIGEN: Event Influence GENeration using Pre-trained Language Models
Reasoning about events and tracking their influences is fundamental to understanding processes.
Neural Language Modeling for Contextualized Temporal Graph Generation
We address this challenge by using existing IE/NLP tools to automatically generate a large quantity (89, 000) of system-produced document-graph pairs, and propose a novel formulation of the contextualized graph generation problem as a sequence-to-sequence mapping task.
JAKET: Joint Pre-training of Knowledge Graph and Language Understanding
Knowledge graphs (KGs) contain rich information about world knowledge, entities and relations.
Unsupervised Parallel Corpus Mining on Web Data
In contrast, there is a large-scale of parallel corpus created by humans on the Internet.
Kernel Stein Generative Modeling
NCK is crucial for successful inference with SVGD in high dimension, as it adapts the kernel to the noise level of the score estimate.
An EM Approach to Non-autoregressive Conditional Sequence Generation
Autoregressive (AR) models have been the dominating approach to conditional sequence generation, but are suffering from the issue of high inference latency.
Knowledge Embedding Based Graph Convolutional Network
Recently, a considerable literature has grown up around the theme of Graph Convolutional Network (GCN).
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
With the success of language pretraining, it is highly desirable to develop more efficient architectures of good scalability that can exploit the abundant unlabeled data at a lower cost.
Ranked #6 on
Reading Comprehension
on RACE
Predicting Performance for Natural Language Processing Tasks
Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly proposed models on each possible experimental setting.
Politeness Transfer: A Tag and Generate Approach
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning.
Practical Comparable Data Collection for Low-Resource Languages via Images
We propose a method of curating high-quality comparable training data for low-resource languages with monolingual annotators.
Correlation-aware Unsupervised Change-point Detection via Graph Neural Networks
Keywords: Multivariate Time Series, Change-point Detection, Graph Neural Networks
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Then, we conduct knowledge transfer from this teacher to MobileBERT.
Ranked #18 on
Semantic Textual Similarity
on MRPC
VIOLIN: A Large-Scale Dataset for Video-and-Language Inference
We introduce a new task, Video-and-Language Inference, for joint multimodal understanding of video and text.
Learning Relation Entailment with Structured and Textual Information
We propose several methods that incorporate both structured and textual information to represent relations for this task.
Pre-training Tasks for Embedding-based Large-scale Retrieval
We consider the large-scale query-document retrieval problem: given a query (e. g., a question), return the set of relevant documents (e. g., paragraphs containing the answer) from a large document corpus.
Graph-Revised Convolutional Network
Graph Convolutional Networks (GCNs) have received increasing attention in the machine learning community for effectively leveraging both the content features of nodes and the linkage patterns across graphs in various applications.
A Re-evaluation of Knowledge Graph Completion Methods
Knowledge Graph Completion (KGC) aims at automatically predicting missing links for large-scale knowledge graphs.
Ranked #25 on
Link Prediction
on FB15k-237
(MR metric)
XL-Editor: Post-editing Sentences with XLNet
While neural sequence generation models achieve initial success for many NLP applications, the canonical decoding procedure with left-to-right generation order (i. e., autoregressive) in one-pass can not reflect the true nature of human revising a sentence to obtain a refined result.
Active Learning for Graph Neural Networks via Node Feature Propagation
Graph Neural Networks (GNNs) for prediction tasks like node classification or edge prediction have received increasing attention in recent machine learning from graphically structured data.
Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework
Learning multilingual representations of text has proven a successful method for many cross-lingual transfer learning tasks.
Active Learning Graph Neural Networks via Node Feature Propagation
Graph Neural Networks (GNNs) for prediction tasks like node classification or edge prediction have received increasing attention in recent machine learning from graphically structured data.
Bridging the domain gap in cross-lingual document classification
We consider the setting of semi-supervised cross-lingual understanding, where labeled data is available in a source language (English), but only unlabeled data is available in the target language.
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text
In this paper, we investigate a simple fix for posterior collapse which yields surprisingly effective results.
DEFT: A corpus for definition extraction in free- and semi-structured text
Definition extraction has been a popular topic in NLP research for well more than a decade, but has been historically limited to well-defined, structured, and narrow conditions.
XLNet: Generalized Autoregressive Pretraining for Language Understanding
With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.
Taming Pretrained Transformers for Extreme Multi-label Text Classification
However, naively applying deep transformer models to the XMC problem leads to sub-optimal performance due to the large output space and the label sparsity issue.
Extreme Multi-Label Classification
General Classification
+3
Transformer-XL: Language Modeling with Longer-Term Dependency
Moreover, Transformer-XL is up to 1, 800+ times faster than vanilla Transformer during evaluation.
Implicit Kernel Learning
While learning the kernel in a data driven way has been investigated, in this paper we explore learning the spectral distribution of kernel via implicit generative models parametrized by deep neural networks.
The ARIEL-CMU Systems for LoReHLT18
This paper describes the ARIEL-CMU submissions to the Low Resource Human Language Technologies (LoReHLT) 2018 evaluations for the tasks Machine Translation (MT), Entity Discovery and Linking (EDL), and detection of Situation Frames in Text and Speech (SF Text and Speech).
Re-examination of the Role of Latent Variables in Sequence Modeling
With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence.
An Adversarial Approach to High-Quality, Sentiment-Controlled Neural Dialogue Generation
In this work, we propose a method for neural dialogue response generation that allows not only generating semantically reasonable responses according to the dialogue history, but also explicitly controlling the sentiment of the response via sentiment labels.
Kernel Change-point Detection with Auxiliary Deep Generative Models
Detecting the emergence of abrupt property changes in time series is a challenging problem.
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling.
Ranked #3 on
Language Modelling
on One Billion Word
Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning
Training task-completion dialogue agents with reinforcement learning usually requires a large number of real user experiences.
SeCSeq: Semantic Coding for Sequence-to-Sequence based Extreme Multi-label Classification
To circumvent the softmax bottleneck, SeCSeq compresses labels into sequences of semantic-aware compact codes, on which Seq2Seq models are trained.
Unsupervised Cross-lingual Transfer of Word Embedding Spaces
Supervised methods for this problem rely on the availability of cross-lingual supervision, either using parallel corpora or bilingual lexicons as the labeled data for training, which may not be available for many low resource languages.
DARTS: Differentiable Architecture Search
This paper addresses the scalability challenge of architecture search by formulating the task in a differentiable manner.
Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
In this paper, we combine the ideas from both stochastic latent variables and dilated convolutions, and propose a new architecture to model sequential data, termed as Stochastic WaveNet, where stochastic latent variables are injected into the WaveNet structure.
Learning Graph Convolution Filters from Data Manifold
Convolution Neural Network (CNN) has gained tremendous success in computer vision tasks with its outstanding ability to capture the local latent features.
Asymmetric Variational Autoencoders
However, freely enriching the family of variational distribution is challenging since the ELBO requires variational likelihood evaluations of the latent variables.
Convolutional Normalizing Flows
Variational inference provides one way to approximate the posterior distribution, however its expressive power is limited and so is the accuracy of resulting approximation.
Learning Depthwise Separable Graph Convolution from Data Manifold
Convolution Neural Network (CNN) has gained tremendous success in computer vision tasks with its outstanding ability to capture the local latent features.
MMD GAN: Towards Deeper Understanding of Moment Matching Network
In this paper, we propose to improve both the model expressiveness of GMMN and its computational efficiency by introducing adversarial kernel learning techniques, as the replacement of a fixed Gaussian kernel in the original GMMN.
Data-driven Random Fourier Features using Stein Effect
Large-scale kernel approximation is an important problem in machine learning research.
Analogical Inference for Multi-Relational Embeddings
Large-scale multi-relational embedding refers to the task of learning the latent representations for entities and relations in large knowledge graphs.
Ranked #24 on
Link Prediction
on WN18
Cross-lingual Distillation for Text Classification
Using soft probabilistic predictions for the documents in a label-rich language as the (induced) supervisory labels in a parallel corpus of documents, we train classifiers successfully for new languages in which labeled training data are not available.
RACE: Large-scale ReAding Comprehension Dataset From Examinations
We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task.
Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
Multivariate time series forecasting is an important machine learning problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation.
Ranked #1 on
Univariate Time Series Forecasting
on Solar-Power
Co-Clustering for Multitask Learning
This paper presents a new multitask learning framework that learns a shared representation among the tasks, incorporating both task and feature clusters.
Leveraging Multilingual Training for Limited Resource Event Extraction
Event extraction has become one of the most important topics in information extraction, but to date, there is very limited work on leveraging cross-lingual training to boost performance.
Adaptive Smoothed Online Multi-Task Learning
This paper addresses the challenge of jointly learning both the per-task model parameters and the inter-task relationships in a multi-task online learning setting.
Cross-Graph Learning of Multi-Relational Associations
Cross-graph Relational Learning (CGRL) refers to the problem of predicting the strengths or labels of multi-relational tuples of heterogeneous object types, through the joint inference over multiple graphs which specify the internal connections among each type of objects.
Bayesian models for Large-scale Hierarchical Classification
A challenging problem in hierarchical classification is to leverage the hierarchical relations among classes for improving classification performance.
