Search Results for author:
Found 30 papers, 22 papers with code
A Gap-Based Framework for Chinese Word Segmentation via Very Deep Convolutional Networks
However, if we consider segmenting a given sentence, the most intuitive idea is to predict whether to segment for each gap between two consecutive characters, which in comparison makes previous approaches seem too complex.
Unsupervised Neural Word Segmentation for Chinese via Segmental Language Modeling
As far as we know, we are the first to propose a neural model for unsupervised CWS and achieve competitive performance to the state-of-the-art statistical models on four different datasets from SIGHAN 2005 bakeoff.
RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
We study the problem of learning representations of entities and relations in knowledge graphs for predicting missing links.
Ranked #2 on
Link Prediction
on FB122
DivGraphPointer: A Graph Pointer Network for Extracting Diverse Keyphrases
Furthermore, we propose a diversified point network to generate a set of diverse keyphrases out of the word graph in the decoding process.
Neural Consciousness Flow
Instead, inspired by the consciousness prior proposed by Yoshua Bengio, we explore reasoning with the notion of attentive awareness from a cognitive perspective, and formulate it in the form of attentive message passing on graphs, called neural consciousness flow (NeuCFlow).
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
In this paper, we provide a novel perspective towards understanding the architecture: we show that the Transformer can be mathematically interpreted as a numerical Ordinary Differential Equation (ODE) solver for a convection-diffusion equation in a multi-particle dynamic system.
Dynamically Pruned Message Passing Networks for Large-Scale Knowledge Graph Reasoning
We propose Dynamically Pruned Message Passing Networks (DPMPN) for large-scale knowledge graph reasoning.
Fast Structured Decoding for Sequence Models
However, these models assume that the decoding process of each token is conditionally independent of others.
A Re-evaluation of Knowledge Graph Completion Methods
Knowledge Graph Completion (KGC) aims at automatically predicting missing links for large-scale knowledge graphs.
Ranked #25 on
Link Prediction
on FB15k-237
(MR metric)
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View.
In particular, how words in a sentence are abstracted into contexts by passing through the layers of the Transformer can be interpreted as approximating multiple particles' movement in the space using the Lie-Trotter splitting scheme and the Euler's method.
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Then, we conduct knowledge transfer from this teacher to MobileBERT.
Ranked #20 on
Semantic Textual Similarity
on MRPC
An EM Approach to Non-autoregressive Conditional Sequence Generation
Autoregressive (AR) models have been the dominating approach to conditional sequence generation, but are suffering from the issue of high inference latency.
Rethinking Transformer-based Set Prediction for Object Detection
DETR is a recently proposed Transformer-based method which views object detection as a set prediction problem and achieves state-of-the-art performance but demands extra-long training time to converge.
Sparse Attention with Learning to Hash
To overcome these issues, this paper proposes a new strategy for sparse attention, namely LHA (Learning-to-Hash Attention), which directly learns separate parameterized hash functions for queries and keys, respectively.
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
PromptSource is a system for creating, sharing, and using natural language prompts.
Recitation-Augmented Language Models
We propose a new paradigm to help Large Language Models (LLMs) generate more accurate factual knowledge without retrieving from an external corpus, called RECITation-augmented gEneration (RECITE).
DIMES: A Differentiable Meta Solver for Combinatorial Optimization Problems
Recently, deep reinforcement learning (DRL) models have shown promising results in solving NP-hard Combinatorial Optimization (CO) problems.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
A Neural PDE Solver with Temporal Stencil Modeling
Numerical simulation of non-linear partial differential equations plays a crucial role in modeling physical science and engineering phenomena, such as weather, climate, and aerodynamics.
DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimization
We evaluate our methods on two well-studied NPC combinatorial optimization problems: Traveling Salesman Problem (TSP) and Maximal Independent Set (MIS).
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
Recent AI-assistant agents, such as ChatGPT, predominantly rely on supervised fine-tuning (SFT) with human annotations and reinforcement learning from human feedback (RLHF) to align the output of large language models (LLMs) with human intentions, ensuring they are helpful, ethical, and reliable.
Active Retrieval Augmented Generation
In this work, we provide a generalized view of active retrieval augmented generation, methods that actively decide when and what to retrieve across the course of the generation.
Accelerating Diffusion-based Combinatorial Optimization Solvers by Progressive Distillation
Graph-based diffusion models have shown promising results in terms of generating high-quality solutions to NP-complete (NPC) combinatorial optimization (CO) problems.
Aligning Large Multimodal Models with Factually Augmented RLHF
Large Multimodal Models (LMM) are built across modalities and the misalignment between two modalities can result in "hallucination", generating textual outputs that are not grounded by the multimodal information in context.
SALMON: Self-Alignment with Instructable Reward Models
Supervised Fine-Tuning (SFT) on response demonstrations combined with Reinforcement Learning from Human Feedback (RLHF) constitutes a powerful paradigm for aligning LLM-based AI agents.
Improving Reinforcement Learning from Human Feedback with Efficient Reward Model Ensemble
Reinforcement Learning from Human Feedback (RLHF) is a widely adopted approach for aligning large language models with human values.
Instruction-tuned Language Models are Better Knowledge Learners
The standard recipe for doing so involves continued pre-training on new documents followed by instruction-tuning on question-answer (QA) pairs.
HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wild
Hallucinations pose a significant challenge to the reliability of large language models (LLMs) in critical domains.
Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision
This paper answers this question in the context of tackling hard reasoning tasks (e. g., level 4-5 MATH problems) via learning from human annotations on easier tasks (e. g., level 1-3 MATH problems), which we term as \textit{easy-to-hard generalization}.
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward
Preference modeling techniques, such as direct preference optimization (DPO), has shown effective in enhancing the generalization abilities of large language model (LLM).
