Search Results for author:
Found 39 papers, 27 papers with code
Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples
Most of the existing defense methods improve the adversarial robustness by making the models adapt to the training set augmented with some adversarial examples.
“Is Whole Word Masking Always Better for Chinese BERT?”: Probing on Chinese Grammatical Error Correction
The meaning of a word in Chinese is different in that a word is a compositional unit consisting of multiple characters.
Case2Code: Learning Inductive Reasoning with Synthetic Data
In this paper, we hope to focus on evaluating and teaching LLMs to conduct inductive reasoning, that is, LLMs are supposed to infer underlying rules by observing examples or sequential transformations.
Unified Active Retrieval for Retrieval Augmented Generation
Experiments on four representative types of user instructions show that UAR significantly outperforms existing work on the retrieval timing judgement and the performance of downstream tasks, which shows the effectiveness of UAR and its helpfulness to downstream tasks.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
Balanced Data Sampling for Language Model Training with Clustering
Data plays a fundamental role in the training of Large Language Models (LLMs).
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images, and music.
Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
To address the dropped tokens and padding, we propose the Rectify-Router, comprising the Intra-GPU Rectification and the Fill-in Rectification.
InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning
We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math.
Query of CC: Unearthing Large Scale Domain-Specific Knowledge from Public Corpora
Large language models have demonstrated remarkable potential in various tasks, however, there remains a significant scarcity of open-source models and data for specific domains.
Can AI Assistants Know What They Don't Know?
To answer this question, we construct a model-specific "I don't know" (Idk) dataset for an assistant, which contains its known and unknown questions, based on existing open-domain question answering datasets.
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
With the rapid development of large language models (LLMs), they are not only used as general-purpose AI assistants but are also customized through further fine-tuning to meet the requirements of different applications.
Super-Resolution on Rotationally Scanned Photoacoustic Microscopy Images Incorporating Scanning Prior
Recently, there is a trend to incorporate deep learning into the scanning process to further increase the scanning speed. Yet, most such attempts are performed for raster scanning while those for rotational scanning are relatively rare.
LLatrieval: LLM-Verified Retrieval for Verifiable Generation
Thus, the LLM can iteratively provide feedback to retrieval and facilitate the retrieval result to fully support verifiable generation.
Watermarking LLMs with Weight Quantization
Abuse of large language models reveals high risks as large language models are being deployed at an astonishing speed.
Character-LLM: A Trainable Agent for Role-Playing
Large language models (LLMs) can be used to serve as agents to simulate human behaviors, given the powerful ability to understand human instructions and provide high-quality generated texts.
PerturbScore: Connecting Discrete and Continuous Perturbations in NLP
Through experimental results, we find that we can build a connection between discrete and continuous perturbations and use the proposed PerturbScore to learn such correlation, surpassing previous methods used in discrete perturbation measuring.
SeqXGPT: Sentence-Level AI-Generated Text Detection
Therefore, it is important to build strong AI-generated text (AIGT) detectors.
Multijugate Dual Learning for Low-Resource Task-Oriented Dialogue System
Dialogue data in real scenarios tend to be sparsely available, rendering data-starved end-to-end dialogue systems trained inadequately.
Improving Contrastive Learning of Sentence Embeddings from AI Feedback
Our method utilizes AI feedback from large pre-trained language models (LLMs) to construct sample pairs with fine-grained sample similarity scores to improve contrastive learning.
Origin Tracing and Detecting of LLMs
The extraordinary performance of large language models (LLMs) heightens the importance of detecting whether the context is generated by an AI system.
Mitigating Negative Style Transfer in Hybrid Dialogue System
As the functionality of dialogue systems evolves, hybrid dialogue systems that accomplish user-specific goals and participate in open-topic chitchat with users are attracting growing attention.
Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator
Besides, we introduce a sentence-level and a session-level score to measure the sentence fluency and session coherence in the interactive evaluation.
Text Adversarial Purification as Defense against Adversarial Attacks
Adversarial purification is a successful defense mechanism against adversarial attacks without requiring knowledge of the form of the incoming attack.
MarkBERT: Marking Word Boundaries Improves Chinese BERT
We present a Chinese BERT model dubbed MarkBERT that uses word information in this work.
Chinese Named Entity Recognition
named-entity-recognition
+7
"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
The meaning of a word in Chinese is different in that a word is a compositional unit consisting of multiple characters.
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier
Pre-trained models are widely used in fine-tuning downstream tasks with linear classifiers optimized by the cross-entropy loss, which might face robustness and stability problems.
Template-free Prompt Tuning for Few-shot NER
Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words.
Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning
\textbf{P}re-\textbf{T}rained \textbf{M}odel\textbf{s} have been widely applied and recently proved vulnerable under backdoor attacks: the released pre-trained weights can be maliciously poisoned with certain triggers.
Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution
Recent studies have shown that deep neural networks are vulnerable to intentionally crafted adversarial examples, and various methods have been proposed to defend against adversarial word-substitution attacks for neural NLP models.
SENT: Sentence-level Distant Relation Extraction via Negative Training
In this work, we propose the use of negative training (NT), in which a model is trained using complementary labels regarding that ``the instance does not belong to these complementary labels".
Certified Robustness to Text Adversarial Attacks by Randomized [MASK]
Recently, few certified defense methods have been developed to provably guarantee the robustness of a text classifier to adversarial synonym substitutions.
COOKIE: Contrastive Cross-Modal Knowledge Sharing Pre-Training for Vision-Language Representation
There are two key designs in it, one is the weight-sharing transformer on top of the visual and textual encoders to align text and image semantically, the other is three kinds of contrastive learning designed for sharing knowledge between different modalities.
Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces
The substitutions in the generated adversarial examples are not characters or words but \textit{'pieces'}, which are more natural to Chinese readers.
TAVAT: Token-Aware Virtual Adversarial Training for Language Understanding
Gradient-based adversarial training is widely used in improving the robustness of neural networks, while it cannot be easily adapted to natural language processing tasks since the embedding space is discrete.
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Adversarial attacks for discrete data (such as texts) have been proved significantly more challenging than continuous data (such as images) since it is difficult to generate adversarial samples with gradient-based methods.
U-Net: Machine Reading Comprehension with Unanswerable Questions
A key subtask is to reliably predict whether the question is unanswerable.
Ranked #12 on
Question Answering
on SQuAD2.0 dev
Quantum anomalous Hall effect in stable 1T-YN$_2$ monolayer with a large nontrivial band gap and high Chern number
The quantum anomalous Hall (QAH) effect is a topologically nontrivial phase, characterized by a non-zero Chern number defined in the bulk and chiral edge states in the boundary.
Materials Science
New Group V Elemental Bilayers: A Tunable Structure Model with 4,6,8-atom Rings
Using first-principles calculations, we propose a series of new elemental bilayers with group V elements (Bi, Sb, As).
Materials Science
