Search Results for author:
Found 63 papers, 29 papers with code
Advancing Prompt-Based Methods for Replay-Independent General Continual Learning
It includes a forgetting-aware initial session adaption that employs pretraining data to initialize prompt parameters and improve generalizability, as well as a non-parametric logit mask of the output layers to mitigate catastrophic forgetting.
WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale
Large language models (LLMs) with extended context windows enable tasks requiring extensive information integration but are limited by the scarcity of high-quality, diverse datasets for long-context instruction tuning.
Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective
Large Language Models (LLMs) have made notable progress in mathematical reasoning, yet they often rely on single-paradigm reasoning that limits their effectiveness across diverse tasks.
Bootstrap Your Own Context Length
We introduce a bootstrapping approach to train long-context language models by exploiting their short-context capabilities only.
RedStone: Curating General, Code, Math, and QA Data for Large Language Models
This study explores the untapped potential of Common Crawl as a comprehensive and flexible resource for pre-training LLMs, addressing both general-purpose language understanding and specialized domain knowledge.
ManiBox: Enhancing Spatial Grasping Generalization via Scalable Simulation Data Generation
To overcome these challenges, we then focus on state-based policy generalization and present \textbf{ManiBox}, a novel bounding-box-guided manipulation method built on a simulation-based teacher-student framework.
Self-Boosting Large Language Models with Synthetic Preference Data
Through alignment with human preferences, Large Language Models (LLMs) have advanced significantly in generating honest, harmless, and helpful responses.
HiDe-PET: Continual Learning via Hierarchical Decomposition of Parameter-Efficient Tuning
The deployment of pre-trained models (PTMs) has greatly advanced the field of continual learning (CL), enabling positive knowledge transfer and resilience to catastrophic forgetting.
Fourier Controller Networks for Real-Time Decision-Making in Embodied Learning
In this paper, we propose to investigate the task from a new perspective of the frequency domain.
PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning
Designing generalizable agents capable of adapting to diverse embodiments has achieved significant attention in Reinforcement Learning (RL), which is critical for deploying RL agents in various real-world applications.
xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
This paper introduces xRAG, an innovative context compression method tailored for retrieval-augmented generation.
CoFInAl: Enhancing Action Quality Assessment with Coarse-to-Fine Instruction Alignment
However, this common strategy yields suboptimal results due to the inherent struggle of these backbones to capture the subtle cues essential for AQA.
MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessment
We propose Continual AQA (CAQA) to refine models using sparse new data.
MathScale: Scaling Instruction Tuning for Mathematical Reasoning
Inspired by the cognitive mechanism in human mathematical learning, it first extracts topics and knowledge points from seed math questions and then build a concept graph, which is subsequently used to generate new math questions.
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
We introduce Generalized Instruction Tuning (called GLAN), a general and scalable method for instruction tuning of Large Language Models (LLMs).
DualTeacher: Bridging Coexistence of Unlabelled Classes for Semi-supervised Incremental Object Detection
A commonly-used strategy for supervised IOD is to encourage the current model (as a student) to mimic the behavior of the old model (as a teacher), but it generally fails in SSIOD because a dominant number of object instances from old and new classes are coexisting and unlabelled, with the teacher only recognizing a fraction of them.
Unleashing the potential of GNNs via Bi-directional Knowledge Transfer
On this basis, BiKT not only allows us to acquire knowledge from both the GNN and its derived model but promotes each other by injecting the knowledge into the other.
Towards a General Framework for Continual Learning with Pre-training
In this work, we present a general framework for continual learning of sequentially arrived tasks with the use of pre-training, which has emerged as a promising direction for artificial intelligence systems to accommodate real-world dynamics.
Tuna: Instruction Tuning using Feedback from Large Language Models
Furthermore, we apply probabilistic ranking and contextual ranking sequentially to the instruction-tuned LLM.
Overcoming Recency Bias of Normalization Statistics in Continual Learning: Balance and Adaptation
Continual learning entails learning a sequence of tasks and balancing their knowledge appropriately.
Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality
Following these empirical and theoretical insights, we propose Hierarchical Decomposition (HiDe-)Prompt, an innovative approach that explicitly optimizes the hierarchical components with an ensemble of task-specific prompts and statistics of both uninstructed and instructed representations, further with the coordination of a contrastive regularization strategy.
Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence
Continual learning aims to empower artificial intelligence (AI) with strong adaptability to the real world.
LongNet: Scaling Transformers to 1,000,000,000 Tokens
Scaling sequence length has become a critical demand in the era of large language models.
A Comprehensive Survey of Continual Learning: Theory, Method and Application
To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime.
Momentum Calibration for Text Generation
The input and output of most text generation tasks can be transformed to two sequences of tokens and they can be modeled using sequence-to-sequence learning modeling tools such as Transformers.
Ranked #3 on
Text Summarization
on SAMSum
Latent Prompt Tuning for Text Summarization
In this paper, we propose Lotus (shorthand for Latent Prompt Tuning for Summarization), which is a single model that can be applied in both controlled and uncontrolled (without control signals) modes.
HVS-Inspired Signal Degradation Network for Just Noticeable Difference Estimation
However, they have a major drawback that the generated JND is assessed in the real-world signal domain instead of in the perceptual domain in the human brain.
CoSCL: Cooperation of Small Continual Learners is Stronger than a Big One
Continual learning requires incremental compatibility with a sequence of tasks.
Diagnosing Ensemble Few-Shot Classifiers
The base learners and labeled samples (shots) in an ensemble few-shot classifier greatly affect the model performance.
Neural Label Search for Zero-Shot Multi-Lingual Extractive Summarization
In zero-shot multilingual extractive text summarization, a model is typically trained on English summarization dataset and then applied on summarization datasets of other languages.
Memory Replay with Data Compression for Continual Learning
In this work, we propose memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer.
Unsupervised Multi-Granularity Summarization
In this paper, we propose the first unsupervised multi-granularity summarization framework, GranuSum.
Auto-Weighted Layer Representation Based View Synthesis Distortion Estimation for 3-D Video Coding
Experimental results show that the VSD can be accurately estimated with the weights learnt by the nonlinear mapping function once its associated S-VSDs are available.
Sequence Level Contrastive Learning for Text Summarization
In this paper, we propose a contrastive learning model for supervised abstractive text summarization, where we view a document, its gold summary and its model generated summaries as different views of the same mean representation and maximize the similarities between them during training.
Double Low-Rank Representation With Projection Distance Penalty for Clustering
This paper presents a novel, simple yet robust self-representation method, i. e., Double Low-Rank Representation with Projection Distance penalty (DLRRPD) for clustering.
Attention Temperature Matters in Abstractive Summarization Distillation
In this paper, we find simply manipulating attention temperatures in Transformers can make pseudo labels easier to learn for student models.
Auto-weighted low-rank representation for clustering
In this paper, a novel unsupervised low-rank representation model, i. e., Auto-weighted Low-Rank Representation (ALRR), is proposed to construct a more favorable similarity graph (SG) for clustering.
Just Noticeable Difference for Deep Machine Vision
Experimental results on image classification demonstrate that we successfully find the JND for deep machine vision.
Unsupervised Fine-tuning for Text Clustering
Fine-tuning with pre-trained language models (e. g. BERT) has achieved great success in many language understanding tasks in supervised settings (e. g. text classification).
Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
We also find in experiments that our model is less dependent on sentence positions.
Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
We propose a novel language-independent approach to improve the efficiency for Grammatical Error Correction (GEC) by dividing the task into two subtasks: Erroneous Span Detection (ESD) and Erroneous Span Correction (ESC).
Taking Modality-free Human Identification as Zero-shot Learning
There have been numerous methods proposed for human identification, such as face identification, person re-identification, and gait identification.
Pre-training for Abstractive Document Summarization by Reinstating Source Text
The main idea is that, given an input text artificially constructed from a document, a model is pre-trained to reinstate the original document.
From Anchor Generation to Distribution Alignment: Learning a Discriminative Embedding Space for Zero-Shot Recognition
In zero-shot learning (ZSL), the samples to be classified are usually projected into side information templates such as attributes.
To See in the Dark: N2DGAN for Background Modeling in Nighttime Scene
Due to the deteriorated conditions of \mbox{illumination} lack and uneven lighting, nighttime images have lower contrast and higher noise than their daytime counterparts of the same scene, which limits seriously the performances of conventional background modeling methods.
Distribution-induced Bidirectional Generative Adversarial Network for Graph Representation Learning
Graph representation learning aims to encode all nodes of a graph into low-dimensional vectors that will serve as input of many compute vision tasks.
Generative Adversarial Network
Graph Representation Learning
Understand Dynamic Regret with Switching Cost for Online Decision Making
We provide a new theoretical analysis framework, which shows an interesting observation, that is, the relation between the switching cost and the dynamic regret is different for settings of OA and OCO.
DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue
More importantly, we can tell which modality (visual or semantic) has more contribution in answering the current question by visualizing the gate values.
Ranked #6 on
Visual Dialog
on VisDial v0.9 val
Defensive Few-shot Learning
(2) how to narrow the distribution gap between clean and adversarial examples under the few-shot setting?
ProLFA: Representative Prototype Selection for Local Feature Aggregation
Given a set of hand-crafted local features, acquiring a global representation via aggregation is a promising technique to boost computational efficiency and improve task performance.
Hierarchical Prototype Learning for Zero-Shot Recognition
Specifically, HPL is able to obtain discriminability on both seen and unseen class domains by learning visual prototypes respectively under the transductive setting.
ATZSL: Defensive Zero-Shot Recognition in the Presence of Adversaries
In this paper, we take an initial attempt, and propose a generic formulation to provide a systematical solution (named ATZSL) for learning a robust ZSL model.
Convolutional Prototype Learning for Zero-Shot Recognition
The key to ZSL is to transfer knowledge from the seen to the unseen classes via auxiliary class attribute vectors.
Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study
Sequence-to-sequence (seq2seq) models have achieved tremendous success in text generation tasks.
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
Neural extractive summarization models usually employ a hierarchical encoder for document encoding and they are trained using sentence-level labels, which are created heuristically using rule-based methods.
Ranked #7 on
Extractive Text Summarization
on CNN / Daily Mail
Neural Latent Extractive Document Summarization
Extractive summarization models require sentence-level labels, which are usually created heuristically (e. g., with rule-based methods) given that most summarization datasets only have document-summary pairs.
Ranked #11 on
Extractive Text Summarization
on CNN / Daily Mail
UParse: the Edinburgh system for the CoNLL 2017 UD shared task
This paper presents our submissions for the CoNLL 2017 UD Shared Task.
Sentence Simplification with Deep Reinforcement Learning
Sentence simplification aims to make sentences easier to read and understand.
Ranked #3 on
Text Simplification
on ASSET
Dependency Parsing as Head Selection
Conventional graph-based dependency parsers guarantee a tree structure both during training and inference.
Top-down Tree Long Short-Term Memory Networks
Long Short-Term Memory (LSTM) networks, a type of recurrent neural network with a more complex computational unit, have been successfully applied to a variety of sequence modeling tasks.
