Search Results for author:
Found 140 papers, 85 papers with code
Tool Learning with Foundation Models
Considering the lack of a systematic tool learning evaluation in prior works, we experiment with 18 representative tools and show the potential of current foundation models in skillfully utilizing tools.
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction
OpenNRE is an open-source and extensible toolkit that provides a unified framework to implement neural models for relation extraction (RE).
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction
Distantly supervised (DS) relation extraction (RE) has attracted much attention in the past few years as it can utilize large-scale auto-labeled data.
OpenKE: An Open Toolkit for Knowledge Embedding
We release an open toolkit for knowledge embedding (OpenKE), which provides a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space.
Knowledge Representation Learning: A Quantitative Review
Knowledge representation learning (KRL) aims to represent entities and relations in knowledge graph in low-dimensional semantic space, which have been widely used in massive knowledge-driven tasks.
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
For data scaling, we introduce a Warmup-Stable-Decay (WSD) learning rate scheduler (LRS), conducive to continuous training and domain adaptation.
CPM: A Large-scale Generative Chinese Pre-trained Language Model
However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available.
ERNIE: Enhanced Language Representation with Informative Entities
Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks.
Ranked #1 on
Entity Linking
on FIGER
Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages
Building a competitive counterpart in other languages is highly challenging due to the low-resource nature of non-English multimodal data (i. e., lack of large-scale, high-quality image-text data).
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
We recruit annotators to search for relevant information using our interface and then answer questions.
FewRel 2.0: Towards More Challenging Few-Shot Relation Classification
We present FewRel 2. 0, a more challenging task to investigate two aspects of few-shot relation classification models: (1) Can they adapt to a new domain with only a handful of instances?
DocRED: A Large-Scale Document-Level Relation Extraction Dataset
Multiple entities in a document generally exhibit complex inter-sentence relations, and cannot be well handled by existing relation extraction (RE) methods that typically focus on extracting intra-sentence relations for single entity pairs.
Ranked #59 on
Relation Extraction
on DocRED
OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication
We then construct a comprehensive benchmark with a total of 16 implemented models to evaluate several information fusion strategies~(i. e. early, late, and intermediate fusion) with state-of-the-art LiDAR detection algorithms.
Ranked #2 on
3D Object Detection
on OPV2V
CPM-2: Large-scale Cost-effective Pre-trained Language Models
We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference.
BMInf: An Efficient Toolkit for Big Model Inference and Tuning
In recent years, large-scale pre-trained language models (PLMs) containing billions of parameters have achieved promising results on various NLP tasks.
The Liver Tumor Segmentation Benchmark (LiTS)
In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018.
Few-NERD: A Few-Shot Named Entity Recognition Dataset
In this paper, we present Few-NERD, a large-scale human-annotated few-shot NER dataset with a hierarchy of 8 coarse-grained and 66 fine-grained entity types.
Ranked #5 on
Named Entity Recognition (NER)
on Few-NERD (SUP)
Networks for Joint Affine and Non-parametric Image Registration
In contrast to existing approaches, our framework combines two registration methods: an affine registration and a vector momentum-parameterized stationary velocity field (vSVF) model.
Ranked #2 on
Image Registration
on Osteoarthritis Initiative
$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
Processing and reasoning over long contexts is crucial for many practical applications of Large Language Models (LLMs), such as document comprehension and agent construction.
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification
Fact verification (FV) is a challenging task which requires to retrieve relevant evidence from plain text and use the evidence to verify given claims.
Ranked #7 on
Fact Verification
on FEVER
PTR: Prompt Tuning with Rules for Text Classification
This indicates that PTR is a promising approach to take advantage of both human prior knowledge and PLMs for those complicated classification tasks.
MAVEN: A Massive General Domain Event Detection Dataset
Most existing datasets exhibit the following issues that limit further development of ED: (1) Data scarcity.
Sampling-based Fast Gradient Rescaling Method for Highly Transferable Adversarial Attacks
Deviation between the original gradient and the generated noises may lead to inaccurate gradient update estimation and suboptimal solutions for adversarial transferability, which is crucial for black-box attacks.
Sampling-based Fast Gradient Rescaling Method for Highly Transferable Adversarial Attacks
In either case, the common gradient-based methods generally use the sign function to generate perturbations on the gradient update, that offers a roughly correct direction and has gained great success.
Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations
Multi-hop knowledge graph (KG) reasoning is an effective and explainable method for predicting the target entity via reasoning paths in query answering (QA) task.
Ranked #3 on
Link Prediction
on NELL-995
Neural Snowball for Few-Shot Relation Learning
To address new relations with few-shot instances, we propose a novel bootstrapping approach, Neural Snowball, to learn new relations by transferring semantic knowledge about existing relations.
UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs
The rapid development of LLMs calls for a lightweight and easy-to-use framework for swift evaluation deployment.
PPT: Pre-trained Prompt Tuning for Few-shot Learning
To ensure the generalization of PPT, we formulate similar classification tasks into a unified task form and pre-train soft prompts for this unified task.
Learning from Context or Names? An Empirical Study on Neural Relation Extraction
We find that (i) while context is the main source to support the predictions, RE models also heavily rely on the information from entity mentions, most of which is type information, and (ii) existing datasets may leak shallow heuristics via entity mentions and thus contribute to the high performance on RE benchmarks.
Ranked #23 on
Relation Extraction
on TACRED
Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention
In this paper, we aim to incorporate the hierarchical information of relations for distantly supervised relation extraction and propose a novel hierarchical attention scheme.
HMEAE: Hierarchical Modular Event Argument Extraction
Existing event extraction methods classify each argument role independently, ignoring the conceptual correlations between different argument roles.
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation
The relation of each sentence is first recognized by distant supervision methods, and then filtered by crowdworkers.
CLEVE: Contrastive Pre-training for Event Extraction
Event extraction (EE) has considerably benefited from pre-trained language models (PLMs) by fine-tuning.
Adversarial Training for Weakly Supervised Event Detection
Modern weakly supervised methods for event detection (ED) avoid time-consuming human annotation and achieve promising results by learning from auto-labeled data.
Fully Hyperbolic Neural Networks
Hyperbolic neural networks have shown great potential for modeling complex data.
MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction
It contains 103, 193 event coreference chains, 1, 216, 217 temporal relations, 57, 992 causal relations, and 15, 841 subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude.
DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction
To demonstrate the effectiveness of DIAG-NRE, we apply it to two real-world datasets and present both significant and interpretable improvements over state-of-the-art methods.
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Experimental results on three knowledge-driven NLP tasks show that existing injection methods are not suitable for the new paradigm, while map-tuning effectively improves the performance of downstream models.
Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph
On the one hand, sparse KGs contain less information, which makes it difficult for the model to choose correct paths.
Ouroboros: Speculative Decoding with Large Model Enhanced Drafting
In this paper, we introduce Ouroboros, which constructs a phrase candidate pool from the verification process of LLMs to provide candidates for draft generation of the small model.
Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data
Open relation extraction (OpenRE) aims to extract relational facts from the open-domain corpus.
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
Notably, the best-performing model, GPT-4V, attains an average score of 17. 23% on OlympiadBench, with a mere 11. 28% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.
Quantifying Similarity between Relations with Fact Distribution
We introduce a conceptually simple and effective method to quantify the similarity between relations in knowledge bases.
GAN Ensemble for Anomaly Detection
Motivated by the observation that GAN ensembles often outperform single GANs in generation tasks, we propose to construct GAN ensembles for anomaly detection.
FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling
To address this gap and promote the development of microscopic traffic flow modeling, we establish a public benchmark dataset for car-following behavior modeling.
Knowledge Inheritance for Pre-trained Language Models
Specifically, we introduce a pre-training framework named "knowledge inheritance" (KI) and explore how could knowledge distillation serve as auxiliary supervision during pre-training to efficiently learn larger PLMs.
Denoising Relation Extraction from Document-level Distant Supervision
Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance.
Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling
The empirical study shows that EDGE is much more efficient than competing methods and can generate large graphs with thousands of nodes.
GACT: Activation Compressed Training for Generic Network Architectures
Training large neural network (NN) models requires extensive memory resources, and Activation Compressed Training (ACT) is a promising approach to reduce training memory footprint.
CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models
In this paper, we propose a novel framework named Coke to dynamically select contextual knowledge and embed knowledge context according to textual context for PLMs, which can avoid the effect of redundant and ambiguous knowledge in KGs that cannot match the input text.
MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization
Scientific data visualization plays a crucial role in research by enabling the direct display of complex information and assisting researchers in identifying implicit patterns.
Visual Distant Supervision for Scene Graph Generation
In this work, we propose visual distant supervision, a novel paradigm of visual relation learning, which can train scene graph models without any human-labeled data.
Order Matters: Probabilistic Modeling of Node Sequence for Graph Generation
A graph generative model defines a distribution over graphs.
Plug-and-Play Document Modules for Pre-trained Models
By inserting document plugins into the backbone PTM for downstream tasks, we can encode a document one time to handle multiple tasks, which is more efficient than conventional encoding-task coupling methods that simultaneously encode documents and input queries using task-specific encoders.
Adversarial Multi-lingual Neural Relation Extraction
To address these issues, we propose an adversarial multi-lingual neural relation extraction (AMNRE) model, which builds both consistent and individual representations for each sentence to consider the consistency and diversity among languages.
Open Hierarchical Relation Extraction
To establish the bidirectional connections between OpenRE and relation hierarchy, we propose the task of open hierarchical relation extraction and present a novel OHRE framework for the task.
Put It Back: Entity Typing with Language Model Enhancement
Entity typing aims to classify semantic types of an entity mention in a specific context.
Local Temperature Scaling for Probability Calibration
Thus, we propose a learning-based calibration method that focuses on multi-label semantic segmentation.
Decoupled Low-light Image Enhancement
On the other hand, since the parameter matrix learned from the first stage is aware of the lightness distribution and the scene structure, it can be incorporated into the second stage as the complementary information.
Brain Extraction from Normal and Pathological Images: A Joint PCA/Image-Reconstruction Approach
Extracting the brain from images with strong pathologies, for example, the presence of a tumor or of a traumatic brain injury, is challenging.
ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
Some recent efforts have explored introducing ReLU or its variants as the substitutive activation function to help LLMs achieve activation sparsity and inference acceleration, but few can simultaneously obtain high sparsity and comparable model performance.
Emergent Modularity in Pre-trained Transformers
In analogy to human brains, we consider two main characteristics of modularity: (1) functional specialization of neurons: we evaluate whether each neuron is mainly specialized in a certain function, and find that the answer is yes.
"Guinea Pig Trials" Utilizing GPT: A Novel Smart Agent-Based Modeling Approach for Studying Firm Competition and Collusion
Our results demonstrate that, in the absence of communication, smart agents consistently reach tacit collusion, leading to prices converging at levels higher than the Bertrand equilibrium price but lower than monopoly or cartel prices.
Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulations
ABM's strength lies in its bottom-up methodology, illuminating emergent phenomena by modeling the behaviors of individual components of a system.
MatSAM: Efficient Extraction of Microstructures of Materials via Visual Large Model
Efficient and accurate extraction of microstructures in micrographs of materials is essential in process optimization and the exploration of structure-property relationships.
CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models
We then perform contrastive semi-supervised learning on both the retrieved unlabeled and original labeled instances to help PLMs capture crucial task-related semantic features.
UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset
Different from previous works that simply translate English instructions, we consider both the language-specific and language-agnostic abilities of LLMs.
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning
To fathom the mystery, we hypothesize that the adaptations of different DETs could all be reparameterized as low-dimensional optimizations in a unified optimization subspace, which could be found by jointly decomposing independent solutions of different DETs.
Meta-Information Guided Meta-Learning for Few-Shot Relation Classification
Few-shot classification requires classifiers to adapt to new classes with only a few training instances.
Towards Accurate Subgraph Similarity Computation via Neural Graph Pruning
In this work, we convert graph pruning to a problem of node relabeling and then relax it to a differentiable problem.
Recyclable Tuning for Continual Pre-training
In pilot studies, we find that after continual pre-training, the upgraded PLM remains compatible with the outdated adapted weights to some extent.
ConPET: Continual Parameter-Efficient Tuning for Large Language Models
First, Static ConPET can adapt former continual learning methods originally designed for relatively smaller models to LLMs through PET and a dynamic replay strategy, which largely reduces the tuning costs and alleviates the over-fitting and forgetting issue.
BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
Effective attention modules have played a crucial role in the success of Transformer-based large language models (LLMs), but the quadratic time and memory complexities of these attention modules also pose a challenge when processing long sequences.
Exploring Mode Connectivity for Pre-trained Language Models
(3) How does the PLM's task knowledge change along the path connecting two minima?
Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
Pre-trained language models (PLMs) have achieved remarkable results on NLP tasks but at the expense of huge parameter sizes and the consequent computational costs.
Robust and Scalable Model Editing for Large Language Models
Large language models (LLMs) can make predictions using parametric knowledge--knowledge encoded in the model weights--or contextual knowledge--knowledge presented in the context.
Collecting Verified COVID-19 Question Answer Pairs
We release a dataset of over 2, 100 COVID19 related Frequently asked Question-Answer pairs scraped from over 40 trusted websites.
Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages
In this paper, we propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages.
MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation
Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-event relationships.
Shall We Talk: Exploring Spontaneous Collaborations of Competing LLM Agents
Recent advancements have shown that agents powered by large language models (LLMs) possess capabilities to simulate human behaviors and societal dynamics.
PatchGT: Transformer over Non-trainable Clusters for Learning Graph Representations
Recently the Transformer structure has shown good performances in graph learning tasks.
Seed-Point Detection of Clumped Convex Objects by Short-Range Attractive Long-Range Repulsive Particle Clustering
Locating the center of convex objects is important in both image processing and unsupervised machine learning/data clustering fields.
Neural Gibbs Sampling for Joint Event Argument Extraction
Existing EAE methods either extract each event argument roles independently or sequentially, which cannot adequately model the joint probability distribution among event arguments and their roles.
Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task
End-to-end text image translation (TIT), which aims at translating the source language embedded in images to the target language, has attracted intensive attention in recent research.
Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning
Since it is non-trivial to directly model the intermediate states and design a running cost function, we propose to use latent stochastic bridges to regularize the intermediate states and use the regularization as the running cost of PETs.
Denoising Distant Supervision for Relation Extraction via Instance-Level Adversarial Training
As shown in the experiments on a large-scale benchmark dataset in relation extraction, our denoising method can effectively filter out noisy instances and achieve significant improvements as compared with the state-of-the-art models.
Efficient Registration of Pathological Images: A Joint PCA/Image-Reconstruction Approach
Registration involving one or more images containing pathologies is challenging, as standard image similarity measures and spatial transforms cannot account for common changes due to pathologies.
Joint Representation Learning of Text and Knowledge for Knowledge Graph Completion
Joint representation learning of text and knowledge within a unified semantic space enables us to perform knowledge graph completion more accurately.
Performance evaluation of wavelet scattering network in image texture classification in various color spaces
Texture plays an important role in many image analysis applications.
Sentence Segmentation for Classical Chinese Based on LSTM with Radical Embedding
In this paper, we develop a low than character feature embedding called radical embedding, and apply it on LSTM model for sentence segmentation of pre modern Chinese texts.
Robust low-rank multilinear tensor approximation for a joint estimation of the multilinear rank and the loading matrices
Based on the low-rank property and an over-estimation of the core tensor, this joint estimation problem is solved by promoting (group) sparsity of the over-estimated core tensor.
VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas Segmentation
Experiments on 3D brain MRI data show that by selecting a good initial atlas set MAS with VoteNet significantly outperforms a number of other label fusion strategies as well as a direct DL segmentation approach.
Efficient training and design of photonic neural network through neuroevolution
Recently, optical neural networks (ONNs) integrated in photonic chips has received extensive attention because they are expected to implement the same pattern recognition tasks in the electronic platforms with high efficiency and low power consumption.
VoteNet+ : An Improved Deep Learning Label Fusion Method for Multi-atlas Segmentation
Specifically, we first illustrate that using a deep convolutional neural network to predict atlas probabilities can better distinguish correct atlas labels from incorrect ones than relying on image intensity difference as is typical in JLF.
Adversarial Language Games for Advanced Natural Language Intelligence
In this work, we propose a challenging adversarial language game called Adversarial Taboo as an example, in which an attacker and a defender compete around a target word.
Unimodal-uniform Constrained Wasserstein Training for Medical Diagnosis
In this paper, we target on this task from the perspective of loss function.
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction
Relational facts are an important component of human knowledge, which are hidden in vast amounts of text.
A Perspective on Deep Learning for Molecular Modeling and Simulations
Deep learning is transforming many areas in science, and it has great potential in modeling molecular systems.
Continual Relation Learning via Episodic Memory Activation and Reconsolidation
Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations.
A Deep Network for Joint Registration and Reconstruction of Images with Pathologies
They are therefore not designed for the registration of images with strong pathologies for example in the context of brain tumors, and traumatic brain injuries.
Mutual Information Regularized Identity-aware Facial ExpressionRecognition in Compressed Video
In the up to two orders of magnitude compressed domain, we can explicitly infer the expression from the residual frames and possibly extract identity factors from the I frame with a pre-trained face recognition network.
Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training
However, the cross entropy loss can not take the different importance of each class in an self-driving system into account.
Deep Reinforcement Learning of Transition States
Combining reinforcement learning (RL) and molecular dynamics (MD) simulations, we propose a machine-learning approach (RL$^\ddag$) to automatically unravel chemical reaction mechanisms.
Identity-aware Facial Expression Recognition in Compressed Video
In the up to two orders of magnitude compressed domain, we can explicitly infer the expression from the residual frames and possible to extract identity factors from the I frame with a pre-trained face recognition network.
Quasi-maximum likelihood estimation of break point in high-dimensional factor models
This paper estimates the break point for large-dimensional factor models with a single structural break in factor loadings at a common unknown date.
Pre-Trained Models: Past, Present and Future
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI).
IsOBS: An Information System for Oracle Bone Script
Oracle bone script (OBS) is the earliest known ancient Chinese writing system and the ancestor of modern Chinese.
Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals
In contrast to the state-of-the-art methods, the proposed methods solve the estimation and testing problem at once with several merits: 1) an accurate sparsity estimation; 2) point estimates with shrinkage/soft-thresholding property; 3) credible intervals for uncertainty quantification; 4) an optimal multiple testing procedure that controls false discovery rate.
Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables
Strong correlations between explanatory variables are problematic for high-dimensional regularized regression methods.
Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference
In this work, we propose a domain generalization (DG) approach to learn on several labeled source domains and transfer knowledge to a target domain that is inaccessible in training.
Prompt-Learning for Fine-Grained Entity Typing
In this work, we investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot and zero-shot scenarios.
Predicting Physics in Mesh-reduced Space with Temporal Attention
Graph-based next-step prediction models have recently been very successful in modeling complex high-dimensional physical systems on irregular meshes.
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
Quantum Kerr Learning
Quantum machine learning is a rapidly evolving field of research that could facilitate important applications for quantum computing and also significantly impact data-driven sciences.
Likelihood ratio test for structural changes in factor models
This paper considers the likelihood ratio (LR) test for a variance change in the estimated factors.
Meta Reinforcement Learning with Successor Feature Based Context
Most reinforcement learning (RL) methods only focus on learning a single task from scratch and are not able to use prior knowledge to learn other tasks more effectively.
Deep Learning for Iris Recognition: A Review
This paper collects 120 relevant papers to summarize the development of iris recognition based on deep learning.
A Majorization-Minimization Gauss-Newton Method for 1-Bit Matrix Completion
In 1-bit matrix completion, the aim is to estimate an underlying low-rank matrix from a partial set of binary observations.
DualGenerator: Information Interaction-based Generative Network for Point Cloud Completion
To tackle these challenges, we propose an information interaction-based generative network for point cloud completion ($\mathbf{DualGenerator}$).
Efficient Cross-Lingual Transfer for Chinese Stable Diffusion with Images as Pivots
IAP optimizes only a separate Chinese text encoder with all other parameters fixed to align Chinese semantics space to the English one in CLIP.
PersonaPKT: Building Personalized Dialogue Agents via Parameter-efficient Knowledge Transfer
Personalized dialogue agents (DAs) powered by large pre-trained language models (PLMs) often rely on explicit persona descriptions to maintain personality consistency.
CPET: Effective Parameter-Efficient Tuning for Compressed Large Language Models
Parameter-efficient tuning (PET) has been widely explored in recent years because it tunes much fewer parameters (PET modules) than full-parameter fine-tuning (FT) while still stimulating sufficient knowledge from large language models (LLMs) for downstream tasks.
EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning
The experimental results illustrate that EnsembleFollower yields improved accuracy of human-like behavior and achieves effectiveness in combining hybrid models, demonstrating that our proposed framework can handle diverse car-following conditions by leveraging the strengths of various low-level models.
QASnowball: An Iterative Bootstrapping Framework for High-Quality Question-Answering Data Generation
Recent years have witnessed the success of question answering (QA), especially its potential to be a foundation paradigm for tackling diverse NLP tasks.
Predicting Emergent Abilities with Infinite Resolution Evaluation
With PassUntil, we conduct a quantitative investigation into the scaling law of task performance.
ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learning
Long-term time series forecasting (LTSF) is important for various domains but is confronted by challenges in handling the complex temporal-contextual relationships.
Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models
Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments, enabling substantial reductions in model storage and memory costs without significant performance compromise.
Bayesian Conditional Diffusion Models for Versatile Spatiotemporal Turbulence Generation
A notable feature of our approach is the method proposed for long-span flow sequence generation, which is based on autoregressive gradient-based conditional sampling, eliminating the need for cumbersome retraining processes.
Traffic Sign Interpretation in Real Road Scene
Following the above issues, we propose a traffic sign interpretation (TSI) task, which aims to interpret global semantic interrelated traffic signs (e. g.,~driving instruction-related texts, symbols, and guide panels) into a natural language for providing accurate instruction support to autonomous or assistant driving.
Generating High-Precision Force Fields for Molecular Dynamics Simulations to Study Chemical Reaction Mechanisms using Molecular Configuration Transformer
We applied this approach to study a Claisen rearrangement reaction and a Carbonyl insertion reaction catalyzed by Manganese.
ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
To find the most efficient activation function for sparse computation, we propose a systematic framework to examine the sparsity of LLMs from three aspects: the trade-off between sparsity and performance, the predictivity of sparsity, and the hardware affinity.
InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory
To alleviate these issues, existing efforts employ sliding attention windows and discard distant tokens to achieve the processing of extremely long sequences.
LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
Most prior works on LoRA combination primarily rely on task-level weights for each involved LoRA, making different examples and tokens share the same LoRA weights.
OneBit: Towards Extremely Low-bit Large Language Models
Model quantification uses low bit-width values to represent the weight matrices of models, which is a promising approach to reduce both storage and computational overheads of deploying highly anticipated LLMs.
Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition
Recent studies have uncovered intriguing phenomena in deep learning, such as grokking, double descent, and emergent abilities in large language models, which challenge human intuition and are crucial for a deeper understanding of neural models.
CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables
For Multivariate Time Series Forecasting (MTSF), recent deep learning applications show that univariate models frequently outperform multivariate ones.
V2X-Real: a Largs-Scale Dataset for Vehicle-to-Everything Cooperative Perception
Recent advancements in Vehicle-to-Everything (V2X) technologies have enabled autonomous vehicles to share sensing information to see through occlusions, greatly boosting the perception capability.
