Search Results for author:
Found 190 papers, 118 papers with code
BMInf: An Efficient Toolkit for Big Model Inference and Tuning
In recent years, large-scale pre-trained language models (PLMs) containing billions of parameters have achieved promising results on various NLP tasks.
Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages
In this paper, we propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages.
Collecting Verified COVID-19 Question Answer Pairs
We release a dataset of over 2, 100 COVID19 related Frequently asked Question-Answer pairs scraped from over 40 trusted websites.
HarmonySeg: Tubular Structure Segmentation with Deep-Shallow Feature Fusion and Growth-Suppression Balanced Loss
Finally, we introduce a topology-preserving loss function that leverages contextual and shape priors to balance the growth and suppression of tubular structures, which also allows the model to handle low-quality and incomplete annotations.
Improving Harmful Text Detection with Joint Retrieval and External Knowledge
Harmful text detection has become a crucial task in the development and deployment of large language models, especially as AI-generated content continues to expand across digital platforms.
MoST: Efficient Monarch Sparse Tuning for 3D Representation Learning
We introduce Monarch Sparse Tuning (MoST), the first reparameterization-based parameter-efficient fine-tuning (PEFT) method tailored for 3D representation learning.
Cost-Optimal Grouped-Query Attention for Long-Context LLMs
However, they overlook the impacts of context length and attention head configuration (the number of query and key-value heads in grouped-query attention) on training and inference.
Singularity-Based Consistent QML Estimation of Multiple Breakpoints in High-Dimensional Factor Models
Under a rotational change, unlike in the single-breakpoint case, the pseudo factor covariance matrix within each regime can be either full rank or singular, yet the QML estimation error for the breakpoints remains stably bounded.
TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators
Despite advances in large language models (LLMs) for conventional code generation, these models struggle to generate accurate, performance-optimized Triton code, as they lack awareness of its specifications and the complexities of GPU programming.
FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling
Speculative sampling has emerged as an important technique for accelerating the auto-regressive generation process of large language models (LLMs) by utilizing a draft-then-verify mechanism to produce multiple tokens per forward pass.
APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
While long-context inference is crucial for advancing large language model (LLM) applications, its prefill speed remains a significant bottleneck.
Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages
While multilingual language models like XLM-R have advanced multilingualism in NLP, they still perform poorly in extremely low-resource languages.
Process Reinforcement through Implicit Rewards
While dense rewards also offer an appealing choice for the reinforcement learning (RL) of LLMs since their fine-grained rewards have the potential to address some inherent issues of outcome rewards, such as training efficiency and credit assignment, this potential remains largely unrealized.
LearningFlow: Automated Policy Learning Workflow for Urban Driving with Large Language Models
LearningFlow includes a curriculum sequence generation process and a reward generation process, which work in tandem to guide the RL policy by generating tailored training curricula and reward functions.
Densing Law of LLMs
This paper introduces the concept of ``\textit{capacity density}'' as a new metric to evaluate the quality of the LLMs across different scales and describes the trend of LLMs in terms of both effectiveness and efficiency.
Fancy123: One Image to High-Quality 3D Mesh Generation via Plug-and-Play Deformation
The fidelity enhancement module deforms the 3D mesh to match the input image.
KBAlign: Efficient Self Adaptation on Specific Knowledge Bases
Humans can utilize techniques to quickly acquire knowledge from specific materials in advance, such as creating self-assessment questions, enabling us to achieving related tasks more efficiently.
SignEye: Traffic Sign Interpretation from Vehicle First-Person View
Meanwhile, we develop a traffic guidance assistant (TGA) scenario application to re-explore the role of traffic signs in ADS as a complement to popular autonomous technologies (such as obstacle perception).
Real-Time Text Detection with Similar Mask in Traffic, Industrial, and Natural Scenes
In addition, to validate the scene robustness of the SM-Net, we conduct experiments on traffic, industrial, and natural scene datasets.
Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
These demonstrate that ReLU is more efficient as the activation function than SiLU and can leverage more training data to improve activation sparsity.
Progressive Compositionality In Text-to-Image Generative Models
Despite the impressive text-to-image (T2I) synthesis capabilities of diffusion models, they often struggle to understand compositional relationships between objects and attributes, especially in complex settings.
The Latent Road to Atoms: Backmapping Coarse-grained Protein Structures with Latent Diffusion
Coarse-grained(CG) molecular dynamics simulations offer computational efficiency for exploring protein conformational ensembles and thermodynamic properties.
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents
In this pipeline, instead of first parsing the document to obtain text, the document is directly embedded using a VLM as an image and then retrieved to enhance the generation of a VLM.
LLM$\times$MapReduce: Simplified Long-Sequence Processing using Large Language Models
The proposed LLM$\times$MapReduce framework splits the entire document into several chunks for LLMs to read and then aggregates the intermediate answers to produce the final output.
Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) mitigates issues of the factual errors and hallucinated outputs generated by Large Language Models (LLMs) in open-domain question-answering tasks (OpenQA) via introducing external knowledge.
Stuffed Mamba: State Collapse and State Capacity of RNN-Based Long-Context Modeling
For the second concern, we train a series of Mamba-2 models on long documents to empirically estimate the recurrent state capacity in language modeling and passkey retrieval.
Autoregressive Moving-average Attention Mechanism for Time Series Forecasting
We propose an Autoregressive (AR) Moving-average (MA) attention structure that can adapt to various linear attention mechanisms, enhancing their ability to capture long-range and local temporal patterns in time series.
Exploring the Benefit of Activation Sparsity in Pre-training
SSD adaptively switches between the Mixtures-of-Experts (MoE) based sparse training and the conventional dense training during the pre-training process, leveraging the efficiency of sparse training and avoiding the static activation correlation of sparse training.
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
Scaling the input context length of a large language model (LLM) incurs a significant increase in computation cost and memory footprint to maintain the attention key-value (KV) cache.
Spotlight Text Detector: Spotlight on Candidate Regions Like a Camera
The irregular contour representation is one of the tough challenges in scene text detection.
Focus Entirety and Perceive Environment for Arbitrary-Shaped Text Detection
The latter extracts region-level information and encourages the model to focus on the distribution of positive samples in the vicinity of a pixel, which perceives environment information.
Enabling Real-Time Conversations with Minimal Training Costs
To address this limitation, researchers have proposed duplex models.
From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents
Since the first instances of online education, where courses were uploaded to accessible and shared online platforms, this form of scaling the dissemination of human knowledge to reach a broader audience has sparked extensive discussion and widespread adoption.
Configurable Foundation Models: Building LLMs from a Modular Perspective
We first formalize modules into emergent bricks - functional neuron partitions that emerge during the pre-training phase, and customized bricks - bricks constructed via additional post-training to improve the capabilities and knowledge of LLMs.
Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slip Scripts
This study presents a multi-modal multi-granularity tokenizer specifically designed for analyzing ancient Chinese scripts, focusing on the Chu bamboo slip (CBS) script used during the Spring and Autumn and Warring States period (771-256 BCE) in Ancient China.
RIDE: Boosting 3D Object Detection for LiDAR Point Clouds via Rotation-Invariant Analysis
Particularly, our RIDE is compatible and easy to plug into the existing one-stage and two-stage 3D detectors, and boosts both detection performance and rotation robustness.
More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding
To address this task, we introduce GreenPLM, which leverages more text data to compensate for the lack of 3D data.
Data-Driven Parametrization of Molecular Mechanics Force Fields for Expansive Chemical Space Coverage
A force field is a critical component in molecular dynamics simulations for computational drug discovery.
Baby Bear: Seeking a Just Right Rating Scale for Scalar Annotations
Using the results of IBWS as a best-desired outcome, we evaluate various direct assessment methods to determine what is both cost-efficient and best correlating to a large scale BWS annotation strategy.
FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection
Open Domain Question Answering (ODQA) has been advancing rapidly in recent times, driven by significant developments in dense passage retrieval and pretrained language models.
MiniCPM-V: A GPT-4V Level MLLM on Your Phone
The recent surge of Multimodal Large Language Models (MLLMs) has fundamentally reshaped the landscape of AI research and industry, shedding light on a promising path toward the next AI milestone.
Ranked #7 on
Multiple-choice
on Neptune-Full
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework
However, evaluating the effectiveness of RAG systems in specialized scenarios remains challenging due to the high costs of data construction and the lack of suitable evaluation metrics.
LIDIA: Precise Liver Tumor Diagnosis on Multi-Phase Contrast-Enhanced CT via Iterative Fusion and Asymmetric Contrastive Learning
The early detection and precise diagnosis of liver tumors are tasks of critical clinical value, yet they pose significant challenges due to the high heterogeneity and variability of liver tumors.
Continual Learning for Adaptable Car-Following in Dynamic Traffic Environments
The continual evolution of autonomous driving technology requires car-following models that can adapt to diverse and dynamic traffic environments.
Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models
Moreover, the growing reliance on multi-modal generation exacerbates this issue because of its susceptibility to adversarial attacks.
EditFollower: Tunable Car Following Models for Customizable Adaptive Cruise Control Systems
To overcome these limitations, we propose the Editable Behavior Generation (EBG) model, a data-driven car-following model that allows for adjusting driving discourtesy levels.
PointDreamer: Zero-shot 3D Textured Mesh Reconstruction from Colored Point Cloud
Regarding this, we propose PointDreamer, a novel framework for textured mesh reconstruction from colored point cloud via diffusion-based 2D inpainting.
Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models
Specifically, we divide the queries and responses of conversations into several time slices and then adopt a time-division-multiplexing (TDM) encoding-decoding strategy to pseudo-simultaneously process these slices.
Fair Text to Medical Image Diffusion Model with Subgroup Distribution Aligned Tuning
For evaluation, we set the target dataset to be enhanced as the BraST18 dataset, and trained a brain magnetic resonance (MR) slice-based gender classifier from it.
Generating and Evolving Reward Functions for Highway Driving with Large Language Models
Reinforcement Learning (RL) plays a crucial role in advancing autonomous driving technologies by maximizing reward functions to achieve the optimal policy.
Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models
Motivated by the long-tail distribution of singular values in the delta weights, we propose a delta quantization approach using mixed-precision.
MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D Priors
Notably, MiniGPT-3D gains an 8. 12 increase on GPT-4 evaluation score for the challenging object captioning task compared to ShapeLLM-13B, while the latter costs 160 total GPU-hours on 8 A800.
Ranked #1 on
3D Object Captioning
on Objaverse
Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model
In contrast, the newly proposed Mamba model, based on state space models (SSM), outperforms Transformer in multiple areas with only linear complexity.
UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs
The rapid development of LLMs calls for a lightweight and easy-to-use framework for swift evaluation deployment.
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
For data scaling, we introduce a Warmup-Stable-Decay (WSD) learning rate scheduler (LRS), conducive to continuous training and domain adaptation.
Robust and Scalable Model Editing for Large Language Models
Large language models (LLMs) can make predictions using parametric knowledge--knowledge encoded in the model weights--or contextual knowledge--knowledge presented in the context.
V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception
Recent advancements in Vehicle-to-Everything (V2X) technologies have enabled autonomous vehicles to share sensing information to see through occlusions, greatly boosting the perception capability.
BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
Effective attention modules have played a crucial role in the success of Transformer-based large language models (LLMs), but the quadratic time and memory complexities of these attention modules also pose a challenge when processing long sequences.
CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables
For Multivariate Time Series Forecasting (MTSF), recent deep learning applications show that univariate models frequently outperform multivariate ones.
Ranked #20 on
Time Series Forecasting
on ETTh1 (336) Multivariate
Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition
Recent studies have uncovered intriguing phenomena in deep learning, such as grokking, double descent, and emergent abilities in large language models, which challenge human intuition and are crucial for a deeper understanding of neural models.
ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
Some recent efforts have explored introducing ReLU or its variants as the substitutive activation function to help LLMs achieve activation sparsity and inference acceleration, but few can simultaneously obtain high sparsity and comparable model performance.
$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
Processing and reasoning over long contexts is crucial for many practical applications of Large Language Models (LLMs), such as document comprehension and agent construction.
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding
To achieve this, we introduce Ouroboros, which can generate draft phrases to parallelize the drafting process and meanwhile lengthen drafts in a training-free manner.
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
Notably, the best-performing model, GPT-4V, attains an average score of 17. 97% on OlympiadBench, with a mere 10. 74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.
Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents
Large Language Models (LLMs) have increasingly been utilized in social simulations, where they are often guided by carefully crafted instructions to stably exhibit human-like behaviors during simulations.
MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization
Scientific data visualization plays a crucial role in research by enabling the direct display of complex information and assisting researchers in identifying implicit patterns.
LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
Most prior works on LoRA combination primarily rely on task-level weights for each involved LoRA, making different examples and tokens share the same LoRA weights.
OneBit: Towards Extremely Low-bit Large Language Models
Model quantification uses low bit-width values to represent the weight matrices of existing models to be quantized, which is a promising approach to reduce both storage and computational overheads of deploying highly anticipated LLMs.
UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset
Different from previous works that simply translate English instructions, we consider both the language-specific and language-agnostic abilities of LLMs.
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
In this paper, we unveil the intrinsic capacity of LLMs for understanding extremely long sequences without any fine-tuning.
ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
To find the most efficient activation function for sparse computation, we propose a systematic framework to examine the sparsity of LLMs from three aspects: the trade-off between sparsity and performance, the predictivity of sparsity, and the hardware affinity.
MatSAM: Efficient Extraction of Microstructures of Materials via Visual Large Model
Efficient and accurate extraction of microstructures in micrographs of materials is essential in process optimization and the exploration of structure-property relationships.
Generating High-Precision Force Fields for Molecular Dynamics Simulations to Study Chemical Reaction Mechanisms using Molecular Configuration Transformer
We applied this approach to study a Claisen rearrangement reaction and a Carbonyl insertion reaction catalyzed by Manganese.
Traffic Sign Interpretation in Real Road Scene
Following the above issues, we propose a traffic sign interpretation (TSI) task, which aims to interpret global semantic interrelated traffic signs (e. g.,~driving instruction-related texts, symbols, and guide panels) into a natural language for providing accurate instruction support to autonomous or assistant driving.
MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation
Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-event relationships.
Bayesian Conditional Diffusion Models for Versatile Spatiotemporal Turbulence Generation
A notable feature of our approach is the method proposed for long-span flow sequence generation, which is based on autoregressive gradient-based conditional sampling, eliminating the need for cumbersome retraining processes.
Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulations
ABM's strength lies in its bottom-up methodology, illuminating emergent phenomena by modeling the behaviors of individual components of a system.
Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
Pre-trained language models (PLMs) have achieved remarkable results on NLP tasks but at the expense of huge parameter sizes and the consequent computational costs.
Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models
Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments, enabling substantial reductions in model storage and memory costs without significant performance compromise.
ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learning
Long-term time series forecasting (LTSF) is important for various domains but is confronted by challenges in handling the complex temporal-contextual relationships.
Predicting Emergent Abilities with Infinite Resolution Evaluation
With PassUntil, we conduct a quantitative investigation into the scaling law of task performance.
ConPET: Continual Parameter-Efficient Tuning for Large Language Models
First, Static ConPET can adapt former continual learning methods originally designed for relatively smaller models to LLMs through PET and a dynamic replay strategy, which largely reduces the tuning costs and alleviates the over-fitting and forgetting issue.
QASnowball: An Iterative Bootstrapping Framework for High-Quality Question-Answering Data Generation
Recent years have witnessed the success of question answering (QA), especially its potential to be a foundation paradigm for tackling diverse NLP tasks.
EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning
The experimental results illustrate that EnsembleFollower yields improved accuracy of human-like behavior and achieves effectiveness in combining hybrid models, demonstrating that our proposed framework can handle diverse car-following conditions by leveraging the strengths of various low-level models.
Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages
Building a competitive counterpart in other languages is highly challenging due to the low-resource nature of non-English multimodal data (i. e., lack of large-scale, high-quality image-text data).
"Guinea Pig Trials" Utilizing GPT: A Novel Smart Agent-Based Modeling Approach for Studying Firm Competition and Collusion
Our results demonstrate that, in the absence of communication, smart agents consistently reach tacit collusion, leading to prices converging at levels higher than the Bertrand equilibrium price but lower than monopoly or cartel prices.
CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices
To address this, there are two key methods available: the first is model compression, which compresses LLMs into smaller sizes; the second is LoRA, which can transfer an LLM to other tasks with very few parameters, avoiding the storage of multiple model variants in multi-task scenarios by only preserving LoRAs.
Sampling-based Fast Gradient Rescaling Method for Highly Transferable Adversarial Attacks
In either case, the common gradient-based methods generally use the sign function to generate perturbations on the gradient update, that offers a roughly correct direction and has gained great success.
PersonaPKT: Building Personalized Dialogue Agents via Parameter-efficient Knowledge Transfer
Personalized dialogue agents (DAs) powered by large pre-trained language models (PLMs) often rely on explicit persona descriptions to maintain personality consistency.
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Experimental results on three knowledge-driven NLP tasks show that existing injection methods are not suitable for the new paradigm, while map-tuning effectively improves the performance of downstream models.
Plug-and-Play Document Modules for Pre-trained Models
By inserting document plugins into the backbone PTM for downstream tasks, we can encode a document one time to handle multiple tasks, which is more efficient than conventional encoding-task coupling methods that simultaneously encode documents and input queries using task-specific encoders.
Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning
Since it is non-trivial to directly model the intermediate states and design a running cost function, we propose to use latent stochastic bridges to regularize the intermediate states and use the regularization as the running cost of PETs.
Emergent Modularity in Pre-trained Transformers
In analogy to human brains, we consider two main characteristics of modularity: (1) functional specialization of neurons: we evaluate whether each neuron is mainly specialized in a certain function, and find that the answer is yes.
FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling
To address this gap and promote the development of microscopic traffic flow modeling, we establish a public benchmark dataset for car-following behavior modeling.
Efficient Cross-Lingual Transfer for Chinese Stable Diffusion with Images as Pivots
IAP optimizes only a separate Chinese text encoder with all other parameters fixed to align Chinese semantics space to the English one in CLIP.
DualGenerator: Information Interaction-based Generative Network for Point Cloud Completion
To tackle these challenges, we propose an information interaction-based generative network for point cloud completion ($\mathbf{DualGenerator}$).
Recyclable Tuning for Continual Pre-training
In pilot studies, we find that after continual pre-training, the upgraded PLM remains compatible with the outdated adapted weights to some extent.
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
We recruit annotators to search for relevant information using our interface and then answer questions.
Efficient and Degree-Guided Graph Generation via Discrete Diffusion Modeling
The empirical study shows that EDGE is much more efficient than competing methods and can generate large graphs with thousands of nodes.
A Majorization-Minimization Gauss-Newton Method for 1-Bit Matrix Completion
In 1-bit matrix completion, the aim is to estimate an underlying low-rank matrix from a partial set of binary observations.
Tool Learning with Foundation Models
Considering the lack of a systematic tool learning evaluation in prior works, we experiment with 18 representative tools and show the potential of current foundation models in skillfully utilizing tools.
Deep Learning for Iris Recognition: A Review
This paper collects 120 relevant papers to summarize the development of iris recognition based on deep learning.
PatchGT: Transformer over Non-trainable Clusters for Learning Graph Representations
Recently the Transformer structure has shown good performances in graph learning tasks.
MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction
It contains 103, 193 event coreference chains, 1, 216, 217 temporal relations, 57, 992 causal relations, and 15, 841 subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude.
Exploring Mode Connectivity for Pre-trained Language Models
(3) How does the PLM's task knowledge change along the path connecting two minima?
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning
To fathom the mystery, we hypothesize that the adaptations of different DETs could all be reparameterized as low-dimensional optimizations in a unified optimization subspace, which could be found by jointly decomposing independent solutions of different DETs.
Towards Accurate Subgraph Similarity Computation via Neural Graph Pruning
In this work, we convert graph pruning to a problem of node relabeling and then relax it to a differentiable problem.
Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task
End-to-end text image translation (TIT), which aims at translating the source language embedded in images to the target language, has attracted intensive attention in recent research.
Meta Reinforcement Learning with Successor Feature Based Context
Most reinforcement learning (RL) methods only focus on learning a single task from scratch and are not able to use prior knowledge to learn other tasks more effectively.
GACT: Activation Compressed Training for Generic Network Architectures
Training large neural network (NN) models requires extensive memory resources, and Activation Compressed Training (ACT) is a promising approach to reduce training memory footprint.
Likelihood ratio test for structural changes in factor models
This paper considers the likelihood ratio (LR) test for a variance change in the estimated factors.
Quantum Kerr Learning
Quantum machine learning is a rapidly evolving field of research that could facilitate important applications for quantum computing and also significantly impact data-driven sciences.
Sampling-based Fast Gradient Rescaling Method for Highly Transferable Adversarial Attacks
Deviation between the original gradient and the generated noises may lead to inaccurate gradient update estimation and suboptimal solutions for adversarial transferability, which is crucial for black-box attacks.
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
Predicting Physics in Mesh-reduced Space with Temporal Attention
Graph-based next-step prediction models have recently been very successful in modeling complex high-dimensional physical systems on irregular meshes.
Decoupled Low-light Image Enhancement
On the other hand, since the parameter matrix learned from the first stage is aware of the lightness distribution and the scene structure, it can be incorporated into the second stage as the complementary information.
OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication
We then construct a comprehensive benchmark with a total of 16 implemented models to evaluate several information fusion strategies~(i. e. early, late, and intermediate fusion) with state-of-the-art LiDAR detection algorithms.
Ranked #2 on
3D Object Detection
on OPV2V
PPT: Pre-trained Prompt Tuning for Few-shot Learning
To ensure the generalization of PPT, we formulate similar classification tasks into a unified task form and pre-train soft prompts for this unified task.
Prompt-Learning for Fine-Grained Entity Typing
In this work, we investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot and zero-shot scenarios.
Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference
In this work, we propose a domain generalization (DG) approach to learn on several labeled source domains and transfer knowledge to a target domain that is inaccessible in training.
CPM-2: Large-scale Cost-effective Pre-trained Language Models
We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference.
Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals
In contrast to the state-of-the-art methods, the proposed methods solve the estimation and testing problem at once with several merits: 1) an accurate sparsity estimation; 2) point estimates with shrinkage/soft-thresholding property; 3) credible intervals for uncertainty quantification; 4) an optimal multiple testing procedure that controls false discovery rate.
Pre-processing with Orthogonal Decompositions for High-dimensional Explanatory Variables
Strong correlations between explanatory variables are problematic for high-dimensional regularized regression methods.
Pre-Trained Models: Past, Present and Future
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI).
Order Matters: Probabilistic Modeling of Node Sequence for Graph Generation
A graph generative model defines a distribution over graphs.
Open Hierarchical Relation Extraction
To establish the bidirectional connections between OpenRE and relation hierarchy, we propose the task of open hierarchical relation extraction and present a novel OHRE framework for the task.
Fully Hyperbolic Neural Networks
Hyperbolic neural networks have shown great potential for modeling complex data.
CLEVE: Contrastive Pre-training for Event Extraction
Event extraction (EE) has considerably benefited from pre-trained language models (PLMs) by fine-tuning.
Knowledge Inheritance for Pre-trained Language Models
Specifically, we introduce a pre-training framework named "knowledge inheritance" (KI) and explore how could knowledge distillation serve as auxiliary supervision during pre-training to efficiently learn larger PLMs.
PTR: Prompt Tuning with Rules for Text Classification
This indicates that PTR is a promising approach to take advantage of both human prior knowledge and PLMs for those complicated classification tasks.
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction
Distantly supervised (DS) relation extraction (RE) has attracted much attention in the past few years as it can utilize large-scale auto-labeled data.
Few-NERD: A Few-Shot Named Entity Recognition Dataset
In this paper, we present Few-NERD, a large-scale human-annotated few-shot NER dataset with a hierarchy of 8 coarse-grained and 66 fine-grained entity types.
Ranked #6 on
Named Entity Recognition (NER)
on Few-NERD (SUP)
Visual Distant Supervision for Scene Graph Generation
In this work, we propose visual distant supervision, a novel paradigm of visual relation learning, which can train scene graph models without any human-labeled data.
Quasi-maximum likelihood estimation of break point in high-dimensional factor models
This paper estimates the break point for large-dimensional factor models with a single structural break in factor loadings at a common unknown date.
CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models
We then perform contrastive semi-supervised learning on both the retrieved unlabeled and original labeled instances to help PLMs capture crucial task-related semantic features.
Identity-aware Facial Expression Recognition in Compressed Video
In the up to two orders of magnitude compressed domain, we can explicitly infer the expression from the residual frames and possible to extract identity factors from the I frame with a pre-trained face recognition network.
GAN Ensemble for Anomaly Detection
Motivated by the observation that GAN ensembles often outperform single GANs in generation tasks, we propose to construct GAN ensembles for anomaly detection.
Meta-Information Guided Meta-Learning for Few-Shot Relation Classification
Few-shot classification requires classifiers to adapt to new classes with only a few training instances.
Neural Gibbs Sampling for Joint Event Argument Extraction
Existing EAE methods either extract each event argument roles independently or sequentially, which cannot adequately model the joint probability distribution among event arguments and their roles.
CPM: A Large-scale Generative Chinese Pre-trained Language Model
However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available.
Deep Reinforcement Learning of Transition States
Combining reinforcement learning (RL) and molecular dynamics (MD) simulations, we propose a machine-learning approach (RL$^\ddag$) to automatically unravel chemical reaction mechanisms.
Denoising Relation Extraction from Document-level Distant Supervision
Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance.
Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training
However, the cross entropy loss can not take the different importance of each class in an self-driving system into account.
Mutual Information Regularized Identity-aware Facial ExpressionRecognition in Compressed Video
In the up to two orders of magnitude compressed domain, we can explicitly infer the expression from the residual frames and possibly extract identity factors from the I frame with a pre-trained face recognition network.
Learning from Context or Names? An Empirical Study on Neural Relation Extraction
We find that (i) while context is the main source to support the predictions, RE models also heavily rely on the information from entity mentions, most of which is type information, and (ii) existing datasets may leak shallow heuristics via entity mentions and thus contribute to the high performance on RE benchmarks.
Ranked #24 on
Relation Extraction
on TACRED
Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph
On the one hand, sparse KGs contain less information, which makes it difficult for the model to choose correct paths.
IsOBS: An Information System for Oracle Bone Script
Oracle bone script (OBS) is the earliest known ancient Chinese writing system and the ancestor of modern Chinese.
CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models
In this paper, we propose a novel framework named Coke to dynamically select contextual knowledge and embed knowledge context according to textual context for PLMs, which can avoid the effect of redundant and ambiguous knowledge in KGs that cannot match the input text.
A Deep Network for Joint Registration and Reconstruction of Images with Pathologies
They are therefore not designed for the registration of images with strong pathologies for example in the context of brain tumors, and traumatic brain injuries.
Local Temperature Scaling for Probability Calibration
Thus, we propose a learning-based calibration method that focuses on multi-label semantic segmentation.
Continual Relation Learning via Episodic Memory Activation and Reconsolidation
Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations.
MAVEN: A Massive General Domain Event Detection Dataset
Most existing datasets exhibit the following issues that limit further development of ED: (1) Data scarcity.
A Perspective on Deep Learning for Molecular Modeling and Simulations
Deep learning is transforming many areas in science, and it has great potential in modeling molecular systems.
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction
Relational facts are an important component of human knowledge, which are hidden in vast amounts of text.
Adversarial Language Games for Advanced Natural Language Intelligence
In this work, we propose a challenging adversarial language game called Adversarial Taboo as an example, in which an attacker and a defender compete around a target word.
Unimodal-uniform Constrained Wasserstein Training for Medical Diagnosis
In this paper, we target on this task from the perspective of loss function.
HMEAE: Hierarchical Modular Event Argument Extraction
Existing event extraction methods classify each argument role independently, ignoring the conceptual correlations between different argument roles.
VoteNet+ : An Improved Deep Learning Label Fusion Method for Multi-atlas Segmentation
Specifically, we first illustrate that using a deep convolutional neural network to predict atlas probabilities can better distinguish correct atlas labels from incorrect ones than relying on image intensity difference as is typical in JLF.
Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data
Open relation extraction (OpenRE) aims to extract relational facts from the open-domain corpus.
FewRel 2.0: Towards More Challenging Few-Shot Relation Classification
We present FewRel 2. 0, a more challenging task to investigate two aspects of few-shot relation classification models: (1) Can they adapt to a new domain with only a handful of instances?
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction
OpenNRE is an open-source and extensible toolkit that provides a unified framework to implement neural models for relation extraction (RE).
Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations
Multi-hop knowledge graph (KG) reasoning is an effective and explainable method for predicting the target entity via reasoning paths in query answering (QA) task.
Ranked #3 on
Link Prediction
on NELL-995
Neural Snowball for Few-Shot Relation Learning
To address new relations with few-shot instances, we propose a novel bootstrapping approach, Neural Snowball, to learn new relations by transferring semantic knowledge about existing relations.
Efficient training and design of photonic neural network through neuroevolution
Recently, optical neural networks (ONNs) integrated in photonic chips has received extensive attention because they are expected to implement the same pattern recognition tasks in the electronic platforms with high efficiency and low power consumption.
Ranked #1 on
Reinforcement Learning
on iris
(using extra training data)
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification
Fact verification (FV) is a challenging task which requires to retrieve relevant evidence from plain text and use the evidence to verify given claims.
Ranked #7 on
Fact Verification
on FEVER
Quantifying Similarity between Relations with Fact Distribution
We introduce a conceptually simple and effective method to quantify the similarity between relations in knowledge bases.
DocRED: A Large-Scale Document-Level Relation Extraction Dataset
Multiple entities in a document generally exhibit complex inter-sentence relations, and cannot be well handled by existing relation extraction (RE) methods that typically focus on extracting intra-sentence relations for single entity pairs.
Ranked #59 on
Relation Extraction
on DocRED
Adversarial Training for Weakly Supervised Event Detection
Modern weakly supervised methods for event detection (ED) avoid time-consuming human annotation and achieve promising results by learning from auto-labeled data.
ERNIE: Enhanced Language Representation with Informative Entities
Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks.
Ranked #1 on
Entity Linking
on FIGER
VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas Segmentation
Experiments on 3D brain MRI data show that by selecting a good initial atlas set MAS with VoteNet significantly outperforms a number of other label fusion strategies as well as a direct DL segmentation approach.
Networks for Joint Affine and Non-parametric Image Registration
In contrast to existing approaches, our framework combines two registration methods: an affine registration and a vector momentum-parameterized stationary velocity field (vSVF) model.
Ranked #2 on
Image Registration
on Osteoarthritis Initiative
The Liver Tumor Segmentation Benchmark (LiTS)
In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018.
Knowledge Representation Learning: A Quantitative Review
Knowledge representation learning (KRL) aims to represent entities and relations in knowledge graph in low-dimensional semantic space, which have been widely used in massive knowledge-driven tasks.
Robust low-rank multilinear tensor approximation for a joint estimation of the multilinear rank and the loading matrices
Based on the low-rank property and an over-estimation of the core tensor, this joint estimation problem is solved by promoting (group) sparsity of the over-estimated core tensor.
DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction
To demonstrate the effectiveness of DIAG-NRE, we apply it to two real-world datasets and present both significant and interpretable improvements over state-of-the-art methods.
OpenKE: An Open Toolkit for Knowledge Embedding
We release an open toolkit for knowledge embedding (OpenKE), which provides a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space.
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation
The relation of each sentence is first recognized by distant supervision methods, and then filtered by crowdworkers.
Sentence Segmentation for Classical Chinese Based on LSTM with Radical Embedding
In this paper, we develop a low than character feature embedding called radical embedding, and apply it on LSTM model for sentence segmentation of pre modern Chinese texts.
Put It Back: Entity Typing with Language Model Enhancement
Entity typing aims to classify semantic types of an entity mention in a specific context.
Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention
In this paper, we aim to incorporate the hierarchical information of relations for distantly supervised relation extraction and propose a novel hierarchical attention scheme.
Adversarial Multi-lingual Neural Relation Extraction
To address these issues, we propose an adversarial multi-lingual neural relation extraction (AMNRE) model, which builds both consistent and individual representations for each sentence to consider the consistency and diversity among languages.
Denoising Distant Supervision for Relation Extraction via Instance-Level Adversarial Training
As shown in the experiments on a large-scale benchmark dataset in relation extraction, our denoising method can effectively filter out noisy instances and achieve significant improvements as compared with the state-of-the-art models.
Seed-Point Detection of Clumped Convex Objects by Short-Range Attractive Long-Range Repulsive Particle Clustering
Locating the center of convex objects is important in both image processing and unsupervised machine learning/data clustering fields.
Brain Extraction from Normal and Pathological Images: A Joint PCA/Image-Reconstruction Approach
Extracting the brain from images with strong pathologies, for example, the presence of a tumor or of a traumatic brain injury, is challenging.
Efficient Registration of Pathological Images: A Joint PCA/Image-Reconstruction Approach
Registration involving one or more images containing pathologies is challenging, as standard image similarity measures and spatial transforms cannot account for common changes due to pathologies.
Joint Representation Learning of Text and Knowledge for Knowledge Graph Completion
Joint representation learning of text and knowledge within a unified semantic space enables us to perform knowledge graph completion more accurately.
Performance evaluation of wavelet scattering network in image texture classification in various color spaces
Texture plays an important role in many image analysis applications.
