Search Results for author:
Found 40 papers, 14 papers with code
CogBERT: Cognition-Guided Pre-trained Language Models
To fill the gap, we propose CogBERT, a framework that can induce fine-grained cognitive features from cognitive data and incorporate cognitive features into BERT by adaptively adjusting the weight of cognitive features for different NLP tasks.
Neural Natural Logic Inference for Interpretable Question Answering
Many open-domain question answering problems can be cast as a textual entailment task, where a question and candidate answers are concatenated to form hypotheses.
Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficient Finetuning
Large Language Models (LLMs) have demonstrated significant potential in performing multiple tasks in multimedia applications, ranging from content generation to interactive entertainment, and artistic creation.
Towards Generalizable and Faithful Logic Reasoning over Natural Language via Resolution Refutation
To address this issue, we propose a novel framework, named Generalizable and Faithful Reasoner (GFaiR), which introduces the paradigm of resolution refutation.
Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
Through pretraining on a corpus with various sources, Large Language Models (LLMs) have gained impressive performance.
Proximity QA: Unleashing the Power of Multi-Modal Large Language Models for Spatial Proximity Analysis
To overcome this limitation in MLLMs, we introduce Proximity Question Answering (Proximity QA), a novel framework designed to enable MLLMs to infer the proximity relationship between objects in images.
VeCAF: Vision-language Collaborative Active Finetuning with Training Objective Awareness
Finetuning a pretrained vision model (PVM) is a common technique for learning downstream vision tasks.
Principled Gradient-based Markov Chain Monte Carlo for Text Generation
Recent papers have demonstrated the possibility of energy-based text generation by adapting gradient-based sampling algorithms, a paradigm of MCMC algorithms that promises fast convergence.
BiPFT: Binary Pre-trained Foundation Transformer with Low-rank Estimation of Binarization Residual Polynomials
Specifically, we first analyze the binarization error in self-attention operations and derive the polynomials of binarization error.
Formal Aspects of Language Modeling
Large language models have become one of the most commonly deployed NLP inventions.
On the Representational Capacity of Recurrent Neural Language Models
We extend the Turing completeness result to the probabilistic case, showing how a rationally weighted RLM with unbounded computation time can simulate any deterministic probabilistic Turing machine (PTM) with rationally weighted transitions.
Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis
Although demonstrating superb performance on various NLP tasks, large language models (LLMs) still suffer from the hallucination problem, which threatens the reliability of LLMs.
FLM-101B: An Open LLM and How to Train It with $100K Budget
We demonstrate that a 101B-parameter LLM with 0. 31T tokens can be trained with a budget of 100K US dollars.
FPTQ: Fine-grained Post-Training Quantization for Large Language Models
In this study, we propose a novel W4A8 post-training quantization method for the available open-sourced LLMs, which combines the advantages of both two recipes.
QD-BEV : Quantization-aware View-guided Distillation for Multi-view 3D Object Detection
Multi-view 3D detection based on BEV (bird-eye-view) has recently achieved significant improvements.
A Formal Perspective on Byte-Pair Encoding
Via submodular functions, we prove that the iterative greedy version is a $\frac{1}{{\sigma(\boldsymbol{\mu}^\star)}}(1-e^{-{\sigma(\boldsymbol{\mu}^\star)}})$-approximation of an optimal merge sequence, where ${\sigma(\boldsymbol{\mu}^\star)}$ is the total backward curvature with respect to the optimal merge sequence $\boldsymbol{\mu}^\star$.
Structured Voronoi Sampling
In this paper, we take an important step toward building a principled approach for sampling from language models with gradient-based methods.
Autoregressive Modeling with Lookahead Attention
To predict the next token, autoregressive models ordinarily examine the past.
A Measure-Theoretic Characterization of Tight Language Models
Language modeling, a central task in natural language processing, involves estimating a probability distribution over strings.
ReCo: Reliable Causal Chain Reasoning via Structural Causal Recurrent Neural Networks
Causal chain reasoning (CCR) is an essential ability for many decision-making AI systems, which requires the model to build reliable causal chains by connecting causal pairs.
CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification
CSQ stabilizes the bit-level mixed-precision training process with a bi-level gradual continuous sparsification on both the bit values of the quantized weights and the bit selection in determining the quantization precision of each layer.
BEV-LGKD: A Unified LiDAR-Guided Knowledge Distillation Framework for BEV 3D Object Detection
In this paper, we propose a unified framework named BEV-LGKD to transfer the knowledge in the teacher-student manner.
NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers
Building on the theoretical insight, NoisyQuant achieves the first success on actively altering the heavy-tailed activation distribution with additive noisy bias to fit a given quantizer.
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning
We propose to select layers based on the variability of their hidden states given a task-specific corpus.
Uncertainty Guided Depth Fusion for Spike Camera
In this paper, we propose a novel Uncertainty-Guided Depth Fusion (UGDF) framework to fuse the predictions of monocular and stereo depth estimation networks for spike camera.
DiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination
To relieve this issue, curriculum learning is proposed to improve model performance and generalization by ordering training samples in a meaningful (e. g., easy to hard) sequence.
Text Difficulty Study: Do machines behave the same as humans regarding text difficulty?
Given a task, human learns from easy to hard, whereas the model learns randomly.
A Graph Enhanced BERT Model for Event Prediction
To this end, we incorporate an additional structured variable into BERT to learn to predict the event connections in the training process.
e-CARE: a New Dataset for Exploring Explainable Causal Reasoning
Understanding causality has vital importance for various Natural Language Processing (NLP) applications.
Binary Neural Networks as a general-propose compute paradigm for on-device computer vision
For binary neural networks (BNNs) to become the mainstream on-device computer vision algorithm, they must achieve a superior speed-vs-accuracy tradeoff than 8-bit quantization and establish a similar degree of general applicability in vision tasks.
Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression
To maintain real-time processing in embedded systems, large on-chip memory is required to buffer the interlayer feature maps.
Learning Event Graph Knowledge for Abductive Reasoning
Abductive reasoning aims at inferring the most plausible explanation for observed events, which would play critical roles in various NLP applications, such as reading comprehension and question answering.
ExCAR: Event Graph Knowledge Enhanced Explainable Causal Reasoning
ExCAR first acquires additional evidence information from a large-scale causal event graph as logical rules for causal reasoning.
Modeling Event Background for If-Then Commonsense Reasoning Using Context-aware Variational Autoencoder
Understanding event and event-centered commonsense reasoning are crucial for natural language processing (NLP).
Training Deep Neural Networks Using Posit Number System
With the increasing size of Deep Neural Network (DNN) models, the high memory space requirements and computational complexity have become an obstacle for efficient DNN implementations.
The Curious Case of Neural Text Degeneration
Despite considerable advancements with deep neural language models, the enigma of neural text degeneration persists when these models are tested as text generators.
An Analog Neural Network Computing Engine using CMOS-Compatible Charge-Trap-Transistor (CTT)
The proposed computing engine is composed of a scalable CTT multiplier array and energy efficient analog-digital interfaces.
A Streaming Accelerator for Deep Convolutional Neural Networks with Image and Feature Decomposition for Resource-limited System Applications
Deep convolutional neural networks (CNN) are widely used in modern artificial intelligence (AI) and smart vision systems but also limited by computation latency, throughput, and energy efficiency on a resource-limited scenario, such as mobile devices, internet of things (IoT), unmanned aerial vehicles (UAV), and so on.
A Reconfigurable Streaming Deep Convolutional Neural Network Accelerator for Internet of Things
To implement image detection using CNN in the internet of things (IoT) devices, a streaming hardware accelerator is proposed.
