Search Results for author:
Found 38 papers, 12 papers with code
LookAhead Tuning: Safer Language Models via Partial Answer Previews
Fine-tuning enables large language models (LLMs) to adapt to specific domains, but often undermines their previously established safety alignment.
Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Models
To better observe performance modifications and characteristics of this threat, we also introduce the TVPI Dataset.
LightThinker: Thinking Step-by-Step Compression
Large language models (LLMs) have shown remarkable performance in complex reasoning tasks, but their efficiency is hindered by the substantial memory and computational costs associated with generating lengthy tokens.
K-ON: Stacking Knowledge On the Head Layer of Large Language Model
Recent advancements in large language models (LLMs) have significantly improved various natural language processing (NLP) tasks.
OntoTune: Ontology-Driven Self-training for Aligning Large Language Models
Compared to existing domain LLMs based on newly collected large-scale domain-specific corpora, our OntoTune, which relies on the existing, long-term developed ontology and LLM itself, significantly reduces data maintenance costs and offers improved generalization ability.
MAQInstruct: Instruction-based Unified Event Relation Extraction
Extracting event relations that deviate from known schemas has proven challenging for previous methods based on multi-class classification, MASK prediction, or prototype matching.
Have We Designed Generalizable Structural Knowledge Promptings? Systematic Evaluation and Rethinking
Large language models (LLMs) have demonstrated exceptional performance in text generation within current NLP research.
OneKE: A Dockerized Schema-Guided LLM Agent-based Knowledge Extraction System
We introduce OneKE, a dockerized schema-guided knowledge extraction system, which can extract knowledge from the Web and raw PDF Books, and support various domains (science, news, etc.).
MKGL: Mastery of a Three-Word Language
Large language models (LLMs) have significantly advanced performance across a spectrum of natural language processing (NLP) tasks.
Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models
Recently, driven by advancements in Multimodal Large Language Models (MLLMs), Vision Language Action Models (VLAMs) are being proposed to achieve better performance in open-vocabulary scenarios for robotic manipulation tasks.
KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation
The recently developed retrieval-augmented generation (RAG) technology has enabled the efficient construction of domain-specific applications.
OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing System
Knowledge representation has been a central aim of AI since its inception.
OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs
This paper introduces a novel and efficient One-pass Generation and retrieval framework (OneGen), designed to improve LLMs' performance on tasks that require both generation and retrieval.
Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformers
This work proposes Quasar-ViT, a hardware-oriented quantization-aware architecture search framework for ViTs, to design efficient ViT models for hardware implementation while preserving the accuracy.
Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approach
Multi-hop question answering is a challenging task with distinct industrial relevance, and Retrieval-Augmented Generation (RAG) methods based on large language models (LLMs) have become a popular approach to tackle this task.
Efficient Knowledge Infusion via KG-LLM Alignment
To tackle the problem of domain-specific knowledge scarcity within large language models (LLMs), knowledge graph-retrievalaugmented method has been proven to be an effective and efficient technique for knowledge infusion.
Continual Few-shot Event Detection via Hierarchical Augmentation Networks
The CFED task is challenging as it involves memorizing previous event types and learning new event types with few-shot samples.
ChatUIE: Exploring Chat-based Unified Information Extraction using Large Language Models
This motivated us to explore domain-specific modeling in chat-based language models as a solution for extracting structured information from natural language.
Zero-Shot Cross-Lingual Document-Level Event Causality Identification with Heterogeneous Graph Contrastive Transfer Learning
Event Causality Identification (ECI) refers to the detection of causal relations between events in texts.
IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpus
Large Language Models (LLMs) demonstrate remarkable potential across various domains; however, they exhibit a significant performance gap in Information Extraction (IE).
Pursing the Sparse Limitation of Spiking Deep Learning Structures
It posits that within dense neural networks, there exist winning tickets or subnetworks that are sparser but do not compromise performance.
Gaining the Sparse Rewards by Exploring Lottery Tickets in Spiking Neural Network
Deploying energy-efficient deep learning algorithms on computational-limited devices, such as robots, is still a pressing issue for real-world applications.
InstructIE: A Bilingual Instruction-based Information Extraction Dataset
Experimental results demonstrate that large language models trained with InstructIE can not only obtain better IE capabilities but also enhance zero-shot performance compared with baselines.
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit their generalization.
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers
While vision transformers (ViTs) have continuously achieved new milestones in the field of computer vision, their sophisticated network architectures with high computation and memory costs have impeded their deployment on resource-limited edge devices.
Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization
Compared with state-of-the-art ViT quantization work (algorithmic approach only without hardware acceleration), our quantization achieves 0. 47% to 1. 36% higher Top-1 accuracy under the same bit-width.
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
To the best of our knowledge, this is the first time quantization has been incorporated into ViT acceleration on FPGAs with the help of a fully automatic framework to guide the quantization strategy on the software side and the accelerator implementations on the hardware side given the target frame rate.
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning
Moreover, our framework can guarantee the identified model to meet resource specifications of mobile devices and FPGA, and even achieve the real-time execution of DeiT-T on mobile platforms.
Ranked #4 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
Specifically, this is the first effort to assign mixed quantization schemes and multiple precisions within layers -- among rows of the DNN weight matrix, for simplified operations in hardware inference, while preserving accuracy.
ILMPQ : An Intra-Layer Multi-Precision Deep Neural Network Quantization framework for FPGA
Our proposed ILMPQ DNN quantization framework achieves 70. 73 Top1 accuracy in ResNet-18 on the ImageNet dataset.
HFSP: A Hardware-friendly Soft Pruning Framework for Vision Transformers
Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while the high computation and memory cost makes its propagation in industrial production difficult.
Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework
Unlike existing methods that use the same quantization scheme for all weights, we propose the first solution that applies different quantization schemes for different rows of the weight matrix.
MSP: An FPGA-Specific Mixed-Scheme, Multi-Precision Deep Neural Network Quantization Framework
To tackle the limited computing and storage resources in edge devices, model compression techniques have been widely used to trim deep neural network (DNN) models for on-device inference execution.
RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices
The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism.
Towards an Efficient and General Framework of Robust Training for Graph Neural Networks
To overcome these limitations, we propose a general framework which leverages the greedy search algorithms and zeroth-order methods to obtain robust GNNs in a generic and an efficient manner.
SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency
Structured weight pruning is a representative model compression technique of DNNs for hardware efficiency and inference accelerations.
Adversarial T-shirt! Evading Person Detectors in A Physical World
To the best of our knowledge, this is the first work that models the effect of deformation for designing physical adversarial examples with respect to-rigid objects such as T-shirts.
Interpreting Adversarial Examples by Activation Promotion and Suppression
It is widely known that convolutional neural networks (CNNs) are vulnerable to adversarial examples: images with imperceptible perturbations crafted to fool classifiers.
