Search Results for author:
Found 38 papers, 12 papers with code
Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns
We present a comprehensive study of sparse attention patterns in Transformer models.
Scalable Differential Privacy with Certified Robustness in Adversarial Learning
In this paper, we aim to develop a scalable algorithm to preserve differential privacy (DP) in adversarial learning for deep neural networks (DNNs), with certified robustness to adversarial examples.
Numerical Pruning for Efficient Autoregressive Models
Transformers have emerged as the leading architecture in deep learning, proving to be versatile and highly effective across diverse domains beyond language and image processing.
Persona-SQ: A Personalized Suggested Question Generation Framework For Real-world Documents
Suggested questions (SQs) provide an effective initial interface for users to engage with their documents in AI-powered reading applications.
SUGAR: Subject-Driven Video Customization in a Zero-Shot Manner
Compared to previous methods, SUGAR achieves state-of-the-art results in identity preservation, video dynamics, and video-text alignment for subject-driven video customization, demonstrating the effectiveness of our proposed method.
LoRA-Contextualizing Adaptation of Large Multimodal Models for Long Document Understanding
In this work, we present a novel framework named LoRA-Contextualizing Adaptation of Large multimodal models (LoCAL), which broadens the capabilities of any LMM to support long-document understanding.
LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models
Large multimodal language models have demonstrated impressive capabilities in understanding and manipulating images.
ARTIST: Improving the Generation of Text-rich Images with Disentangled Diffusion Models and Large Language Models
Diffusion models have demonstrated exceptional capabilities in generating a broad spectrum of visual content, yet their proficiency in rendering text is still limited: they often generate inaccurate characters or words that fail to blend well with the underlying image.
Toffee: Efficient Million-Scale Dataset Construction for Subject-Driven Text-to-Image Generation
Our dataset is 5 times the size of previous largest dataset, yet our cost is tens of thousands of GPU hours lower.
DocSynthv2: A Practical Autoregressive Modeling for Document Generation
While the generation of document layouts has been extensively explored, comprehensive document generation encompassing both layout and content presents a more complex challenge.
TRINS: Towards Multimodal Language Models that Can Read
In this work, we introduce TRINS: a Text-Rich image INStruction dataset, with the objective of enhancing the reading ability of the multimodal large language model.
Improve Temporal Awareness of LLMs for Sequential Recommendation
Large language models (LLMs) have demonstrated impressive zero-shot abilities in solving a wide range of general-purpose tasks.
Automatic Layout Planning for Visually-Rich Documents with Instruction-Following Models
Recent advancements in instruction-following models have made user interactions with models more user-friendly and efficient, broadening their applicability.
SOHES: Self-supervised Open-world Hierarchical Entity Segmentation
Using raw images as the sole training data, our method achieves unprecedented performance in self-supervised open-world segmentation, marking a significant milestone towards high-quality open-world entity segmentation in the absence of human-annotated masks.
3DPFIX: Improving Remote Novices' 3D Printing Troubleshooting through Human-AI Collaboration
While troubleshooting plays an essential part of 3D printing, the process remains challenging for many remote novices even with the help of well-developed online sources, such as online troubleshooting archives and online community help.
Customization Assistant for Text-to-image Generation
Some existing methods do not require fine-tuning, while their performance are unsatisfactory.
Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information
Our work aims to address this concern by introducing a novel approach to detecting adversarial prompts at a token level, leveraging the LLM's capability to predict the next token's probability.
AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models
Safety alignment of Large Language Models (LLMs) can be compromised with manual jailbreak attacks and (automatic) adversarial attacks.
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
Instruction tuning unlocks the superior capability of Large Language Models (LLM) to interact with humans.
Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels
Remarkably, by incorporating conditional information from the powerful CLIP model, our method can boost the current SOTA accuracy by 10-20 absolute points in many cases.
Ranked #1 on
Image Classification
on Food-101N
(using extra training data)
Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approach
However, generating images of novel concept provided by the user input image is still a challenging task.
Adversarial Attacks and Defenses in Machine Learning-Powered Networks: A Contemporary Survey
This survey provides a comprehensive overview of the recent advancements in the field of adversarial attack and defense techniques, with a focus on deep neural network-based classification models.
MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding
In contrast, region-level models attempt to encode regions corresponding to paragraphs or text blocks into a single embedding, but they perform worse with additional word-level features.
User-Entity Differential Privacy in Learning Natural Language Models
In this paper, we introduce a novel concept of user-entity differential privacy (UeDP) to provide formal privacy protection simultaneously to both sensitive entities in textual data and data owners in learning natural language models (NLMs).
Unified Pretraining Framework for Document Understanding
Document intelligence automates the extraction of information from documents and supports many business applications.
Ranked #8 on
Document Layout Analysis
on PubLayNet val
Aligning Eyes between Humans and Deep Neural Network through Interactive Attention Alignment
We propose a novel framework of Interactive Attention Alignment (IAA) that aims at realizing human-steerable Deep Neural Networks (DNNs).
Towards Language-Free Training for Text-to-Image Generation
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality text-image pairs.
UniDoc: Unified Pretraining Framework for Document Understanding
Document intelligence automates the extraction of information from documents and supports many business applications.
LAFITE: Towards Language-Free Training for Text-to-Image Generation
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality image-text pairs.
Ranked #2 on
Text-to-Image Generation
on Multi-Modal-CelebA-HQ
Bit-aware Randomized Response for Local Differential Privacy in Federated Learning
In this paper, we develop BitRand, a bit-aware randomized response algorithm, to preserve local differential privacy (LDP) in federated learning (FL).
CLVSA: A Convolutional LSTM Based Variational Sequence-to-Sequence Model with Attention for Predicting Trends of Financial Markets
Financial markets are a complex dynamical system.
Financial Markets Prediction with Deep Learning
Financial markets are difficult to predict due to its complex systems dynamics.
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
These two observations are further employed to formulate a measurement which can quantify the shortcut degree of each training sample.
Self-Supervised Relationship Probing
Structured representations of images that model visual relationships are beneficial for many vision and vision-language applications.
Open-Domain Question Answering with Pre-Constructed Question Spaces
Our reader-retriever first uses an offline reader to read the corpus and generate collections of all answerable questions associated with their answers, and then uses an online retriever to respond to user queries by searching the pre-constructed question spaces for answers that are most likely to be asked in the given way.
Cross-Domain Document Object Detection: Benchmark Suite and Method
We establish a benchmark suite consisting of different types of PDF document datasets that can be utilized for cross-domain DOD model training and evaluation.
Long-Term Memory Networks for Question Answering
Question answering is an important and difficult task in the natural language processing domain, because many basic natural language processing tasks can be cast into a question answering task.
Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks
Existing work solves this problem by employing recurrent neural networks (RNNs) to model EHR data and utilizing simple attention mechanism to interpret the results.
