Search Results for author:
Found 46 papers, 16 papers with code
DocTime: A Document-level Temporal Dependency Graph Parser
We introduce DocTime - a novel temporal dependency graph (TDG) parser that takes as input a text document and produces a temporal dependency graph.
Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns
We present a comprehensive study of sparse attention patterns in Transformer models.
SOHES: Self-supervised Open-world Hierarchical Entity Segmentation
Using raw images as the sole training data, our method achieves unprecedented performance in self-supervised open-world segmentation, marking a significant milestone towards high-quality open-world entity segmentation in the absence of human-annotated masks.
Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
Instruction tuning is critical to large language models (LLMs) for achieving better instruction following and task adaptation capabilities but its success heavily relies on the training data quality.
Fourier Circuits in Neural Networks: Unlocking the Potential of Large Language Models in Mathematical Reasoning and Modular Arithmetic
Our research presents a thorough analytical characterization of the features learned by stylized one-hidden layer neural networks and one-layer Transformers in addressing this task.
Customization Assistant for Text-to-image Generation
Some existing methods do not require fine-tuning, while their performance are unsatisfactory.
LRM: Large Reconstruction Model for Single Image to 3D
We propose the first Large Reconstruction Model (LRM) that predicts the 3D model of an object from a single input image within just 5 seconds.
Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances
To achieve state-of-the-art performance, one still needs to train NER models on large-scale, high-quality annotated data, an asset that is both costly and time-intensive to accumulate.
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning
Recent advancements in Large Language Models (LLMs) have expanded the horizons of natural language understanding and generation.
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
Instruction tuning unlocks the superior capability of Large Language Models (LLM) to interact with humans.
AIMS: All-Inclusive Multi-Level Segmentation
Despite the progress of image segmentation for accurate visual entity segmentation, completing the diverse requirements of image editing applications for different-level region-of-interest selections remains unsolved.
Learning the Visualness of Text Using Large Vision-Language Models
Visual text evokes an image in a person's mind, while non-visual text fails to do so.
LayerDoc: Layer-wise Extraction of Spatial Hierarchical Structure in Visually-Rich Documents
Experiments show that our approach outperforms competitive baselines by 10-15% on three diverse datasets of forms and mobile app screen layouts for the tasks of spatial region classification, higher-order group identification, layout hierarchy extraction, reading order detection, and word grouping.
High Quality Entity Segmentation
Given the high-quality and -resolution nature of the dataset, we propose CropFormer which is designed to tackle the intractability of instance-level segmentation on high-resolution images.
MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding
In contrast, region-level models attempt to encode regions corresponding to paragraphs or text blocks into a single embedding, but they perform worse with additional word-level features.
Delving into Out-of-Distribution Detection with Vision-Language Representations
Recognizing out-of-distribution (OOD) samples is critical for machine learning systems deployed in the open world.
High-Quality Entity Segmentation
It improves mask prediction by fusing high-res image crops that provide more fine-grained image details and the full image.
User-Entity Differential Privacy in Learning Natural Language Models
In this paper, we introduce a novel concept of user-entity differential privacy (UeDP) to provide formal privacy protection simultaneously to both sensitive entities in textual data and data owners in learning natural language models (NLMs).
Improving the Reliability for Confidence Estimation
Confidence estimation, a task that aims to evaluate the trustworthiness of the model's prediction output during deployment, has received lots of research attention recently, due to its importance for the safe deployment of deep models.
Meta Spatio-Temporal Debiasing for Video Scene Graph Generation
Video scene graph generation (VidSGG) aims to parse the video content into scene graphs, which involves modeling the spatio-temporal contextual information in the video.
Unified Pretraining Framework for Document Understanding
Document intelligence automates the extraction of information from documents and supports many business applications.
Ranked #7 on
Document Layout Analysis
on PubLayNet val
Towards Language-Free Training for Text-to-Image Generation
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality text-image pairs.
EI-CLIP: Entity-Aware Interventional Contrastive Learning for E-Commerce Cross-Modal Retrieval
recommendation, and marketing services.
CA-SSL: Class-Agnostic Semi-Supervised Learning for Detection and Segmentation
To improve instance-level detection/segmentation performance, existing self-supervised and semi-supervised methods extract either task-unrelated or task-specific training signals from unlabeled data.
UniDoc: Unified Pretraining Framework for Document Understanding
Document intelligence automates the extraction of information from documents and supports many business applications.
LAFITE: Towards Language-Free Training for Text-to-Image Generation
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality image-text pairs.
Ranked #2 on
Text-to-Image Generation
on Multi-Modal-CelebA-HQ
Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling
To address this, we propose a cross-modal pseudo-labeling framework, which generates training pseudo masks by aligning word semantics in captions with visual features of object masks in images.
Bit-aware Randomized Response for Local Differential Privacy in Federated Learning
In this paper, we develop BitRand, a bit-aware randomized response algorithm, to preserve local differential privacy (LDP) in federated learning (FL).
Multi-Scale Aligned Distillation for Low-Resolution Detection
However, this option traditionally hurts the detection performance much.
Open-World Entity Segmentation
By removing the need of class label prediction, the models trained for such task can focus more on improving segmentation quality.
Exploiting Semantic Embedding and Visual Feature for Facial Action Unit Detection
The learned AU semantic embeddings are then used as guidance for the generation of attention maps through a cross-modality attention network.
SelfDoc: Self-Supervised Document Representation Learning
For downstream usage, we propose a novel modality-adaptive attention mechanism for multimodal feature fusion by adaptively emphasizing language and vision signals.
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
These two observations are further employed to formulate a measurement which can quantify the shortcut degree of each training sample.
Self-Supervised Relationship Probing
Structured representations of images that model visual relationships are beneficial for many vision and vision-language applications.
UNISON: Unpaired Cross-lingual Image Captioning
In this work, we present a novel unpaired cross-lingual method to generate image captions without relying on any caption corpus in the source or the target language.
Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning
In this paper, we propose a novel visual encoder to explicitly distinguish viewpoint changes from semantic changes in the change captioning task.
Resilient Load Restoration in Microgrids Considering Mobile Energy Storage Fleets: A Deep Reinforcement Learning Approach
Mobile energy storage systems (MESSs) provide mobility and flexibility to enhance distribution system resilience.
Watch It Twice: Video Captioning with a Refocused Video Encoder
With the rapid growth of video data and the increasing demands of various applications such as intelligent video search and assistance toward visually-impaired people, video captioning task has received a lot of attention recently in computer vision and natural language processing fields.
Scene Graph Generation with External Knowledge and Image Reconstruction
Scene graph generation has received growing attention with the advancements in image understanding tasks such as object detection, attributes and relationship prediction,~\etc.
Unpaired Image Captioning via Scene Graph Alignments
Most of current image captioning models heavily rely on paired image-caption datasets.
Video Captioning with Boundary-aware Hierarchical Language Decoding and Joint Video Prediction
In this paper, we propose a boundary-aware hierarchical language decoder for video captioning, which consists of a high-level GRU based language decoder, working as a global (caption-level) language model, and a low-level GRU based language decoder, working as a local (phrase-level) language model.
Unpaired Image Captioning by Language Pivoting
Image captioning is a multimodal task involving computer vision and natural language processing, where the goal is to learn a mapping from the image to its natural language description.
Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models
Textual-visual cross-modal retrieval has been a hot research topic in both computer vision and natural language processing communities.
Stack-Captioning: Coarse-to-Fine Learning for Image Captioning
On the other hand, multi-stage image caption model is hard to train due to the vanishing gradient problem.
An Empirical Study of Language CNN for Image Captioning
Language Models based on recurrent neural networks have dominated recent image caption generation tasks.
Recent Advances in Convolutional Neural Networks
In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing.
