Search Results for author:
Found 24 papers, 15 papers with code
Automatic Gloss Dictionary for Sign Language Learners
A multi-language dictionary is a fundamental tool for language learning, allowing the learner to look up unfamiliar words.
BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing
Then we design a subject representation learning task which enables a diffusion model to leverage such visual representation and generates new subject renditions.
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
In this paper, we conduct a systematic and comprehensive study on vision-language instruction tuning based on the pre-trained BLIP-2 models.
Toeplitz Neural Network for Sequence Modeling
Sequence modeling has important applications in natural language processing and computer vision.
Joint Task and Data Oriented Semantic Communications: A Deep Separate Source-channel Coding Scheme
This paper proposes a deep separate source-channel coding (DSSCC) framework for the joint task and data oriented semantic communications (JTD-SC) and utilizes the variational autoencoder approach to solve the rate-distortion problem with semantic distortion.
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
The cost of vision-and-language pre-training has become increasingly prohibitive due to end-to-end training of large-scale models.
Ranked #1 on
Image Retrieval
on COCO
From Images to Textual Prompts: Zero-Shot Visual Question Answering With Frozen Large Language Models
To address this issue, we propose Img2Prompt, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models
To address this issue, we propose \emph{Img2Prompt}, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
PCRED: Zero-shot Relation Triplet Extraction with Potential Candidate Relation Selection and Entity Boundary Detection
To address the above issues, we propose a novel method named PCRED for ZeroRTE with Potential Candidate Relation Selection and Entity Boundary Detection.
The Devil in Linear Transformer
In this paper, we examine existing kernel-based linear transformers and identify two key issues that lead to such performance gaps: 1) unbounded gradients in the attention computation adversely impact the convergence of linear transformer models; 2) attention dilution which trivially distributes attention scores over long sequences while neglecting neighbouring structures.
TODE-Trans: Transparent Object Depth Estimation with Transformer
We observe that the global characteristics of the transformer make it easier to extract contextual information to perform depth estimation of transparent areas.
LAVIS: A Library for Language-Vision Intelligence
We introduce LAVIS, an open-source deep learning library for LAnguage-VISion research and applications.
cosFormer: Rethinking Softmax in Attention
As one of its core components, the softmax attention helps to capture long-range dependencies yet prohibits its scale-up due to the quadratic space and time complexity to the sequence length.
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision.
Ranked #4 on
Image Captioning
on nocaps-val-out-domain
Transcribing Natural Languages for The Deaf via Neural Editing Programs
This work studies the task of glossification, of which the aim is to em transcribe natural spoken language sentences for the Deaf (hard-of-hearing) community to ordered sign language glosses.
Align and Prompt: Video-and-Language Pre-training with Entity Prompts
To achieve this, we first introduce an entity prompter module, which is trained with VTC to produce the similarity between a video crop and text prompts instantiated with entity names.
Ranked #11 on
Zero-Shot Video Retrieval
on DiDeMo
EDFace-Celeb-1M: Benchmarking Face Hallucination with a Million-scale Dataset
It is thus unclear how these algorithms perform on public face hallucination datasets.
Ranked #1 on
Image Super-Resolution
on WLFW
Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework
Video deraining is an important task in computer vision as the unwanted rain hampers the visibility of videos and deteriorates the robustness of most outdoor vision systems.
Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal
In addition, to further refine the result, a Differential-driven Dual Attention-in-Attention Model (D-DAiAM) is proposed with a "heavy-to-light" scheme to remove rain via addressing the unsatisfying deraining regions.
ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring
Video deblurring models exploit consecutive frames to remove blurs from camera shakes and object motions.
Benchmarking Ultra-High-Definition Image Super-Resolution
Increasingly, modern mobile devices allow capturing images at Ultra-High-Definition (UHD) resolution, which includes 4K and 8K images.
TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translation
Sign language translation (SLT) aims to interpret sign video sequences into text-based natural language sentences.
Transferring Cross-domain Knowledge for Video Sign Language Recognition
To this end, we extract news signs using a base WSLR model, and then design a classifier jointly trained on news and isolated signs to coarsely align these two domain features.
Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison
Based on this new large-scale dataset, we are able to experiment with several deep learning methods for word-level sign recognition and evaluate their performances in large scale scenarios.
Ranked #3 on
Sign Language Recognition
on WLASL100
