Search Results for author:
Found 48 papers, 28 papers with code
GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI
Despite significant advancements in general artificial intelligence, such as GPT-4, their effectiveness in the medical domain (general medical AI, GMAI) remains constrained due to the absence of specialized medical knowledge.
MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Models
Recent advancements in text-to-image (T2I) diffusion models have enabled the creation of high-quality images from text prompts, but they still struggle to generate images with precise control over specific visual concepts.
Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models
For clearness enhancement, we use real-world data, utilizing a dual-step strategy with pseudo-labels assessed by vision-language models and weather prompt learning.
Unveiling Deep Shadows: A Survey on Image and Video Shadow Detection, Removal, and Generation in the Era of Deep Learning
Shadows are formed when light encounters obstacles, leading to areas of diminished illumination.
Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling
Predicting multivariate time series is crucial, demanding precise modeling of intricate patterns, including inter-series dependencies and intra-series variations.
TrafficMOT: A Challenging Dataset for Multi-Object Tracking in Complex Traffic Scenarios
Multi-object tracking in traffic videos is a crucial research area, offering immense potential for enhancing traffic monitoring accuracy and promoting road safety measures through the utilisation of advanced machine learning algorithms.
IDRNet: Intervention-Driven Relation Network for Semantic Segmentation
Next, a deletion diagnostics procedure is conducted to model relations of these semantic-level representations via perceiving the network outputs and the extracted relations are utilized to guide the semantic-level representations to interact with each other.
SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels
Existing shadow detection datasets often contain missing or mislabeled shadows, which can hinder the performance of deep learning models trained directly on such data.
Video Dehazing via a Multi-Range Temporal Alignment Network with Physical Prior
Video dehazing aims to recover haze-free frames with high visibility and contrast.
Learning Weather-General and Weather-Specific Features for Image Restoration Under Multiple Adverse Weather Conditions
Inspired by this observation, we design an efficient unified framework with a two-stage training strategy to explore the weather-general and weather-specific features.
Matching Is Not Enough: A Two-Stage Framework for Category-Agnostic Pose Estimation
To calibrate the inaccurate matching results, we introduce a two-stage framework, where matched keypoints from the first stage are viewed as similarity-aware position proposals.
Ranked #5 on
2D Pose Estimation
on MP-100
Video Instance Shadow Detection Under the Sun and Sky
Instance shadow detection, crucial for applications such as photo editing and light direction estimation, has undergone significant advancements in predicting shadow instances, object instances, and their associations.
Sparse2Dense: Learning to Densify 3D Features for 3D Object Detection
Importantly, we formulate the lightweight plug-in S2D module and the point cloud reconstruction module in SDet to densify 3D features and train SDet to produce 3D features, following the dense 3D features in DDet.
Demystify Transformers & Convolutions in Modern Image Deep Networks
Our experiments on various tasks and an analysis of inductive bias show a significant performance boost due to advanced network-level and block-level designs, but performance differences persist among different STMs.
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state.
Ranked #1 on
Instance Segmentation
on COCO test-dev
(AP50 metric, using extra
training data)
Learning Shadow Correspondence for Video Shadow Detection
We further design a new evaluation metric to evaluate the temporal stability of the video shadow detection results.
NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis
In this paper, we present NUWA-Infinity, a generative model for infinite visual synthesis, which is defined as the task of generating arbitrarily-sized high-resolution images or long-duration videos.
Ranked #1 on
Image Outpainting
on LHQC
Instance Shadow Detection with A Single-Stage Detector
This paper formulates a new problem, instance shadow detection, which aims to detect shadow instance and the associated object instance that cast each shadow in the input image.
Ranked #1 on
Instance Shadow Detection
on SOBA
Dynamic Message Propagation Network for RGB-D Salient Object Detection
This paper presents a novel deep neural network framework for RGB-D salient object detection by controlling the message passing between the RGB images and depth maps on the feature level and exploring the long-range semantic contexts and geometric information on both RGB and depth features to infer salient objects.
GLIPv2: Unifying Localization and Vision-Language Understanding
We present GLIPv2, a grounded VL understanding model, that serves both localization tasks (e. g., object detection, instance segmentation) and Vision-Language (VL) understanding tasks (e. g., VQA, image captioning).
Ranked #1 on
Phrase Grounding
on Flickr30k Entities Test
(using extra training data)
GIT: A Generative Image-to-text Transformer for Vision and Language
In this paper, we design and train a Generative Image-to-text Transformer, GIT, to unify vision-language tasks such as image/video captioning and question answering.
Ranked #1 on
Image Captioning
on nocaps-XD near-domain
Spatial Autoregressive Coding for Graph Neural Recommendation
To adequately leverage neighbor proximity and high-order information, we design a novel spatial autoregressive paradigm.
K-LITE: Learning Transferable Visual Models with External Knowledge
We propose K-LITE, a simple strategy to leverage external knowledge for building transferable visual systems: In training, it enriches entities in text with WordNet and Wiktionary knowledge, leading to an efficient and scalable approach to learning image representations that uses knowledge about the visual concepts.
Enhancing Pseudo Label Quality for Semi-Supervised Domain-Generalized Medical Image Segmentation
With these augmentations as perturbations, we feed the input to a confidence-aware cross pseudo supervision network to measure the variance of pseudo labels and regularize the network to learn with more confident pseudo labels.
Injecting Semantic Concepts into End-to-End Image Captioning
In this paper, we are concerned with a better-performing detector-free image captioning model, and propose a pure vision transformer-based image captioning model, dubbed as ViTCAP, in which grid representations are used without extracting the regional features.
Scaling Up Vision-Language Pre-training for Image Captioning
In this paper, we present LEMON, a LargE-scale iMage captiONer, and provide the first empirical study on the scaling behavior of VLP for image captioning.
Ranked #3 on
Image Captioning
on nocaps-XD entire
(using extra training data)
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling
On grounded captioning, UniTAB presents a simpler solution with a single output head, and significantly outperforms state of the art in both grounding and captioning evaluations.
UFO: A UniFied TransfOrmer for Vision-Language Representation Learning
In this paper, we propose a single UniFied transfOrmer (UFO), which is capable of processing either unimodal inputs (e. g., image or language) or multimodal inputs (e. g., the concatenation of the image and the question), for vision-language (VL) representation learning.
Strategic Inventories in a Supply Chain with Downstream Cournot Duopoly
The inventories carried in a supply chain as a strategic tool to influence the competing firms are considered to be strategic inventories (SI).
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA
To address this challenge, we propose PICa, a simple yet effective method that Prompts GPT3 via the use of Image Captions, for knowledge-based VQA.
Ranked #21 on
Visual Question Answering (VQA)
on OK-VQA
(using extra training data)
Single-Stage Instance Shadow Detection with Bidirectional Relation Learning
Instance shadow detection aims to find shadow instances paired with the objects that cast the shadows.
Ranked #2 on
Instance Shadow Detection
on SOBA
Compressing Visual-linguistic Model via Knowledge Distillation
In this paper, we study knowledge distillation (KD) to effectively compress a transformer-based large VL model into a small VL model.
Global Guidance Network for Breast Lesion Segmentation in Ultrasound Images
The BD modules learn additional breast lesion boundary map to enhance the boundary quality of a segmentation result refinement.
Deep Texture-Aware Features for Camouflaged Object Detection
Camouflaged object detection is a challenging task that aims to identify objects having similar texture to the surroundings.
Relief and Stimulus in A Cross-sector Multi-product Scarce Resource Supply Chain Network
In the era of a growing population, systemic changes to the world, and the rising risk of crises, humanity has been facing an unprecedented challenge of resource scarcity.
Incorporating Vision Bias into Click Models for Image-oriented Search Engine
Most typical click models assume that the probability of a document to be examined by users only depends on position, such as PBM and UBM.
VinVL: Revisiting Visual Representations in Vision-Language Models
In our experiments we feed the visual features generated by the new object detection model into a Transformer-based VL fusion model \oscar \cite{li2020oscar}, and utilize an improved approach \short\ to pre-train the VL model and fine-tune it on a wide range of downstream VL tasks.
Ranked #2 on
Image-text matching
on CommercialAdsDataset
MiniVLM: A Smaller and Faster Vision-Language Model
We design a Two-stage Efficient feature Extractor (TEE), inspired by the one-stage EfficientDet network, to significantly reduce the time cost of visual feature extraction by $95\%$, compared to a baseline model.
VIVO: Visual Vocabulary Pre-Training for Novel Object Captioning
It is highly desirable yet challenging to generate image captions that can describe novel objects which are unseen in caption-labeled training data, a capability that is evaluated in the novel object captioning challenge (nocaps).
Ranked #3 on
Image Captioning
on nocaps-XD out-of-domain
Revisiting Shadow Detection: A New Benchmark Dataset for Complex World
Shadow detection in general photos is a nontrivial problem, due to the complexity of the real world.
Ranked #10 on
Shadow Detection
on CUHK-Shadow
Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration
This paper presents a novel framework for unsupervised 3D brain image registration by capturing the feature-level transformation relationships between the unaligned image and reference image.
Depth-Attentional Features for Single-Image Rain Removal
Rain is a common weather phenomenon, where object visibility varies with depth from the camera and objects faraway are visually blocked more by fog than by rain streaks.
Ranked #5 on
Single Image Deraining
on RainCityscapes
Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data
This paper presents a new method for shadow removal using unpaired data, enabling us to avoid tedious annotations and obtain more diverse training samples.
Ranked #6 on
Shadow Removal
on SRD
SAC-Net: Spatial Attenuation Context for Salient Object Detection
This paper presents a new deep neural network design for salient object detection by maximizing the integration of local and global image context within, around, and beyond the salient objects.
Direction-aware Spatial Context Features for Shadow Detection and Removal
This paper presents a novel deep neural network design for shadow detection and removal by analyzing the spatial image context in a direction-aware manner.
Ranked #1 on
Shadow Removal
on SRD
SINet: A Scale-insensitive Convolutional Neural Network for Fast Vehicle Detection
Based on these findings, we present a scale-insensitive convolutional neural network (SINet) for fast detecting vehicles with a large variance of scales.
Direction-aware Spatial Context Features for Shadow Detection
To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent neural network (RNN) by introducing attention weights when aggregating spatial context features in the RNN.
Ranked #2 on
RGB Salient Object Detection
on SBU / SBU-Refine
(Bandit) Convex Optimization with Biased Noisy Gradient Oracles
Algorithms for bandit convex optimization and online learning often rely on constructing noisy gradient estimates, which are then used in appropriately adjusted first-order algorithms, replacing actual gradients.
