Search Results for author:
Found 301 papers, 117 papers with code
Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition
Recognizing irregular text in natural scene images is challenging due to the large variance in text appearance, such as curvature, orientation and distortion.
Ranked #26 on
Scene Text Recognition
on ICDAR2015
NAS-FCOS: Fast Neural Architecture Search for Object Detection
The success of deep neural networks relies on significant architecture engineering.
Ranked #113 on
Object Detection
on COCO test-dev
Qwen Technical Report
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans.
Ranked #3 on
Multi-Label Text Classification
on CC3M-TagMask
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
In this work, we pursue a unified paradigm for multimodal pretraining to break the scaffolds of complex task/modality-specific customization.
Ranked #1 on
Visual Question Answering
on VQA v2 test-std
(yes/no metric)
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
In this work, we explore a scalable way for building a general representation model toward unlimited modalities.
Ranked #1 on
Semantic Segmentation
on ADE20K
(using extra training data)
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
In this work, we introduce the Qwen-VL series, a set of large-scale vision-language models (LVLMs) designed to perceive and understand both texts and images.
Ranked #3 on
Visual Question Answering
on MM-Vet
DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population
We present an open-source and extensible knowledge extraction toolkit DeepKE, supporting complicated low-resource, document-level and multimodal scenarios in the knowledge base population.
Verified Low-Level Programming Embedded in F*
Low* is a shallow embedding of a small, sequential, well-behaved subset of C in F*, a dependently-typed variant of ML aimed at program verification.
Programming Languages Cryptography and Security
Prompt Tuning for Generative Multimodal Pretrained Models
Prompt tuning has become a new paradigm for model tuning and it has demonstrated success in natural language pretraining and even vision pretraining.
Ranked #2 on
Visual Entailment
on SNLI-VE test
Transferring General Multimodal Pretrained Models to Text Recognition
This paper proposes a new method, OFA-OCR, to transfer multimodal pretrained models to text recognition.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
In NeuS, we propose to represent a surface as the zero-level set of a signed distance function (SDF) and develop a new volume rendering method to train a neural SDF representation.
Multi-Label Image Recognition with Graph Convolutional Networks
The task of multi-label image recognition is to predict a set of object labels that present in an image.
Ranked #12 on
Multi-Label Classification
on PASCAL VOC 2007
Editing Large Language Models: Problems, Methods, and Opportunities
Our objective is to provide valuable insights into the effectiveness and feasibility of each editing technique, thereby assisting the community in making informed decisions on the selection of the most appropriate method for a specific task or context.
EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models
Large Language Models (LLMs) usually suffer from knowledge cutoff or fallacy issues, which means they are unaware of unseen events or generate text with incorrect facts owing to outdated/noisy data.
A Comprehensive Study of Knowledge Editing for Large Language Models
In this paper, we first define the knowledge editing problem and then provide a comprehensive review of cutting-edge approaches.
Ranked #1 on
knowledge editing
on zsRE
(using extra training data)
StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
We perform volume rendering only to produce a low-resolution feature map and progressively apply upsampling in 2D to address the first issue.
Multi-Granularity Prediction for Scene Text Recognition
In this work, we first draw inspiration from the recent progress in Vision Transformer (ViT) to construct a conceptually simple yet powerful vision STR model, which is built upon ViT and outperforms previous state-of-the-art models for scene text recognition, including both pure vision models and language-augmented methods.
Ranked #1 on
Scene Text Recognition
on Uber-Text
(using extra training data)
Levenshtein OCR
A novel scene text recognizer based on Vision-Language Transformer (VLT) is presented.
Multi-Granularity Prediction with Learnable Fusion for Scene Text Recognition
Specifically, MGP-STR achieves an average recognition accuracy of $94\%$ on standard benchmarks for scene text recognition.
LISTER: Neighbor Decoding for Length-Insensitive Scene Text Recognition
The diversity in length constitutes a significant characteristic of text.
F$^{2}$-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories
Based on our analysis, we further propose a novel space-warping method called perspective warping, which allows us to handle arbitrary trajectories in the grid-based NeRF framework.
MVDream: Multi-view Diffusion for 3D Generation
We introduce MVDream, a diffusion model that is able to generate consistent multi-view images from a given text prompt.
The ApolloScape Open Dataset for Autonomous Driving and its Application
In this paper, we provide a sensor fusion scheme integrating camera videos, consumer-grade motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robust self-localization and semantic segmentation for autonomous driving.
NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
We present a neural rendering-based method called NeRO for reconstructing the geometry and the BRDF of reflective objects from multiview images captured in an unknown environment.
Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network
Depth estimation from a single image is a fundamental problem in computer vision.
Learning Depth with Convolutional Spatial Propagation Network
In this paper, we propose a simple yet effective convolutional spatial propagation network (CSPN) to learn the affinity matrix for various depth estimation tasks.
Visual Question Answering: A Survey of Methods and Datasets
Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities.
MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion
This paper introduces MVDiffusion, a simple yet effective method for generating consistent multi-view images from text prompts given pixel-to-pixel correspondences (e. g., perspective crops from a panorama or multi-view images given depth maps and poses).
Neural Rays for Occlusion-aware Image-based Rendering
On such a 3D point, these generalization methods will include inconsistent image features from invisible views, which interfere with the radiance field construction.
SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views
We introduce SparseNeuS, a novel neural rendering based method for the task of surface reconstruction from multi-view images.
Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphs
From the ASG, we propose a novel ASG2Caption model, which is able to recognise user intentions and semantics in the graph, and therefore generate desired captions according to the graph structure.
Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video
We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera.
NAS-FCOS: Efficient Search for Object Detection Architectures
Neural Architecture Search (NAS) has shown great potential in effectively reducing manual effort in network design by automatically discovering optimal architectures.
BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fields
Our approach models the physical image formation process of a motion blurred image, and jointly learns the parameters of NeRF and recovers the camera motion trajectories during exposure time.
Real-time Segmentation and Facial Skin Tones Grading
Modern approaches for semantic segmention usually pay too much attention to the accuracy of the model, and therefore it is strongly recommended to introduce cumbersome backbones, which brings heavy computation burden and memory footprint.
PERF: Panoramic Neural Radiance Field from a Single Panorama
In this paper, we present PERF, a 360-degree novel view synthesis framework that trains a panoramic neural radiance field from a single panorama.
OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models
As a starting point, we provide presets of 7 different modalities and 23 highly-diverse example tasks in OFASys, with which we also develop a first-in-kind, single model, OFA+, that can handle text, image, speech, video, and motion data.
HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers
Last, we proposed an efficient fine-grained search strategy to train HR-NAS, which effectively explores the search space, and finds optimal architectures given various tasks and computation resources.
Joint Unsupervised Learning of Optical Flow and Depth by Watching Stereo Videos
Then the whole scene is decomposed into moving foreground and static background by compar- ing the estimated optical flow and rigid flow derived from the depth and ego-motion.
Self-Taught Convolutional Neural Networks for Short Text Clustering
Short text clustering is a challenging problem due to its sparseness of text representation.
Ranked #2 on
Short Text Clustering
on Stackoverflow
DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings
To tackle that, we propose an effective knowledge distillation framework for contrastive sentence embeddings, termed DistilCSE.
Speech2Video Synthesis with 3D Skeleton Regularization and Expressive Body Poses
In this paper, we propose a novel approach to convert given speech audio to a photo-realistic speaking video of a specific person, where the output video has synchronized, realistic, and expressive rich body dynamics.
Anisotropic Convolutional Networks for 3D Semantic Scene Completion
In contrast to the standard 3D convolution that is limited to a fixed 3D receptive field, our module is capable of modeling the dimensional anisotropy voxel-wisely.
BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting
In this paper, we introduce a novel approach, named BAD-Gaussians (Bundle Adjusted Deblur Gaussian Splatting), which leverages explicit Gaussian representation and handles severe motion-blurred images with inaccurate camera poses to achieve high-quality scene reconstruction.
Give Me Something to Eat: Referring Expression Comprehension with Commonsense Knowledge
In this case, we need to use commonsense knowledge to identify the objects in the image.
LEGO: Learning Edge with Geometry all at Once by Watching Videos
In our framework, the predicted depths, normals and edges are forced to be consistent all the time.
Deep Learning and Its Applications to Machine Health Monitoring: A Survey
Since 2006, deep learning (DL) has become a rapidly growing research direction, redefining state-of-the-art performances in a wide range of areas such as object recognition, image segmentation, speech recognition and machine translation.
Discriminative and Robust Online Learning for Siamese Visual Tracking
The problem of visual object tracking has traditionally been handled by variant tracking paradigms, either learning a model of the object's appearance exclusively online or matching the object with the target in an offline-trained embedding space.
TouchStone: Evaluating Vision-Language Models by Language Models
Large vision-language models (LVLMs) have recently witnessed rapid advancements, exhibiting a remarkable capacity for perceiving, understanding, and processing visual information by connecting visual receptor with large language models (LLMs).
VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection
With the benefit of dual branch, VadCLIP achieves both coarse-grained and fine-grained video anomaly detection by transferring pre-trained knowledge from CLIP to WSVAD task.
Simple is not Easy: A Simple Strong Baseline for TextVQA and TextCaps
Texts appearing in daily scenes that can be recognized by OCR (Optical Character Recognition) tools contain significant information, such as street name, product brand and prices.
DeLS-3D: Deep Localization and Segmentation with a 3D Semantic Map
The uniqueness of our design is a sensor fusion scheme which integrates camera videos, motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robustness and efficiency of the system.
Label Relation Graphs Enhanced Hierarchical Residual Network for Hierarchical Multi-Granularity Classification
Hierarchical multi-granularity classification (HMC) assigns hierarchical multi-granularity labels to each object and focuses on encoding the label hierarchy, e. g., ["Albatross", "Laysan Albatross"] from coarse-to-fine levels.
Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings
Previous knowledge graph embedding approaches usually map entities to representations and utilize score functions to predict the target entities, yet they typically struggle to reason rare or emerging unseen entities.
Ranked #1 on
Link Prediction
on FB15k-237-ind
WikiAsp: A Dataset for Multi-domain Aspect-based Summarization
In this paper, we propose WikiAsp, a large-scale dataset for multi-domain aspect-based summarization that attempts to spur research in the direction of open-domain aspect-based summarization.
GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Tasks
Large language models (LLMs) like ChatGPT, exhibit powerful zero-shot and instruction-following capabilities, have catalyzed a revolutionary transformation across diverse fields, especially for open-ended tasks.
VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis
The Gaussian reconstruction kernels have been proposed by Westover (1990) and studied by the computer graphics community back in the 90s, which gives an alternative representation of object 3D geometry from meshes and point clouds.
Semantic Instance Segmentation via Deep Metric Learning
We propose a new method for semantic instance segmentation, by first computing how likely two pixels are to belong to the same object, and then by grouping similar pixels together.
Glocal Energy-based Learning for Few-Shot Open-Set Recognition
Few-shot open-set recognition (FSOR) is a challenging task of great practical value.
Semi-Supervised Wide-Angle Portraits Correction by Multi-Scale Transformer
We propose a semi-supervised network for wide-angle portraits correction.
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model
To make the final image feature concentrate more on the target visual concept, a Class-Aware Visual Prompt Tuning (CAVPT) scheme is further proposed in our DPT, where the class-aware visual prompt is generated dynamically by performing the cross attention between text prompts features and image patch token embeddings to encode both the downstream task-related information and visual instance information.
AerialVLN: Vision-and-Language Navigation for UAVs
Navigating in the sky is more complicated than on the ground because agents need to consider the flying height and more complex spatial relationship reasoning.
Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding
Performance on the five tasks of depth estimation, optical flow estimation, odometry, moving object segmentation and scene flow estimation shows that our approach outperforms other SoTA methods.
NightLab: A Dual-level Architecture with Hardness Detection for Segmentation at Night
Notably, NightLab contains models at two levels of granularity, i. e. image and regional, and each level is composed of light adaptation and segmentation modules.
Node Representation Learning in Graph via Node-to-Neighbourhood Mutual Information Maximization
In this work, we present a simple-yet-effective self-supervised node representation learning strategy via directly maximizing the mutual information between the hidden representations of nodes and their neighbourhood, which can be theoretically justified by its link to graph smoothing.
Pushing the Performance Limit of Scene Text Recognizer without Human Annotation
A robust consistency regularization based semi-supervised framework is proposed for STR, which can effectively solve the instability issue due to domain inconsistency between synthetic and real images.
Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignment
Considering the excessive memory overheads of contrastive learning, we further propose a negative-free solution, where the main contribution is a Graph Signal Decorrelation (GSD) constraint to avoid representation collapse and over-smoothing.
Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction
To obtain high-quality positron emission tomography (PET) scans while reducing radiation exposure to the human body, various approaches have been proposed to reconstruct standard-dose PET (SPET) images from low-dose PET (LPET) images.
HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigation
Pre-training has been adopted in a few of recent works for Vision-and-Language Navigation (VLN).
Ranked #4 on
Visual Navigation
on R2R
Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing
Consequently, effectively adapting large pre-trained models to downstream tasks in an efficient manner has become a prominent research area.
FastRE: Towards Fast Relation Extraction with Convolutional Encoder and Improved Cascade Binary Tagging Framework
Recent work for extracting relations from texts has achieved excellent performance.
A geometry-aware deep network for depth estimation in monocular endoscopy
To address this issue, we introduce a gradient loss to penalize edge fluctuations ambiguous around stepped edge structures and a normal loss to explicitly express the sensitivity to frequently small structures, and propose a geometric consistency loss to spreads the spatial information across the sample grids to constrain the global geometric anatomy structures.
Revisiting Prototypical Network for Cross Domain Few-Shot Learning
Prototypical Network is a popular few-shot solver that aims at establishing a feature metric generalizable to novel few-shot classification (FSC) tasks using deep neural networks.
Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation
For autonomous driving, the dynamics and states of vehicle parts such as doors, the trunk, and the bonnet can provide meaningful semantic information and interaction states, which are essential to ensuring the safety of the self-driving vehicle.
A Simple and Robust Correlation Filtering Method for Text-based Person Search
Text-based person search aims to associate pedestrian images with natural language descriptions.
Ranked #8 on
Text based Person Retrieval
on ICFG-PEDES
SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
In this paper, we focus on image inpainting task, aiming at recovering the missing area of an incomplete image given the context information.
Hyperspectral Classification Based on Lightweight 3-D-CNN With Transfer Learning
In contrast to previous approaches, we do not impose restrictions over the source data sets, in which they do not have to be collected by the same sensors as the target data sets.
Flash: Efficient Dynamic Routing for Offchain Networks
Mice payments are directly sent by looking up a routing table with a few precomputed paths to reduce probing overhead.
Networking and Internet Architecture
Person Re-identification in Aerial Imagery
Our method can learn a discriminative and compact feature representation for ReID in aerial imagery and can be trained in an end-to-end fashion efficiently.
Knowledge Rumination for Pre-trained Language Models
Previous studies have revealed that vanilla pre-trained language models (PLMs) lack the capacity to handle knowledge-intensive NLP tasks alone; thus, several works have attempted to integrate external knowledge into PLMs.
AdaXpert: Adapting Neural Architecture for Growing Data
To address this, we present a neural architecture adaptation method, namely Adaptation eXpert (AdaXpert), to efficiently adjust previous architectures on the growing data.
Domain Adaptation for Deep Entity Resolution: A Design Space Exploration
Entity resolution (ER) is a core problem of data integration.
Ranked #2 on
Entity Resolution
on WDC Watches-small
Learning Conditional Attributes for Compositional Zero-Shot Learning
Compositional Zero-Shot Learning (CZSL) aims to train models to recognize novel compositional concepts based on learned concepts such as attribute-object combinations.
Ranked #1 on
Compositional Zero-Shot Learning
on MIT-States
Unicorn: A Unified Multi-tasking Model for Supporting Matching Tasks in Data Integration
The widely used practice is to build task-specific or even dataset-specific solutions, which are hard to generalize and disable the opportunities of knowledge sharing that can be learned from different datasets and multiple tasks.
Structural Recurrent Neural Network for Traffic Speed Prediction
We use a graph of a vehicular road network with recurrent neural networks (RNNs) to infer the interaction between adjacent road segments as well as the temporal dynamics.
Scalable Learning With a Structural Recurrent Neural Network for Short-Term Traffic Prediction
With the real traffic speed data measured in the city of Santander, we demonstrate the proposed SRNN outperforms the image-based approaches using the capsule network (CapsNet) by 14. 1% and the convolutional neural network (CNN) by 5. 87%, respectively, in terms of root mean squared error (RMSE).
Structured Multimodal Attentions for TextVQA
In this paper, we propose an end-to-end structured multimodal attention (SMA) neural network to mainly solve the first two issues above.
USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields
USB-NeRF is able to correct rolling shutter distortions and recover accurate camera motion trajectory simultaneously under the framework of NeRF, by modeling the physical image formation process of a RS camera.
A Holistic Representation Guided Attention Network for Scene Text Recognition
In this work, we propose a simple yet strong approach for scene text recognition.
Ground-to-Aerial Person Search: Benchmark Dataset and Approach
In this work, we construct a large-scale dataset for Ground-to-Aerial Person Search, named G2APS, which contains 31, 770 images of 260, 559 annotated bounding boxes for 2, 644 identities appearing in both of the UAVs and ground surveillance cameras.
Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach
Parameter-efficient fine-tuning for pre-trained Vision Transformers aims to adeptly tailor a model to downstream tasks by learning a minimal set of new adaptation parameters while preserving the frozen majority of pre-trained parameters.
SCL-VI: Self-supervised Context Learning for Visual Inspection of Industrial Defects
Current unsupervised models struggle to strike a balance between detecting texture and object defects, lacking the capacity to discern latent representations and intricate features.
A Capsule Network for Traffic Speed Prediction in Complex Road Networks
Traffic flow data from induction loop sensors are essentially a time series, which is also spatially related to traffic in different road segments.
AutoRemover: Automatic Object Removal for Autonomous Driving Videos
To deal with shadows, we build up an autonomous driving shadow dataset and design a deep neural network to detect shadows automatically.
PCEE-BERT: Accelerating BERT Inference via Patient and Confident Early Exiting
In this work, we propose Patient and Confident Early Exiting BERT (PCEE-BERT), an off-the-shelf sample-dependent early exiting method that can work with different PLMs and can also work along with popular model compression methods.
The Law of Parsimony in Gradient Descent for Learning Deep Linear Networks
Second, it allows us to better understand deep representation learning by elucidating the linear progressive separation and concentration of representations from shallow to deep layers.
Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples
To solve this problem, we propose an end-to-end trainable deep network which is inspired by the state-of-the-art fine-grained recognition model and is tailored for the FSFG task.
Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Model
In this context, we propose a novel task called Video Anomaly Retrieval (VAR), which aims to pragmatically retrieve relevant anomalous videos by cross-modalities, e. g., language descriptions and synchronous audios.
Link Prediction in Social Networks: the State-of-the-Art
Finally, some future challenges of the link prediction in social networks are discussed.
Social and Information Networks Physics and Society
CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning
Large pre-trained VLMs like CLIP have demonstrated superior zero-shot recognition ability, and a number of recent studies leverage this ability to mitigate catastrophic forgetting in CL, but they focus on closed-set CL in a single domain dataset.
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold
When training overparameterized deep networks for classification tasks, it has been widely observed that the learned features exhibit a so-called "neural collapse" phenomenon.
Watch out Venomous Snake Species: A Solution to SnakeCLEF2023
The SnakeCLEF2023 competition aims to the development of advanced algorithms for snake species identification through the analysis of images and accompanying metadata.
Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination
To the best of our knowledge, this is the first quantitative characterization of feature evolution in hierarchical representations of deep linear networks.
Learning Graph Convolutional Networks for Multi-Label Recognition and Applications
The task of multi-label image recognition is to predict a set of object labels that present in an image.
Multi-Label Classification
Multi-label Image Recognition with Partial Labels
Convergence and Recovery Guarantees of the K-Subspaces Method for Subspace Clustering
The K-subspaces (KSS) method is a generalization of the K-means method for subspace clustering.
Attribute-Aware Deep Hashing with Self-Consistency for Large-Scale Fine-Grained Image Retrieval
Our work focuses on tackling large-scale fine-grained image retrieval as ranking the images depicting the concept of interests (i. e., the same sub-category labels) highest based on the fine-grained details in the query.
Continual Referring Expression Comprehension via Dual Modular Memorization
In this paper, we propose Continual Referring Expression Comprehension (CREC), a new setting for REC, where a model is learning on a stream of incoming tasks.
Adaptive Importance Learning for Improving Lightweight Image Super-resolution Network
In this study, we revisit this problem from an orthog- onal view, and propose a novel learning strategy to maxi- mize the pixel-wise fitting capacity of a given lightweight network architecture.
Training a Binary Weight Object Detector by Knowledge Transfer for Autonomous Driving
Autonomous driving has harsh requirements of small model size and energy efficiency, in order to enable the embedded system to achieve real-time on-board object detection.
View Extrapolation of Human Body from a Single Image
We study how to synthesize novel views of human body from a single image.
Occlusion Aware Unsupervised Learning of Optical Flow
Especially on KITTI dataset where abundant unlabeled samples exist, our unsupervised method outperforms its counterpart trained with supervised learning.
Visual Question Answering with Memory-Augmented Networks
In this paper, we exploit a memory-augmented neural network to predict accurate answers to visual questions, even when those answers occur rarely in the training set.
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features
Within each region of interest, MaskLab performs foreground/background segmentation by combining semantic and direction prediction.
Ranked #85 on
Instance Segmentation
on COCO test-dev
(using extra training data)
Fine-grained Pattern Matching Over Streaming Time Series
Hence, we formulate a new problem, called "fine-grained pattern matching", which allows users to specify varied granularities of matching deviation to different segments of a given pattern, and fuzzy regions for adaptive breakpoints determination between consecutive segments.
Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
The Visual Dialogue task requires an agent to engage in a conversation about an image with a human.
Ranked #4 on
Visual Dialog
on VisDial v0.9 val
Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency
Learning to reconstruct depths in a single image by watching unlabeled videos via deep convolutional network (DCN) is attracting significant attention in recent years.
Towards End-to-End Car License Plates Detection and Recognition with Deep Neural Networks
In contrast to existing approaches which take license plate detection and recognition as two separate tasks and settle them step by step, our method jointly solves these two tasks by a single network.
Joint Multi-Person Pose Estimation and Semantic Part Segmentation
To refine part segments, the refined pose and the original part potential are integrated through a Part FCN, where the skeleton feature from pose serves as additional regularization cues for part segments.
Ranked #5 on
Human Part Segmentation
on PASCAL-Part
FVQA: Fact-based Visual Question Answering
We evaluate several baseline models on the FVQA dataset, and describe a novel model which is capable of reasoning about an image on the basis of supporting facts.
Ranked #2 on
Visual Question Answering (VQA)
on F-VQA
Towards End-to-end Text Spotting with Convolutional Recurrent Neural Networks
In this work, we jointly address the problem of text detection and recognition in natural scene images based on convolutional recurrent neural networks.
A Fast and Compact Saliency Score Regression Network Based on Fully Convolutional Network
In this paper, we tackle this problem by proposing a fast and compact saliency score regression network which employs fully convolutional network, a special deep convolutional neural network, to estimate the saliency of objects in images.
Compositional Model based Fisher Vector Coding for Image Classification
To handle this limitation, in this paper we break the convention which assumes that a local feature is drawn from one of few Gaussian distributions.
Image Captioning and Visual Question Answering Based on Attributes and External Knowledge
Much recent progress in Vision-to-Language problems has been achieved through a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).
The VQA-Machine: Learning How to Use Existing Vision Algorithms to Answer New Questions
To train a method to perform even one of these operations accurately from {image, question, answer} tuples would be challenging, but to aim to achieve them all with a limited set of such training data seems ambitious at best.
Exploiting Temporal Information for DCNN-based Fine-Grained Object Classification
Fine-grained classification is a relatively new field that has concentrated on using information from a single image, while ignoring the enormous potential of using video data to improve classification.
DOC: Deep OCclusion Estimation From a Single Image
In this paper we propose a deep network architecture, called DOC, which acts on a single image, detects object boundaries and estimates the border ownership (i. e. which side of the boundary is foreground and which is background).
Where to Focus: Query Adaptive Matching for Instance Retrieval Using Convolutional Feature Maps
Instance retrieval requires one to search for images that contain a particular object within a large corpus.
Pushing the Limits of Deep CNNs for Pedestrian Detection
In this work, we show that by re-using the convolutional feature maps (CFMs) of a deep convolutional neural network (DCNN) model as image features to train an ensemble of boosted decision models, we are able to achieve the best reported accuracy without using specially designed learning algorithms.
Large-scale Binary Quadratic Optimization Using Semidefinite Relaxation and Applications
Two standard relaxation methods are widely used for solving general BQPs--spectral methods and semidefinite programming (SDP), each with their own advantages and disadvantages.
Ask Me Anything: Free-form Visual Question Answering Based on Knowledge from External Sources
Priming a recurrent neural network with this combined information, and the submitted question, leads to a very flexible visual question answering approach.
Zoom Better to See Clearer: Human and Object Parsing with Hierarchical Auto-Zoom Net
To tackle these difficulties, we propose a "Hierarchical Auto-Zoom Net" (HAZN) for object part parsing which adapts to the local scales of objects and parts.
Ranked #8 on
Human Part Segmentation
on PASCAL-Part
Hi Detector, What's Wrong with that Object? Identifying Irregular Object From Images by Modelling the Detection Score Distribution
To address this problem, we propose a novel approach by inspecting the distribution of the detection scores at multiple image regions based on the detector trained from the "regular object" and "other objects".
Order-aware Convolutional Pooling for Video Based Action Recognition
Most video based action recognition approaches create the video-level representation by temporally pooling the features extracted at each frame.
Pose-Guided Human Parsing with Deep Learned Features
Parsing human body into semantic regions is crucial to human-centric analysis.
Explicit Knowledge-based Reasoning for Visual Question Answering
We describe a method for visual question answering which is capable of reasoning about contents of an image on the basis of information extracted from a large-scale knowledge base.
Efficient Semidefinite Branch-and-Cut for MAP-MRF Inference
We propose a Branch-and-Cut (B&C) method for solving general MAP-MRF inference problems.
Implementation of Training Convolutional Neural Networks
Deep learning refers to the shining branch of machine learning that is based on learning levels of representations.
Joint Object and Part Segmentation using Deep Learned Potentials
Segmenting semantic objects from images and parsing them into their respective semantic parts are fundamental steps towards detailed object understanding in computer vision.
Temporal Pyramid Pooling Based Convolutional Neural Networks for Action Recognition
One challenge is that video contains a varying number of frames which is incompatible to the standard input format of CNNs.
Efficient SDP Inference for Fully-connected CRFs Based on Low-rank Decomposition
Conditional Random Fields (CRF) have been widely used in a variety of computer vision tasks.
A Fast Semidefinite Approach to Solving Binary Quadratic Problems
Second, compared with conventional SDP methods, the new SDP formulation leads to a significantly more efficient and scalable dual optimization approach, which has the same degree of complexity as spectral methods.
Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understanding
The four types of information, i. e. 2D flow, camera pose, segment mask and depth maps, are integrated into a differentiable holistic 3D motion parser (HMP), where per-pixel 3D motion for rigid background and moving objects are recovered.
Towards Effective Deep Embedding for Zero-Shot Learning
Towards this goal, we present a simple but effective two-branch network to simultaneously map semantic descriptions and visual samples into a joint space, on which visual embeddings are forced to regress to their class-level semantic embeddings and the embeddings crossing classes are required to be distinguishable by a trainable classifier.
RGB-D Based Action Recognition with Light-weight 3D Convolutional Networks
Different from RGB videos, depth data in RGB-D videos provide key complementary information for tristimulus visual data which potentially could achieve accuracy improvement for action recognition.
ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving
Specifically, we first segment each car with a pre-trained Mask R-CNN, and then regress towards its 3D pose and shape based on a deformable 3D car model with or without using semantic keypoints.
Visual Question Answering as Reading Comprehension
In contrast to struggling on multimodal feature fusion, in this paper, we propose to unify all the input information by natural language so as to convert VQA into a machine reading comprehension problem.
Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networks
Being composed of node attention component and edge attention component, the proposed graph attention mechanism explicitly represents inter-object relationships, and properties with a flexibility and power impossible with competing approaches.
CVTE at IJCNLP-2017 Task 1: Character Checking System for Chinese Grammatical Error Diagnosis Task
Grammatical error diagnosis is an important task in natural language processing.
SURGE: Surface Regularized Geometry Estimation from a Single Image
This paper introduces an approach to regularize 2. 5D surface normal and depth predictions at each pixel given a single input image.
RPC: A Large-Scale Retail Product Checkout Dataset
The main challenge of this problem comes from the large scale and the fine-grained nature of the product categories as well as the difficulty for collecting training images that reflect the realistic checkout scenarios due to continuous update of the products.
Supervised Kernel Descriptors for Visual Recognition
In visual recognition tasks, the design of low level image feature representation is fundamental.
Towards Unified Depth and Semantic Prediction From a Single Image
By allowing for interactions between the depth and semantic information, the joint network provides more accurate depth prediction than a state-of-the-art CNN trained solely for depth prediction [5].
What's Wrong With That Object? Identifying Images of Unusual Objects by Modelling the Detection Score Distribution
The key observation motivating our approach is that "regular object" images, "unusual object" images and "other objects" images exhibit different region-level scores in terms of both the score values and the spatial distributions.
Multi-Attention Network for One Shot Learning
One-shot learning is a challenging problem where the aim is to recognize a class identified by a single training image.
Pixel-aware Deep Function-mixture Network for Spectral Super-Resolution
To address this problem, we propose a pixel-aware deep function-mixture network for SSR, which is composed of a new class of modules, termed function-mixture (FM) blocks.
Vehicle Re-identification in Aerial Imagery: Dataset and Approach
It is capable of explicitly detecting discriminative parts for each specific vehicle and significantly outperforms the evaluated baselines and state-of-the-art vehicle ReID approaches.
Towards End-to-End Text Spotting in Natural Scenes
Text spotting in natural scene images is of great importance for many image understanding tasks.
Evaluating Local Geometric Feature Representations for 3D Rigid Data Matching
The outcomes present interesting findings that may shed new light on this community and provide complementary perspectives to existing evaluations on the topic of local geometric feature description.
A Performance Evaluation of Correspondence Grouping Methods for 3D Rigid Data Matching
Seeking consistent point-to-point correspondences between 3D rigid data (point clouds, meshes, or depth maps) is a fundamental problem in 3D computer vision.
EPNAS: Efficient Progressive Neural Architecture Search
In this paper, we propose Efficient Progressive Neural Architecture Search (EPNAS), a neural architecture search (NAS) that efficiently handles large search space through a novel progressive search policy with performance prediction based on REINFORCE~\cite{Williams. 1992. PG}.
V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices
One of the primary challenges faced by deep learning is the degree to which current methods exploit superficial statistics and dataset bias, rather than learning to generalise over the specific representations they have experienced.
A Method to Learn Embedding of a Probabilistic Medical Knowledge Graph: Algorithm Development
This paper proposes an algorithm named as PrTransH to learn embedding vectors from real world EMR data based medical knowledge.
Efficient Automatic Meta Optimization Search for Few-Shot Learning
NAS automatically generates and evaluates meta-learner's architecture for few-shot learning problems, while the meta-learner uses meta-learning algorithm to optimize its parameters based on the distribution of learning tasks.
Attend to the Difference: Cross-Modality Person Re-identification via Contrastive Correlation
The problem of cross-modality person re-identification has been receiving increasing attention recently, due to its practical significance.
Cross-Modality Person Re-identification
Person Re-Identification
Resilient Load Restoration in Microgrids Considering Mobile Energy Storage Fleets: A Deep Reinforcement Learning Approach
Mobile energy storage systems (MESSs) provide mobility and flexibility to enhance distribution system resilience.
CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion
In this paper, we propose CSPN++, which further improves its effectiveness and efficiency by learning adaptive convolutional kernel sizes and the number of iterations for the propagation, thus the context and computational resources needed at each pixel could be dynamically assigned upon requests.
To Balance or Not to Balance: A Simple-yet-Effective Approach for Learning with Long-Tailed Distributions
Such imbalanced distribution causes a great challenge for learning a deep neural network, which can be boiled down into a dilemma: on the one hand, we prefer to increase the exposure of tail class samples to avoid the excessive dominance of head classes in the classifier training.
Using Sampled Network Data With The Autologistic Actor Attribute Model
Social science research increasingly benefits from statistical methods for understanding the structured nature of social life, including for social network data.
Social and Information Networks Methodology
Deep Domain Adaptive Object Detection: a Survey
Deep learning (DL) based object detection has achieved great progress.
Cops-Ref: A new Dataset and Task on Compositional Referring Expression Comprehension
To bridge the gap, we propose a new dataset for visual reasoning in context of referring expression comprehension with two main features.
TEDL: A Text Encryption Method Based on Deep Learning
Firstly, both communication parties establish a word vector table by training a deep learning model according to specified hyperparameters.
Toward Interpretability of Dual-Encoder Models for Dialogue Response Suggestions
To improve the interpretability in the dual encoder models, we design a novel regularization loss to minimize the mutual information between unimportant words and desired labels, in addition to the original attention method, so that important words are emphasized while unimportant words are de-emphasized.
Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System
Prior work in standardized science exams requires support from large text corpus, such as targeted science corpus fromWikipedia or SimpleWikipedia.
A Robust Attentional Framework for License Plate Recognition in the Wild
On the other hand, the 2D attentional based license plate recognizer with an Xception-based CNN encoder is capable of recognizing license plates with different patterns under various scenarios accurately and robustly.
Non-Convex Exact Community Recovery in Stochastic Block Model
Community detection in graphs that are generated according to stochastic block models (SBMs) has received much attention lately.
ODE-CNN: Omnidirectional Depth Extension Networks
Omnidirectional 360{\deg} camera proliferates rapidly for autonomous robots since it significantly enhances the perception ability by widening the field of view(FoV).
Semi-Supervised Crowd Counting via Self-Training on Surrogate Tasks
Most existing crowd counting systems rely on the availability of the object location annotation which can be expensive to obtain.
A Nearly-Linear Time Algorithm for Exact Community Recovery in Stochastic Block Model
In this paper, we focus on the problem of exactly recovering the communities in a binary symmetric SBM, where a graph of $n$ vertices is partitioned into two equal-sized communities and the vertices are connected with probability $p = \alpha\log(n)/n$ within communities and $q = \beta\log(n)/n$ across communities for some $\alpha>\beta>0$.
Disentangled Neural Architecture Search
However, existing methods rely heavily on a black-box controller to search architectures, which suffers from the serious problem of lacking interpretability.
Few-shot Action Recognition with Implicit Temporal Alignment and Pair Similarity Optimization
Few-shot learning aims to recognize instances from novel classes with few labeled samples, which has great value in research and application.
Localization and delocalization of light in photonic moire lattices
Moire lattices consist of two identical periodic structures overlaid with a relative rotation angle.
Optics
Where to Look and How to Describe: Fashion Image Retrieval with an Attentional Heterogeneous Bilinear Network
Fashion products typically feature in compositions of a variety of styles at different clothing parts.
Quantum Dynamics of Optimization Problems
The mathematical relationship between the objective function and the wave function is established, and the quantum interpretation of the optimization problem is realized.
Fully-Automated Liver Tumor Localization and Characterization from Multi-Phase MR Volumes Using Key-Slice ROI Parsing: A Physician-Inspired Approach
Using radiological scans to identify liver tumors is crucial for proper patient treatment.
Derive Lovelock Gravity from String Theory in Cosmological Background
It was proved more than three decades ago, that the first order $\alpha'$ correction of string effective theory could be written as the Gauss-Bonnet term, which is the quadratic term of Lovelock gravity.
High Energy Physics - Theory General Relativity and Quantum Cosmology High Energy Physics - Phenomenology
MeisterMorxrc at SemEval-2020 Task 9: Fine-Tune Bert and Multitask Learning for Sentiment Analysis of Code-Mixed Tweets
Natural language processing (NLP) has been applied to various fields including text classification and sentiment analysis.
A Collaborative Visual SLAM Framework for Service Robots
We present a collaborative visual simultaneous localization and mapping (SLAM) framework for service robots.
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.
Instance and Pair-Aware Dynamic Networks for Re-Identification
The proposed model is composed of three main branches where a self-guided dynamic branch is constructed to strengthen instance-specific features, focusing on every single image.
Pluggable Weakly-Supervised Cross-View Learning for Accurate Vehicle Re-Identification
Learning cross-view consistent feature representation is the key for accurate vehicle Re-identification (ReID), since the visual appearance of vehicles changes significantly under different viewpoints.
Real-Time Visual Object Tracking via Few-Shot Learning
Visual Object Tracking (VOT) can be seen as an extended task of Few-Shot Learning (FSL).
Higher Performance Visual Tracking with Dual-Modal Localization
Visual Object Tracking (VOT) has synchronous needs for both robustness and accuracy.
Hetero-Modal Learning and Expansive Consistency Constraints for Semi-Supervised Detection from Multi-Sequence Data
Lesion detection serves a critical role in early diagnosis and has been well explored in recent years due to methodological advancesand increased data availability.
Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification
Learning discriminative image representations plays a vital role in long-tailed image classification because it can ease the classifier learning in imbalanced cases.
Ranked #10 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
An Adversarial Human Pose Estimation Network Injected with Graph Structure
Because of the invisible human keypoints in images caused by illumination, occlusion and overlap, it is likely to produce unreasonable human pose prediction for most of the current human pose estimation methods.
Residual Gaussian Process: A Tractable Nonparametric Bayesian Emulator for Multi-fidelity Simulations
The resulting model is equipped with a closed-form solution for the predictive posterior, making it applicable to advanced, high-dimensional tasks that require uncertainty estimation.
PURE: Passive mUlti-peRson idEntification via Deep Footstep Separation and Recognition
Recently, \textit{passive behavioral biometrics} (e. g., gesture or footstep) have become promising complements to conventional user identification methods (e. g., face or fingerprint) under special situations, yet existing sensing technologies require lengthy measurement traces and cannot identify multiple users at the same time.
CAT: Cross-Attention Transformer for One-Shot Object Detection
Given a query patch from a novel class, one-shot object detection aims to detect all instances of that class in a target image through the semantic similarity comparison.
