Search Results for author:
Found 215 papers, 91 papers with code
Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation
Such training objective is sub-optimal when the target sequence is not perfect, e. g., when the target sequence is corrupted with noises, or when only weak sequence supervision is available.
NLIP: Noise-robust Language-Image Pre-training
Large-scale cross-modal pre-training paradigms have recently shown ubiquitous success on a wide range of downstream tasks, e. g., zero-shot classification, retrieval and image captioning.
UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression
Naturally, we also present a unified multi-task Geometric Transformer framework, Geoformer, to tackle calculation and proving problems simultaneously in the form of sequence generation, which finally shows the reasoning ability can be improved on both two tasks by unifying formulation.
CoupAlign: Coupling Word-Pixel with Sentence-Mask Alignments for Referring Image Segmentation
Then in the Sentence-Mask Alignment (SMA) module, the masks are weighted by the sentence embedding to localize the referred object, and finally projected back to aggregate the pixels for the target.
3D-TOGO: Towards Text-Guided Cross-Category 3D Object Generation
Here, we make the first attempt to achieve generic text-guided cross-category 3D object generation via a new 3D-TOGO model, which integrates a text-to-views generation module and a views-to-3D generation module.
Towards Hard-pose Virtual Try-on via 3D-aware Global Correspondence Learning
Existing methods are restricted in this setting as they estimate garment warping flows mainly based on 2D poses and appearance, which omits the geometric prior of the 3D human body shape.
Structure-Preserving 3D Garment Modeling with Neural Sewing Machines
In this paper, we propose a novel Neural Sewing Machine (NSM), a learning-based framework for structure-preserving 3D garment modeling, which is capable of learning representations for garments with diverse shapes and topologies and is successfully applied to 3D garment reconstruction and controllable manipulation.
P$^3$OVD: Fine-grained Visual-Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection
Inspired by the success of visual-language methods (VLMs) in zero-shot classification, recent works attempt to extend this line of work into object detection by leveraging the localization ability of pre-trained VLMs and generating pseudo labels for unseen classes in a self-training manner.
MetaLogic: Logical Reasoning Explanations with Fine-Grained Structure
To this end, we propose a comprehensive logical reasoning explanation form.
Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers
Despite the success, its development and application on self-supervised vision transformers have been hindered by several barriers, including the high search cost, the lack of supervision, and the unsuitable search space.
MARLlib: Extending RLlib for Multi-agent Reinforcement Learning
Despite the fast development of multi-agent reinforcement learning (MARL) methods, there is a lack of commonly-acknowledged baseline implementation and evaluation platforms.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Improving Multi-turn Emotional Support Dialogue Generation with Lookahead Strategy Planning
Providing Emotional Support (ES) to soothe people in emotional distress is an essential capability in social interactions.
DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection
We further design a concept dictionary~(with descriptions) from various online sources and detection datasets to provide prior knowledge for each concept.
Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving
Aiming towards a holistic understanding of multiple downstream tasks simultaneously, there is a need for extracting features with better transferability.
ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Design
ARMANI discretizes an image into uniform tokens based on a learned cross-modal codebook in its first stage and uses a Transformer to model the distribution of image tokens for a real image given the tokens of the control signals in its second stage.
Composable Text Controls in Latent Space with ODEs
This paper proposes a new efficient approach for composable text operations in the compact latent space of text.
SiRi: A Simple Selective Retraining Mechanism for Transformer-based Visual Grounding
Particularly, SiRi conveys a significant principle to the research of visual grounding, i. e., a better initialized vision-language encoder would help the model converge to a better local minimum, advancing the performance accordingly.
PASTA-GAN++: A Versatile Framework for High-Resolution Unpaired Virtual Try-on
In this work, we take a step forwards to explore versatile virtual try-on solutions, which we argue should possess three main properties, namely, they should support unsupervised training, arbitrary garment categories, and controllable garment editing.
Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding
To bridge the gap between supervised semantic segmentation and real-world applications that acquires one model to recognize arbitrary new concepts, recent zero-shot segmentation attracts a lot of attention by exploring the relationships between unseen and seen object categories, yet requiring large amounts of densely-annotated data with diverse base classes.
Discourse-Aware Graph Networks for Textual Logical Reasoning
In this work, we propose logic structural-constraint modeling to solve the logical reasoning QA and introduce discourse-aware graph networks (DAGNs).
Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval
Our goal in this research is to study a more realistic environment in which we can conduct weakly-supervised multi-modal instance-level product retrieval for fine-grained product categories.
Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation
To endow models with the capability of incorporating expert knowledge, we propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG), in which clinical relation triples are injected into the visual features as prior knowledge to drive the decoding procedure.
Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL
In this study, we quantify the agent's behavior difference and build its relationship with the policy performance via {\bf Role Diversity}, a metric to measure the characteristics of MARL tasks.
ADAPT: Vision-Language Navigation with Modality-Aligned Action Prompts
Vision-Language Navigation (VLN) is a challenging task that requires an embodied agent to perform action-level modality alignment, i. e., make instruction-asked actions sequentially in complex visual environments.
Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection
There are two critical sensors for 3D perception in autonomous driving, the camera and the LiDAR.
ZeroGen$^+$: Self-Guided High-Quality Data Generation in Efficient Zero-Shot Learning
To address this problem, we propose a noise-robust bi-level re-weighting framework which is able to learn the per-sample weights measuring the data quality without requiring any gold data.
Knowledge Distillation via the Target-aware Transformer
To this end, we propose a novel one-to-all spatial matching knowledge distillation approach.
Unbiased Math Word Problems Benchmark for Mitigating Solving Bias
However, current solvers exist solving bias which consists of data bias and learning bias due to biased dataset and improper training strategy.
LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning
To address this issue and make a step towards interpretable MWP solving, we first construct a high-quality MWP dataset named InterMWP which consists of 11, 495 MWPs and annotates interpretable logical formulas based on algebraic knowledge as the grounded linguistic logic of each solution equation.
Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism
To make ROSETTA automatically determine which experience is available and useful, a prototypical task correlation guided Gating Diversity Controller(GDC) is introduced to adaptively adjust the diversity of gates for the new task based on class-specific prototypes.
"My nose is running.""Are you also coughing?": Building A Medical Diagnosis Agent with Interpretable Inquiry Logics
In this paper, we explore how to bring interpretability to data-driven DSMD.
Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search
It is able to find top 0. 16\% and 0. 29\% architectures on average on two search spaces under the budget of only 50 models.
Dressing in the Wild by Watching Dance Videos
While significant progress has been made in garment transfer, one of the most applicable directions of human-centric image generation, existing works overlook the in-the-wild imagery, presenting severe garment-person misalignment as well as noticeable degradation in fine texture details.
Automated Progressive Learning for Efficient Training of Vision Transformers
First, we develop a strong manual baseline for progressive learning of ViTs, by introducing momentum growth (MoGrow) to bridge the gap brought by model growth.
Beyond Fixation: Dynamic Window Visual Transformer
Recently, a surge of interest in visual transformers is to reduce the computational cost by limiting the calculation of self-attention to a local window.
Laneformer: Object-aware Row-Column Transformers for Lane Detection
We present Laneformer, a conceptually simple yet powerful transformer-based architecture tailored for lane detection that is a long-standing research topic for visual perception in autonomous driving.
elBERto: Self-supervised Commonsense Learning for Question Answering
Commonsense question answering requires reasoning about everyday situations and causes and effects implicit in context.
CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving
One main reason that impedes the development of truly reliably self-driving systems is the lack of public datasets for evaluating the performance of object detectors on corner cases.
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration
To improve the ability of fast cross-domain adaptation, we propose Prompt-based Environmental Self-exploration (ProbES), which can self-explore the environments by sampling trajectories and automatically generates structured instructions via a large-scale cross-modal pretrained model (CLIP).
Modern Augmented Reality: Applications, Trends, and Future Directions
Augmented reality (AR) is one of the relatively old, yet trending areas in the intersection of computer vision and computer graphics with numerous applications in several areas, from gaming and entertainment, to education and healthcare.
Revisiting Over-smoothing in BERT from the Perspective of Graph
Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields.
Wukong: A 100 Million Large-scale Chinese Cross-modal Pre-training Benchmark
Experiments show that Wukong can serve as a promising Chinese pre-training dataset and benchmark for different cross-modal learning methods.
Ranked #6 on
Zero-shot Image Retrieval
on COCO-CN
Exploring Inter-Channel Correlation for Diversity-preserved KnowledgeDistillation
Extensive experiments on two vision tasks, includ-ing ImageNet classification and Pascal VOC segmentation, demonstrate the superiority of our ICKD, which consis-tently outperforms many existing methods, advancing thestate-of-the-art in the fields of Knowledge Distillation.
BodyGAN: General-Purpose Controllable Neural Human Body Generation
This is because current methods mainly rely on a single pose/appearance model, which is limited in disentangling various poses and appearance in human images.
Contrastive Instruction-Trajectory Learning for Vision-Language Navigation
The vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction.
Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN
Image-based virtual try-on is one of the most promising applications of human-centric image generation due to its tremendous real-world potential.
FILIP: Fine-grained Interactive Language-Image Pre-Training
In this paper, we introduce a large-scale Fine-grained Interactive Language-Image Pre-training (FILIP) to achieve finer-level alignment through a cross-modal late interaction mechanism, which uses a token-wise maximum similarity between visual and textual tokens to guide the contrastive objective.
UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model
However, the popular DensePose-COCO dataset relies on a sophisticated manual annotation system, leading to severe limitations in acquiring the denser and more accurate annotated pose resources.
Image Comes Dancing with Collaborative Parsing-Flow Video Synthesis
The integration of human parsing and appearance flow effectively guides the generation of video frames with realistic appearance.
IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
Also, we develop a strong IconQA baseline Patch-TRM that applies a pyramid cross-modal Transformer with input diagram embeddings pre-trained on the icon dataset.
Ranked #1 on
Visual Question Answering
on IconQA
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers
Dynamic networks have shown their promising capability in reducing theoretical computation complexity by adapting their architectures to the input during inference.
Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition
Specifically, we unify a pre-trained acoustic model (wav2vec 2. 0) and a language model (BERT) into an end-to-end trainable framework.
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation
In this paper, we have a critical insight that improving the feed-forward network (FFN) in BERT has a higher gain than improving the multi-head attention (MHA) since the computational cost of FFN is 2$\sim$3 times larger than MHA.
M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining
Despite the potential of multi-modal pre-training to learn highly discriminative feature representations from complementary data modalities, current progress is being slowed by the lack of large-scale modality-diverse datasets.
Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection
To resolve the problems, we propose a novel second-stage module, named pyramid RoI head, to adaptively learn the features from the sparse points of interest.
Ranked #2 on
3D Object Detection
on waymo vehicle
(AP metric)
Voxel Transformer for 3D Object Detection
We present Voxel Transformer (VoTr), a novel and effective voxel-based Transformer backbone for 3D object detection from point clouds.
Ranked #3 on
3D Object Detection
on waymo vehicle
(L1 mAP metric)
Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift
We attribute this ranking correlation problem to the supernet training consistency shift, including feature shift and parameter shift.
FFA-IR: Towards an Explainable and Reliable Medical Report Generation Benchmark
Researchers have explored advanced methods from computer vision and natural language processing to incorporate medical domain knowledge for the generation of readable medical reports.
M3D-VTON: A Monocular-to-3D Virtual Try-On Network
Virtual 3D try-on can provide an intuitive and realistic view for online shopping and has a huge potential commercial value.
Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning
Medical imaging technologies, including computed tomography (CT) or chest X-Ray (CXR), are largely employed to facilitate the diagnosis of the COVID-19.
NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models
Fine-tuning from pre-trained ImageNet models has been a simple, effective, and popular approach for various computer vision tasks.
WAS-VTON: Warping Architecture Search for Virtual Try-on Network
Despite recent progress on image-based virtual try-on, current methods are constraint by shared warping networks and thus fail to synthesize natural try-on results when faced with clothing categories that require different warping operations.
Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-modal Pretraining
In this paper, we investigate a more realistic setting that aims to perform weakly-supervised multi-modal instance-level product retrieval among fine-grained product categories.
Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation
Specifically, we propose a Dynamic Reinforced Instruction Attacker (DR-Attacker), which learns to mislead the navigator to move to the wrong target by destroying the most instructive information in instructions at different timesteps.
AutoBERT-Zero: Evolving BERT Backbone from Scratch
Transformer-based pre-trained language models like BERT and its variants have recently achieved promising performance in various natural language processing (NLP) tasks.
Ranked #10 on
Semantic Textual Similarity
on MRPC
Deep Learning for Embodied Vision Navigation: A Survey
A navigation agent is supposed to have various intelligent skills, such as visual perceiving, mapping, planning, exploring and reasoning, etc.
Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks
Previous math word problem solvers following the encoder-decoder paradigm fail to explicitly incorporate essential math symbolic constraints, leading to unexplainable and unreasonable predictions.
Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation
Such training objective is sub-optimal when the target sequence is not perfect, e. g., when the target sequence is corrupted with noises, or when only weak sequence supervision is available.
SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Driving
Experiments show that SODA10M can serve as a promising pre-training dataset for different self-supervised learning methods, which gives superior performance when fine-tuning with different downstream tasks (i. e., detection, semantic/instance segmentation) in autonomous driving domain.
One Million Scenes for Autonomous Driving: ONCE Dataset
To facilitate future research on exploiting unlabeled data for 3D detection, we additionally provide a benchmark in which we reproduce and evaluate a variety of self-supervised and semi-supervised methods on the ONCE dataset.
Prototypical Graph Contrastive Learning
However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer from the critical sampling bias issue, i. e., the negatives likely having the same semantic structure with the query, leading to performance degradation.
Vision-Language Navigation with Random Environmental Mixup
Then, we cross-connect the key views of different scenes to construct augmented scenes.
Ranked #35 on
Vision and Language Navigation
on VLN Challenge
Towards Quantifiable Dialogue Coherence Evaluation
To address these limitations, we propose Quantifiable Dialogue Coherence Evaluation (QuantiDCE), a novel framework aiming to train a quantifiable dialogue coherence metric that can reflect the actual human rating standards.
GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning
Therefore, we propose a Geometric Question Answering dataset GeoQA, containing 4, 998 geometric problems with corresponding annotated programs, which illustrate the solving process of the given problems.
TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search
While existing NAS methods mostly design architectures on a single task, algorithms that look beyond single-task search are surging to pursue a more efficient and universal solution across various tasks.
Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning
We further propose a novel geometry solving approach with formal language and symbolic reasoning, called Interpretable Geometry Problem Solver (Inter-GPS).
Ranked #1 on
Mathematical Question Answering
on GeoS
SOON: Scenario Oriented Object Navigation with Graph-based Exploration
In this task, an agent is required to navigate from an arbitrary position in a 3D embodied environment to localize a target following a scene description.
DAGN: Discourse-Aware Graph Network for Logical Reasoning
The model encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse-aware features via a graph network for downstream QA tasks.
Ranked #24 on
Reading Comprehension
on ReClor
Dynamic Slimmable Network
Here, we explore a dynamic network slimming regime, named Dynamic Slimmable Network (DS-Net), which aims to achieve good hardware-efficiency via dynamically adjusting filter numbers of networks at test time with respect to different inputs, while keeping filters stored statically and contiguously in hardware to prevent the extra burden.
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search
In this work, we present Block-wisely Self-supervised Neural Architecture Search (BossNAS), an unsupervised NAS method that addresses the problem of inaccurate architecture rating caused by large weight-sharing space and biased supervision in previous methods.
Ranked #1 on
Neural Architecture Search
on ImageNet
(Top 5 Accuracy metric)
A Data-Centric Framework for Composable NLP Workflows
Empirical natural language processing (NLP) systems in application domains (e. g., healthcare, finance, education) involve interoperation among multiple components, ranging from data ingestion, human annotation, to text retrieval, analysis, generation, and visualization.
SparseBERT: Rethinking the Importance Analysis in Self-attention
A surprising result is that diagonal elements in the attention map are the least important compared with other attention positions.
Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search
For object detection, the well-established classification and regression loss functions have been carefully designed by considering diverse learning challenges.
GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer
Non-parallel text style transfer has attracted increasing research interests in recent years.
Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer
Prior highly-tuned image parsing models are usually studied in a certain domain with a specific set of semantic labels and can hardly be adapted into other scenarios (e. g., sharing discrepant label granularity) without extensive re-training.
UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers
Recent advances in multi-agent reinforcement learning have been largely limited in training one model from scratch for every new task.
Unifying Relational Sentence Generation and Retrieval for Medical Image Report Composition
Beyond generating long and topic-coherent paragraphs in traditional captioning tasks, the medical image report composition task poses more task-oriented challenges by requiring both the highly-accurate medical term diagnosis and multiple heterogeneous forms of information including impression and findings.
Erasure for Advancing: Dynamic Self-Supervised Learning for Commonsense Reasoning
To solve this issue, we propose a novel DynamIc Self-sUperviSed Erasure (DISUSE) which adaptively erases redundant and artifactual clues in the context and questions to learn and establish the correct corresponding pair relations between the questions and their clues.
CAT-SAC: Soft Actor-Critic with Curiosity-Aware Entropy Temperature
The curiosity is added to the target entropy to increase the entropy temperature for unfamiliar states and decrease the target entropy for familiar states.
NASOA: Towards Faster Task-oriented Online Fine-tuning
The resulting model zoo is more training efficient than SOTA NAS models, e. g. 6x faster than RegNetY-16GF, and 1. 7x faster than EfficientNetB3.
TransNAS-Bench-101: Improving Transferrability and Generalizability of Cross-Task Neural Architecture Search
While existing NAS methods mostly design architectures on one single task, algorithms that look beyond single-task search are surging to pursue a more efficient and universal solution across various tasks.
UPDeT: Universal Multi-agent RL via Policy Decoupling with Transformers
Recent advances in multi-agent reinforcement learning have been largely limited in training one model from scratch for every new task.
Exploring Inter-Channel Correlation for Diversity-Preserved Knowledge Distillation
Extensive experiments on two vision tasks, including ImageNet classification and Pascal VOC segmentation, demonstrate the superiority of our ICKD, which consistently outperforms many existing methods, advancing the state-of-the-art in the fields of Knowledge Distillation.
Ranked #8 on
Knowledge Distillation
on ImageNet
Linguistically Routing Capsule Network for Out-of-Distribution Visual Question Answering
The experimental results show that our proposed method can improve current VQA models on OOD split without losing performance on the in-domain test data.
Self-Motivated Communication Agent for Real-World Vision-Dialog Navigation
Vision-Dialog Navigation (VDN) requires an agent to ask questions and navigate following the human responses to find target objects.
Exploring Geometry-Aware Contrast and Clustering Harmonization for Self-Supervised 3D Object Detection
Here we present a novel self-supervised 3D Object detection framework that seamlessly integrates the geometry-aware contrast and clustering harmonization to lift the unsupervised 3D representation learning, named GCC-3D.
REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement
It is crucial since the quality of the evidence is the key to answering commonsense questions, and even determines the upper bound on the QA systems performance.
Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation
Besides, we develop a Graph-Evolving Meta-Learning (GEML) framework that learns to evolve the commonsense graph for reasoning disease-symptom correlations in a new disease, which effectively alleviates the needs of a large number of dialogues.
Adversarial Meta Sampling for Multilingual Low-Resource Speech Recognition
When sampling tasks in MML-ASR, AMS adaptively determines the task sampling probability for each source language.
Knowledge-Routed Visual Question Reasoning: Challenges for Deep Representation Embedding
Specifically, we generate the question-answer pair based on both the Visual Genome scene graph and an external knowledge base with controlled programs to disentangle the knowledge from other biases.
Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation
Panoptic segmentation that unifies instance segmentation and semantic segmentation has recently attracted increasing attention.
Ranked #16 on
Panoptic Segmentation
on COCO test-dev
AutoSync: Learning to Synchronize for Data-Parallel Distributed Deep Learning
Synchronization is a key step in data-parallel distributed machine learning (ML).
Continuous Transition: Improving Sample Efficiency for Continuous Control Problems via MixUp
Although deep reinforcement learning (RL) has been successfully applied to a variety of robotic control tasks, it's still challenging to apply it to real-world tasks, due to the poor sample efficiency.
Towards Robust Partially Supervised Multi-Structure Medical Image Segmentation on Small-Scale Data
To bridge the methodological gaps in partially supervised learning (PSL) under data scarcity, we propose Vicinal Labels Under Uncertainty (VLUU), a simple yet efficient framework utilizing the human structure similarity for partially supervised medical image segmentation.
Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation
In this work, we propose an efficient, cooperative and highly automated framework to simultaneously search for all main components including backbone, segmentation branches, and feature fusion module in a unified panoptic segmentation pipeline based on the prevailing one-shot Network Architecture Search (NAS) paradigm.
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables
In this paper, we develop a general framework for interpretable natural language understanding that requires only a small set of human annotated explanations for training.
Iterative Graph Self-Distillation
Recently, there has been increasing interest in the challenge of how to discriminatively vectorize graphs.
MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generation
To push forward the future research on building expert-sensitive medical dialogue system, we proposes two kinds of medical dialogue tasks based on MedDG dataset.
Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems
A practical automatic textual math word problems (MWPs) solver should be able to solve various textual MWPs while most existing works only focused on one-unknown linear MWPs.
Ranked #9 on
Math Word Problem Solving
on Math23K
GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems
Capitalized on the topic-level dialogue graph, we propose a new evaluation metric GRADE, which stands for Graph-enhanced Representations for Automatic Dialogue Evaluation.
CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending
In this paper, we propose a novel lane-sensitive architecture search framework named CurveLane-NAS to automatically capture both long-ranged coherent and accurate short-range curve information while unifying both architecture search and post-processing on curve lane predictions via point blending.
Ranked #4 on
Lane Detection
on CurveLanes
CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search
In spite of its remarkable progress, many algorithms are restricted to particular search spaces.
Ranked #12 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation
Firstly, the regions of primary interest to radiologists are usually located in a small area of the global image, meaning that the remainder parts of the image could be considered as irrelevant noise in the training procedure.
Bidirectional Graph Reasoning Network for Panoptic Segmentation
We introduce a Bidirectional Graph Reasoning Network (BGRNet), which incorporates graph structure into the conventional panoptic segmentation network to mine the intra-modular and intermodular relations within and between foreground things and background stuff classes.
Linguistically Driven Graph Capsule Network for Visual Question Reasoning
Inspired by the property of a capsule network that can carve a tree structure inside a regular convolutional neural network (CNN), we propose a hierarchical compositional reasoning model called the "Linguistically driven Graph Capsule Network", where the compositional process is guided by the linguistic parse tree.
Vision-Dialog Navigation by Exploring Cross-modal Memory
Benefiting from the collaborative learning of the L-mem and the V-mem, our CMN is able to explore the memory about the decision making of historical navigation actions which is for the current step.
Towards Causality-Aware Inferring: A Sequential Discriminative Approach for Medical Diagnosis
To eliminate this bias and inspired by the propensity score matching technique with causal diagram, we propose a propensity-based patient simulator to effectively answer unrecorded inquiry by drawing knowledge from the other records; Bias (ii) inherently comes along with the passively collected data, and is one of the key obstacles for training the agent towards "learning how" rather than "remembering what".
ElixirNet: Relation-aware Network Architecture Adaptation for Medical Lesion Detection
Is a hand-crafted detection network tailored for natural image undoubtedly good enough over a discrepant medical lesion domain?
Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN
Finally, an InterDomain Transfer Module is proposed to exploit diverse transfer dependencies across all domains and enhance the regional feature representation by attending and transferring semantic contexts globally.
Dynamic Knowledge Routing Network For Target-Guided Open-Domain Conversation
Target-guided open-domain conversation aims to proactively and naturally guide a dialogue agent or human to achieve specific goals, topics or keywords during open-ended conversations.
Blockwisely Supervised Neural Architecture Search with Knowledge Distillation
Moreover, we find that the knowledge of a network model lies not only in the network parameters but also in the network architecture.
Ranked #1 on
Neural Architecture Search
on CIFAR-100
SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection
In this paper, we present a two-stage coarse-to-fine searching strategy named Structural-to-Modular NAS (SM-NAS) for searching a GPU-friendly design of both an efficient combination of modules and better modular-level architecture for object detection.
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
In this paper, we introduce Auxiliary Reasoning Navigation (AuxRN), a framework with four self-supervised auxiliary reasoning tasks to take advantage of the additional training signals derived from the semantic information.
Ranked #11 on
Vision and Language Navigation
on VLN Challenge
Heterogeneous Graph Learning for Visual Commonsense Reasoning
Our HGL consists of a primal vision-to-answer heterogeneous graph (VAHG) module and a dual question-to-answer heterogeneous graph (QAHG) module to interactively refine reasoning paths for semantic agreement.
Layout-Graph Reasoning for Fashion Landmark Detection
Each Layout-Graph Reasoning(LGR) layer aims to map feature representations into structural graph nodes via a Map-to-Node module, performs reasoning over structural graph nodes to achieve global layout coherency via a layout-graph reasoning module, and then maps graph nodes back to enhance feature representations via a Node-to-Map module.
Meta R-CNN : Towards General Solver for Instance-level Few-shot Learning
Resembling the rapid learning capability of human, few-shot learning empowers vision systems to understand new concepts by training with few samples.
Ranked #15 on
Few-Shot Object Detection
on MS-COCO (30-shot)
Explainable High-order Visual Question Reasoning: A New Benchmark and Knowledge-routed Network
Explanation and high-order reasoning capabilities are crucial for real-world visual question answering with diverse levels of inference complexity (e. g., what is the dog that is near the girl playing with?)
Multivariate-Information Adversarial Ensemble for Scalable Joint Distribution Matching
A broad range of cross-$m$-domain generation researches boil down to matching a joint distribution by deep generative models (DGMs).
Blending-target Domain Adaptation by Adversarial Meta-Adaptation Networks
(Unsupervised) Domain Adaptation (DA) seeks for classifying target instances when solely provided with source labeled and target unlabeled examples for training.
Ranked #2 on
Multi-target Domain Adaptation
on Office-Home
Multi-target Domain Adaptation
Unsupervised Domain Adaptation
Fashion Editing with Adversarial Parsing Learning
Interactive fashion image manipulation, which enables users to edit images with sketches and color strokes, is an interesting research problem with great application value.
Learning Personalized Modular Network Guided by Structured Knowledge
Learning semantic configurations and activation of modules to align well with structured knowledge can be regarded as a decision-making procedure, which is solved by a new graph-based reinforcement learning algorithm.
Target-Guided Open-Domain Conversation
We study the problem of imposing conversational goals on open-domain chat agents.
Graph Transformer
Graph neural networks (GNN) have gained increasing research interests as a mean to the challenging goal of robust and universal graph learning.
Graphonomy: Universal Human Parsing via Graph Transfer Learning
By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity.
Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generation
Generating long and semantic-coherent reports to describe medical images poses great challenges towards bridging visual and linguistic modalities, incorporating medical domain knowledge, and generating realistic and accurate descriptions.
Towards Multi-pose Guided Virtual Try-on Network
Given an input person image, a desired clothes image, and a desired pose, the proposed Multi-pose Guided Virtual Try-on Network (MG-VTON) can generate a new person image after fitting the desired clothes into the input image and manipulating human poses.
Ranked #1 on
Virtual Try-on
on Deep-Fashion
End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis
Besides the challenges for conversational dialogue systems (e. g. topic transition coherency and question understanding), automatic medical diagnosis further poses more critical requirements for the dialogue rationality in the context of medical knowledge and symptom-disease relations.
Data-to-Text Generation with Style Imitation
That is, the model learns to imitate the writing style of any given exemplar sentence, with automatic adaptions to faithfully describe the content record.
Symbolic Graph Reasoning Meets Convolutions
To cooperate with local convolutions, each SGR is constituted by three modules: a) a primal local-to-semantic voting module where the features of all symbolic nodes are generated by voting from local representations; b) a graph reasoning module propagates information over knowledge graph to achieve global semantic coherency; c) a dual semantic-to-local mapping module learns new associations of the evolved symbolic nodes with local representations, and accordingly enhances local features.
Ranked #79 on
Semantic Segmentation
on ADE20K val
Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis
Despite remarkable advances in image synthesis research, existing works often fail in manipulating images under the context of large geometric transformations.
AutoLoss: Learning Discrete Schedules for Alternate Optimization
Many machine learning problems involve iteratively and alternately optimizing different task objectives with respect to different sets of parameters.
AutoLoss: Learning Discrete Schedule for Alternate Optimization
Many machine learning problems involve iteratively and alternately optimizing different task objectives with respect to different sets of parameters.
Interpretable Visual Question Answering by Reasoning on Dependency Trees
Collaborative reasoning for understanding image-question pairs is a very critical but underexplored topic in interpretable visual question answering systems.
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
The versatile toolkit also fosters technique sharing across different text generation tasks.
RCAA: Relational Context-Aware Agents for Person Search
In this paper, we address this problem by training relational context-aware agents which learn the actions to localize the target person from the gallery of whole scene images.
Adversarial Geometry-Aware Human Motion Prediction
We explore an approach to forecasting human motion in a few milliseconds given an input 3D skeleton sequence based on a recurrent encoder-decoder framework.
Generative Semantic Manipulation with Mask-Contrasting GAN
Despite the promising results on paired/unpaired image-to-image translation achieved by Generative Adversarial Networks (GANs), prior works often only transfer the low-level information (e. g. color or texture changes), but fail to manipulate high-level semantic meanings (e. g., geometric structure or content) of different object regions.
Jointly Deep Multi-View Learning for Clustering Analysis
To our best knowledge, this is the first work to model the multi-view clustering in a deep joint framework, which will provide a meaningful thinking in unsupervised multi-view learning.
Adaptive Temporal Encoding Network for Video Instance-level Human Parsing
Beyond the existing single-person and multiple-person human parsing tasks in static images, this paper makes the first attempt to investigate a more realistic video instance-level human parsing that simultaneously segments out each person instance and parses each instance into more fine-grained parts (e. g., head, leg, dress).
Instance-level Human Parsing via Part Grouping Network
Instance-level human parsing towards real-world human analysis scenarios is still under-explored due to the absence of sufficient data resources and technical difficulty in parsing multiple instances in a single pass.
Ranked #3 on
Human Part Segmentation
on CIHP
Reinforced Auto-Zoom Net: Towards Accurate and Fast Breast Cancer Segmentation in Whole-slide Images
Motivated by the zoom-in operation of a pathologist using a digital microscope, RAZN learns a policy network to decide whether zooming is required in a given region of interest.
Toward Characteristic-Preserving Image-based Virtual Try-On Network
Second, to alleviate boundary artifacts of warped clothes and make the results more realistic, we employ a Try-On Module that learns a composition mask to integrate the warped clothes and the rendered image to ensure smoothness.
Query-Conditioned Three-Player Adversarial Network for Video Summarization
Video summarization plays an important role in video understanding by selecting key frames/shots.
