Search Results for author:
Found 133 papers, 63 papers with code
RAST: Domain-Robust Dialogue Rewriting as Sequence Tagging
The task of dialogue rewriting aims to reconstruct the latest dialogue utterance by copying the missing content from the dialogue context.
Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding
Efforts to enhance MLLMs, such as incorporating point cloud features, have been made, yet a considerable gap remains between the models' learned representations and the inherent complexity of 3D scenes.
Bridging Geometric States via Geometric Diffusion Bridge
The accurate prediction of geometric state evolution in complex systems is critical for advancing scientific domains such as quantum chemistry and material modeling.
TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
By treating model parameters as tokens, we replace all the linear projections in Transformers with our token-parameter attention layer, where input tokens act as queries and model parameters as keys and values.
How Numerical Precision Affects Mathematical Reasoning Capabilities of LLMs
Despite the remarkable success of Transformer-based Large Language Models (LLMs) across various domains, understanding and enhancing their mathematical capabilities remains a significant challenge.
Enhancing Temporal Modeling of Video LLMs via Time Gating
However, most existing Video LLMs neglect temporal information in video data, leading to struggles with temporal-aware video understanding.
Explainable Diagnosis Prediction through Neuro-Symbolic Integration
Diagnosis prediction is a critical task in healthcare, where timely and accurate identification of medical conditions can significantly impact patient outcomes.
Making Long-Context Language Models Better Multi-Hop Reasoners
Recent advancements in long-context modeling have enhanced language models (LMs) for complex tasks across multiple NLP applications.
Knowledge-driven AI-generated data for accurate and interpretable breast ultrasound diagnoses
In the prospective external evaluation, our diagnostic model outperforms the average performance of nine radiologists by 33. 5% in specificity with the same sensitivity, improving their performance by providing predictions with an interpretable decision-making process.
Large Language Models Struggle in Token-Level Clinical Named Entity Recognition
However, there is a significant research gap when it comes to employing token-level NER for clinical texts, especially with the use of local open-source LLMs.
GeoMFormer: A General Architecture for Geometric Molecular Representation Learning
We argue that there is a strong need for a general and flexible framework for learning both invariant and equivariant features.
Quantum Algorithms and Lower Bounds for Finite-Sum Optimization
In addition, when $F$ is nonconvex, our quantum algorithm can find an $\epsilon$-critial point using $\tilde{O}(n+\ell(d^{1/3}n^{1/3}+\sqrt{d})/\epsilon^2)$ queries.
The Power of Active Multi-Task Learning in Reinforcement Learning from Human Feedback
We demonstrate that the sample complexity of source tasks in multi-task RLHF can be reduced by considering task relevance and allocating different sample sizes to source tasks with varying task relevance.
DPO Meets PPO: Reinforced Token Optimization for RLHF
For its practical implementation, \texttt{RTO} innovatively integrates Direct Preference Optimization (DPO) and PPO.
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution prediction", diverging from the standard raster-scan "next-token prediction".
Ranked #15 on
Image Generation
on ImageNet 256x256
Beyond Embeddings: The Promise of Visual Table in Visual Reasoning
Visual representation learning has been a cornerstone in computer vision, involving typical forms such as visual embeddings, structural symbols, and text-based representations.
Ranked #85 on
Visual Question Answering
on MM-Vet
GiT: Towards Generalist Vision Transformer through Universal Language Interface
Due to its simple design, this paradigm holds promise for narrowing the architectural gap between vision and language.
Ranked #2 on
Video Captioning
on MSVD-CTN
(using extra training data)
On Cyclical MCMC Sampling
Cyclical MCMC is a novel MCMC framework recently proposed by Zhang et al. (2019) to address the challenge posed by high-dimensional multimodal posterior distributions like those arising in deep learning.
Do Efficient Transformers Really Save Computation?
Our results show that while these models are expressive enough to solve general DP tasks, contrary to expectations, they require a model size that scales with the problem size.
DOF: Accelerating High-order Differential Operators with Forward Propagation
Solving partial differential equations (PDEs) efficiently is essential for analyzing complex physical systems.
Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation
In this work, we leverage the intrinsic segmentation of language sequences and design a new positional encoding method called Bilevel Positional Encoding (BiPE).
Deep Learning with Information Fusion and Model Interpretation for Health Monitoring of Fetus based on Long-term Prenatal Electronic Fetal Heart Rate Monitoring Data
This kind of continuous monitoring, in contrast to the short-term one, collects an extended period of fetal heart data.
Beyond Weisfeiler-Lehman: A Quantitative Framework for GNN Expressiveness
Specifically, we identify a fundamental expressivity measure termed homomorphism expressivity, which quantifies the ability of GNN models to count graphs under homomorphism.
End-to-End Crystal Structure Prediction from Powder X-Ray Diffraction
XtalNet represents a significant advance in CSP, enabling the prediction of complex structures from PXRD data without the need for external databases or manual intervention.
Horizon-Free and Instance-Dependent Regret Bounds for Reinforcement Learning with General Function Approximation
To tackle long planning horizon problems in reinforcement learning with general function approximation, we propose the first algorithm, termed as UCRL-WVTR, that achieves both \emph{horizon-free} and \emph{instance-dependent}, since it eliminates the polynomial dependency on the planning horizon.
Towards Learning a Generalist Model for Embodied Navigation
We conduct extensive experiments to evaluate the performance and generalizability of our model.
3D Question Answering (3D-QA)
Embodied Question Answering
+3
Mixed-Variable Global Sensitivity Analysis For Knowledge Discovery And Efficient Combinatorial Materials Design
So far, global sensitivity studies have often been limited to design spaces with only quantitative (numerical) design variables.
A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling
However, most existing MF methods rely on the hierarchical assumption of fidelity levels or fail to capture the intercorrelation between multiple fidelity levels and utilize it to quantify the value of the future samples and navigate the adaptive sampling.
CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity
With distributed machine learning being a prominent technique for large-scale machine learning tasks, communication complexity has become a major bottleneck for speeding up training and scaling up machine numbers.
VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Control
In particular, our VL-PET-large with lightweight PET module designs significantly outperforms VL-Adapter by 2. 92% (3. 41%) and LoRA by 3. 37% (7. 03%) with BART-base (T5-base) on image-text tasks.
UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation
Jointly processing information from multiple sensors is crucial to achieving accurate and robust perception for reliable autonomous driving systems.
Ranked #8 on
3D Object Detection
on nuScenes
GRU-D-Weibull: A Novel Real-Time Individualized Endpoint Prediction
Our findings demonstrate the considerable potential of GRU-D-Weibull as the next-generation architecture for endpoint risk management, capable of generating various endpoint estimates for real-time monitoring using clinical data.
CLEVA: Chinese Language Models EVAluation Platform
With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model's capabilities has become an increasingly significant issue.
Topology-Preserving Automatic Labeling of Coronary Arteries via Anatomy-aware Connection Classifier
Automatic labeling of coronary arteries is an essential task in the practical diagnosis process of cardiovascular diseases.
Forward Laplacian: A New Computational Framework for Neural Network-based Variational Monte Carlo
Neural network-based variational Monte Carlo (NN-VMC) has emerged as a promising cutting-edge technique of ab initio quantum chemistry.
Data-Driven Design for Metamaterials and Multiscale Systems: A Review
Metamaterials are artificial materials designed to exhibit effective material parameters that go beyond those found in nature.
Boosting Breast Ultrasound Video Classification by the Guidance of Keyframe Feature Centers
The coherence loss uses the feature centers generated by the static images to guide the frame attention in the video model.
Tackling Heavy-Tailed Rewards in Reinforcement Learning with Function Approximation: Minimax Optimal and Instance-Dependent Regret Bounds
Our algorithm, termed as \textsc{Heavy-LSVI-UCB}, achieves the \emph{first} computationally efficient \emph{instance-dependent} $K$-episode regret of $\tilde{O}(d \sqrt{H \mathcal{U}^*} K^\frac{1}{1+\epsilon} + d \sqrt{H \mathcal{V}^* K})$.
Offline Meta Reinforcement Learning with In-Distribution Online Adaptation
We find a return-based uncertainty quantification for IDAQ that performs effectively.
Mining Negative Temporal Contexts For False Positive Suppression In Real-Time Ultrasound Lesion Detection
To address this issue, we propose to extract contexts from previous frames, including NTC, with the guidance of inverse optical flow.
Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective
By using circuit complexity theory, we first give impossibility results showing that bounded-depth Transformers are unable to directly produce correct answers for basic arithmetic/equation tasks unless the model size grows super-polynomially with respect to the input length.
ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning
In this work, we first investigate the effects of synthetic data in synthetic-to-real novel view synthesis and surprisingly observe that models trained with synthetic data tend to produce sharper but less accurate volume densities.
A Cross-institutional Evaluation on Breast Cancer Phenotyping NLP Algorithms on Electronic Health Records
We developed three types of NLP models (i. e., conditional random field, bi-directional long short-term memory and CancerBERT) to extract cancer phenotypes from clinical texts.
Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regret
While quantum reinforcement learning (RL) has attracted a surge of attention recently, its theoretical understanding is limited.
A Complete Expressiveness Hierarchy for Subgraph GNNs via Subgraph Weisfeiler-Lehman Tests
Recently, subgraph GNNs have emerged as an important direction for developing expressive graph neural networks (GNNs).
Ranked #1 on
Subgraph Counting - C6
on Synthetic Graph
A Reduction-based Framework for Sequential Decision Making with Delayed Feedback
We study stochastic delayed feedback in general multi-agent sequential decision making, which includes bandits, single-agent Markov decision processes (MDPs), and Markov games (MGs).
Rethinking the Expressive Power of GNNs via Graph Biconnectivity
In this paper, we take a fundamentally different perspective to study the expressive power of GNNs beyond the WL test.
DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets
However, due to the sparse characteristics of point clouds, it is non-trivial to apply a standard transformer on sparse points.
Ranked #1 on
3D Object Detection
on waymo cyclist
Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling
This is the first use of sparse convolution for 2D masked modeling.
Ranked #1 on
Instance Segmentation
on COCO 2017 val
Discrimination, calibration, and point estimate accuracy of GRU-D-Weibull architecture for real-time individualized endpoint prediction
Real-time individual endpoint prediction has always been a challenging task but of great clinic utility for both patients and healthcare providers.
A Unified Mutual Supervision Framework for Referring Expression Segmentation and Generation
Our mutual supervision contains two directions.
GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond
The proposed algorithm modifies the standard posterior sampling algorithm in two aspects: (i) we use an optimistic prior distribution that biases towards hypotheses with higher values and (ii) a loglikelihood function is set to be the empirical loss evaluated on the historical data, where the choice of loss function supports both model-free and model-based learning.
Eliciting Knowledge from Large Pre-Trained Models for Unsupervised Knowledge-Grounded Conversation
Recent advances in large-scale pre-training provide large models with the potential to learn knowledge from the raw text.
A 5G Enabled Adaptive Computing Workflow for Greener Power Grid
5G wireless technology can deliver higher data speeds, ultra low latency, more reliability, massive network capacity, increased availability, and a more uniform user experience to users.
Provable Sim-to-real Transfer in Continuous Domain with Partial Observations
Sim-to-real transfer trains RL agents in the simulated environments and then deploys them in the real world.
On the Power of Pre-training for Generalization in RL: Provable Benefits and Hardness
Generalization in Reinforcement Learning (RL) aims to learn an agent during training that generalizes to the target environment.
Denoising Masked AutoEncoders Help Robust Classification
In this paper, we propose a new self-supervised method, which is called Denoising Masked AutoEncoders (DMAE), for learning certified robust classifiers of images.
CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds
We present a novel two-stage fully sparse convolutional 3D object detection framework, named CAGroup3D.
Ranked #1 on
3D Object Detection
on SUN-RGBD
One Transformer Can Understand Both 2D & 3D Molecular Data
To achieve this goal, in this work, we develop a novel Transformer-based Molecular model called Transformer-M, which can take molecular data of 2D or 3D formats as input and generate meaningful semantic representations.
Ranked #4 on
Graph Regression
on PCQM4Mv2-LSC
Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective
Designing neural networks with bounded Lipschitz constant is a promising way to obtain certifiably robust classifiers against adversarial examples.
Check and Link: Pairwise Lesion Correspondence Guides Mammogram Mass Detection
In mammogram mass detection, modeling pairwise lesion correspondence explicitly is particularly important.
PointScatter: Point Set Representation for Tubular Structure Extraction
Inspired by this, we propose PointScatter, an alternative to the segmentation models for the tubular structure extraction task.
Boosting 3D Object Detection via Object-Focused Image Fusion
Given a set of point features and image feature maps, DeMF adaptively aggregates image features by taking the projected 2D location of the 3D point as reference.
Ranked #5 on
3D Object Detection
on SUN-RGBD val
DecoupleNet: Decoupled Network for Domain Adaptive Semantic Segmentation
Unsupervised domain adaptation in semantic segmentation has been raised to alleviate the reliance on expensive pixel-wise annotations.
Is $L^2$ Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?
In particular, we leverage the concept of stability in the literature of partial differential equation to study the asymptotic behavior of the learned solution as the loss approaches zero.
Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
We also extend our techniques to the two-player zero-sum Markov games (MGs), and establish a new performance lower bound for MGs, which tightens the existing result, and verifies the nearly minimax optimality of the proposed algorithm.
Voxel Field Fusion for 3D Object Detection
In this work, we present a conceptually simple yet effective framework for cross-modality 3D object detection, named voxel field fusion.
Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power
Moreover, we establish an improved upper bound of $\exp({\mathcal{O}}(k))$ for the network size to achieve low robust generalization error when the data lies on a manifold with intrinsic dimension $k$ ($k \ll d$).
Your Transformer May Not be as Powerful as You Expect
Extensive experiments covering typical architectures and tasks demonstrate that our model is parameter-efficient and can achieve superior performance to strong baselines in a wide range of applications.
Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation
To the best of our knowledge, this is the first theoretical result for PbRL with (general) function approximation.
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency
Structured pruning has been extensively studied on monolingual pre-trained language models and is yet to be fully evaluated on their multilingual counterparts.
Multi-View Transformer for 3D Visual Grounding
The multi-view space enables the network to learn a more robust multi-modal representation for 3D visual grounding and eliminates the dependence on specific views.
RBGNet: Ray-based Grouping for 3D Object Detection
In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers.
Ranked #13 on
3D Object Detection
on SUN-RGBD val
Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors
The proposed stochastic function is sampled from a Gaussian process prior to (1) provide infinite number of joint Gaussian distributions of random context variables (diversity-promoting) and (2) explicitly model dependency between context variables (accurate-encoding).
Stratified Transformer for 3D Point Cloud Segmentation
In this paper, we propose Stratified Transformer that is able to capture long-range contexts and demonstrates strong generalization ability and high performance.
Ranked #17 on
Semantic Segmentation
on ScanNet
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
Reconstruction Task Finds Universal Winning Tickets
Pruning well-trained neural networks is effective to achieve a promising accuracy-efficiency trade-off in computer vision regimes.
t-METASET: Tailoring Property Bias of Large-Scale Metamaterial Datasets through Active Learning
Distinctly, we seek a solution to a commonplace yet frequently overlooked scenario at early stages of data-driven design of metamaterials: when a massive (~O(10^4 )) shape-only library has been prepared with no properties evaluated.
Learning Physics-Informed Neural Networks without Stacked Back-propagation
In this work, we develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks.
Pessimistic Minimax Value Iteration: Provably Efficient Equilibrium Learning from Offline Datasets
When the dataset does not have uniform coverage over all policy pairs, finding an approximate NE involves challenges in three aspects: (i) distributional shift between the behavior policy and the optimal policy, (ii) function approximation to handle large state space, and (iii) minimax optimization for equilibrium solving.
FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows
Hence, we propose segment act, an extension of dialog act from utterance level to segment level, and crowdsource a large-scale dataset for it.
Nearly Optimal Policy Optimization with Stable at Any Time Guarantee
Policy optimization methods are one of the most widely used classes of Reinforcement Learning (RL) algorithms.
Remixing Functionally Graded Structures: Data-Driven Topology Optimization with Multiclass Shape Blending
We propose to inherit the advantages of both through a data-driven framework for multiclass functionally graded structures that mixes several families, i. e., classes, of microstructure topologies to create spatially-varying designs with guaranteed feasibility.
Breaking the Moments Condition Barrier: No-Regret Algorithm for Bandits with Super Heavy-Tailed Payoffs
Despite a large amount of effort in dealing with heavy-tailed error in machine learning, little is known when moments of the error can become non-existential: the random noise $\eta$ satisfies Pr$\left[|\eta| > |y|\right] \le 1/|y|^{\alpha}$ for some $\alpha > 0$.
Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis
Distributionally robust optimization (DRO) is a widely-used approach to learn models that are robust against distribution shift.
An Open Natural Language Processing Development Framework for EHR-based Clinical Research: A case demonstration using the National COVID Cohort Collaborative (N3C)
Although we use COVID-19 as a use case in this effort, our framework is general enough to be applied to other domains of interest in clinical NLP.
Boosting the Certified Robustness of L-infinity Distance Nets
Recently, Zhang et al. (2021) developed a new neural network architecture based on $\ell_\infty$-distance functions, which naturally possesses certified $\ell_\infty$ robustness by its construction.
Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning
The confidence bank is leveraged as an indicator to tilt training towards under-performing categories, instantiated in three strategies: 1) adaptive Copy-Paste and CutMix data augmentation approaches which give more chance for under-performing categories to be copied or cut; 2) an adaptive data sampling approach to encourage pixels from under-performing category to be sampled; 3) a simple yet effective re-weighting method to alleviate the training noise raised by pseudo-labeling.
Near-Optimal Reward-Free Exploration for Linear Mixture MDPs with Plug-in Solver
In particular, we take a plug-in solver approach, where we focus on learning a model in the exploration phase and demand that \emph{any planning algorithm} on the learned model can give a near-optimal policy.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
Understanding Domain Randomization for Sim-to-real Transfer
Reinforcement learning encounters many challenges when applied directly in the real world.
Deep Structured Instance Graph for Distilling Object Detectors
Effectively structuring deep knowledge plays a pivotal role in transfer from teacher to student, especially in semantic vision tasks.
CancerBERT: a BERT model for Extracting Breast Cancer Phenotypes from Electronic Health Records
This data used in the study included 21, 291 breast cancer patients diagnosed from 2010 to 2020, patients' clinical notes and pathology reports were collected from the University of Minnesota Clinical Data Repository (UMN).
Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision
In particular, Panoptic FCN encodes each object instance or stuff category with the proposed kernel generator and produces the prediction by convolving the high-resolution feature directly.
A fast asynchronous MCMC sampler for sparse Bayesian inference
We propose a very fast approximate Markov Chain Monte Carlo (MCMC) sampling framework that is applicable to a large class of sparse Bayesian inference problems, where the computational cost per iteration in several models is of order $O(ns)$, where $n$ is the sample size, and $s$ the underlying sparsity of the model.
Conditional Temporal Variational AutoEncoder for Action Video Prediction
To synthesize a realistic action sequence based on a single human image, it is crucial to model both motion patterns and diversity in the action video.
Collaborative Visual Navigation
As a fundamental problem for Artificial Intelligence, multi-agent system (MAS) is making rapid progress, mainly driven by multi-agent reinforcement learning (MARL) techniques.
Semi-supervised Semantic Segmentation with Directional Context-aware Consistency
Semantic segmentation has made tremendous progress in recent years.
Scalable Gaussian Processes for Data-Driven Design using Big Data with Categorical Factors
In addition, an interpretable latent space is obtained to draw insights into the effect of categorical factors, such as those associated with building blocks of architectures and element choices in metamaterial and materials design.
Multi-stage Optimization based Adversarial Training
In this work, we aim to avoid the catastrophic overfitting by introducing multi-step adversarial examples during the single-step adversarial training.
Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding
Since in many state-of-the-art models, relative positional encoding is used as default, designing efficient Transformers that can incorporate RPE is appealing.
A Reduction-Based Framework for Conservative Bandits and Reinforcement Learning
We also obtain a new upper bound for conservative low-rank MDP.
FedCM: Federated Learning with Client-level Momentum
Federated Learning is a distributed machine learning approach which enables model training without data sharing.
Self-Supervised 3D Mesh Reconstruction From Single Images
Recent single-view 3D reconstruction methods reconstruct object's shape and texture from a single image with only 2D image-level annotation.
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs
Existing pre-trained models for knowledge-graph-to-text (KG-to-text) generation simply fine-tune text-to-text pre-trained models such as BART or T5 on KG-to-text datasets, which largely ignore the graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments.
Ranked #1 on
KG-to-Text Generation
on WebQuestions
Data-Driven Multiscale Design of Cellular Composites with Multiclass Microstructures for Natural Frequency Maximization
After integrating LVGP with the density-based TO, an efficient data-driven cellular composite optimization process is developed to enable concurrent exploration of microstructure concepts and the associated volume fractions for natural frequency optimization.
Towards a Theoretical Framework of Out-of-Distribution Generalization
We also introduce a new concept of expansion function, which characterizes to what extent the variance is amplified in the test domains over the training domains, and therefore give a quantitative meaning of invariant features.
SAT: 2D Semantics Assisted Training for 3D Visual Grounding
3D visual grounding aims at grounding a natural language description about a 3D scene, usually represented in the form of 3D point clouds, to the targeted object region.
Dense Relation Distillation with Context-aware Aggregation for Few-Shot Object Detection
In this work, aiming to fully exploit features of annotated novel object and capture fine-grained features of query object, we propose Dense Relation Distillation with Context-aware Aggregation (DCNet) to tackle the few-shot detection problem.
DAGN: Discourse-Aware Graph Network for Logical Reasoning
The model encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse-aware features via a graph network for downstream QA tasks.
Ranked #24 on
Reading Comprehension
on ReClor
Revisiting Language Encoding in Learning Multilingual Representations
The language embedding can be either added to the word embedding or attached at the beginning of the sentence.
Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons
This directly provides a rigorous guarantee of the certified robustness based on the margin of prediction outputs.
Near-optimal Representation Learning for Linear Bandits and Linear RL
This paper studies representation learning for multi-task linear bandits and multi-task episodic RL with linear value function approximation.
Region-Aware Contrastive Learning for Semantic Segmentation
Inspired by recent progress in unsupervised contrastive learning, we propose the region-aware contrastive learning (RegionContrast) for semantic segmentation in the supervised manner.
Pretrain-to-Finetune Adversarial Training via Sample-wise Randomized Smoothing
For certification, we carefully allocate specific robust regions for each test sample.
Learning Contextual Perturbation Budgets for Training Robust Neural Networks
We also demonstrate that the perturbation budget generator can produce semantically-meaningful budgets, which implies that the generator can capture contextual information and the sensitivity of different features in a given image.
AT-GAN: An Adversarial Generative Model for Non-constrained Adversarial Examples
A recent work targets unrestricted adversarial example using generative model but their method is based on a search in the neighborhood of input noise, so actually their output is still constrained by input.
Robust Dialogue Utterance Rewriting as Sequence Tagging
The task of dialogue rewriting aims to reconstruct the latest dialogue utterance by copying the missing content from the dialogue context.
Fully Convolutional Networks for Panoptic Segmentation
In this paper, we present a conceptually simple, strong, and efficient framework for panoptic segmentation, called Panoptic FCN.
Ranked #1 on
Panoptic Segmentation
on COCO minival
(SQ metric)
Learnable Boundary Guided Adversarial Training
Previous adversarial training raises model robustness under the compromise of accuracy on natural data.
Ranked #1 on
Adversarial Defense
on CIFAR-100
Semi-Supervised Learning for Video Captioning
In the proposed study, we make the first attempt to train the video captioning model on labeled data and unlabeled data jointly, in a semi-supervised learning manner.
LID 2020: The Learning from Imperfect Data Challenge Results
The purpose of the Learning from Imperfect Data (LID) workshop is to inspire and facilitate the research in developing novel approaches that would harness the imperfect data and improve the data-efficiency during training.
Improved Analysis of Clipping Algorithms for Non-convex Optimization
Gradient clipping is commonly used in training deep neural networks partly due to its practicability in relieving the exploding gradient problem.
Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation
Our core innovation is the learning of a region-phrase score function, based on which an image-sentence score function is further constructed.
Deep Generative Modeling for Mechanistic-based Learning and Design of Metamaterial Systems
For microstructure design, the tuning of mechanical properties and complex manipulations of microstructures are easily achieved by simple vector operations in the latent space.
Data-Driven Topology Optimization with Multiclass Microstructures using Latent Variable Gaussian Process
With this model, we can easily obtain a continuous and differentiable transition between different microstructure concepts that can render gradient information for multiscale topology optimization.
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View.
In particular, how words in a sentence are abstracted into contexts by passing through the layers of the Transformer can be interpreted as approximating multiple particles' movement in the space using the Lie-Trotter splitting scheme and the Euler's method.
Defective Convolutional Layers Learn Robust CNNs
We first show that predictions made by the defective CNN are less dependent on textural information, but more on shape information, and further find that adversarial examples generated by the defective CNN appear to have semantic shapes.
Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation
The theory gives a complete characterization of the structure of neuron activation subspace matches, where the core concepts are maximum match and simple match which describe the overall and the finest similarity between sets of neurons in two networks respectively.
Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space
This paper explores image caption generation using conditional variational auto-encoders (CVAEs).
Learning Two-Branch Neural Networks for Image-Text Matching Tasks
Image-language matching tasks have recently attracted a lot of attention in the computer vision field.
Learning Deep Structure-Preserving Image-Text Embeddings
This paper proposes a method for learning joint embeddings of images and text using a two-branch neural network with multiple layers of linear projections followed by nonlinearities.
Ranked #14 on
Phrase Grounding
on Flickr30k Entities Test
Training Deeper Convolutional Networks with Deep Supervision
One of the most promising ways of improving the performance of deep convolutional neural networks is by increasing the number of convolutional layers.
Sufficient Conditions for Agnostic Active Learnable
We study pool-based active learning in the presence of noise, i. e. the agnostic setting.
