Search Results for author:
Found 257 papers, 92 papers with code
Variational Inference for Crowdsourcing
Crowdsourcing has become a popular paradigm for labeling large datasets.
Variational Algorithms for Marginal MAP
The marginal maximum a posteriori probability (MAP) estimation problem, which calculates the mode of the marginal posterior distribution of a subset of variables with the remaining variables marginalized, is an important inference problem in many models, such as those with hidden variables or uncertain parameters.
Scoring Workers in Crowdsourcing: How Many Control Questions are Enough?
We study the problem of estimating continuous quantities, such as prices, probabilities, and point spreads, using a crowdsourcing approach.
Variational Planning for Graph-based MDPs
The KL divergence is optimized using the belief propagation algorithm, with complexity exponential in only the cluster size of the graph.
Marginal Structured SVM with Hidden Variables
In this work, we propose the marginal structured SVM (MSSVM) for structured prediction with hidden variables.
Distributed Estimation, Information Loss and Exponential Families
Distributed learning of probabilistic models from multiple data repositories with minimum communication is increasingly important.
A Convolutional Click Prediction Model
The explosion in online advertisement urges to better estimate the click prediction of ads.
A Convolutional Click Prediction Model
The explosion in online advertisement urges to better estimate the click prediction of ads.
Cheaper and Better: Selecting Good Workers for Crowdsourcing
Crowdsourcing provides a popular paradigm for data collection at scale.
Communication-efficient sparse regression: a one-shot approach
We devise a one-shot approach to distributed sparse regression in the high-dimensional setting.
Regularized Minimax Conditional Entropy for Crowdsourcing
There is a rapidly increasing interest in crowdsourcing for data labeling.
Decomposition Bounds for Marginal MAP
Marginal MAP inference involves making MAP predictions in systems defined with latent variables or missing information.
Probabilistic Variational Bounds for Graphical Models
We propose a simple Monte Carlo based inference method that augments convex variational bounds by adding importance sampling (IS).
A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation
We derive a new discrepancy statistic for measuring differences between two probability distributions based on combining Stein's identity with the reproducing kernel Hilbert space theory.
Local Perturb-and-MAP for Structured Prediction
In this work, we present a new Local Perturb-and-MAP (locPMAP) framework that replaces the global optimization with a local optimization by exploiting our observed connection between locPMAP and the pseudolikelihood of the original CRF model.
Bootstrap Model Aggregation for Distributed Statistical Learning
In distributed, or privacy-preserving learning, we are often given a set of probabilistic models estimated from different local repositories, and asked to combine them into a single model that gives efficient statistical estimation.
Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm
We propose a general purpose variational inference algorithm that forms a natural counterpart of gradient descent for optimization.
Context-aware Sequential Recommendation
Recently, Recurrent Neural Networks (RNN) based methods have been successfully applied in several sequential modeling tasks.
ICE: Information Credibility Evaluation on Social Media via Representation Learning
In this paper, we propose a novel representation learning method, Information Credibility Evaluation (ICE), to learn representations of information credibility on social media.
Black-box Importance Sampling
Importance sampling is widely used in machine learning and statistics, but its power is limited by the restriction of using simple proposals for which the importance weights can be tractably calculated.
Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
We propose a simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference.
Ranked #19 on
Conditional Image Generation
on CIFAR-10
(Inception score metric)
Two Methods For Wild Variational Inference
Variational inference provides a powerful tool for approximate probabilistic in- ference on complex, structured models.
Approximate Inference with Amortised MCMC
We propose a novel approximate inference algorithm that approximates a target distribution by amortising the dynamics of a user-selected MCMC sampler.
Stein Variational Policy Gradient
Policy gradient methods have been successfully applied to many complex reinforcement learning problems.
Stein Variational Adaptive Importance Sampling
We propose a novel adaptive importance sampling algorithm which incorporates Stein variational gradient decent algorithm (SVGD) with importance sampling (IS).
Stein Variational Gradient Descent as Gradient Flow
Stein variational gradient descent (SVGD) is a deterministic sampling algorithm that iteratively transports a set of particles to approximate given distributions, based on an efficient gradient-based update that guarantees to optimally decrease the KL divergence within a function space.
Learning Deep Energy Models: Contrastive Divergence vs. Amortized MLE
We propose a number of new algorithms for learning deep energy models and demonstrate their properties.
Learning to Draw Samples with Amortized Stein Variational Gradient Descent
We propose a simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference.
Energy-efficient Amortized Inference with Cascaded Deep Classifiers
Deep neural networks have been remarkable successful in various AI tasks but often cast high computation and energy cost for energy-constrained applications such as mobile sensing.
Learning Infinite RBMs with Frank-Wolfe
In this work, we propose an infinite restricted Boltzmann machine~(RBM), whose maximum likelihood estimation~(MLE) corresponds to a constrained convex optimization.
Stochastic Variance Reduction for Policy Gradient Estimation
Recent advances in policy gradient methods and deep learning have demonstrated their applicability for complex reinforcement learning problems.
Efficient Localized Inference for Large Graphical Models
We propose a new localized inference algorithm for answering marginalization queries in large graphical models with the correlation decay property.
Action-depedent Control Variates for Policy Optimization via Stein's Identity
Policy gradient methods have achieved remarkable successes in solving challenging reinforcement learning problems.
On the Discrimination-Generalization Tradeoff in GANs
When evaluated with neural distance, our bounds show that generalization is guaranteed as long as the discriminator set is small enough, regardless of the size of the generator or hypothesis set.
Stein Variational Message Passing for Continuous Graphical Models
We propose a novel distributed inference algorithm for continuous graphical models, by extending Stein variational gradient descent (SVGD) to leverage the Markov dependency structure of the distribution of interest.
Action-dependent Control Variates for Policy Optimization via Stein Identity
Policy gradient methods have achieved remarkable successes in solving challenging reinforcement learning problems.
Adaptive Scan Gibbs Sampler for Large Scale Inference Problems
For large scale on-line inference problems the update strategy is critical for performance.
BEBP: An Poisoning Method Against Machine Learning Based IDSs
In this paper, we adopt the Edge Pattern Detection (EPD) algorithm to design a novel poisoning method that attack against several machine learning algorithms used in IDSs.
Learning to Explore with Meta-Policy Gradient
The performance of off-policy learning, including deep Q-learning and deep deterministic policy gradient (DDPG), critically depends on the choice of the exploration policy.
Learning Self-Imitating Diverse Policies
Improving the efficiency of RL algorithms in real-world problems with sparse or episodic rewards is therefore a pressing need.
Stein Variational Gradient Descent Without Gradient
Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for complex distributions.
Goodness-of-fit Testing for Discrete Distributions via Stein Discrepancy
Recent work has combined Stein’s method with reproducing kernel Hilbert space theory to develop nonparametric goodness-of-fit tests for un-normalized probability distributions.
Learning to Explore via Meta-Policy Gradient
The performance of off-policy learning, including deep Q-learning and deep deterministic policy gradient (DDPG), critically depends on the choice of the exploration policy.
Off-Policy Evaluation and Learning from Logged Bandit Feedback: Error Reduction via Surrogate Policy
Such an error reduction phenomenon is somewhat surprising as the estimated surrogate policy is less accurate than the given historical policy.
Shrinkage-based Bias-Variance Trade-off for Deep Reinforcement Learning
Deep reinforcement learning has achieved remarkable successes in solving various challenging artificial intelligence tasks.
Network localization is unalterable by infections in bursts
To shed light on the disease localization phenomenon, we study a bursty susceptible-infected-susceptible (SIS) model and analyze the model under the mean-field approximation.
Physics and Society Social and Information Networks
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel
We prove that for infinite-width two-layer nets, noisy gradient descent optimizes the regularized neural net loss to a global minimum in polynomial iterations.
Stein Variational Gradient Descent as Moment Matching
Stein variational gradient descent (SVGD) is a non-parametric inference algorithm that evolves a set of particles to fit a given distribution of interest.
Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation
We consider the off-policy estimation problem of estimating the expected reward of a target policy using samples collected by a different behavior policy.
Variational Inference with Tail-adaptive f-Divergence
Variational inference with {\alpha}-divergences has been widely used in modern probabilistic machine learning.
On the Margin Theory of Feedforward Neural Networks
We establish: 1) for multi-layer feedforward relu networks, the global minimizer of a weakly-regularized cross-entropy loss has the maximum normalized margin among all networks, 2) as a result, increasing the over-parametrization improves the normalized margin and generalization error bounds for deep networks.
A Kernel Loss for Solving the Bellman Equation
Value function learning plays a central role in many state-of-the-art reinforcement-learning algorithms.
Sequence Modeling of Temporal Credit Assignment for Episodic Reinforcement Learning
Recent advances in deep reinforcement learning algorithms have shown great potential and success for solving many challenging real-world problems, including Go game and robotic applications.
Exploration via Hindsight Goal Generation
Goal-oriented reinforcement learning has recently been a practical framework for robotic manipulation tasks, in which an agent is required to reach a certain goal defined by a function on the state space.
Improving Neural Language Modeling via Adversarial Training
Theoretically, we show that our adversarial mechanism effectively encourages the diversity of the embedding vectors, helping to increase the robustness of models.
Ranked #5 on
Language Modelling
on Penn Treebank (Word Level)
Learning Belief Representations for Imitation Learning in POMDPs
We consider the problem of imitation learning from expert demonstrations in partially observable Markov decision processes (POMDPs).
Training Robust Deep Neural Networks via Adversarial Noise Propagation
Various adversarial defense methods have accordingly been developed to improve adversarial robustness for deep models.
Filling the Soap Bubbles: Efficient Black-Box Adversarial Certification with Non-Gaussian Smoothing
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning.
Statistical Adaptive Stochastic Optimization
We investigate statistical methods for automatically scheduling the learning rate (step size) in stochastic optimization.
Automatically Learning Feature Crossing from Model Interpretation for Tabular Data
Automatically feature generation is a major topic of automated machine learning.
Splitting Steepest Descent for Growing Neural Architectures
We develop a progressive training approach for neural networks which adaptively grows the network structure by splitting existing neurons to multiple off-springs.
Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent
Designing energy-efficient networks is of critical importance for enabling state-of-the-art deep learning in mobile and edge settings where the computation and energy budgets are highly limited.
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation
Our method is doubly robust in that the bias vanishes when either the density ratio or the value function estimation is perfect.
Stein Variational Gradient Descent With Matrix-Valued Kernels
Stein variational gradient descent (SVGD) is a particle-based inference algorithm that leverages gradient information for efficient approximate inference.
Learning Preferences and Demands in Visual Recommendation
For modeling users' demands on different categories of items, the problem can be formulated as recommendation with contextual and sequential information.
Implicit Regularization and Convergence for Weight Normalization
For certain stepsizes of g and w , we show that they can converge close to the minimum norm solution.
The Black-Scholes-Merton dual equation
We derive the Black-Scholes-Merton dual equation, which has exactly the same form as the Black-Scholes-Merton equation.
Unifying Graph Convolutional Networks as Matrix Factorization
In recent years, substantial progress has been made on graph convolutional networks (GCN).
Deep Interaction Machine: A Simple but Effective Model for High-order Feature Interactions
IM is an efficient and exact implementation of high-order FM, whose time complexity linearly grows with the order of interactions and the number of feature fields.
Deep Interaction Machine: A Simple but Effective Model for High-order Feature Interactions
IM is an efficient and exact implementation of high-order FM, whose time complexity linearly grows with the order of interactions and the number of feature fields.
Attributed Multi-Relational Attention Network for Fact-checking URL Recommendation
To combat fake news, researchers mostly focused on detecting fake news and journalists built and maintained fact-checking sites (e. g., Snopes. com and Politifact. com).
Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision
We propose multipoint quantization, a quantization method that approximates a full-precision weight vector using a linear combination of multiple vectors of low-bit numbers; this is in contrast to typical quantization methods that approximate each weight using a single low precision number.
MaxUp: A Simple Way to Improve Generalization of Neural Network Training
The idea is to generate a set of augmented data with some random perturbations or transforms and minimize the maximum, or worst case loss over the augmented data.
Ranked #187 on
Image Classification
on ImageNet
Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Framework
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning.
Detection and Classification of Astronomical Targets with Deep Neural Networks in Wide Field Small Aperture Telescopes
To increase the generalization ability of our framework, we use both simulated and real observation images to train the neural network.
Stein Self-Repulsive Dynamics: Benefits From Past Samples
Our idea is to introduce Stein variational gradient as a repulsive force to push the samples of Langevin dynamics away from the past trajectories.
Statistical Adaptive Stochastic Gradient Methods
We propose a statistical adaptive procedure called SALSA for automatically scheduling the learning rate (step size) in stochastic gradient methods.
Stein Variational Inference for Discrete Distributions
In addition, such transform can be straightforwardly employed in gradient-free kernelized Stein discrepancy to perform goodness-of-fit (GOF) test on discrete distributions.
Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection
This differs from the existing methods based on backward elimination, which remove redundant neurons from the large network.
Med7: a transferable clinical natural language processing model for electronic health records
In this work we introduced a named-entity recognition model for clinical natural language processing.
Medical Named Entity Recognition
named-entity-recognition
+4
Steepest Descent Neural Architecture Optimization: Escaping Local Optimum with Signed Neural Splitting
Recently, Liu et al.[19] proposed a splitting steepest descent (S2D) method that jointly optimizes the neural parameters and architectures based on progressively growing network structures by splitting neurons into multiple copies in a steepest descent fashion.
Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning
Off-policy estimation for long-horizon problems is important in many real-life applications such as healthcare and robotics, where high-fidelity simulators may not be available and on-policy evaluation is expensive or impossible.
Dimension Independent Generalization Error by Stochastic Gradient Descent
One classical canon of statistics is that large models are prone to overfitting, and model selection procedures are necessary for high dimensional data.
An Empirical Study on Feature Discretization
When dealing with continuous numeric features, we usually adopt feature discretization.
TAGNN: Target Attentive Graph Neural Networks for Session-based Recommendation
However, these methods compress a session into one fixed representation vector without considering the target items to be predicted.
Ranked #3 on
Session-Based Recommendations
on yoochoose1
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
For security reasons, it is of critical importance to develop models with certified robustness that can provably guarantee that the prediction is can not be altered by any possible synonymous word substitution.
Deep Graph Contrastive Representation Learning
Moreover, our unsupervised method even surpasses its supervised counterparts on transductive tasks, demonstrating its great potential in real-world applications.
Ranked #1 on
Node Classification
on DBLP
TFNet: Multi-Semantic Feature Interaction for CTR Prediction
The CTR (Click-Through Rate) prediction plays a central role in the domain of computational advertising and recommender systems.
Ranked #31 on
Click-Through Rate Prediction
on Criteo
Go Wide, Then Narrow: Efficient Training of Deep Thin Networks
This is achieved by layerwise imitation, that is, forcing the thin network to mimic the intermediate outputs of the wide network from layer to layer.
Simplification of Graph Convolutional Networks: A Matrix Factorization-based Perspective
Moreover, we have also conducted experiments on a typical task of graph embedding, i. e., community detection, and the proposed UCMF model outperforms several representative graph embedding models.
Deep Active Learning by Model Interpretability
In this paper, inspired by piece-wise linear interpretability in DNN, we introduce the linearly separable regions of samples to the problem of active learning, and propose a novel Deep Active learning approach by Model Interpretability (DAMI).
Accountable Off-Policy Evaluation With Kernel Bellman Statistics
We consider off-policy evaluation (OPE), which evaluates the performance of a new policy from observed data collected from previous experiments, without requiring the execution of the new policy.
DeepSlicing: Deep Reinforcement Learning Assisted Resource Allocation for Network Slicing
Network slicing enables multiple virtual networks run on the same physical infrastructure to support various use cases in 5G and beyond.
Disentangled Item Representation for Recommender Systems
In this way, the items are represented at the attribute level, which can provide fine-grained information of items in recommendation.
DNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data
Simple classifiers, e. g., Logistic Regression (LR), are globally interpretable, but not powerful enough to model complex nonlinear interactions among features in tabular data.
Modeling Dyadic Conversations for Personality Inference
We adjust the formulation of each layer of a conventional GRU with sequence to sequence learning and personal information of both sides of the conversation.
An efficient representation of chronological events in medical texts
In this work we addressed the problem of capturing sequential information contained in longitudinal electronic health records (EHRs).
Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
To the best of our knowledge, this is the first work to provide in-depth analysis and discussion of applying pruning to online recommendation systems with non-stationary data distribution.
Population Gradients improve performance across data-sets and architectures in object classification
The most successful methods such as ReLU transfer functions, batch normalization, Xavier initialization, dropout, learning rate decay, or dynamic optimizers, have become standards in the field due, particularly, to their ability to increase the performance of Neural Networks (NNs) significantly and in almost all situations.
Graph Contrastive Learning with Adaptive Augmentation
On the node attribute level, we corrupt node features by adding more noise to unimportant node features, to enforce the model to recognize underlying semantic information.
Off-Policy Interval Estimation with Lipschitz Value Iteration
Off-policy evaluation provides an essential tool for evaluating the effects of different policies or treatments using only observed data.
Greedy Optimization Provably Wins the Lottery: Logarithmic Number of Winning Tickets is Enough
Despite the great success of deep learning, recent works show that large deep neural networks are often highly redundant and can be significantly reduced in size.
When Contrastive Learning Meets Active Learning: A Novel Graph Active Learning Paradigm with Self-Supervision
To this end, we present a minimax selection scheme that explicitly harnesses neighborhood information and discover homophilous subgraphs to facilitate active selection.
Debiasing Convolutional Neural Networks via Meta Orthogonalization
While deep learning models often achieve strong task performance, their successes are hampered by their inability to disentangle spurious correlations from causative factors, such as when they use protected attributes (e. g., race, gender, etc.)
Point Spread Function Estimation for Wide Field Small Aperture Telescopes with Deep Neural Networks and Calibration Data
Then we use these data to train a DNN (Tel--Net).
Smart obervation method with wide field small aperture telescopes for real time transient detection
The ARGUS uses a deep learning based astronomical detection algorithm implemented in embedded devices in each WFSATs to detect astronomical targets.
Certified Monotonic Neural Networks
In this work, we propose to certify the monotonicity of the general piece-wise linear neural networks by solving a mixed integer linear programming problem. This provides a new general approach for learning monotonic neural networks with arbitrary model structures.
AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
Semi-supervised learning (SSL) is a key approach toward more data-efficient machine learning by jointly leverage both labeled and unlabeled data.
KeepAugment: A Simple Information-Preserving Data Augmentation Approach
Data augmentation (DA) is an essential technique for training state-of-the-art deep learning systems.
Eudoxus: Characterizing and Accelerating Localization in Autonomous Machines
We develop and commercialize autonomous machines, such as logistic robots and self-driving cars, around the globe.
Self-Driving Cars
Hardware Architecture
LiveMap: Real-Time Dynamic Map in Automotive Edge Computing
In this paper, we propose LiveMap, a real-time dynamic map, that detects, matches, and tracks objects on the road with crowdsourcing data from connected vehicles in sub-second.
Speeding up Deep Learning Training by Sharing Weights and Then Unsharing
It has been widely observed that increasing deep learning model sizes often leads to significant performance improvements on a variety of natural language processing and computer vision tasks.
A Coach-Player Framework for Dynamic Team Composition
The performance of our method is comparable or even better than the setting where all players have a full view of the environment, but no coach.
Varying Coefficient Neural Network with Functional Targeted Regularization for Estimating Continuous Treatment Effects
With the rising abundance of observational data with continuous treatments, we investigate the problem of estimating average dose-response curve (ADRF).
Fast Training of Contrastive Learning with Intermediate Contrastive Loss
We apply our method to recently-proposed MOCO, SimCLR, SwAV and notice that we can reduce the computational cost with little loss on the performance of ImageNet linear classification and other downstream tasks.
Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction
To better model complex feature interaction, in this paper we propose a novel DisentanglEd Self-atTentIve NEtwork (DESTINE) framework for CTR prediction that explicitly decouples the computation of unary feature importance from pairwise interaction.
Coherent ray-wave structured light based on (helical) Ince-Gaussian modes
The topological evolution of classic eigenmodes including Hermite-Laguerre-Gaussian and (helical) InceGaussian modes is exploited to construct coherent state modes, which unifies the representations of travelingwave (TW) and standing-wave (SW) ray-wave structured light for the first time and realizes the TW-SW unified ray-wave geometric beam with topology of raytrajectories splitting effect, breaking the boundary of TW and SW structured light.
Optics
STAN: Spatio-Temporal Attention Network for Next Location Recommendation
The next location recommendation is at the core of various location-based applications.
Ranked #1 on
point of interests
on Gowalla
AlphaNet: Improved Training of Supernets with Alpha-Divergence
Weight-sharing NAS builds a supernet that assembles all the architectures as its sub-networks and jointly trains the supernet with the sub-networks.
Ranked #12 on
Neural Architecture Search
on ImageNet
Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks
We propose firefly neural architecture descent, a general framework for progressively and dynamically growing neural networks to jointly optimize the networks' parameters and architectures.
Centroid Transformers: Learning to Abstract with Attention
Self-attention, as the key block of transformers, is a powerful mechanism for extracting features from the inputs.
Ranked #618 on
Image Classification
on ImageNet
DNN2LR: Automatic Feature Crossing for Credit Scoring
Accordingly, we can design an automatic feature crossing method to find feature interactions in DNN, and use them as cross features in LR.
A Survey on Graph Structure Learning: Progress and Opportunities
Specifically, we first formulate a general pipeline of GSL and review state-of-the-art methods classified by the way of modeling graph structures, followed by applications of GSL across domains.
Non-asymptotic Confidence Intervals of Off-policy Evaluation: Primal and Dual Bounds
Off-policy evaluation (OPE) is the task of estimating the expected reward of a given policy based on offline data previously collected under different policies.
VCNet and Functional Targeted Regularization For Learning Causal Effects of Continuous Treatments
Motivated by the rising abundance of observational data with continuous treatments, we investigate the problem of estimating the average dose-response curve (ADRF).
GraphDIVE: Graph Classification by Mixture of Diverse Experts
Graph classification is a challenging research problem in many applications across a broad range of domains.
DyGCN: Dynamic Graph Embedding with Graph Convolutional Network
We naturally generalizes the embedding propagation scheme of GCN to dynamic setting in an efficient manner, which is to propagate the change along the graph to update node embeddings.
Dynamic Graph Neural Networks for Sequential Recommendation
We propose a new method named Dynamic Graph Neural Network for Sequential Recommendation (DGSR), which connects different user sequences through a dynamic graph structure, exploring the interactive behavior of users and items with time and order information.
Mining Latent Structures for Multimedia Recommendation
To be specific, in the proposed LATTICE model, we devise a novel modality-aware structure learning layer, which learns item-item structures for each modality and aggregates multiple modalities to obtain latent item graphs.
Vision Transformers with Patch Diversification
To alleviate this problem, in this work, we introduce novel loss functions in vision transformer training to explicitly encourage diversity across patch representations for more discriminative feature extraction.
Ranked #19 on
Semantic Segmentation
on Cityscapes val
Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition
Specifically, we 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
Sampling with Trusthworthy Constraints: A Variational Gradient Framework
In this work, we propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint specified by a general nonlinear function.
Automatic and Harmless Regularization with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach
In this work, we consider constrained optimization as a more principled approach for trading off two losses, with a special emphasis on lexicographic optimization, a degenerated limit of constrained optimization which optimizes a secondary loss inside the optimal set of the main loss.
Profiling Pareto Front With Multi-Objective Stein Variational Gradient Descent
Finding diverse and representative Pareto solutions from the Pareto front is a key challenge in multi-objective optimization (MOO).
Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations
Especially, the discovery of PDEs with highly nonlinear coefficients from low-quality data remains largely under-addressed.
Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)
Partial differential equations (PDEs) are concise and understandable representations of domain knowledge, which are essential for deepening our understanding of physical processes and predicting future responses.
MaxUp: Lightweight Adversarial Training With Data Augmentation Improves Neural Network Training
The idea is to generate a set of augmented data with some random perturbations or transforms, and minimize the maximum, or worst case loss over the augmented data.
PNet -- A Deep Learning Based Photometry and Astrometry Bayesian Framework
Time domain astronomy has emerged as a vibrant research field in recent years, focusing on celestial objects that exhibit variable magnitudes or positions.
Real-Time Anchor-Free Single-Stage 3D Detection with IoU-Awareness
Extended from our last year's award-winning model AFDet, we have made a handful of modifications to the base model, to improve the accuracy and at the same time to greatly reduce the latency.
Deep Active Learning for Text Classification with Diverse Interpretations
To train high-performing models with the minimal annotation cost, active learning is proposed to select and label the most informative samples, yet it is still challenging to measure informativeness of samples used in DNNs.
Deep Contrastive Multiview Network Embedding
In our work, different views can be obtained based on the various relations among nodes.
Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning
Recently, heterogeneous Graph Neural Networks (GNNs) have become a de facto model for analyzing HGs, while most of them rely on a relative large number of labeled data.
An Empirical Study of Graph Contrastive Learning
We envision this work to provide useful empirical evidence of effective GCL algorithms and offer several insights for future research.
Pareto Navigation Gradient Descent: a First Order Algorithm for Optimization in Pareto Set
The notion of the Pareto set allows us to focus on the set of (often infinite number of) models that cannot be strictly improved.
NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training
In this work, we observe that the poor performance is due to a gradient conflict issue: the gradients of different sub-networks conflict with that of the supernet more severely in ViTs than CNNs, which leads to early saturation in training and inferior convergence.
Ranked #7 on
Neural Architecture Search
on ImageNet
Neural Energy Minimization for Molecular Conformation Optimization
Assuming different forms of the underlying potential energy function, we can not only reinterpret and unify many of the existing models but also derive new variants of SE(3)-equivariant neural networks in a principled manner.
Speeding up Deep Model Training by Sharing Weights and Then Unsharing
Our approach exploits the special structure of BERT that contains a stack of repeated modules (i. e., transformer encoders).
Relation-aware Heterogeneous Graph for User Profiling
User profiling has long been an important problem that investigates user interests in many real applications.
Exciting-Inhibition Network for Person Reidentification in Internet of Things
Index Terms—Channel-spatial attention block (CSAB), exciting-inhibition network (EINet), Internet of Things (IoT), person reidentification (re-ID), soft batch dropblock.
Ranked #55 on
Person Re-Identification
on Market-1501
Pareto Navigation Gradient Descent: a First-Order Algorithm for Optimization in Pareto Set
The notion of the Pareto set allows us to focus on the set of (often infinite number of) models that cannot be strictly improved.
Centroid Approximation for Bootstrap: Improving Particle Quality at Inference
In this work, we propose an efficient method to explicitly \emph{optimize} a small set of high quality ``centroid'' points to better approximate the ideal bootstrap distribution.
Conflict-Averse Gradient Descent for Multi-task Learning
The goal of multi-task learning is to enable more efficient learning than single task learning by sharing model structures for a diverse set of tasks.
Latent Structure Mining with Contrastive Modality Fusion for Multimedia Recommendation
Although having access to multiple modalities might allow us to capture rich information, we argue that the simple coarse-grained fusion by linear combination or concatenation in previous work is insufficient to fully understand content information and item relationships. To this end, we propose a latent structure MIning with ContRastive mOdality fusion method (MICRO for brevity).
OnSlicing: Online End-to-End Network Slicing with Reinforcement Learning
As online learning is converged, OnSlicing reduces 12. 5% usage without any violations as compared to the state-of-the-art online DRL solution.
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts
We present a unified Vision-Language pretrained Model (VLMo) that jointly learns a dual encoder and a fusion encoder with a modular Transformer network.
Ranked #2 on
Image Retrieval
on PhotoChat
Automatic and Harmless Regularization with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach
In this work, we consider constrained optimization as a more principled approach for trading off two losses, with a special emphasis on lexicographic optimization, a degenerated limit of constrained optimization which optimizes a secondary loss inside the optimal set of the main loss.
Sampling with Trusthworthy Constraints: A Variational Gradient Framework
In this work, we propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint specified by a general nonlinear function.
Profiling Pareto Front With Multi-Objective Stein Variational Gradient Descent
Finding diverse and representative Pareto solutions from the Pareto front is a key challenge in multi-objective optimization (MOO).
argmax centroid
We propose a general method to construct centroid approximation for the distribution of maximum points of a random function (a. k. a.
FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization
We approach text-to-image generation by combining the power of the retrained CLIP representation with an off-the-shelf image generator (GANs), optimizing in the latent space of GAN to find images that achieve maximum CLIP score with the given input text.
Ranked #48 on
Text-to-Image Generation
on MS COCO
Network Compression via Central Filter
In this paper, by exploring the similarities between feature maps, we propose a novel filter pruning method, Central Filter (CF), which suggests that a filter is approximately equal to a set of other filters after appropriate adjustments.
AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds
From this observation, we have devised a single-stage anchor-free network that can fulfill these requirements.
Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements.
Operator Deep Q-Learning: Zero-Shot Reward Transferring in Reinforcement Learning
The benefit of learning the operator is that we can incorporate any new reward function as input and attain its corresponding value function in a zero-shot manner.
Evidence-aware Fake News Detection with Graph Neural Networks
In this paper, we focus on the evidence-based fake news detection, where several evidences are utilized to probe the veracity of news (i. e., a claim).
EdgeMap: CrowdSourcing High Definition Map in Automotive Edge Computing
High definition (HD) map needs to be updated frequently to capture road changes, which is constrained by limited specialized collection vehicles.
How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity
Although traditional optimization methods focus on finding a single optimal solution, most objective functions in modern machine learning problems, especially those in deep learning, often have multiple or infinite numbers of optima.
A Dual Neighborhood Hypergraph Neural Network for Change Detection in VHR Remote Sensing Images
To address this limitation, a dual neighborhood hypergraph neural network is proposed in this article, which combines the multiscale superpixel segmentation and hypergraph convolution to model and exploit the complex relationships.
A Survey on Deep Graph Generation: Methods and Applications
In this paper, we conduct a comprehensive review on the existing literature of deep graph generation from a variety of emerging methods to its wide application areas.
WCL-BBCD: A Contrastive Learning and Knowledge Graph Approach to Named Entity Recognition
The model first trains the sentence pairs in the text, calculate similarity between sentence pairs, and fine-tunes BERT used for the named entity recognition task according to the similarity, so as to alleviate word ambiguity.
Continual Learning and Private Unlearning
As intelligent agents become autonomous over longer periods of time, they may eventually become lifelong counterparts to specific people.
Estimating spot volatility under infinite variation jumps with dependent market microstructure noise
Jumps and market microstructure noise are stylized features of high-frequency financial data.
RMT-Net: Reject-aware Multi-Task Network for Modeling Missing-not-at-random Data in Financial Credit Scoring
Accordingly, we for the first time propose to model the biased credit scoring data with Multi-Task Learning (MTL).
CAINNFlow: Convolutional block Attention modules and Invertible Neural Networks Flow for anomaly detection and localization tasks
Detection of object anomalies is crucial in industrial processes, but unsupervised anomaly detection and localization is particularly important due to the difficulty of obtaining a large number of defective samples and the unpredictable types of anomalies in real life.
A Langevin-like Sampler for Discrete Distributions
We propose discrete Langevin proposal (DLP), a simple and scalable gradient-based proposal for sampling complex high-dimensional discrete distributions.
HOPE: Hierarchical Spatial-temporal Network for Occupancy Flow Prediction
In this report, we introduce our solution to the Occupancy and Flow Prediction challenge in the Waymo Open Dataset Challenges at CVPR 2022, which ranks 1st on the leaderboard.
Split Localized Conformal Prediction
The modified score inherits the spirit of split conformal methods, which is simple and efficient and can scale to high dimensional settings.
Network Pruning via Feature Shift Minimization
In this paper, we propose a novel Feature Shift Minimization (FSM) method to compress CNN models, which evaluates the feature shift by converging the information of both features and filters.
Improving Multi-Interest Network with Stable Learning
Modeling users' dynamic preferences from historical behaviors lies at the core of modern recommender systems.
Metric Residual Networks for Sample Efficient Goal-Conditioned Reinforcement Learning
Furthermore, we introduce the metric residual network (MRN) that deliberately decomposes the action-value function Q(s, a, g) into the negated summation of a metric plus a residual asymmetric component.
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
A big convergence of language, vision, and multimodal pretraining is emerging.
Ranked #1 on
Visual Reasoning
on NLVR2 Test
Second-Order Global Attention Networks for Graph Classification and Regression
Graph Neural Networks (GNNs) are powerful to learn representation of graph-structured data, which fuse both attributive and topological information.
Let us Build Bridges: Understanding and Extending Diffusion Generative Models
Diffusion-based generative models have achieved promising results recently, but raise an array of open questions in terms of conceptual understanding, theoretical analysis, algorithm improvement and extensions to discrete, structured, non-Euclidean domains.
Diffusion-based Molecule Generation with Informative Prior Bridges
AI-based molecule generation provides a promising approach to a large area of biomedical sciences and engineering, such as antibody design, hydrolase engineering, or vaccine development.
First Hitting Diffusion Models for Generating Manifold, Graph and Categorical Data
We propose a family of First Hitting Diffusion Models (FHDM), deep generative models that generate data with a diffusion process that terminates at a random first hitting time.
Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems
One of the key challenges of learning an online recommendation model is the temporal domain shift, which causes the mismatch between the training and testing data distribution and hence domain generalization error.
Deep Stable Representation Learning on Electronic Health Records
The unstable correlation between procedures and diagnoses existed in the training distribution can cause spurious correlation between historical EHR and future diagnosis.
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
The idea of rectified flow is to learn the ODE to follow the straight paths connecting the points drawn from \pi_0 and \pi_1 as much as possible.
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach
Bilevel optimization (BO) is useful for solving a variety of important machine learning problems including but not limited to hyperparameter optimization, meta-learning, continual learning, and reinforcement learning.
Rectified Flow: A Marginal Preserving Approach to Optimal Transport
We present a flow-based approach to the optimal transport (OT) problem between two continuous distributions $\pi_0,\pi_1$ on $\mathbb{R}^d$, of minimizing a transport cost $\mathbb{E}[c(X_1-X_0)]$ in the set of couplings $(X_0, X_1)$ whose marginal distributions on $X_0, X_1$ equals $\pi_0,\pi_1$, respectively, where $c$ is a cost function.
Improving Molecular Pretraining with Complementary Featurizations
Molecular pretraining, which learns molecular representations over massive unlabeled data, has become a prominent paradigm to solve a variety of tasks in computational chemistry and drug discovery.
Neural Volumetric Mesh Generator
We first propose a diffusion-based generative model to tackle this problem by generating voxelized shapes with close-to-reality outlines and structures.
Adversarial Contrastive Learning for Evidence-aware Fake News Detection with Graph Neural Networks
Comprehensive experiments have demonstrated the superiority of GETRAL over the state-of-the-arts and validated the efficacy of semantic mining with graph structure and contrastive learning.
Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descent
Sampling methods, as important inference and learning techniques, are typically designed for unconstrained domains.
The Devil is in the Conflict: Disentangled Information Graph Neural Networks for Fraud Detection
In this paper, we argue that the performance degradation is mainly attributed to the inconsistency between topology and attribute.
Sampling with Mollified Interaction Energy Descent
These energies rely on mollifier functions -- smooth approximations of the Dirac delta originated from PDE theory.
Atlas: Automate Online Service Configuration in Network Slicing
First, we design a learning-based simulator to reduce the sim-to-real discrepancy, which is accomplished by a new parameter searching method based on Bayesian optimization.
Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report
While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints.
Distributed Node Covering Optimization for Large Scale Networks and Its Application on Social Advertising
Combinatorial optimizations are usually complex and inefficient, which limits their applications in large-scale networks with billions of links.
