Search Results for author:
Found 84 papers, 28 papers with code
Learning to Generalize from Sparse and Underspecified Rewards
The parameters of the auxiliary reward function are optimized with respect to the validation performance of a trained policy.
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
However, this progress has largely focused on the architecture of neural networks, where it has relied on sophisticated expert-designed layers as building blocks---or similarly restrictive search spaces.
The Evolved Transformer
Recent works have highlighted the strength of the Transformer architecture on sequence tasks while, at the same time, neural architecture search (NAS) has begun to outperform human-designed models.
Ranked #1 on
Machine Translation
on WMT2014 English-Czech
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
Harnessing the statistical power of neural networks to perform language understanding and symbolic reasoning is difficult, when it requires executing efficient discrete operations against a large knowledge-base.
Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing
We present Memory Augmented Policy Optimization (MAPO), a simple and novel way to leverage a memory buffer of promising trajectories to reduce the variance of policy gradient estimate.
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision
We study the open-domain named entity recognition (NER) problem under distant supervision.
GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models
Going beyond this, we propose GMMSeg, a new family of segmentation models that rely on a dense generative classifier for the joint distribution p(pixel feature, class).
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Quantization is an indispensable technique for serving Large Language Models (LLMs) and has recently found its way into LoRA fine-tuning.
Visual Abductive Reasoning
In this paper, we propose a new task and dataset, Visual Abductive Reasoning (VAR), for examining abductive reasoning ability of machine intelligence in everyday visual situations.
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
We propose MoEBERT, which uses a Mixture-of-Experts structure to increase model capacity and inference speed.
Local-Global Context Aware Transformer for Language-Guided Video Segmentation
We explore the task of language-guided video segmentation (LVS).
Ranked #7 on
Referring Expression Segmentation
on A2D Sentences
Referring Expression Segmentation
Referring Video Object Segmentation
+5
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Large Transformer-based models have exhibited superior performance in various natural language processing and computer vision tasks.
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
Analysis shows that the proposed schedule indeed reduces the redundancy and improves generalization performance.
Distractor Generation for Multiple Choice Questions Using Learning to Rank
We investigate how machine learning models, specifically ranking models, can be used to select useful distractors for multiple choice questions.
DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization
We propose DeepChannel, a robust, data-efficient, and interpretable neural model for extractive document summarization.
UniTS: A Universal Time Series Analysis Framework with Self-supervised Representation Learning
Machine learning has emerged as a powerful tool for time series analysis.
AugLiChem: Data Augmentation Library of Chemical Structures for Machine Learning
Inspired by the success of data augmentations in computer vision and natural language processing, we developed AugLiChem: the data augmentation library for chemical structures.
Definition Modeling: Learning to define word embeddings in natural language
Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks.
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
As such, TED reduces the knowledge gap between the two models and helps the student to fit better on the target task.
Contrastive Shapelet Learning for Unsupervised Multivariate Time Series Representation Learning
Recent studies have shown great promise in unsupervised representation learning (URL) for multivariate time series, because URL has the capability in learning generalizable representation for many downstream tasks without using inaccessible labels.
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
The Lottery Ticket Hypothesis suggests that an over-parametrized network consists of ``lottery tickets'', and training a certain collection of them (i. e., a subnetwork) can match the performance of the full model.
Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast
On most benchmarks, the generic GNN pre-trained by iMolCLR rivals or even surpasses supervised learning models with sophisticated architecture designs and engineered features.
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
Adversarial regularization has been shown to improve the generalization performance of deep learning models in various natural language processing tasks.
Unified Functional Hashing in Automatic Machine Learning
In this paper, we show that large efficiency gains can be obtained by employing a fast unified functional hash, especially through the functional equivalence caching technique, which we also present.
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
To retain ensemble benefits while maintaining a low memory cost, we propose a consistency-regularized ensemble learning approach based on perturbed models, named CAMERO.
ARCH: Efficient Adversarial Regularized Training with Caching
Adversarial regularization can improve model generalization in many natural language processing tasks.
Unsupervised semantic segmentation of high-resolution UAV imagery for road scene parsing
In this paper, an unsupervised road parsing framework that leverages recent advances in vision language models and fundamental computer vision model is introduced. Initially, a vision language model is employed to efficiently process ultra-large resolution UAV images to quickly detect road regions of interest in the images.
A Fully Convolutional Tri-branch Network (FCTN) for Domain Adaptation
We evaluate the proposed network on large-scale domain adaptation experiments using both synthetic (GTA) and real (Cityscapes) images.
Active Learning of Strict Partial Orders: A Case Study on Concept Prerequisite Relations
Strict partial order is a mathematical structure commonly seen in relational data.
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (Short Version)
In this work, we propose the Manager-Programmer-Computer framework, which integrates neural networks with non-differentiable memory to support abstract, scalable and precise operations through a friendly neural computer interface.
Multi-Scale FCN With Cascaded Instance Aware Segmentation for Arbitrary Oriented Word Spotting in the Wild
Scene text detection has attracted great attention these years.
Learning to Guide: Guidance Law Based on Deep Meta-learning and Model Predictive Path Integral Control
However the traditional MPPI framework assumes the actual environment similar to the training dataset for the deep neural network which is impractical in practice with different maneuvering of target, other perturbations and actuator failures.
Robotics Systems and Control
Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing
To overcome this limitation, we propose a novel multi-domain NMT model using individual modules for each domain, on which we apply word-level, adaptive and layer-wise domain mixing.
Uncovering Insurance Fraud Conspiracy with Network Learning
In order to detect and prevent fraudulent insurance claims, we developed a novel data-driven procedure to identify groups of organized fraudsters, one of the major contributions to financial losses, by learning network information.
InfDetect: a Large Scale Graph-based Fraud Detection System for E-Commerce Insurance
The insurance industry has been creating innovative products around the emerging online shopping activities.
Compositional Generalization via Neural-Symbolic Stack Machines
Despite achieving tremendous success, existing deep learning models have exposed limitations in compositional generalization, the capability to learn compositional rules and apply them to unseen cases in a systematic manner.
Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension
Integrating distributed representations with symbolic operations is essential for reading comprehension requiring complex reasoning, such as counting, sorting and arithmetics, but most existing approaches are hard to scale to more domains or more complex reasoning.
Ranked #5 on
Question Answering
on DROP Test
Evolving Machine Learning Algorithms From Scratch
However, this progress has largely focused on the architecture of neural networks, where it has relied on sophisticated expert-designed layers as building blocks---or similarly restrictive search spaces.
A Broad Learning Approach for Context-Aware Mobile Application Recommendation
With the rapid development of mobile apps, the availability of a large number of mobile apps in application stores brings challenge to locate appropriate apps for users.
Line Failure Localization of Power Networks Part I: Non-cut Outages
Transmission line failures in power systems propagate non-locally, making the control of the resulting outages extremely difficult.
Adaptive Network Response to Line Failures in Power Systems
Transmission line failures in power systems propagate and cascade non-locally.
Line Failure Localization of Power Networks Part II: Cut Set Outages
Transmission line failure in power systems prop-agate non-locally, making the control of the resulting outages extremely difficult.
PyGlove: Symbolic Programming for Automated Machine Learning
As a result, AutoML can be reformulated as an automated process of symbolic manipulation.
LinkLouvain: Link-Aware A/B Testing and Its Application on Online Marketing Campaign
In this paper, we analyze the network A/B testing problem under a real-world online marketing campaign, describe our proposed LinkLouvain method, and evaluate it on real-world data.
ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentation
Text-based video segmentation is a challenging task that segments out the natural language referred objects in videos.
Ranked #4 on
Referring Expression Segmentation
on J-HMDB
(Precision@0.9 metric)
Token-wise Curriculum Learning for Neural Machine Translation
Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of "easy" samples from training data at the early training stage.
Carbon Emissions and Large Neural Network Training
To help reduce the carbon footprint of ML, we believe energy usage and CO2e should be a key metric in evaluating models, and we are collaborating with MLPerf developers to include energy usage during training and inference in this industry standard benchmark.
Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation
Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference.
VSPW: A Large-scale Dataset for Video Scene Parsing in the Wild
To the best of our knowledge, our VSPW is the first attempt to tackle the challenging video scene parsing task in the wild by considering diverse scenarios.
Self-Training with Differentiable Teacher
In self-training, the student contributes to the prediction performance, and the teacher controls the training process by generating pseudo-labels.
Contrastive Video-Language Segmentation
We focus on the problem of segmenting a certain object referred by a natural language sentence in video content, at the core of formulating a pinpoint vision-language relation.
Neural Program Planner for Structured Predictions
We consider the problem of weakly supervised structured prediction (SP) with reinforcement learning (RL) – for example, given a database table and a question, perform a sequence of computation actions on the table, which generates a response and receives a binary success-failure reward.
S+PAGE: A Speaker and Position-Aware Graph Neural Network Model for Emotion Recognition in Conversation
Emotion recognition in conversation (ERC) has attracted much attention in recent years for its necessity in widespread applications.
Ranked #1 on
Emotion Recognition in Conversation
on DailyDialog
A General Traffic Shaping Protocol in E-Commerce
Our algorithm can straightforwardly optimize the linear programming in the prime space, and its solution can be simply applied by a stochastic strategy to fulfill the optimized objective and the constraints in expectation.
TPAD: Identifying Effective Trajectory Predictions Under the Guidance of Trajectory Anomaly Detection Model
They are lack of self-evaluation ability, that is, to examine the rationality of their prediction results, thus failing to guide users to identify high-quality ones from their candidate results.
A neural probabilistic model for context based citation recommendation
It is not always easy for knowledgeable researchers to give an accurate citation context for a cited paper or to find the right paper to cite given context.
The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink
Four best practices can reduce ML training energy by up to 100x and CO2 emissions up to 1000x.
Interface Networks for Failure Localization in Power Systems
Transmission power systems usually consist of interconnected sub-grids that are operated relatively independently.
ZHIXIAOBAO at SemEval-2022 Task 10: Apporoaching Structured Sentiment with Graph Parsing
This paper presents our submission to task 10, Structured Sentiment Analysis of the SemEval 2022 competition.
Multi-Task Mixture Density Graph Neural Networks for Predicting Cu-based Single-Atom Alloy Catalysts for CO2 Reduction Reaction
Graph neural networks (GNNs) have drawn more and more attention from material scientists and demonstrated a high capacity to establish connections between the structure and properties.
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers
Since the teacher model has a significantly larger capacity and stronger representation power than the student model, it is very difficult for the student to produce predictions that match the teacher's over a massive amount of open-domain training data.
DR-Label: Improving GNN Models for Catalysis Systems by Label Deconstruction and Reconstruction
Reversely, the model Reconstructs a more robust equilibrium state prediction by transforming edge-level predictions to node-level with a sphere-fitting algorithm.
Initial Structure to Relaxed Energy (IS2RE), Direct
Property Prediction
Optimization design of a micro-perforated panel absorber with 8.6 octave bands
In order to improve low-frequency characteristics of micro-perforated panel absorbers, sound absorption structures composed of micro-perforated panels and expansion chambers are design, and an optimization design method is constructed based on the transfer function model and the simulated annealing algorithm.
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Pruning enhances the diversity of low-rank approximations, and low-rank approximation prevents pruning from losing too many expressive neurons.
Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation
Recent advances in semi-supervised semantic segmentation have been heavily reliant on pseudo labeling to compensate for limited labeled data, disregarding the valuable relational knowledge among semantic concepts.
Towards Ubiquitous Intelligent Hand Interaction
The development of ubiquitous computing and sensing devices has brought about novel interaction scenarios such as mixed reality and IoT (e. g., smart home), which pose new demands for the next generation of natural user interfaces (NUI).
Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta
To address these challenges, we present Scaling User Modeling (SUM), a framework widely deployed in Meta's ads ranking system, designed to facilitate efficient and scalable sharing of online user representation across hundreds of ads models.
TARGET: Template-Transferable Backdoor Attack Against Prompt-based NLP Models via GPT4
Most of the existing attack methods focus on inserting manually predefined templates as triggers in the pre-training phase to train the victim model and utilize the same triggers in the downstream task to perform inference, which tends to ignore the transferability and stealthiness of the templates.
Symbolic Discovery of Optimization Algorithms
On diffusion models, Lion outperforms Adam by achieving a better FID score and reducing the training compute by up to 2. 3x.
Unsupervised Multi-modal Feature Alignment for Time Series Representation Learning
In contrast to conventional methods that fuse features from multiple modalities, our proposed approach simplifies the neural architecture by retaining a single time series encoder, consequently leading to preserved scalability.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
A Multi-day Needs-based Modeling Approach for Activity and Travel Demand Analysis
The utility includes both the benefit in the inventory gained and the cost in time, monetary expense as well as maintenance of safety stock.
A Novel Dual-Stage Evolutionary Algorithm for Finding Robust Solutions
The primary objective of the peak-detection stage is to identify peaks in the fitness landscape of the original optimization problem.
Grayscale Image Colorization with GAN and CycleGAN in Different Image Domain
Automatic colorization of grayscale image has been a challenging task.
Market Responses to Genuine Versus Strategic Generosity: An Empirical Examination of NFT Charity Fundraisers
Questions may arise about the motivations of donors in these charity fundraisers, resulting in a negative social image.
Accelerated Cloud for Artificial Intelligence (ACAI)
Training an effective Machine learning (ML) model is an iterative process that requires effort in multiple dimensions.
Communication Efficient Distributed Training with Distributed Lion
The Lion optimizer has been a promising competitor with the AdamW for training large AI models, with advantages on memory, computation, and sample efficiency.
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
