Search Results for author:
Found 72 papers, 27 papers with code
Evolving Machine Learning Algorithms From Scratch
However, this progress has largely focused on the architecture of neural networks, where it has relied on sophisticated expert-designed layers as building blocks---or similarly restrictive search spaces.
ZHIXIAOBAO at SemEval-2022 Task 10: Apporoaching Structured Sentiment with Graph Parsing
This paper presents our submission to task 10, Structured Sentiment Analysis of the SemEval 2022 competition.
TARGET: Template-Transferable Backdoor Attack Against Prompt-based NLP Models via GPT4
Most of the existing attack methods focus on inserting manually predefined templates as triggers in the pre-training phase to train the victim model and utilize the same triggers in the downstream task to perform inference, which tends to ignore the transferability and stealthiness of the templates.
Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta
To address these challenges, we present Scaling User Modeling (SUM), a framework widely deployed in Meta's ads ranking system, designed to facilitate efficient and scalable sharing of online user representation across hundreds of ads models.
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Quantization is an indispensable technique for serving Large Language Models (LLMs) and has recently found its way into LoRA fine-tuning.
Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation
Recent advances in semi-supervised semantic segmentation have been heavily reliant on pseudo labeling to compensate for limited labeled data, disregarding the valuable relational knowledge among semantic concepts.
Towards Ubiquitous Intelligent Hand Interaction
The development of ubiquitous computing and sensing devices has brought about novel interaction scenarios such as mixed reality and IoT (e. g., smart home), which pose new demands for the next generation of natural user interfaces (NUI).
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Pruning enhances the diversity of low-rank approximations, and low-rank approximation prevents pruning from losing too many expressive neurons.
Contrastive Shapelet Learning for Unsupervised Multivariate Time Series Representation Learning
Recent studies have shown great promise in unsupervised representation learning (URL) for multivariate time series, because URL has the capability in learning generalizable representation for many downstream tasks without using inaccessible labels.
Optimization design of a micro-perforated panel absorber with 8.6 octave bands
In order to improve low-frequency characteristics of micro-perforated panel absorbers, sound absorption structures composed of micro-perforated panels and expansion chambers are design, and an optimization design method is constructed based on the transfer function model and the simulated annealing algorithm.
UniTS: A Universal Time Series Analysis Framework with Self-supervised Representation Learning
Machine learning has emerged as a powerful tool for time series analysis.
DR-Label: Improving GNN Models for Catalysis Systems by Label Deconstruction and Reconstruction
Reversely, the model Reconstructs a more robust equilibrium state prediction by transforming edge-level predictions to node-level with a sphere-fitting algorithm.
Initial Structure to Relaxed Energy (IS2RE), Direct
Property Prediction
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers
Since the teacher model has a significantly larger capacity and stronger representation power than the student model, it is very difficult for the student to produce predictions that match the teacher's over a massive amount of open-domain training data.
Unified Functional Hashing in Automatic Machine Learning
In this paper, we show that large efficiency gains can be obtained by employing a fast unified functional hash, especially through the functional equivalence caching technique, which we also present.
GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models
Going beyond this, we propose GMMSeg, a new family of segmentation models that rely on a dense generative classifier for the joint distribution p(pixel feature, class).
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
As such, TED reduces the knowledge gap between the two models and helps the student to fit better on the target task.
Multi-Task Mixture Density Graph Neural Networks for Predicting Cu-based Single-Atom Alloy Catalysts for CO2 Reduction Reaction
Graph neural networks (GNNs) have drawn more and more attention from material scientists and demonstrated a high capacity to establish connections between the structure and properties.
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Large Transformer-based models have exhibited superior performance in various natural language processing and computer vision tasks.
Interface Networks for Failure Localization in Power Systems
Transmission power systems usually consist of interconnected sub-grids that are operated relatively independently.
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
We propose MoEBERT, which uses a Mixture-of-Experts structure to increase model capacity and inference speed.
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
To retain ensemble benefits while maintaining a low memory cost, we propose a consistency-regularized ensemble learning approach based on perturbed models, named CAMERO.
The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink
Four best practices can reduce ML training energy by up to 100x and CO2 emissions up to 1000x.
Visual Abductive Reasoning
In this paper, we propose a new task and dataset, Visual Abductive Reasoning (VAR), for examining abductive reasoning ability of machine intelligence in everyday visual situations.
Local-Global Context Aware Transformer for Language-Guided Video Segmentation
In light of this, we present Locater (local-global context aware Transformer), which augments the Transformer architecture with a finite memory so as to query the entire video with the language expression in an efficient manner.
Ranked #7 on
Referring Expression Segmentation
on A2D Sentences
Referring Expression Segmentation
Referring Video Object Segmentation
+5
Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast
On most benchmarks, the generic GNN pre-trained by iMolCLR rivals or even surpasses supervised learning models with sophisticated architecture designs and engineered features.
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
Analysis shows that the proposed schedule indeed reduces the redundancy and improves generalization performance.
TPAD: Identifying Effective Trajectory Predictions Under the Guidance of Trajectory Anomaly Detection Model
They are lack of self-evaluation ability, that is, to examine the rationality of their prediction results, thus failing to guide users to identify high-quality ones from their candidate results.
A General Traffic Shaping Protocol in E-Commerce
Our algorithm can straightforwardly optimize the linear programming in the prime space, and its solution can be simply applied by a stochastic strategy to fulfill the optimized objective and the constraints in expectation.
S+PAGE: A Speaker and Position-Aware Graph Neural Network Model for Emotion Recognition in Conversation
Emotion recognition in conversation (ERC) has attracted much attention in recent years for its necessity in widespread applications.
Ranked #1 on
Emotion Recognition in Conversation
on DailyDialog
AugLiChem: Data Augmentation Library of Chemical Structures for Machine Learning
Inspired by the success of data augmentations in computer vision and natural language processing, we developed AugLiChem: the data augmentation library for chemical structures.
Contrastive Video-Language Segmentation
We focus on the problem of segmenting a certain object referred by a natural language sentence in video content, at the core of formulating a pinpoint vision-language relation.
Self-Training with Differentiable Teacher
In self-training, the student contributes to the prediction performance, and the teacher controls the training process by generating pseudo-labels.
ARCH: Efficient Adversarial Regularized Training with Caching
Adversarial regularization can improve model generalization in many natural language processing tasks.
VSPW: A Large-scale Dataset for Video Scene Parsing in the Wild
To the best of our knowledge, our VSPW is the first attempt to tackle the challenging video scene parsing task in the wild by considering diverse scenarios.
Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation
Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference.
One-shot visual object segmentation
Referring Video Object Segmentation
+2
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
The Lottery Ticket Hypothesis suggests that an over-parametrized network consists of ``lottery tickets'', and training a certain collection of them (i. e., a subnetwork) can match the performance of the full model.
Carbon Emissions and Large Neural Network Training
To help reduce the carbon footprint of ML, we believe energy usage and CO2e should be a key metric in evaluating models, and we are collaborating with MLPerf developers to include energy usage during training and inference in this industry standard benchmark.
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
Adversarial regularization has been shown to improve the generalization performance of deep learning models in various natural language processing tasks.
Token-wise Curriculum Learning for Neural Machine Translation
Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of "easy" samples from training data at the early training stage.
ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentation
Text-based video segmentation is a challenging task that segments out the natural language referred objects in videos.
Ranked #4 on
Referring Expression Segmentation
on J-HMDB
(Precision@0.9 metric)
LinkLouvain: Link-Aware A/B Testing and Its Application on Online Marketing Campaign
In this paper, we analyze the network A/B testing problem under a real-world online marketing campaign, describe our proposed LinkLouvain method, and evaluate it on real-world data.
PyGlove: Symbolic Programming for Automated Machine Learning
As a result, AutoML can be reformulated as an automated process of symbolic manipulation.
Compositional Generalization via Neural-Symbolic Stack Machines
Despite achieving tremendous success, existing deep learning models have exposed limitations in compositional generalization, the capability to learn compositional rules and apply them to unseen cases in a systematic manner.
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision
We study the open-domain named entity recognition (NER) problem under distant supervision.
Line Failure Localization of Power Networks Part II: Cut Set Outages
Transmission line failure in power systems prop-agate non-locally, making the control of the resulting outages extremely difficult.
Adaptive Network Response to Line Failures in Power Systems
Transmission line failures in power systems propagate and cascade non-locally.
Line Failure Localization of Power Networks Part I: Non-cut Outages
Transmission line failures in power systems propagate non-locally, making the control of the resulting outages extremely difficult.
Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension
Integrating distributed representations with symbolic operations is essential for reading comprehension requiring complex reasoning, such as counting, sorting and arithmetics, but most existing approaches are hard to scale to more domains or more complex reasoning.
Ranked #5 on
Question Answering
on DROP Test
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
However, this progress has largely focused on the architecture of neural networks, where it has relied on sophisticated expert-designed layers as building blocks---or similarly restrictive search spaces.
InfDetect: a Large Scale Graph-based Fraud Detection System for E-Commerce Insurance
The insurance industry has been creating innovative products around the emerging online shopping activities.
Uncovering Insurance Fraud Conspiracy with Network Learning
In order to detect and prevent fraudulent insurance claims, we developed a novel data-driven procedure to identify groups of organized fraudsters, one of the major contributions to financial losses, by learning network information.
Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing
To overcome this limitation, we propose a novel multi-domain NMT model using individual modules for each domain, on which we apply word-level, adaptive and layer-wise domain mixing.
Learning to Guide: Guidance Law Based on Deep Meta-learning and Model Predictive Path Integral Control
However the traditional MPPI framework assumes the actual environment similar to the training dataset for the deep neural network which is impractical in practice with different maneuvering of target, other perturbations and actuator failures.
Robotics Systems and Control
Neural Program Planner for Structured Predictions
We consider the problem of weakly supervised structured prediction (SP) with reinforcement learning (RL) – for example, given a database table and a question, perform a sequence of computation actions on the table, which generates a response and receives a binary success-failure reward.
Learning to Generalize from Sparse and Underspecified Rewards
The parameters of the auxiliary reward function are optimized with respect to the validation performance of a trained policy.
The Evolved Transformer
Recent works have highlighted the strength of the Transformer architecture on sequence tasks while, at the same time, neural architecture search (NAS) has begun to outperform human-designed models.
Ranked #1 on
Machine Translation
on WMT2014 English-Czech
DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization
We propose DeepChannel, a robust, data-efficient, and interpretable neural model for extractive document summarization.
Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing
We present Memory Augmented Policy Optimization (MAPO), a simple and novel way to leverage a memory buffer of promising trajectories to reduce the variance of policy gradient estimate.
Distractor Generation for Multiple Choice Questions Using Learning to Rank
We investigate how machine learning models, specifically ranking models, can be used to select useful distractors for multiple choice questions.
Active Learning of Strict Partial Orders: A Case Study on Concept Prerequisite Relations
Strict partial order is a mathematical structure commonly seen in relational data.
A Fully Convolutional Tri-branch Network (FCTN) for Domain Adaptation
We evaluate the proposed network on large-scale domain adaptation experiments using both synthetic (GTA) and real (Cityscapes) images.
A Broad Learning Approach for Context-Aware Mobile Application Recommendation
With the rapid development of mobile apps, the availability of a large number of mobile apps in application stores brings challenge to locate appropriate apps for users.
Multi-Scale FCN With Cascaded Instance Aware Segmentation for Arbitrary Oriented Word Spotting in the Wild
Scene text detection has attracted great attention these years.
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (Short Version)
In this work, we propose the Manager-Programmer-Computer framework, which integrates neural networks with non-differentiable memory to support abstract, scalable and precise operations through a friendly neural computer interface.
Definition Modeling: Learning to define word embeddings in natural language
Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks.
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
Harnessing the statistical power of neural networks to perform language understanding and symbolic reasoning is difficult, when it requires executing efficient discrete operations against a large knowledge-base.
A neural probabilistic model for context based citation recommendation
It is not always easy for knowledgeable researchers to give an accurate citation context for a cited paper or to find the right paper to cite given context.
