Search Results for author:
Found 105 papers, 41 papers with code
Evolving Machine Learning Algorithms From Scratch
However, this progress has largely focused on the architecture of neural networks, where it has relied on sophisticated expert-designed layers as building blocks---or similarly restrictive search spaces.
ZHIXIAOBAO at SemEval-2022 Task 10: Apporoaching Structured Sentiment with Graph Parsing
This paper presents our submission to task 10, Structured Sentiment Analysis of the SemEval 2022 competition.
CRYSIM: Prediction of Symmetric Structures of Large Crystals with GPU-based Ising Machines
Solving black-box optimization problems with Ising machines is increasingly common in materials science.
KDSelector: A Knowledge-Enhanced and Data-Efficient Model Selector Learning Framework for Time Series Anomaly Detection
Model selection has been raised as an essential problem in the area of time series anomaly detection (TSAD), because there is no single best TSAD model for the highly heterogeneous time series in real-world applications.
COSMOS: A Hybrid Adaptive Optimizer for Memory-Efficient Training of LLMs
In this paper, we propose COSMOS, a novel hybrid optimizer that leverages the varying importance of eigensubspaces in the gradient matrix to achieve memory efficiency without compromising optimization performance.
Event Argument Extraction with Enriched Prompts
This work aims to delve deeper into prompt-based event argument extraction (EAE) models.
ChatDiT: A Training-Free Baseline for Task-Agnostic Free-Form Chatting with Diffusion Transformers
Building upon this foundation, we present ChatDiT, a zero-shot, general-purpose, and interactive visual generation framework that leverages pretrained diffusion transformers in their original form, requiring no additional tuning, adapters, or modifications.
IDEA-Bench: How Far are Generative Models from Professional Designing?
Real-world design tasks - such as picture book creation, film storyboard development using character sets, photo retouching, visual effects, and font transfer - are highly diverse and complex, requiring deep interpretation and extraction of various elements from instructions, descriptions, and reference images.
In-Context LoRA for Diffusion Transformers
While task-specific in terms of tuning data, our framework remains task-agnostic in architecture and pipeline, offering a powerful tool for the community and providing valuable insights for further research on product-level task-agnostic generation systems.
Group Diffusion Transformers are Unsupervised Multitask Learners
In this work, we introduce Group Diffusion Transformers (GDTs), a novel framework that unifies diverse visual generation tasks by redefining them as a group generation problem.
ERCache: An Efficient and Reliable Caching Framework for Large-Scale User Representations in Meta's Ads System
The increasing complexity of deep learning models used for calculating user representations presents significant challenges, particularly with limited computational resources and strict service-level agreements (SLAs).
How Does the Disclosure of AI Assistance Affect the Perceptions of Writing?
To understand how people perceive writings that are produced under this paradigm, in this paper, we conduct an experimental study to understand whether and how the disclosure of the level and type of AI assistance in the writing process would affect people's perceptions of the writing on various aspects, including their evaluation on the quality of the writing and their ranking of different writings.
PoseAugment: Generative Human Pose Data Augmentation with Physical Plausibility for IMU-based Motion Capture
However, effective data augmentation for IMU-based motion capture is challenging, since it has to capture the physical relations and constraints of the human body, while maintaining the data distribution and quality.
Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation
Chain-of-thought prompting significantly boosts the reasoning ability of large language models but still faces three issues: hallucination problem, restricted interpretability, and uncontrollable generation.
Rethinking Video Segmentation with Masked Video Consistency: Did the Model Learn as Intended?
Video segmentation aims at partitioning video sequences into meaningful segments based on objects or regions of interest within frames.
Molecule Graph Networks with Many-body Equivariant Interactions
In this study, we develop Equivariant N-body Interaction Networks (ENINet) that explicitly integrates l = 1 equivariant many-body interactions to enhance directional symmetric information in the message passing scheme.
WindowMixer: Intra-Window and Inter-Window Modeling for Time Series Forecasting
Time series forecasting (TSF) is crucial in fields like economic forecasting, weather prediction, traffic flow analysis, and public health surveillance.
Tokenize features, enhancing tables: the FT-TABPFN model for tabular classification
Traditional methods for tabular classification usually rely on supervised learning from scratch, which requires extensive training data to determine model parameters.
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
When finetuned on 4K-length sequences, Samba efficiently extrapolates to a 256K context length with perfect memory recall on the Passkey Retrieval task, and exhibits superior retrieval extrapolation on the challenging Phonebook task compared to full-attention models.
Semantic Segmentation on VSPW Dataset through Masked Video Consistency
MVC enforces the consistency between predictions of masked frames where random patches are withheld.
Non-confusing Generation of Customized Concepts in Diffusion Models
We tackle the common challenge of inter-concept visual confusion in compositional concept generation using text-guided diffusion models (TGDMs).
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
Ranked #5 on
MMR total
on MRR-Benchmark
(using extra training data)
Communication Efficient Distributed Training with Distributed Lion
The Lion optimizer has been a promising competitor with the AdamW for training large AI models, with advantages on memory, computation, and sample efficiency.
Applying Unsupervised Semantic Segmentation to High-Resolution UAV Imagery for Enhanced Road Scene Parsing
There are two challenges presented in parsing road scenes from UAV images: the complexity of processing high-resolution images and the dependency on extensive manual annotations required by traditional supervised deep learning methods to train robust and accurate models.
Accelerated Cloud for Artificial Intelligence (ACAI)
Training an effective Machine learning (ML) model is an iterative process that requires effort in multiple dimensions.
The Market Consequences of Perceived Strategic Generosity: An Empirical Examination of NFT Charity Fundraisers
Questions may arise about donors' motivations in these charity fundraisers, potentially resulting in a negative social image.
Grayscale Image Colorization with GAN and CycleGAN in Different Image Domain
Automatic colorization of grayscale image has been a challenging task.
A Novel Dual-Stage Evolutionary Algorithm for Finding Robust Solutions
The primary objective of the peak-detection stage is to identify peaks in the fitness landscape of the original optimization problem.
A Multi-day Needs-based Modeling Approach for Activity and Travel Demand Analysis
The utility includes both the benefit in the inventory gained and the cost in time, monetary expense as well as maintenance of safety stock.
Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracy
Instead of generating a single token at a time, we propose a Trie-based retrieval and verification mechanism to be able to accept several tokens at a forward step.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Visual Question Answering (VQA)
on InfographicVQA
Unsupervised Multi-modal Feature Alignment for Time Series Representation Learning
In contrast to conventional methods that fuse features from multiple modalities, our proposed approach simplifies the neural architecture by retaining a single time series encoder, consequently leading to preserved scalability.
TARGET: Template-Transferable Backdoor Attack Against Prompt-based NLP Models via GPT4
Most of the existing attack methods focus on inserting manually predefined templates as triggers in the pre-training phase to train the victim model and utilize the same triggers in the downstream task to perform inference, which tends to ignore the transferability and stealthiness of the templates.
Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta
To address these challenges, we present Scaling User Modeling (SUM), a framework widely deployed in Meta's ads ranking system, designed to facilitate efficient and scalable sharing of online user representation across hundreds of ads models.
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Quantization is an indispensable technique for serving Large Language Models (LLMs) and has recently found its way into LoRA fine-tuning.
Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation
Recent advances in semi-supervised semantic segmentation have been heavily reliant on pseudo labeling to compensate for limited labeled data, disregarding the valuable relational knowledge among semantic concepts.
Towards Ubiquitous Intelligent Hand Interaction
The development of ubiquitous computing and sensing devices has brought about novel interaction scenarios such as mixed reality and IoT (e. g., smart home), which pose new demands for the next generation of natural user interfaces (NUI).
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Pruning enhances the diversity of low-rank approximations, and low-rank approximation prevents pruning from losing too many expressive neurons.
A Shapelet-based Framework for Unsupervised Multivariate Time Series Representation Learning
Recent studies have shown great promise in unsupervised representation learning (URL) for multivariate time series, because URL has the capability in learning generalizable representation for many downstream tasks without using inaccessible labels.
Optimization design of a micro-perforated panel absorber with 8.6 octave bands
In order to improve low-frequency characteristics of micro-perforated panel absorbers, sound absorption structures composed of micro-perforated panels and expansion chambers are design, and an optimization design method is constructed based on the transfer function model and the simulated annealing algorithm.
UniTS: A Universal Time Series Analysis Framework Powered by Self-Supervised Representation Learning
Machine learning has emerged as a powerful tool for time series analysis.
DR-Label: Improving GNN Models for Catalysis Systems by Label Deconstruction and Reconstruction
Reversely, the model Reconstructs a more robust equilibrium state prediction by transforming edge-level predictions to node-level with a sphere-fitting algorithm.
Graph Neural Network
Initial Structure to Relaxed Energy (IS2RE), Direct
+1
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers
Since the teacher model has a significantly larger capacity and stronger representation power than the student model, it is very difficult for the student to produce predictions that match the teacher's over a massive amount of open-domain training data.
Symbolic Discovery of Optimization Algorithms
On diffusion models, Lion outperforms Adam by achieving a better FID score and reducing the training compute by up to 2. 3x.
Unified Functional Hashing in Automatic Machine Learning
In this paper, we show that large efficiency gains can be obtained by employing a fast unified functional hash, especially through the functional equivalence caching technique, which we also present.
GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models
Going beyond this, we propose GMMSeg, a new family of segmentation models that rely on a dense generative classifier for the joint distribution p(pixel feature, class).
Ranked #4 on
Out-of-Distribution Detection
on ADE-OoD
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
As such, TED reduces the knowledge gap between the two models and helps the student to fit better on the target task.
Multi-Task Mixture Density Graph Neural Networks for Predicting Cu-based Single-Atom Alloy Catalysts for CO2 Reduction Reaction
Graph neural networks (GNNs) have drawn more and more attention from material scientists and demonstrated a high capacity to establish connections between the structure and properties.
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Large Transformer-based models have exhibited superior performance in various natural language processing and computer vision tasks.
Interface Networks for Failure Localization in Power Systems
Transmission power systems usually consist of interconnected sub-grids that are operated relatively independently.
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
We propose MoEBERT, which uses a Mixture-of-Experts structure to increase model capacity and inference speed.
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
To retain ensemble benefits while maintaining a low memory cost, we propose a consistency-regularized ensemble learning approach based on perturbed models, named CAMERO.
The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink
Four best practices can reduce ML training energy by up to 100x and CO2 emissions up to 1000x.
Visual Abductive Reasoning
In this paper, we propose a new task and dataset, Visual Abductive Reasoning (VAR), for examining abductive reasoning ability of machine intelligence in everyday visual situations.
Local-Global Context Aware Transformer for Language-Guided Video Segmentation
We explore the task of language-guided video segmentation (LVS).
Ranked #7 on
Referring Expression Segmentation
on A2D Sentences
Referring Expression Segmentation
Referring Video Object Segmentation
+5
Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrast
On most benchmarks, the generic GNN pre-trained by iMolCLR rivals or even surpasses supervised learning models with sophisticated architecture designs and engineered features.
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
Analysis shows that the proposed schedule indeed reduces the redundancy and improves generalization performance.
TPAD: Identifying Effective Trajectory Predictions Under the Guidance of Trajectory Anomaly Detection Model
They are lack of self-evaluation ability, that is, to examine the rationality of their prediction results, thus failing to guide users to identify high-quality ones from their candidate results.
A General Traffic Shaping Protocol in E-Commerce
Our algorithm can straightforwardly optimize the linear programming in the prime space, and its solution can be simply applied by a stochastic strategy to fulfill the optimized objective and the constraints in expectation.
S+PAGE: A Speaker and Position-Aware Graph Neural Network Model for Emotion Recognition in Conversation
Emotion recognition in conversation (ERC) has attracted much attention in recent years for its necessity in widespread applications.
Ranked #1 on
Emotion Recognition in Conversation
on DailyDialog
AugLiChem: Data Augmentation Library of Chemical Structures for Machine Learning
Inspired by the success of data augmentations in computer vision and natural language processing, we developed AugLiChem: the data augmentation library for chemical structures.
Contrastive Video-Language Segmentation
We focus on the problem of segmenting a certain object referred by a natural language sentence in video content, at the core of formulating a pinpoint vision-language relation.
Self-Training with Differentiable Teacher
In self-training, the student contributes to the prediction performance, and the teacher controls the training process by generating pseudo-labels.
ARCH: Efficient Adversarial Regularized Training with Caching
Adversarial regularization can improve model generalization in many natural language processing tasks.
VSPW: A Large-scale Dataset for Video Scene Parsing in the Wild
To the best of our knowledge, our VSPW is the first attempt to tackle the challenging video scene parsing task in the wild by considering diverse scenarios.
Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation
Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference.
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
The Lottery Ticket Hypothesis suggests that an over-parametrized network consists of ``lottery tickets'', and training a certain collection of them (i. e., a subnetwork) can match the performance of the full model.
Carbon Emissions and Large Neural Network Training
To help reduce the carbon footprint of ML, we believe energy usage and CO2e should be a key metric in evaluating models, and we are collaborating with MLPerf developers to include energy usage during training and inference in this industry standard benchmark.
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
Adversarial regularization has been shown to improve the generalization performance of deep learning models in various natural language processing tasks.
Token-wise Curriculum Learning for Neural Machine Translation
Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of "easy" samples from training data at the early training stage.
ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentation
Text-based video segmentation is a challenging task that segments out the natural language referred objects in videos.
Ranked #4 on
Referring Expression Segmentation
on J-HMDB
(Precision@0.9 metric)
LinkLouvain: Link-Aware A/B Testing and Its Application on Online Marketing Campaign
In this paper, we analyze the network A/B testing problem under a real-world online marketing campaign, describe our proposed LinkLouvain method, and evaluate it on real-world data.
PyGlove: Symbolic Programming for Automated Machine Learning
As a result, AutoML can be reformulated as an automated process of symbolic manipulation.
Compositional Generalization via Neural-Symbolic Stack Machines
Despite achieving tremendous success, existing deep learning models have exposed limitations in compositional generalization, the capability to learn compositional rules and apply them to unseen cases in a systematic manner.
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision
We study the open-domain named entity recognition (NER) problem under distant supervision.
Line Failure Localization of Power Networks Part II: Cut Set Outages
Transmission line failure in power systems prop-agate non-locally, making the control of the resulting outages extremely difficult.
Adaptive Network Response to Line Failures in Power Systems
Transmission line failures in power systems propagate and cascade non-locally.
Line Failure Localization of Power Networks Part I: Non-cut Outages
Transmission line failures in power systems propagate non-locally, making the control of the resulting outages extremely difficult.
Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension
Integrating distributed representations with symbolic operations is essential for reading comprehension requiring complex reasoning, such as counting, sorting and arithmetics, but most existing approaches are hard to scale to more domains or more complex reasoning.
Ranked #5 on
Question Answering
on DROP Test
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
However, this progress has largely focused on the architecture of neural networks, where it has relied on sophisticated expert-designed layers as building blocks---or similarly restrictive search spaces.
InfDetect: a Large Scale Graph-based Fraud Detection System for E-Commerce Insurance
The insurance industry has been creating innovative products around the emerging online shopping activities.
Uncovering Insurance Fraud Conspiracy with Network Learning
In order to detect and prevent fraudulent insurance claims, we developed a novel data-driven procedure to identify groups of organized fraudsters, one of the major contributions to financial losses, by learning network information.
Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing
To overcome this limitation, we propose a novel multi-domain NMT model using individual modules for each domain, on which we apply word-level, adaptive and layer-wise domain mixing.
Learning to Guide: Guidance Law Based on Deep Meta-learning and Model Predictive Path Integral Control
However the traditional MPPI framework assumes the actual environment similar to the training dataset for the deep neural network which is impractical in practice with different maneuvering of target, other perturbations and actuator failures.
Robotics Systems and Control
Neural Program Planner for Structured Predictions
We consider the problem of weakly supervised structured prediction (SP) with reinforcement learning (RL) – for example, given a database table and a question, perform a sequence of computation actions on the table, which generates a response and receives a binary success-failure reward.
Learning to Generalize from Sparse and Underspecified Rewards
The parameters of the auxiliary reward function are optimized with respect to the validation performance of a trained policy.
The Evolved Transformer
Recent works have highlighted the strength of the Transformer architecture on sequence tasks while, at the same time, neural architecture search (NAS) has begun to outperform human-designed models.
Ranked #1 on
Machine Translation
on WMT2014 English-Czech
DeepChannel: Salience Estimation by Contrastive Learning for Extractive Document Summarization
We propose DeepChannel, a robust, data-efficient, and interpretable neural model for extractive document summarization.
Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing
We present Memory Augmented Policy Optimization (MAPO), a simple and novel way to leverage a memory buffer of promising trajectories to reduce the variance of policy gradient estimate.
Distractor Generation for Multiple Choice Questions Using Learning to Rank
We investigate how machine learning models, specifically ranking models, can be used to select useful distractors for multiple choice questions.
Active Learning of Strict Partial Orders: A Case Study on Concept Prerequisite Relations
Strict partial order is a mathematical structure commonly seen in relational data.
A Fully Convolutional Tri-branch Network (FCTN) for Domain Adaptation
We evaluate the proposed network on large-scale domain adaptation experiments using both synthetic (GTA) and real (Cityscapes) images.
A Broad Learning Approach for Context-Aware Mobile Application Recommendation
With the rapid development of mobile apps, the availability of a large number of mobile apps in application stores brings challenge to locate appropriate apps for users.
Multi-Scale FCN With Cascaded Instance Aware Segmentation for Arbitrary Oriented Word Spotting in the Wild
Scene text detection has attracted great attention these years.
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (Short Version)
In this work, we propose the Manager-Programmer-Computer framework, which integrates neural networks with non-differentiable memory to support abstract, scalable and precise operations through a friendly neural computer interface.
Definition Modeling: Learning to define word embeddings in natural language
Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks.
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
Harnessing the statistical power of neural networks to perform language understanding and symbolic reasoning is difficult, when it requires executing efficient discrete operations against a large knowledge-base.
A neural probabilistic model for context based citation recommendation
It is not always easy for knowledgeable researchers to give an accurate citation context for a cited paper or to find the right paper to cite given context.
