Search Results for author:
Found 17 papers, 9 papers with code
Red Teaming Language Model Detectors with Language Models
The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users.
Symbol tuning improves in-context learning in language models
We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e. g., "positive/negative sentiment") are replaced with arbitrary symbols (e. g., "foo/bar").
Symbolic Discovery of Optimization Algorithms
On diffusion models, Lion outperforms Adam by achieving a better FID score and reducing the training compute by up to 2. 3x.
Towards Efficient and Scalable Sharpness-Aware Minimization
Recently, Sharpness-Aware Minimization (SAM), which connects the geometry of the loss landscape and generalization, has demonstrated significant performance boosts on training large-scale models such as vision transformers.
Can Vision Transformers Perform Convolution?
Several recent studies have demonstrated that attention-based networks, such as Vision Transformer (ViT), can outperform Convolutional Neural Networks (CNNs) on several computer vision tasks without using convolutional layers.
Learning to Schedule Learning rate with Graph Neural Networks
By constructing a directed graph for the underlying neural network of the target problem, GNS encodes current dynamics with a graph message passing network and trains an agent to control the learning rate accordingly via reinforcement learning.
Sharpness-Aware Minimization in Large-Batch Training: Training Vision Transformer In Minutes
Large-batch training is an important direction for distributed machine learning, which can improve the utilization of large-scale clusters and therefore accelerate the training process.
RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving
Predictor-based algorithms have achieved remarkable performance in the Neural Architecture Search (NAS) tasks.
Rethinking Architecture Selection in Differentiable NAS
Differentiable Neural Architecture Search is one of the most popular Neural Architecture Search (NAS) methods for its search efficiency and simplicity, accomplished by jointly optimizing the model weight and architecture parameters in a weight-sharing supernet via gradient-based algorithms.
When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations
Vision Transformers (ViTs) and MLPs signal further efforts on replacing hand-wired features or inductive biases with general-purpose neural architectures.
Concurrent Adversarial Learning for Large-Batch Training
Current methods usually use extensive data augmentation to increase the batch size, but we found the performance gain with data augmentation decreases as batch size increases, and data augmentation will become insufficient after certain point.
2.5D Visual Relationship Detection
To enable progress on this task, we create a new dataset consisting of 220k human-annotated 2. 5D relationships among 512K objects from 11K images.
Robust and Accurate Object Detection via Adversarial Learning
Data augmentation has become a de facto component for training high-performance deep image classifiers, but its potential is under-explored for object detection.
Ranked #17 on
Object Detection
on COCO-O
DrNAS: Dirichlet Neural Architecture Search
This paper proposes a novel differentiable architecture search method by formulating it into a distribution learning problem.
Stabilizing Differentiable Architecture Search via Perturbation-based Regularization
Furthermore, we mathematically show that SDARTS implicitly regularizes the Hessian norm of the validation loss, which accounts for a smoother loss landscape and improved performance.
Efficient Neural Interaction Function Search for Collaborative Filtering
Motivated by the recent success of automated machine learning (AutoML), we propose in this paper the search for simple neural interaction functions (SIF) in CF.
Learning to Recommend with Multiple Cascading Behaviors
To fully exploit the signal in the data of multiple types of behaviors, we perform a joint optimization based on the multi-task learning framework, where the optimization on a behavior is treated as a task.
