Search Results for author:
Found 61 papers, 17 papers with code
Efficient Pareto Manifold Learning with Low-Rank Structure
However, it suffers from scalability issues when the number of tasks is large.
Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategy
Pre-training followed by fine-tuning is widely adopted among practitioners.
Communication-Efficient and Privacy-Preserving Decentralized Meta-Learning
Distributed learning, which does not require gathering training data in a central location, has become increasingly important in the big-data era.
Mixup Augmentation with Multiple Interpolations
Mixup and its variants form a popular class of data augmentation techniques. Using a random sample pair, it generates a new sample by linear interpolation of the inputs and labels.
Direct Alignment of Language Models via Quality-Aware Self-Refinement
To alleviate this problem, we investigate the use of intrinsic knowledge within the on-the-fly fine-tuning LLM to obtain relative qualities and help to refine the loss function.
Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
As the capabilities of large language models (LLMs) have expanded dramatically, aligning these models with human values presents a significant challenge.
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation
To construct robust MLLMs, we propose ECSO (Eyes Closed, Safety On), a novel training-free protecting approach that exploits the inherent safety awareness of MLLMs, and generates safer responses via adaptively transforming unsafe images into texts to activate the intrinsic safety mechanism of pre-aligned LLMs in MLLMs.
Task-customized Masked AutoEncoder via Mixture of Cluster-conditional Experts
It also obtains new state-of-the-art self-supervised learning results on detection and segmentation.
KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion
Knowledge Graph Completion (KGC) is crucial for addressing knowledge graph incompleteness and supporting downstream applications.
GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning
Large Language Models (LLMs) are increasingly used for various tasks with graph structures.
Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning
Instruction tuning of Large Vision-language Models (LVLMs) has revolutionized the development of versatile models with zero-shot generalization across a wide range of downstream vision-language tasks.
Aggregation Weighting of Federated Learning via Generalization Bound Estimation
In this paper, we replace the aforementioned weighting method with a new strategy that considers the generalization bounds of each local model.
BYOM: Building Your Own Multi-Task Model For Free
Recently, various merging methods have been proposed to build a multi-task model from task-specific finetuned models without retraining.
Domain-Guided Conditional Diffusion Model for Unsupervised Domain Adaptation
However, the performance of existing UDA methods is constrained by the large domain shift and limited target domain data.
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
Our MetaMath-7B model achieves 66. 4% on GSM8K and 19. 4% on MATH, exceeding the state-of-the-art models of the same size by 11. 5% and 8. 7%.
Ranked #51 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Dual-Balancing for Multi-Task Learning
Multi-task learning (MTL), a learning paradigm to learn multiple related tasks simultaneously, has achieved great success in various fields.
Forward-Backward Reasoning in Large Language Models for Mathematical Verification
Instead of using forward or backward reasoning alone, we propose FOBAR to combine FOrward and BAckward Reasoning for verification.
Effective Structured Prompting by Meta-Learning and Representative Verbalizer
Combining meta-learning the prompt pool and RepVerb, we propose MetaPrompter for effective structured prompting.
A Survey on Time-Series Pre-Trained Models
Time-Series Mining (TSM) is an important research area since it shows great potential in practical applications.
Searching a High-Performance Feature Extractor for Text Recognition Network
We design a domain-specific search space by exploring principles for having good feature extractors.
Neural Architecture Search
Vocal Bursts Intensity Prediction
A Survey of Learning on Small Data: Generalization, Optimization, and Challenge
In future, learning on small data that approximates the generalization ability of big data is one of the ultimate purposes of AI, which requires machines to recognize objectives and scenarios relying on small data as humans.
Black-box Generalization of Machine Teaching
An inherent assumption is that this learning manner can derive those updates into the optimal hypothesis.
Revisiting Over-smoothing in BERT from the Perspective of Graph
Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields.
Dropout's Dream Land: Generalization from Learned Simulators to Reality
By training the World Model using dropout, the dream environment is capable of creating a nearly infinite number of different dream environments.
Pyramidal Dense Attention Networks for Lightweight Image Super-Resolution
Recently, deep convolutional neural network methods have achieved an excellent performance in image superresolution (SR), but they can not be easily applied to embedded devices due to large memory cost.
Feedback Pyramid Attention Networks for Single Image Super-Resolution
Recently, convolutional neural network (CNN) based image super-resolution (SR) methods have achieved significant performance improvement.
TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation
To this end, we propose a target orientated hypothesis adaptation network (TOHAN) to solve the FHA problem, where we generate highly-compatible unlabeled data (i. e., an intermediate domain) to help train a target-domain classifier.
SparseBERT: Rethinking the Importance Analysis in Self-attention
A surprising result is that diagonal elements in the attention map are the least important compared with other attention positions.
A Survey of Label-noise Representation Learning: Past, Present and Future
Classical machine learning implicitly assumes that labels of the training data are sampled from a clean distribution, which can be too restrictive for real-world scenarios.
A Scalable, Adaptive and Sound Nonconvex Regularizer for Low-rank Matrix Completion
Extensive low-rank matrix completion experiments on a number of synthetic and real-world data sets show that the proposed method obtains state-of-the-art recovery performance while being the fastest in comparison to existing low-rank matrix learning methods.
Effective Decoding in Graph Auto-Encoder using Triadic Closure
We propose the triad decoder, which considers and predicts the three edges involved in a local triad together.
Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
In this work, we propose BONAS (Bayesian Optimized Neural Architecture Search), a sample-based NAS framework which is accelerated using weight-sharing to evaluate multiple related architectures simultaneously.
Multi-objective Neural Architecture Search via Predictive Network Performance Optimization
Inspired by the nature of the graph structure of a neural network, we propose BOGCN-NAS, a NAS algorithm using Bayesian Optimization with Graph Convolutional Network (GCN) predictor.
Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback
In particular, on distributed ResNet training with 7 workers on the ImageNet, the proposed algorithm achieves the same testing accuracy as momentum SGD using full-precision gradients, but with $46\%$ less wall clock time.
Blockwise Adaptivity: Faster Training and Better Generalization in Deep Learning
Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks.
Analysis of Quantized Models
We show that (i) weight-quantized networks converge to an error related to the weight quantization resolution and weight dimension; (ii) quantizing gradients slows convergence by a factor related to the gradient quantization resolution and dimension; and (iii) clipping the gradient before quantization renders this factor dimension-free, thus allowing the use of fewer bits for gradient quantization.
General Convolutional Sparse Coding with Unknown Noise
However, existing CSC methods can only model noises from Gaussian distribution, which is restrictive and unrealistic.
Differential Private Stack Generalization with an Application to Diabetes Prediction
To meet the standard of differential privacy, noise is usually added into the original data, which inevitably deteriorates the predicting performance of subsequent learning algorithms.
FasTer: Fast Tensor Completion with Nonconvex Regularization
Due to the easy optimization, the convex overlapping nuclear norm has been popularly used for tensor completion.
Lightweight Stochastic Optimization for Minimizing Finite Sums with Infinite Data
The memory cost of SSAG does not depend on the sample size, while that of S-SAGA is the same as those of variance reduction methods on un- perturbed data.
Power Law in Sparsified Deep Neural Networks
The power law has been observed in the degree distributions of many biological neural networks.
Online Convolutional Sparse Coding with Sample-Dependent Dictionary
Convolutional sparse coding (CSC) has been popularly used for the learning of shift-invariant dictionaries in image and signal processing.
Loss-aware Weight Quantization of Deep Networks
The huge size of deep networks hinders their use in small computing devices.
Large-Scale Low-Rank Matrix Learning with Nonconvex Regularizers
Based on it, we develop a proximal gradient algorithm (and its accelerated variant) with inexact proximal splitting and prove that a convergence rate of O(1/T) where T is the number of iterations is guaranteed.
Follow the Moving Leader in Deep Learning
Deep networks are highly nonlinear and difficult to optimize.
Scalable Online Convolutional Sparse Coding
Convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary from the data.
Multi-Label Learning with Global and Local Label Correlation
In fact, in the real-world applications, both cases may occur that some label correlations are globally applicable and some are shared only in a local group of instances.
Efficient Inexact Proximal Gradient Algorithm for Nonconvex Problems
The proximal gradient algorithm has been popularly used for convex optimization.
Optimization and Control
Loss-aware Binarization of Deep Networks
Deep neural network models, though very powerful and highly successful, are computationally expensive in terms of space and time.
Fast Learning with Nonconvex L1-2 Regularization
In this paper, we show that a closed-form solution can be derived for the proximal step associated with this regularizer.
Learning of Generalized Low-Rank Models: A Greedy Approach
Learning of low-rank matrices is fundamental to many machine learning applications.
Efficient Learning with a Family of Nonconvex Regularizers by Redistributing Nonconvexity
The nonconvex regularizer is then transformed to a familiar convex regularizer, while the resultant loss function can still be guaranteed to be smooth.
Stochastic Variance-Reduced ADMM
The alternating direction method of multipliers (ADMM) is a powerful optimization solver in machine learning.
Fast Nonsmooth Regularized Risk Minimization with Continuation
In regularized risk minimization, the associated optimization problem becomes particularly difficult when both the loss and regularizer are nonsmooth.
Fast Low-Rank Matrix Learning with Nonconvex Regularization
This allows the use of power method to approximate the SVD efficiently.
Asynchronous Distributed Semi-Stochastic Gradient Optimization
With the recent proliferation of large-scale learning problems, there have been a lot of interest on distributed machine learning algorithms, particularly those that are based on stochastic gradient descent (SGD) and its variants.
Fast Stochastic Alternating Direction Method of Multipliers
This matches the convergence rate of the batch ADMM algorithm, but without the need to visit all the samples in each iteration.
Convex and Scalable Weakly Labeled SVMs
In this paper, we study the problem of learning from weakly labeled data, where labels of the training examples are incomplete.
Mandatory Leaf Node Prediction in Hierarchical Multilabel Classification
However, while there have been a lot of MLNP methods in hierarchical multiclass classification, performing MLNP in hierarchical multilabel classification is much more difficult.
Priors for Diversity in Generative Latent Variable Models
We show how to perform MAP inference with DPP priors in latent Dirichlet allocation and in mixture models, leading to better intuition for the latent variable representation and quantitatively improved unsupervised feature extraction, without compromising the generative aspects of the model.
Accelerated Gradient Methods for Stochastic Optimization and Online Learning
Regularized risk minimization often involves non-smooth optimization, either because of the loss function (e. g., hinge loss) or the regularizer (e. g., $\ell_1$-regularizer).
