Search Results for author:
Found 79 papers, 35 papers with code
CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology
Additionally, we develop a specialized pathology CLIP-based visual processor for CPath-Omni, CPath-CLIP, which, for the first time, integrates different vision models and incorporates a large language model as a text encoder to build a more powerful CLIP model, which achieves SOTA performance on nine zero-shot and four few-shot datasets.
Generative Modeling with Explicit Memory
Recent studies indicate that the denoising process in deep generative diffusion models implicitly learns and memorizes semantic information from the data distribution.
Ranked #1 on
Image Generation
on CIFAR-10
Learn How to Query from Unlabeled Data Streams in Federated Learning
Federated learning (FL) enables collaborative learning among decentralized clients while safeguarding the privacy of their local data.
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
We confirm that our experts make progress in the environments given 8 hours, with 82% of expert attempts achieving a non-zero score and 24% matching or exceeding our strong reference solutions.
Distribution-Aware Compensation Design for Sustainable Data Rights in Machine Learning
Modern distributed learning systems face a critical challenge when clients request the removal of their data influence from trained models, as this process can significantly destabilize system performance and affect remaining participants.
MorphAgent: Empowering Agents through Self-Evolving Profiles and Decentralized Collaboration
Large Language Model (LLM) based multi-agent systems (MAS) have shown promise in tackling complex tasks, but often rely on predefined roles and centralized coordination, limiting their adaptability to evolving challenges.
Out-of-Bounding-Box Triggers: A Stealthy Approach to Cheat Object Detectors
In recent years, the study of adversarial robustness in object detection systems, particularly those based on deep neural networks (DNNs), has become a pivotal area of research.
ELICIT: LLM Augmentation via External In-Context Capability
By externally storing and reusing vectors that represent in-context learned capabilities, \alg not only demonstrates the potential to operate modular capabilities but also significantly enhances the performance, versatility, adaptability, and scalability of large language models.
CollabEdit: Towards Non-destructive Collaborative Knowledge Editing
Collaborative learning of large language models (LLMs) has emerged as a new paradigm for utilizing private data from different parties to guarantee efficiency and privacy.
Baichuan-Omni Technical Report
The salient multimodal capabilities and interactive experience of GPT-4o highlight its critical role in practical applications, yet it lacks a high-performing open-source counterpart.
Ranked #20 on
Visual Question Answering
on MM-Vet
Information Design with Unknown Prior
In this model, we design a learning algorithm for the information designer with $O(\log T)$ regret in the general case, and another algorithm with $\Theta(\log \log T)$ regret in the case where the receiver has only two actions.
User-Creator Feature Polarization in Recommender Systems with Dual Influence
Recommender systems serve the dual purpose of presenting relevant content to users and helping content creators reach their target audience.
Leveraging large language models for nano synthesis mechanism explanation: solid foundations or mere conjectures?
With the rapid development of artificial intelligence (AI), large language models (LLMs) such as GPT-4 have garnered significant attention in the scientific community, demonstrating great potential in advancing scientific discovery.
Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning
Fine-tuning large pre-trained foundation models, such as the 175B GPT-3, has attracted more attention for downstream tasks recently.
PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration
Vision Language Models (VLMs) like CLIP have attracted substantial attention in pathology, serving as backbones for applications such as zero-shot image classification and Whole Slide Image (WSI) analysis.
Cognitive Insights and Stable Coalition Matching for Fostering Multi-Agent Cooperation
Cognitive abilities, such as Theory of Mind (ToM), play a vital role in facilitating cooperation in human social interactions.
Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexing
To demonstrate the effectiveness of the proposed Client2Vec method, we conduct three case studies that assess the impact of the client index on the FL training process.
Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
The Sparse Mixture of Experts (SMoE) has been widely employed to enhance the efficiency of training and inference for Transformer-based foundational models, yielding promising results.
Ranked #170 on
Visual Question Answering
on MM-Vet
GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost
We first conduct a comprehensive comparison of various loss functions for soft label utilization in dataset distillation, revealing that the model trained on the synthetic dataset exhibits high sensitivity to the choice of loss function for soft label utilization.
Efficiency for Free: Ideal Data Are Transportable Representations
Data, the seminal opportunity and challenge in modern machine learning, currently constrains the scalability of representation learning and impedes the pace of model evolution.
Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software Learning
We introduce a governance framework centered on federated learning (FL), designed to foster the joint development and maintenance of open-source AI code models while safeguarding data privacy and security.
DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference
Large language models (LLMs) are increasingly employed for complex tasks that process multiple generation calls in a tree structure with shared prefixes of tokens, including few-shot prompting, multi-step reasoning, speculative decoding, etc.
Generalized Principal-Agent Problem with a Learning Agent
(2) If the agent uses contextual no-swap-regret learning algorithms with swap-regret $\mathrm{SReg}(T)$, then the principal cannot obtain utility more than $U^* + O(\frac{\mathrm{SReg(T)}}{T})$.
Switch EMA: A Free Lunch for Better Flatness and Sharpness
Exponential Moving Average (EMA) is a widely used weight averaging (WA) regularization to learn flat optima for better generalizations without extra cost in deep neural network (DNN) optimization.
Multi-Sender Persuasion: A Computational Perspective
We consider the multi-sender persuasion problem: multiple players with informational advantage signal to convince a single self-interested actor to take certain actions.
Training-time Neuron Alignment through Permutation Subspace for Improving Linear Mode Connectivity and Model Fusion
In deep learning, stochastic gradient descent often yields functionally similar yet widely scattered solutions in the weight space even under the same initialization, causing barriers in the Linear Mode Connectivity (LMC) landscape.
Federated Unlearning: a Perspective of Stability and Fairness
Our key contribution lies in a comprehensive theoretical analysis of the trade-offs in FU and provides insights into data heterogeneity's impacts on FU.
PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathology
To address this, we introduce PathMMU, the largest and highest-quality expert-validated pathology benchmark for Large Multimodal Models (LMMs).
Learning Thresholds with Latent Values and Censored Feedback
In this paper, we investigate a problem of actively learning threshold in latent space, where the unknown reward $g(\gamma, v)$ depends on the proposed threshold $\gamma$ and latent value $v$ and it can be $only$ achieved if the threshold is lower than or equal to the unknown latent value.
On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm
Contemporary machine learning requires training large neural networks on massive datasets and thus faces the challenges of high computational demands.
Towards Robust Multi-Modal Reasoning via Model Selection
The reasoning capabilities of LLM (Large Language Model) are widely acknowledged in recent research, inspiring studies on tool learning and autonomous agents.
Find Your Optimal Assignments On-the-fly: A Holistic Framework for Clustered Federated Learning
To this end, this paper presents a comprehensive investigation into current clustered FL methods and proposes a four-tier framework, namely HCFL, to encompass and extend existing approaches.
Revisiting Implicit Models: Sparsity Trade-offs Capability in Weight-tied Model for Vision Tasks
Implicit models such as Deep Equilibrium Models (DEQs) have garnered significant attention in the community for their ability to train infinite layer models with elegant solution-finding procedures and constant memory footprint.
On Pitfalls of Test-Time Adaptation
Test-Time Adaptation (TTA) has recently emerged as a promising approach for tackling the robustness challenge under distribution shifts.
Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging
In this work, we tackle this challenge by marrying prompt tuning with FL to snapshot compressive imaging for the first time and propose an federated hardware-prompt learning (FedHP) method.
Prediction with Incomplete Data under Agnostic Mask Distribution Shift
We focus on the case where the underlying joint distribution of complete features and label is invariant, but the missing pattern, i. e., mask distribution may shift agnostically between training and testing.
No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier
Recent advances in neural collapse have shown that the classifiers and feature prototypes under perfect training scenarios collapse into an optimal structure called simplex equiangular tight frame (ETF).
Revisiting Weighted Aggregation in Federated Learning with Neural Networks
In federated learning (FL), weighted aggregation of local models is conducted to generate a global model, and the aggregation weights are normalized (the sum of weights is 1) and proportional to the local data sizes.
Persuading a Behavioral Agent: Approximately Best Responding and Learning
We show that, under natural assumptions, (1) the sender can find a signaling scheme that guarantees itself an expected utility almost as good as its optimal utility in the classic model, no matter what approximately best-responding strategy the receiver uses; (2) on the other hand, there is no signaling scheme that gives the sender much more utility than its optimal utility in the classic model, even if the receiver uses the approximately best-responding strategy that is best for the sender.
FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering
In this paper, we identify the learning challenges posed by the simultaneous occurrence of diverse distribution shifts and propose a clustering principle to overcome these challenges.
Decentralized Gradient Tracking with Local Steps
Gradient tracking (GT) is an algorithm designed for solving decentralized optimization problems over a network (such as training a machine learning model).
How Does Independence Help Generalization? Sample Complexity of ERM on Product Distributions
While many classical notions of learnability (e. g., PAC learnability) are distribution-free, utilizing the specific structures of an input distribution may improve learning performance.
FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reduction
As a remedy, we propose FedBR, a novel unified algorithm that reduces the local learning bias on features and classifiers to tackle these challenges.
Test-Time Robust Personalization for Federated Learning
Personalized FL additionally adapts the global model to different clients, achieving promising results on consistent local training and test distributions.
Adversarial Training for High-Stakes Reliability
We found that adversarial training increased robustness to the adversarial attacks that we trained on -- doubling the time for our contractors to find adversarial examples both with our tool (from 13 to 26 minutes) and without (from 20 to 44 minutes) -- without affecting in-distribution performance.
Learning Disentangled Behaviour Patterns for Wearable-based Human Activity Recognition
To address this issue, in this work, we proposed a Behaviour Pattern Disentanglement (BPD) framework, which can disentangle the behavior patterns from the irrelevant noises such as personal styles or environmental noises, etc.
An Improved Analysis of Gradient Tracking for Decentralized Machine Learning
We consider decentralized machine learning over a network where the training data is distributed across $n$ agents, each of which can compute stochastic model updates on their local data.
Towards Federated Learning on Time-Evolving Heterogeneous Data
Federated Learning (FL) is a learning paradigm that protects privacy by keeping client data on edge devices.
Learning by Active Forgetting for Neural Networks
This paper presents a learning model by active forgetting mechanism with artificial neural networks.
Nash Convergence of Mean-Based Learning Algorithms in First Price Auctions
Specifically, the results depend on the number of bidders with the highest value: - If the number is at least three, the bidding dynamics almost surely converges to a Nash equilibrium of the auction, both in time-average and in last-iterate.
RelaySum for Decentralized Deep Learning on Heterogeneous Data
A key challenge, primarily in decentralized deep learning, remains the handling of differences between the workers' local data distributions.
Networked Time Series Prediction with Incomplete Data via Generative Adversarial Network
An important task in such applications is to predict the future values of a NETS based on its historical values and the underlying graph.
Representation Memorization for Fast Learning New Knowledge without Forgetting
In this paper, we consider scenarios that require learning new classes or data distributions quickly and incrementally over time, as it often occurs in real-world dynamic environments.
The Multi-Modal Video Reasoning and Analyzing Competition
In this paper, we introduce the Multi-Modal Video Reasoning and Analyzing Competition (MMVRAC) workshop in conjunction with ICCV 2021.
The Optimal Size of an Epistemic Congress
We analyze the optimal size of a congress in a representative democracy.
Deep Learning for IoT
Besides, this paper presents a research on data retrieval solution to avoid hacking by adversaries in the fields of adversary machine leaning.
Consensus Control for Decentralized Deep Learning
Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters.
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
In this paper, we investigate and identify the limitation of several decentralized optimization algorithms for different degrees of data heterogeneity.
On the Effect of Consensus in Decentralized Deep Learning
Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters.
AST-GCN: Attribute-Augmented Spatiotemporal Graph Convolutional Network for Traffic Forecasting
Traffic forecasting is a fundamental and challenging task in the field of intelligent transportation.
A Game-Theoretic Analysis of the Empirical Revenue Maximization Algorithm with Endogenous Sampling
The Empirical Revenue Maximization (ERM) is one of the most important price learning algorithms in auction design: as the literature shows it can learn approximately optimal reserve prices for revenue-maximizing auctioneers in both repeated auctions and uniform-price auctions.
XCM: An Explainable Convolutional Neural Network for Multivariate Time Series Classification
Then, we illustrate how XCM reconciles performance and explainability on a synthetic dataset and show that XCM enables a more precise identification of the regions of the input data that are important for predictions compared to the current deep learning MTS classifier also providing faithful explainability.
Learning Utilities and Equilibria in Non-Truthful Auctions
In non-truthful auctions, agents' utility for a strategy depends on the strategies of the opponents and also the prior distribution over their private types; the set of Bayes Nash equilibria generally has an intricate dependence on the prior.
On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them
We analyze the influence of adversarial training on the loss landscape of machine learning models.
Ensemble Distillation for Robust Model Fusion in Federated Learning
In most of the current training schemes the central model is refined by averaging the parameters of the server model and the updated parameters from the client side.
Extrapolation for Large-batch Training in Deep Learning
Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data.
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning.
Deep Collaborative Embedding for information cascade prediction
First, the existing works often assume an underlying information diffusion model, which is impractical in real world due to the complexity of information diffusion.
Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation
It is due to two facts: first, as the model learns more tasks, the intersection of the low-error parameter subspace satisfying for these tasks becomes smaller or even does not exist; second, when the model learns a new task, the cumulative error keeps increasing as the model tries to protect the parameter configuration of previous tasks from interference.
Decentralized Deep Learning with Arbitrary Communication Compression
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks, as well as for efficient scaling to large compute clusters.
Exploring Interpretable LSTM Neural Networks over Multi-Variable Data
In this paper, we explore the structure of LSTM recurrent neural networks to learn variable-wise hidden states, with the aim to capture different dynamics in multi-variable time series and distinguish the contribution of variables to the prediction.
T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction
However, traffic forecasting has always been considered an open scientific issue, owing to the constraints of urban road network topological structure and the law of dynamic change with time, namely, spatial dependence and temporal dependence.
Ranked #4 on
Traffic Prediction
on SZ-Taxi
Exploring the interpretability of LSTM neural networks over multi-variable data
In learning a predictive model over multivariate time series consisting of target and exogenous variables, the forecasting performance and interpretability of the model are both essential for deployment and uncovering knowledge behind the data.
Don't Use Large Mini-Batches, Use Local SGD
Mini-batch stochastic gradient methods (SGD) are state of the art for distributed training of deep neural networks.
Multi-variable LSTM neural network for autoregressive exogenous model
In this paper, we propose multi-variable LSTM capable of accurate forecasting and variable importance interpretation for time series with exogenous variables.
An interpretable LSTM neural network for autoregressive exogenous model
In this paper, we propose an interpretable LSTM recurrent neural network, i. e., multi-variable LSTM for time series with exogenous variables.
Training DNNs with Hybrid Block Floating Point
We identify block floating point (BFP) as a promising alternative representation since it exhibits wide dynamic range and enables the majority of DNN operations to be performed with fixed-point logic.
RubyStar: A Non-Task-Oriented Mixture Model Dialog System
RubyStar is a dialog system designed to create "human-like" conversation by combining different response generation strategies.
