Search Results for author:
Found 88 papers, 40 papers with code
InfoFlood: Jailbreaking Large Language Models with Information Overload
Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains.
PREMISE: Scalable and Strategic Prompt Optimization for Efficient Mathematical Reasoning in Large Models
Large reasoning models (LRMs) such as Claude 3. 7 Sonnet and OpenAI o1 achieve strong performance on mathematical benchmarks using lengthy chain-of-thought (CoT) reasoning, but the resulting traces are often unnecessarily verbose.
Reasoning Can Hurt the Inductive Abilities of Large Language Models
Large Language Models (LLMs) have shown remarkable progress across domains, yet their ability to perform inductive reasoning - inferring latent rules from sparse examples - remains limited.
From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Models
Large foundation models (LFMs) are susceptible to two distinct vulnerabilities: hallucinations and jailbreak attacks.
SIPDO: Closed-Loop Prompt Optimization via Synthetic Data Feedback
Prompt quality plays a critical role in the performance of large language models (LLMs), motivating a growing body of work on prompt optimization.
Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation
While prior work has applied intent detection to enhance LLMs' moderation guardrails, showing a significant success against content-level jailbreaks, the robustness of these intent-aware guardrails under malicious manipulations remains under-explored.
Beamforming-Codebook-Aware Channel Knowledge Map Construction for Multi-Antenna Systems
Channel knowledge map (CKM) has emerged as a crucial technology for next-generation communication, enabling the construction of high-fidelity mappings between spatial environments and channel parameters via electromagnetic information analysis.
Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems
In this work, we bring attention to prompt stability-the consistency of model responses across repeated executions-as a key factor for building robust and effective prompt generation systems.
Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems
The advent of large language models (LLMs) has catalyzed a transformative shift in artificial intelligence, paving the way for advanced intelligent agents capable of sophisticated reasoning, robust perception, and versatile action across diverse domains.
Ranked #1 on
Continual Learning
on AIDS
(using extra training data)
IMPROVE: Iterative Model Pipeline Refinement and Optimization Leveraging LLM Agents
These findings establish Iterative Refinement as an effective new strategy for LLM-driven ML automation and position IMPROVE as an accessible solution for building high-quality computer vision models without requiring ML expertise.
CTR-Driven Advertising Image Generation with Multimodal Large Language Models
In web data, advertising images are crucial for capturing user attention and improving advertising effectiveness.
Examining Alignment of Large Language Models through Representative Heuristics: The Case of Political Stereotypes
Drawing on cognitive science findings related to representativeness heuristics -- where individuals readily recall the representative attribute of a target group in a way that leads to exaggerated beliefs -- we scrutinize LLM responses through this heuristics lens.
Revolve: Optimizing AI Systems by Tracking Response Evolution in Textual Optimization
However, relying solely on such feedback can be limited when the adjustments made in response to this feedback are either too small or fluctuate irregularly, potentially slowing down or even stalling the optimization process.
Conflict-Aware Adversarial Training
Adversarial training is the most effective method to obtain adversarial robustness for deep neural networks by directly involving adversarial samples in the training procedure.
DistDD: Distributed Data Distillation Aggregation through Gradient Matching
In this paper, we introduce DistDD, a novel approach within the federated learning framework that reduces the need for repetitive communication by distilling data directly on clients' devices.
MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification
We further compare the performance of MemeCLIP and zero-shot GPT-4 on the hate classification task.
Ranked #4 on
Hateful Meme Classification
on PrideMM
Simple Unsupervised Knowledge Distillation With Space Similarity
In this paper, instead of heuristically constructing preservation worthy relationships between samples, we directly motivate the student to model the teacher's embedding manifold.
DS-ViT: Dual-Stream Vision Transformer for Cross-Task Distillation in Alzheimer's Early Diagnosis
In the field of Alzheimer's disease diagnosis, segmentation and classification tasks are inherently interconnected.
A Quantitative Approach for Evaluating Disease Focus and Interpretability of Deep Learning Models for Alzheimer's Disease Classification
To bridge such gap, in this study, we developed a quantitative disease-focusing strategy to first enhance the interpretability of DL models using saliency maps and brain segmentations; then we propose a disease-focus (DF) score that quantifies how much a DL model focuses on brain areas relevant to AD pathology based on clinically known MRI-based pathological regions of AD.
Towards Reliable Advertising Image Generation Using Human Feedback
In the e-commerce realm, compelling advertising images are pivotal for attracting customer attention.
Quantitative Evaluation of the Saliency Map for Alzheimer's Disease Classifier with Anatomical Segmentation
In order to leverage the fact that the brain volume shrinkage happens in AD patients during disease progression, we define a new evaluation metric, brain volume change score (VCS), by computing the average Pearson correlation of the brain volume changes and the saliency values of a model in different brain regions for each patient.
JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models
The rapid evolution of artificial intelligence (AI) through developments in Large Language Models (LLMs) and Vision-Language Models (VLMs) has brought significant advancements across various technological domains.
Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey
Large Language Models (LLMs) have achieved unparalleled success across diverse language modeling tasks in recent years.
From Tissue Plane to Organ World: A Benchmark Dataset for Multimodal Biomedical Image Registration using Deep Co-Attention Networks
To gain the most information from this multimodal, multiscale approach, it is desirable to identify precisely where a histologic tissue section was taken from within the organ in order to correlate with the tissue features in exactly the same organ region.
Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters
To address this issue, we introduce JAMBench, a harmful behavior benchmark designed to trigger and evaluate moderation guardrails.
Approximate Nullspace Augmented Finetuning for Robust Vision Transformers
Enhancing the robustness of deep learning models, particularly in the realm of vision transformers (ViTs), is crucial for their real-world deployment.
Towards Adversarially Robust Dataset Distillation by Curvature Regularization
Extensive empirical experiments suggest that our method not only outperforms standard adversarial training on both accuracy and robustness with less computation overhead but is also capable of generating robust distilled datasets that can withstand various adversarial attacks.
Beyond Finite Data: Towards Data-free Out-of-distribution Generalization via Extrapolation
Starting with the class of interest, we query the LLMs to extract relevant knowledge for these novel domains.
Toward a Team of AI-made Scientists for Scientific Discovery from Gene Expression Data
For instance, it has facilitated the identification of disease-predictive genes from gene expression data, significantly advancing healthcare.
GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models
Our system of different roles will leverage this knowledge graph to generate new jailbreaks, which have proved effective in inducing LLMs to generate unethical or guideline-violating responses.
Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks
Despite advances in AI alignment, large language models (LLMs) remain vulnerable to adversarial attacks or jailbreaking, in which adversaries can modify prompts to induce unwanted behavior.
ADAPT: Alzheimer Diagnosis through Adaptive Profiling Transformers
Automated diagnosis of Alzheimer Disease(AD) from brain imaging, such as magnetic resonance imaging (MRI), has become increasingly important and has attracted the community to contribute many deep learning methods.
Generate E-commerce Product Background by Integrating Category Commonality and Personalized Style
Furthermore, for products with specific and fine-grained requirements in layout, elements, etc, a Personality-Wise Generator is devised to learn such personalized style directly from a reference image to resolve textual ambiguities, and is trained in a self-supervised manner for more efficient training data usage.
Dataset Distillation via the Wasserstein Metric
Dataset Distillation (DD) emerges as a powerful strategy to encapsulate the expansive information of large datasets into significantly smaller, synthetic equivalents, thereby preserving model performance with reduced computational overhead.
Beyond Pixels: Exploring Human-Readable SVG Generation for Simple Images with Vision Language Models
Our method focuses on producing SVGs that are both accurate and simple, aligning with human readability and understanding.
Choosing Wisely and Learning Deeply: Selective Cross-Modality Distillation via CLIP for Domain Generalization
SCMD leverages the capabilities of large vision-language models, specifically CLIP, to train a more efficient model, ensuring it acquires robust generalization capabilities across unseen domains.
Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models
We propose a conceptually simple and lightweight framework for improving the robustness of vision models through the combination of knowledge distillation and data augmentation.
Ranked #13 on
Domain Generalization
on ImageNet-Sketch
Adaptive Test-Time Personalization for Federated Learning
To tackle this challenge, we propose a novel algorithm called ATP to adaptively learns the adaptation rates for each module in the model from distribution shifts among source domains.
ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning
When personalized federated learning (FL) meets large foundation models, new challenges arise from various limitations in resources.
Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models
By leveraging the in-context learning ability of LMs, we integrate Monte Carlo Tree Search into LATS to enable LMs as agents, along with LM-powered value functions and self-reflections for proficient exploration and enhanced decision-making.
Ranked #16 on
Code Generation
on MBPP
Towards Understanding Adversarial Transferability in Federated Learning
We investigate a specific security risk in FL: a group of malicious clients has impacted the model during training by disguising their identities and acting as benign clients but later switching to an adversarial role.
A Sentence Speaks a Thousand Images: Domain Generalization through Distilling CLIP with Language Guidance
Domain generalization studies the problem of training a model with samples from several domains (or distributions) and then testing the model with samples from a new, unseen domain.
Ranked #21 on
Domain Generalization
on PACS
Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Machine learning has demonstrated remarkable performance over finite datasets, yet whether the scores over the fixed benchmarks can sufficiently indicate the model's performance in the real world is still in discussion.
Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives
Accordingly, this survey presents the background of trustworthy machine learning development using a unified set of concepts, connects this language to Pearl's causal hierarchy, and finally discusses methods explicitly inspired by causality literature.
Optimizing the Collaboration Structure in Cross-Silo Federated Learning
In federated learning (FL), multiple clients collaborate to train machine learning models together while keeping their data decentralized.
Leveraging Large Language Models for Scalable Vector Graphics-Driven Image Understanding
To study what the LLM can do with this XML-based textual description of images, we test the LLM on three broad computer vision tasks: (i) visual reasoning and question answering, (ii) image classification under distribution shift, few-shot learning, and (iii) generating new images using visual prompting.
BadLabel: A Robust Perspective on Evaluating and Enhancing Label-noise Learning
To address the challenge posed by BadLabel, we further propose a robust LNL method that perturbs the labels in an adversarial manner at each epoch to make the loss values of clean and noisy labels again distinguishable.
Calibrated Teacher for Sparsely Annotated Object Detection
Recent works on sparsely annotated object detection alleviate this problem by generating pseudo labels for the missing annotations.
Iterative Few-shot Semantic Segmentation from Image Label Text
Few-shot semantic segmentation aims to learn to segment unseen class objects with the guidance of only a few support images.
Ranked #49 on
Few-Shot Semantic Segmentation
on COCO-20i (1-shot)
Expeditious Saliency-guided Mix-up through Random Gradient Thresholding
Mix-up training approaches have proven to be effective in improving the generalization ability of Deep Neural Networks.
Toward Robust Diagnosis: A Contour Attention Preserving Adversarial Defense for COVID-19 Detection
As the COVID-19 pandemic puts pressure on healthcare systems worldwide, the computed tomography image based AI diagnostic system has become a sustainable solution for early diagnosis.
A Principled Evaluation Protocol for Comparative Investigation of the Effectiveness of DNN Classification Models on Similar-but-non-identical Datasets
Deep Neural Network (DNN) models are increasingly evaluated using new replication test datasets, which have been carefully created to be similar to older and popular benchmark datasets.
MRCLens: an MRC Dataset Bias Detection Toolkit
Many recent neural models have shown remarkable empirical results in Machine Reading Comprehension, but evidence suggests sometimes the models take advantage of dataset biases to predict and fail to generalize on out-of-sample data.
Robustar: Interactive Toolbox Supporting Precise Data Annotation for Robust Vision Learning
We introduce the initial release of our software Robustar, which aims to improve the robustness of vision classification machine learning models through a data-driven perspective.
Efficiently Leveraging Multi-level User Intent for Session-based Recommendation via Atten-Mixer Network
Based on this observation, we intuitively propose to remove the GNN propagation part, while the readout module will take on more responsibility in the model reasoning process.
Bear the Query in Mind: Visual Grounding with Query-conditioned Convolution
With our proposed QCM, the downstream fusion module receives visual features that are more discriminative and focused on the desired object described in the expression, leading to more accurate predictions.
Toward Learning Robust and Invariant Representations with Alignment Regularization and Data Augmentation
Finally, we test this simple technique we identify (worst-case data augmentation with squared l2 norm alignment regularization) and show that the benefits of this method outrun those of the specially designed methods.
The Two Dimensions of Worst-case Training and the Integrated Effect for Out-of-domain Generalization
Training with an emphasis on "hard-to-learn" components of the data has been proven as an effective method to improve the generalization of machine learning models, especially in the settings where robustness (e. g., generalization across distributions) is valued.
The Two Dimensions of Worst-Case Training and Their Integrated Effect for Out-of-Domain Generalization
Training with an emphasis on "hard-to-learn" components of the data has been proven as an effective method to improve the generalization of machine learning models, especially in the settings where robustness (e. g., generalization across distributions) is valued.
CatchBackdoor: Backdoor Detection via Critical Trojan Neural Path Fuzzing
Extensive experiments on MINST, CIFAR-10, and a-ImageNet datasets and 7 models (LeNet, ResNet, and VGG) demonstrate the superiority of CatchBackdoor over the state-of-the-art methods, in terms of (1) \emph{effective} - it shows better detection performance, especially on stealthy attacks ($\sim$ $\times$ 2 on average); (2) \emph{extensible} - it is robust to trigger size and can conduct detection without benign examples.
Measure and Improve Robustness in NLP Models: A Survey
Despite robustness being an increasingly studied topic, it has been separately explored in applications like vision and NLP, with various definitions, evaluation and mitigation strategies in multiple lines of research.
Tradeoffs of Linear Mixed Models in Genome-wide Association Studies
Motivated by empirical arguments that are well-known from the genome-wide association studies (GWAS) literature, we study the statistical properties of linear mixed models (LMMs) applied to GWAS.
Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
Machine learning has demonstrated remarkable prediction accuracy over i. i. d data, but the accuracy often drops when tested with data from another distribution.
Robust Contrastive Learning Using Negative Samples with Diminished Semantics
We also show that model bias favors texture and shape features differently under different test settings.
Enabling the Network to Surf the Internet
Few-shot learning is challenging due to the limited data and labels.
Active Learning to Classify Macromolecular Structures in situ for Less Supervision in Cryo-Electron Tomography
Deep learning based subtomogram classification have played critical roles for such tasks.
On the Consistency Loss for Leveraging Augmented Data to Learn Robust and Invariant Representations
Data augmentation is one of the most popular techniques for improving the robustness of neural networks.
Learning Robust Models by Countering Spurious Correlations
In this paper, we formally study the generalization error bound for this setup with the knowledge of how the spurious features are associated with the label.
Squared $\ell_2$ Norm as Consistency Loss for Leveraging Augmented Data to Learn Robust and Invariant Representations
Data augmentation is one of the most popular techniques for improving the robustness of neural networks.
Word Shape Matters: Robust Machine Translation with Visual Embedding
Neural machine translation has achieved remarkable empirical performance over standard benchmark datasets, yet recent evidence suggests that the models can still fail easily dealing with substandard inputs such as misspelled words, To overcome this issue, we introduce a new encoding heuristic of the input symbols for character-level NLP models: it encodes the shape of each character through the images depicting the letters when printed.
Self-Challenging Improves Cross-Domain Generalization
We introduce a simple training heuristic, Representation Self-Challenging (RSC), that significantly improves the generalization of CNN to the out-of-domain data.
Ranked #35 on
Domain Generalization
on PACS
High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
We investigate the relationship between the frequency spectrum of image data and the generalization behavior of convolutional neural networks (CNN).
Smooth Kernels Improve Adversarial Robustness and Perceptually-Aligned Gradients
Recent research has shown that CNNs are often overly sensitive to high-frequency textural patterns.
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
We first learn a biased model that only uses features that are known to relate to dataset bias.
Learning Robust Global Representations by Penalizing Local Predictive Power
Despite their renowned predictive power on i. i. d.
Ranked #114 on
Domain Generalization
on PACS
High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
We investigate the relationship between the frequency spectrum of image data and the generalization behavior of convolutional neural networks (CNN).
Learning Robust Representations by Projecting Superficial Statistics Out
We test our method on the battery of standard domain generalization data sets and, interestingly, achieve comparable or better performance as compared to other domain generalization methods that explicitly require samples from the target distribution for training.
Ranked #123 on
Domain Generalization
on PACS
What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
In this paper, we further investigate the statistical irregularities, what we refer as confounding factors, of the NLI data sets.
Removing Confounding Factors Associated Weights in Deep Neural Networks Improves the Prediction Accuracy for Healthcare Applications
The proliferation of healthcare data has brought the opportunities of applying data-driven approaches, such as machine learning methods, to assist diagnosis.
On the Origin of Deep Learning
This paper is a review of the evolutionary history of deep learning models.
SeDMiD for Confusion Detection: Uncovering Mind State from Time Series Brain Wave Data
In this paper, we propose an extension of State Space Model to work with different sources of information together with its learning and inference algorithms.
Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysis
In this paper, we propose a Select-Additive Learning (SAL) procedure that improves the generalizability of trained neural networks for multimodal sentiment analysis.
Evaluating Protein-protein Interaction Predictors with a Novel 3-Dimensional Metric
In order for the predicted interactions to be directly adopted by biologists, the ma- chine learning predictions have to be of high precision, regardless of recall.
A Survey: Time Travel in Deep Learning Space: An Introduction to Deep Learning Models and How Deep Learning Models Evolved from the Initial Ideas
Further, we will also look into the development history of modelling time series data with neural networks.
Evaluation of Protein-protein Interaction Predictors with Noisy Partially Labeled Data Sets
In this paper, we focused on the problem that non-availability of accurately labeled testing data sets in the domain of protein-protein interaction (PPI) prediction may lead to biased evaluation results.
Multimodal Transfer Deep Learning with Applications in Audio-Visual Recognition
We propose a transfer deep learning (TDL) framework that can transfer the knowledge obtained from a single-modal neural network to a network with a different modality.
Discovery of Important Crossroads in Road Network using Massive Taxi Trajectories
However, none of existing approaches addresses the problem of identifying network-wide important crossroads in real road network.
