Search Results for author:
Found 12 papers, 6 papers with code
Language models scale reliably with over-training and on downstream tasks
We fit scaling laws that extrapolate in both the number of model parameters and the ratio of training tokens to parameters.
Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuning
Despite vision-language models' (VLMs) remarkable capabilities as versatile visual assistants, two substantial challenges persist within the existing VLM frameworks: (1) lacking task diversity in pretraining and visual instruction tuning, and (2) annotation error and bias in GPT-4 synthesized instruction tuning data.
DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training
FlashAttention (Dao, 2023) effectively reduces the quadratic peak memory usage to linear in training transformer-based large language models (LLMs) on a single GPU.
VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use
These descriptions enable 1) collecting human-verified reference outputs for each instance; and 2) automatic evaluation of candidate multimodal generations using a text-only LLM, aligning with human judgment.
Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment
To tackle this problem, we show that relation alignment can be enforced by encouraging the directed language attention from 'mug' to 'grass' (capturing the semantic relation 'in') to match the directed visual attention from the mug to the grass.
Does Structural Attention Improve Compositional Representations in Vision-Language Models?
Although scaling self-supervised approaches has gained widespread success in Vision-Language pre-training, a number of works providing structural knowledge of visually-grounded semantics have recently shown incremental performance gains.
Ranked #27 on
Visual Reasoning
on Winoground
MPCFormer: fast, performant and private Transformer inference with MPC
Through extensive evaluations, we show that MPCFORMER significantly speeds up Transformer inference in MPC settings while achieving similar ML performance to the input model.
How and When Adversarial Robustness Transfers in Knowledge Distillation?
Our comprehensive analysis shows several novel insights that (1) With KDIGA, students can preserve or even exceed the adversarial robustness of the teacher model, even when their models have fundamentally different architectures; (2) KDIGA enables robustness to transfer to pre-trained students, such as KD from an adversarially trained ResNet to a pre-trained ViT, without loss of clean accuracy; and (3) Our derived local linearity bounds for characterizing adversarial robustness in KD are consistent with the empirical results.
On the Adversarial Robustness of Vision Transformers
Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision.
Robust Text CAPTCHAs Using Adversarial Examples
At the second stage, we design and apply a highly transferable adversarial attack for text CAPTCHAs to better obstruct CAPTCHA solvers.
Stochastic Channel-Based Federated Learning for Medical Data Privacy Preserving
Specifically, we design, implement and evaluate a channel-based update algorithm for the central server in a distributed system, which selects the channels with regard to the most active features in a training loop and uploads them as learned information from local datasets.
Privacy Preserving Stochastic Channel-Based Federated Learning with Neural Network Pruning
Artificial neural network has achieved unprecedented success in a wide variety of domains such as classifying, predicting and recognizing objects.
