Search Results for author:
Found 44 papers, 32 papers with code
AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
This work represents a crucial step toward establishing global standards for AI risk and reliability evaluation while acknowledging the need for continued development in areas such as multiturn interactions, multimodal understanding, coverage of additional languages, and emerging hazard categories.
MSTS: A Multimodal Safety Test Suite for Vision-Language Models
Finally, we explore the automation of VLM safety assessments, finding even the best safety classifiers to be lacking.
LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Gaps
Building safe Large Language Models (LLMs) across multiple languages is essential in ensuring both safe access and linguistic diversity.
SCAR: Sparse Conditioned Autoencoders for Concept Detection and Steering in LLMs
Large Language Models (LLMs) have demonstrated remarkable capabilities in generating human-like text, but their output may not be aligned with the user or even produce harmful content.
Core Tokensets for Data-efficient Sequential Training of Transformers
Deep networks are frequently tuned to novel tasks and continue learning from ongoing data streams.
Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning
Prompt injection (both direct and indirect) and jailbreaking are now recognized as significant issues for large language models (LLMs), particularly due to their potential for harm in application-integrated contexts.
T-FREE: Subword Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings
Tokenizers are crucial for encoding information in Large Language Models, but their development has recently stagnated, and they contain inherent weaknesses.
LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Models
This paper introduces LlavaGuard, a suite of VLM-based vision safeguards that address the critical need for reliable guardrails in the era of large-scale data and models.
Introducing v0.5 of the AI Safety Benchmark from MLCommons
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming
When building Large Language Models (LLMs), it is paramount to bear safety in mind and protect them with guardrails.
DeiSAM: Segment Anything with Deictic Prompting
To remedy this issue, we propose DeiSAM -- a combination of large pre-trained neural networks with differentiable logic reasoners -- for deictic promptable segmentation.
Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You
Our results show that not only do models exhibit strong gender biases but they also behave differently across languages.
LEDITS++: Limitless Image Editing using Text-to-Image Models
Our results demonstrate the capabilities of LEDITS++ and its improvements over previous methods.
Divergent Token Metrics: Measuring degradation to prune away LLM components -- and optimize quantization
Large Language Models (LLMs) have reshaped natural language processing with their impressive capabilities.
Distilling Adversarial Prompts from Safety Benchmarks: Report for the Adversarial Nibbler Challenge
Text-conditioned image generation models have recently achieved astonishing image quality and alignment results.
Mitigating Inappropriateness in Image Generation: Can there be Value in Reflecting the World's Ugliness?
Text-conditioned image generation models have recently achieved astonishing results in image quality and text alignment and are consequently employed in a fast-growing number of applications.
MultiFusion: Fusing Pre-Trained Models for Multi-Lingual, Multi-Modal Image Generation
The recent popularity of text-to-image diffusion models (DM) can largely be attributed to the intuitive interface they provide to users.
Class Attribute Inference Attacks: Inferring Sensitive Class Information by Diffusion-Based Attribute Manipulations
Neural network-based image classifiers are powerful tools for computer vision tasks, but they inadvertently reveal sensitive attribute information about their classes, raising concerns about their privacy.
Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness
Generative AI models have recently achieved astonishing results in quality and are consequently employed in a fast-growing number of applications.
SEGA: Instructing Text-to-Image Models using Semantic Guidance
This leaves the user with little semantic control.
AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
Generative transformer models have become increasingly complex, with large numbers of parameters and the ability to process multiple input modalities.
The Stable Artist: Steering Semantics in Diffusion Latent Space
Large, text-conditioned generative diffusion models have recently gained a lot of attention for their impressive performance in generating high-fidelity images from text alone.
Speaking Multiple Languages Affects the Moral Bias of Language Models
Do the models capture moral norms from English and impose them on other languages?
Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
Text-conditioned image generation models have recently achieved astonishing results in image quality and text alignment and are consequently employed in a fast-growing number of applications.
Revision Transformers: Instructing Language Models to Change their Values
In this work, we question the current common practice of storing all information in the model parameters and propose the Revision Transformer (RiT) to facilitate easy model updating.
LAION-5B: An open large-scale dataset for training next generation image-text models
We show successful replication and fine-tuning of foundational models like CLIP, GLIDE and Stable Diffusion using the dataset, and discuss further experiments enabled with an openly available dataset of this scale.
Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis
Models for text-to-image synthesis, such as DALL-E~2 and Stable Diffusion, have recently drawn a lot of interest from academia and the general public.
Does CLIP Know My Face?
Our large-scale experiments on CLIP demonstrate that individuals used for training can be identified with very high accuracy.
LogicRank: Logic Induced Reranking for Generative Text-to-Image Systems
Text-to-image models have recently achieved remarkable success with seemingly accurate samples in photo-realistic quality.
ILLUME: Rationalizing Vision-Language Models through Human Interactions
Bootstrapping from pre-trained language models has been proven to be an efficient approach for building vision-language models (VLM) for tasks such as image captioning or visual question answering.
Do Multilingual Language Models Capture Differing Moral Norms?
Massively multilingual sentence representations are trained on large corpora of uncurated data, with a very imbalanced proportion of languages included in the training.
A Typology for Exploring the Mitigation of Shortcut Behavior
In addition, we discuss existing and introduce novel measures and benchmarks for evaluating the overall abilities of a XIL method.
Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?
This calls for increased dataset documentation, e. g., using datasheets.
Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations
In this work, we show the advantages of prototype representations for understanding and revising the latent space of neural concept learners.
Inferring Offensiveness In Images From Natural Language Supervision
Probing or fine-tuning (large-scale) pre-trained models results in state-of-the-art performance for many NLP tasks and, more recently, even for computer vision tasks when combined with image data.
Interactively Providing Explanations for Transformer Language Models
Transformer language models are state of the art in a multitude of NLP tasks.
Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do
That is, we show that these norms can be captured geometrically by a direction, which can be computed, e. g., by a PCA, in the embedding space, reflecting well the agreement of phrases to social norms implicitly expressed in the training texts and providing a path for attenuating or even preventing toxic degeneration in LMs.
Adaptive Rational Activations to Boost Deep Reinforcement Learning
Latest insights from biology show that intelligence not only emerges from the connections between neurons but that individual neurons shoulder more computational responsibility than previously anticipated.
Ranked #3 on
Atari Games
on Atari 2600 Skiing
(using extra training data)
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations
Most explanation methods in deep learning map importance estimates for a model's prediction back to the original input space.
Making deep neural networks right for the right scientific reasons by interacting with their explanations
Deep neural networks have shown excellent performances in many real-world applications.
BERT has a Moral Compass: Improvements of ethical and moral values of machines
But has BERT also a better moral compass?
Meta-Learning Runge-Kutta
However, by studying the characteristics of the local error function we show that including the partial derivatives of the initial value problem is favorable.
Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks
The performance of deep network learning strongly depends on the choice of the non-linear activation function associated with each neuron.
Neural Conditional Gradients
The move from hand-designed to learned optimizers in machine learning has been quite successful for gradient-based and -free optimizers.
