Search Results for author:
Found 75 papers, 46 papers with code
Transformer Explainer: Interactive Learning of Text-Generative Models
Transformers have revolutionized machine learning, yet their inner workings remain opaque to many.
MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation
To address the pressing need for client-side dense retrieval, we introduce MeMemo, the first open-source JavaScript toolkit that adapts the state-of-the-art approximate nearest neighbor search technique HNSW to browser environments.
Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models
Safety alignment is the key to guiding the behaviors of large language models (LLMs) that are in line with human preferences and restrict harmful behaviors at inference time, but recent studies show that it can be easily compromised by finetuning with only a few adversarially designed training examples.
LLM Attributor: Interactive Visual Attribution for LLM Generation
Our library offers a new way to quickly attribute an LLM's text generation to training data points to inspect model behaviors, enhance its trustworthiness, and compare model-generated text with user-provided text.
UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining
We present UniTable, a training framework that unifies both the training paradigm and training objective of TR.
Self-Supervised Pre-Training for Table Structure Recognition Transformer
We discover that the performance gap between the linear projection transformer and the hybrid CNN-transformer can be mitigated by SSP of the visual encoder in the TSR model.
Mobile Fitting Room: On-device Virtual Try-on via Diffusion Models
The growing digital landscape of fashion e-commerce calls for interactive and user-friendly interfaces for virtually trying on clothes.
Wordflow: Social Prompt Engineering for Large Language Models
To investigate social prompt engineering, we introduce Wordflow, an open-source and social text editor that enables everyday users to easily create, run, share, and discover LLM prompts.
High-Performance Transformers for Table Structure Recognition Need Early Convolutions
This allows it to "see" an appropriate portion of the table and "store" the complex table structure within sufficient context length for the subsequent transformer.
Ranked #4 on
Table Recognition
on PubTabNet
REVAMP: Automated Simulations of Adversarial Attacks on Arbitrary Objects in Realistic Scenes
Generating these adversarial objects in the digital space has been extensively studied, however successfully transferring these attacks from the digital realm to the physical realm has proven challenging when controlling for real-world environmental factors.
ObjectComposer: Consistent Generation of Multiple Objects Without Fine-tuning
We build upon the recent BLIP-Diffusion model, which can generate images of single objects specified by reference images.
Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models
The generative process of Diffusion Models (DMs) has recently set state-of-the-art on many AI generation benchmarks.
Robust Principles: Architectural Design Principles for Adversarially Robust CNNs
Our research aims to unify existing works' diverging opinions on how architectural components affect the adversarial robustness of CNNs.
LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked
We test LLM Self Defense on GPT 3. 5 and Llama 2, two of the current most prominent LLMs against various types of attacks, such as forcefully inducing affirmative responses to prompts and prompt engineering attacks.
ManimML: Communicating Machine Learning Architectures with Animation
A user can take a preexisting neural network architecture and easily write a specification for an animation in ManimML, which will then automatically compose animations for different components of the system into a final animation of the entire neural network.
WizMap: Scalable Interactive Visualization for Exploring Large Machine Learning Embeddings
Machine learning models often learn latent embedding representations that capture the domain semantics of their training data.
SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks
Through this study, we identify key design implications and trade-offs, such as leveraging multimodal data in notebooks as well as balancing the degree of visualization-notebook integration.
Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion
Diffusion Explainer tightly integrates a visual overview of Stable Diffusion's complex structure with explanations of the underlying operations.
WebSHAP: Towards Explaining Any Machine Learning Models Anywhere
As machine learning (ML) is increasingly integrated into our everyday Web experience, there is a call for transparent and explainable web-based ML.
GAM Coach: Towards Interactive and User-centered Algorithmic Recourse
Machine learning (ML) recourse techniques are increasingly used in high-stakes domains, providing end users with actions to alter ML predictions, but they assume ML developers understand what input variables can be changed.
Energy Transformer
Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory.
Lessons from the Development of an Anomaly Detection Interface on the Mars Perseverance Rover using the ISHMAP Framework
While anomaly detection stands among the most important and valuable problems across many scientific domains, anomaly detection research often focuses on AI methods that can lack the nuance and interpretability so critical to conducting scientific inquiry.
RobArch: Designing Robust Architectures against Adversarial Attacks
Adversarial Training is the most effective approach for improving the robustness of Deep Neural Networks (DNNs).
DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models
With recent advancements in diffusion models, users can generate high-quality images by writing text prompts in natural language.
NeuroMapper: In-browser Visualizer for Neural Network Training
We present our ongoing work NeuroMapper, an in-browser visualization tool that helps machine learning (ML) developers interpret the evolution of a model during training, providing a new way to monitor the training process and visually discover reasons for suboptimal training.
IMB-NAS: Neural Architecture Search for Imbalanced Datasets
In this paper, we propose a new and complementary direction for improving performance on long tailed datasets - optimizing the backbone architecture through neural architecture search (NAS).
TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization
Given thousands of equally accurate machine learning (ML) models, how can users choose among them?
Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values
Machine learning (ML) interpretability techniques can reveal undesirable patterns in data that models exploit to make predictions--potentially causing harms once deployed.
Visual Auditor: Interactive Visualization for Detection and Summarization of Model Biases
Recent research has developed algorithms for effectively identifying intersectional bias in the form of interpretable, underperforming subsets (or slices) of the data.
VisCUIT: Visual Auditor for Bias in CNN Image Classifier
CNN image classifiers are widely used, thanks to their efficiency and accuracy.
Hear No Evil: Towards Adversarial Robustness of Automatic Speech Recognition via Multi-Task Learning
In this work, we investigate the impact of performing such multi-task learning on the adversarial robustness of ASR models in the speech domain.
SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning
Person tracking using computer vision techniques has wide ranging applications such as autonomous driving, home security and sports analytics.
Concept Evolution in Deep Learning Training: A Unified Interpretation Framework and Discoveries
We present ConceptEvo, a unified interpretation framework for deep neural networks (DNNs) that reveals the inception and evolution of learned concepts during training.
GAM Changer: Editing Generalized Additive Models with Interactive Visualization
Recent strides in interpretable machine learning (ML) research reveal that models exploit undesirable patterns in the data to make predictions, which potentially causes harms in deployment.
A Search Engine for Discovery of Scientific Challenges and Directions
To address this problem, we present a novel task of extraction and search of scientific challenges and directions, to facilitate rapid knowledge discovery.
NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks
Through a large-scale human evaluation, we demonstrate that our technique discovers neuron groups that represent coherent, human-meaningful concepts.
Quantifying the Impact of Human Capital, Job History, and Language Factors on Job Seniority with a Large-scale Analysis of Resumes
As job markets worldwide have become more competitive and applicant selection criteria have become more opaque, and different (and sometimes contradictory) information and advice is available for job seekers wishing to progress in their careers, it has never been more difficult to determine which factors in a r\'esum\'e most effectively help career progression.
EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models
The advent of larger machine learning (ML) models have improved state-of-the-art (SOTA) performance in various modeling tasks, ranging from computer vision to natural language.
Dodrio: Exploring Transformer Models with Interactive Visualization
Why do large pre-trained transformer-based models perform so well across a wide variety of NLP tasks?
Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learning
Training deep neural networks for automatic speech recognition (ASR) requires large amounts of transcribed speech.
RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization
With the widespread use of toxic language online, platforms are increasingly using automated systems that leverage advances in natural language processing to automatically flag and remove toxic comments.
MalNet: A Large-Scale Image Database of Malicious Software
Computer vision is playing an increasingly important role in automated malware detection with the rise of the image-based binary representation.
SkeletonVis: Interactive Visualization for Understanding Adversarial Attacks on Human Action Recognition Models
Skeleton-based human action recognition technologies are increasingly used in video based applications, such as home robotics, healthcare on aging population, and surveillance.
A Large-Scale Database for Graph Representation Learning
With the rapid emergence of graph representation learning, the construction of new large-scale datasets is necessary to distinguish model capabilities and accurately assess the strengths and weaknesses of each technique.
A Comparative Analysis of Industry Human-AI Interaction Guidelines
We have then used this framework to compare each of the surveyed companies to find differences in areas of emphasis.
Human-Computer Interaction
Examining the Ordering of Rhetorical Strategies in Persuasive Requests
We find that specific (orderings of) strategies interact uniquely with a request's content to impact success rate, and thus the persuasiveness of a request.
Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations
This paper systematically derives design dimensions for the structured evaluation of explainable artificial intelligence (XAI) approaches.
Bluff: Interactively Deciphering Adversarial Attacks on Deep Neural Networks
Deep neural networks (DNNs) are now commonly used in many domains.
Mapping Researchers with PeopleMap
Discovering research expertise at universities can be a difficult task.
ELF: An Early-Exiting Framework for Long-Tailed Classification
The natural world often follows a long-tailed data distribution where only a few classes account for most of the examples.
Ranked #40 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
PeopleMap: Visualization Tool for Mapping Out Researchers using Natural Language Processing
Discovering research expertise at institutions can be a difficult task.
Evaluating Graph Vulnerability and Robustness using TIGER
By democratizing the tools required to study network robustness, our goal is to assist researchers and practitioners in analyzing their own networks; and facilitate the development of new research in the field.
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization
Deep learning's great success motivates many practitioners and students to learn about this exciting technology.
UnMask: Adversarial Detection and Defense Through Robust Feature Alignment
UnMask detects such attacks and defends the model by rectifying the misclassification, re-classifying the image based on its robust features.
REST: Robust and Efficient Neural Networks for Sleep Monitoring in the Wild
By deploying these models to an Android application on a smartphone, we quantitatively observe that REST allows models to achieve up to 17x energy reduction and 9x faster inference.
Massif: Interactive Interpretation of Adversarial Attacks on Deep Learning
Deep neural networks (DNNs) are increasingly powering high-stakes applications such as autonomous cars and healthcare; however, DNNs are often treated as "black boxes" in such applications.
CNN 101: Interactive Visual Learning for Convolutional Neural Networks
The success of deep learning solving previously-thought hard problems has inspired many non-experts to learn and understand this exciting technology.
RECAST: Interactive Auditing of Automatic Toxicity Detection Models
As toxic language becomes nearly pervasive online, there has been increasing interest in leveraging the advancements in natural language processing (NLP), from very large transformer models to automatically detecting and removing toxic comments.
ElectroLens: Understanding Atomistic Simulations Through Spatially-resolved Visualization of High-dimensional Features
In recent years, machine learning (ML) has gained significant popularity in the field of chemical informatics and electronic structure theory.
NeuralDivergence: Exploring and Understanding Neural Networks by Comparing Activation Distributions
As deep neural networks are increasingly used in solving high-stake problems, there is a pressing need to understand their internal decision mechanisms.
Talk Proposal: Towards the Realistic Evaluation of Evasion Attacks using CARLA
In this talk we describe our content-preserving attack on object detectors, ShapeShifter, and demonstrate how to evaluate this threat in realistic scenarios.
FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
We present FairVis, a mixed-initiative visual analytics system that integrates a novel subgroup discovery technique for users to audit the fairness of machine learning models.
Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations
Deep learning is increasingly used in decision-making tasks.
GOGGLES: Automatic Image Labeling with Affinity Coding
We build the GOGGLES system that implements affinity coding for labeling image datasets by designing a novel set of reusable affinity functions for images, and propose a novel hierarchical generative model for class inference using a small development set.
The Efficacy of SHIELD under Different Threat Models
To evaluate the robustness of the defense against an adaptive attacker, we consider the targeted-attack success rate of the Projected Gradient Descent (PGD) attack, which is a strong gradient-based adversarial attack proposed in adversarial machine learning research.
GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
Recent success in deep learning has generated immense interest among practitioners and students, inspiring many to learn about this new technology.
Interactive Classification for Deep Learning Interpretation
We present an interactive system enabling users to manipulate images to explore the robustness and sensitivity of deep learning image classifiers.
ADAGIO: Interactive Experimentation with Adversarial Attack and Defense for Audio
Adversarial machine learning research has recently demonstrated the feasibility to confuse automatic speech recognition (ASR) models by introducing acoustically imperceptible perturbations to audio samples.
ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector
Given the ability to directly manipulate image pixels in the digital input space, an adversary can easily generate imperceptible perturbations to fool a Deep Neural Network (DNN) image classifier, as demonstrated in prior work.
Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression
The rapidly growing body of research in adversarial machine learning has demonstrated that deep neural networks (DNNs) are highly vulnerable to adversarially generated images.
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
We present a survey of the role of visual analytics in deep learning research, which highlights its short yet impactful history and thoroughly summarizes the state-of-the-art using a human-centered interrogative framework, focusing on the Five W's and How (Why, Who, What, How, When, and Where).
Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression
Deep neural networks (DNNs) have achieved great success in solving a variety of machine learning (ML) problems, especially in the domain of image recognition.
ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models
While deep learning models have achieved state-of-the-art accuracies for many prediction tasks, understanding these models remains a challenge.
M3: Scaling Up Machine Learning via Memory Mapping
To process data that do not fit in RAM, conventional wisdom would suggest using distributed approaches.
Communication Efficient Distributed Agnostic Boosting
We consider the problem of learning from distributed data in the agnostic setting, i. e., in the presence of arbitrary forms of noise.
