Search Results for author:
Found 21 papers, 9 papers with code
GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation
We first benchmark MM-Navigator on our collected iOS screen dataset.
GPT-4V(ision) as a Generalist Evaluator for Vision-Language Tasks
Automatically evaluating vision-language tasks is challenging, especially when it comes to reflecting human judgments due to limitations in accounting for fine-grained details.
Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving
In the context of human-assisted or autonomous driving, explainability models can help user acceptance and understanding of decisions made by the autonomous vehicle, which can be used to rationalize and explain driver or vehicle behavior.
MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation
Curated datasets for healthcare are often limited due to the need of human annotations from experts.
Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models
Medical image classification is a critical problem for healthcare, with the potential to alleviate the workload of doctors and facilitate diagnoses of patients.
Learning Concise and Descriptive Attributes for Visual Recognition
Recent advances in foundation models present new opportunities for interpretable visual recognition -- one can first query Large Language Models (LLMs) to obtain a set of attributes that describe each class, then apply vision-language models to classify images via these attributes.
Comparing Apples to Apples: Generating Aspect-Aware Comparative Sentences from User Reviews
It is time-consuming to find the best product among many similar alternatives.
"Nothing Abnormal": Disambiguating Medical Reports via Contrastive Knowledge Infusion
Based on our analysis, we define a disambiguation rewriting task to regenerate an input to be unambiguous while preserving information about the original content.
CLIP also Understands Text: Prompting CLIP for Phrase Understanding
However, the application of its text encoder solely for text understanding has been less explored.
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation
Recent advances in text-to-image synthesis make it possible to visualize machine imaginations for a given context.
Personalized Showcases: Generating Multi-Modal Explanations for Recommendations
In this paper, to further enrich explanations, we propose a new task named personalized showcases, in which we provide both textual and visual information to explain our recommendations.
Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation
Radiology report generation aims at generating descriptive text from radiology images automatically, which may present an opportunity to improve radiology reporting and interpretation.
ImaginE: An Imagination-Based Automatic Evaluation Metric for Natural Language Generation
Automatic evaluations for natural language generation (NLG) conventionally rely on token-level or embedding-level comparisons with text references.
L2C: Describing Visual Differences Needs Semantic Understanding of Individuals
Recent advances in language and vision push forward the research of captioning a single image to describing visual differences between image pairs.
Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation
Outdoor vision-and-language navigation (VLN) is such a task where an agent follows natural language instructions and navigates a real-life urban environment.
Ranked #4 on
Vision and Language Navigation
on Touchdown Dataset
(using extra training data)
Cross-Lingual Vision-Language Navigation
Commanding a robot to navigate with natural language instructions is a long-term goal for grounded language understanding and robotics.
CosRec: 2D Convolutional Neural Networks for Sequential Recommendation
Sequential patterns play an important role in building modern recommender systems.
FairST: Equitable Spatial and Temporal Demand Prediction for New Mobility Systems
Emerging transportation modes, including car-sharing, bike-sharing, and ride-hailing, are transforming urban mobility but have been shown to reinforce socioeconomic inequities.
PA3D: Pose-Action 3D Machine for Video Recognition
Recent studies have witnessed the successes of using 3D CNNs for video action recognition.
Ranked #2 on
Skeleton Based Action Recognition
on J-HMDB
Predicting Abandonment in Online Coding Tutorials
Learners regularly abandon online coding tutorials when they get bored or frustrated, but there are few techniques for anticipating this abandonment to intervene.
