Search Results for author:
Found 10 papers, 5 papers with code
A Picture Is Worth a Graph: Blueprint Debate on Graph for Multimodal Reasoning
The study addresses two key challenges: the trivialization of opinions resulting from excessive summarization and the diversion of focus caused by distractor concepts introduced from images.
Generative Active Learning for Image Synthesis Personalization
The primary challenge in conducting active learning on generative models lies in the open-ended nature of querying, which differs from the closed form of querying in discriminative models that typically target a single concept.
Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue
Tuning pretrained language models for dialogue generation has been a prevalent paradigm for building capable dialogue agents.
Compositional Inversion for Stable Diffusion Models
It stems from the fact that during inversion, the irrelevant semantics in the user images are also encoded, forcing the inverted concepts to occupy locations far from the core distribution in the embedding space.
Untargeted Black-box Attacks for Social Recommendations
To perform untargeted attacks on social recommender systems, attackers can construct malicious social relationships for fake users to enhance the attack performance.
Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective
In this work, we propose a novel LLM-based framework (MolReGPT) for molecule-caption translation, where an In-Context Few-Shot Molecule Learning paradigm is introduced to empower molecule discovery with LLMs like ChatGPT to perform their in-context learning capability without domain-specific pre-training and fine-tuning.
Ranked #3 on
Text-based de novo Molecule Generation
on ChEBI-20
Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey
We also give visualization and analysis of the model parameters and results on representative downstream tasks.
Conceptor Learning for Class Activation Mapping
Class Activation Mapping (CAM) has been widely adopted to generate saliency maps which provides visual explanations for deep neural networks (DNNs).
Attention on Attention for Image Captioning
In this paper, we propose an Attention on Attention (AoA) module, which extends the conventional attention mechanisms to determine the relevance between attention results and queries.
ParNet: Position-aware Aggregated Relation Network for Image-Text matching
The first step, we call it intra-modal relation mechanism, in which we computes responses between different objects in an image or different words in a sentence separately; The second step, we call it inter-modal relation mechanism, in which the query plays a role of textual context to refine the relationship among object proposals in an image.
