Search Results for author:
Found 45 papers, 16 papers with code
AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding
We evaluated AURORA both on a set of 12 apps with known tarpits from prior work, and on a new set of five of the most popular apps from the Google Play store.
From Canteen Food to Daily Meals: Generalizing Food Recognition to More Practical Scenarios
These two datasets are used to evaluate the transferability of approaches from the well-curated food image domain to the everyday-life food image domain.
LLMBind: A Unified Modality-Task Integration Framework
In the multi-modal domain, the dependence of various models on specific input formats leads to user confusion and hinders progress.
Video Editing for Video Retrieval
The teacher model is employed to edit the clips in the training set whereas the student model trains on the edited clips.
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
In this work, we propose a simple yet effective training strategy MoE-Tuning for LVLMs.
Ranked #52 on
Visual Question Answering
on MM-Vet
FoodLMM: A Versatile Food Assistant using Large Multi-modal Model
In the second stage, we construct a multi-round conversation dataset and a reasoning segmentation dataset to fine-tune the model, enabling it to conduct professional dialogues and generate segmentation masks based on complex reasoning in the food domain.
Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models
Based on the evaluation, our work makes a key analysis of the underlying reason for the success of the attack, namely the inability of LLMs to distinguish between instructions and external content and the absence of LLMs' awareness to not execute instructions within external content.
CAR: Consolidation, Augmentation and Regulation for Recipe Retrieval
Learning recipe and food image representation in common embedding space is non-trivial but crucial for cross-modal recipe retrieval.
Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models
Video-based large language models (Video-LLMs) have been recently introduced, targeting both fundamental improvements in perception and comprehension, and a diverse range of user inquiries.
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
In this work, we unify visual representation into the language feature space to advance the foundational LLM towards a unified LVLM.
Ranked #2 on
Zero-Shot Video Question Answer
on TGIF-QA
X-Transfer: A Transfer Learning-Based Framework for GAN-Generated Fake Image Detection
Generative adversarial networks (GANs) have remarkably advanced in diverse domains, especially image generation and editing.
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
Ranked #1 on
Zero-shot Audio Classification
on VGG-Sound
(using extra training data)
Improving Cross-dataset Deepfake Detection with Deep Information Decomposition
Deepfake technology poses a significant threat to security and social trust.
Controlling Neural Style Transfer with Deep Reinforcement Learning
Controlling the degree of stylization in the Neural Style Transfer (NST) is a little tricky since it usually needs hand-engineering on hyper-parameters.
Image-to-Image Translation with Deep Reinforcement Learning
The key feature in the RL-I2IT framework is to decompose a monolithic learning process into small steps with a lightweight model to progressively transform a source image successively to a target image.
CgT-GAN: CLIP-guided Text GAN for Image Captioning
Particularly, we use adversarial training to teach CgT-GAN to mimic the phrases of an external text corpus and CLIP-based reward to provide semantic guidance.
MKL-$L_{0/1}$-SVM
This paper presents a Multiple Kernel Learning (abbreviated as MKL) framework for the Support Vector Machine (SVM) with the $(0, 1)$ loss function.
Towards Attack-tolerant Federated Learning via Critical Parameter Analysis
Federated learning is used to train a shared model in a decentralized way without clients sharing private data with each other.
FedDefender: Client-Side Attack-Tolerant Federated Learning
Evaluations of real-world scenarios across multiple datasets show that the proposed method enhances the robustness of federated learning against model poisoning attacks.
Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark
Companies have begun to offer Embedding as a Service (EaaS) based on these LLMs, which can benefit various natural language processing (NLP) tasks for customers.
Harnessing the Power of Text-image Contrastive Models for Automatic Detection of Online Misinformation
As growing usage of social media websites in the recent decades, the amount of news articles spreading online rapidly, resulting in an unprecedented scale of potentially fraudulent information.
An ADMM Solver for the MKL-$L_{0/1}$-SVM
We formulate the Multiple Kernel Learning (abbreviated as MKL) problem for the support vector machine with the infamous $(0, 1)$-loss function.
Attacking Important Pixels for Anchor-free Detectors
Existing adversarial attacks on object detection focus on attacking anchor-based detectors, which may not work well for anchor-free detectors.
On the Statistical Consistency of a Generalized Cepstral Estimator
We consider the problem to estimate the generalized cepstral coefficients of a stationary stochastic process or stationary multidimensional random field.
RMBench: Benchmarking Deep Reinforcement Learning for Robotic Manipulator Control
Reinforcement learning is applied to solve actual complex tasks from high-dimensional, sensory inputs.
Text-driven Video Prediction
Specifically, appearance and motion components are provided by the image and caption separately.
EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations
VISOR annotates videos from EPIC-KITCHENS, which comes with a new set of challenges not encountered in current video segmentation datasets.
Robust Quantity-Aware Aggregation for Federated Learning
Federated learning (FL) enables multiple clients to collaboratively train models without sharing their local data, and becomes an important privacy-preserving machine learning framework.
Cross-lingual Adaptation for Recipe Retrieval with Mixup
To bridge the domain gap, recipe mixup loss is proposed to enforce the intermediate domain to locate in the shortest geodesic path between source and target domains in the recipe embedding space.
Improving robustness of language models from a geometry-aware perspective
On top of FADA, we propose geometry-aware adversarial training (GAT) to perform adversarial training on friendly adversarial data so that we can save a large number of search steps.
OneLabeler: A Flexible System for Building Data Labeling Tools
In this paper, we propose a conceptual framework for data labeling and OneLabeler based on the conceptual framework to support easy building of labeling tools for diverse usage scenarios.
UA-FedRec: Untargeted Attack on Federated News Recommendation
Our study reveals a critical security issue in existing federated news recommendation systems and calls for research efforts to address the issue.
TREATED:Towards Universal Defense against Textual Adversarial Attacks
Recent work shows that deep neural networks are vulnerable to adversarial examples.
Transferable Adversarial Examples for Anchor Free Object Detection
Deep neural networks have been demonstrated to be vulnerable to adversarial attacks: subtle perturbation can completely change prediction result.
Imperceptible Adversarial Examples for Fake Image Detection
Fooling people with highly realistic fake images generated with Deepfake or GANs brings a great social disturbance to our society.
Pyramid Fusion Dark Channel Prior for Single Image Dehazing
In this paper, we propose the pyramid fusion dark channel prior (PF-DCP) for single image dehazing.
An Optimized H.266/VVC Software Decoder On Mobile Platform
As the successor of H. 265/HEVC, the new versatile video coding standard (H. 266/VVC) can provide up to 50% bitrate saving with the same subjective quality, at the cost of increased decoding complexity.
New Strong Bounds on sub-GeV Dark Matter from Boosted and Migdal Effects
Due to the low nuclear recoils, sub-GeV dark matter (DM) is usually beyond the sensitivity of the conventional DM direct detection experiments.
High Energy Physics - Phenomenology Cosmology and Nongalactic Astrophysics
Line Spectrum Representation for Vector Processes With Application to Frequency Estimation
A positive semidefinite Toeplitz matrix, which often arises as the finite covariance matrix of a stationary random process, can be decomposed as the sum of a nonnegative multiple of the identity corresponding to a white noise, and a singular term corresponding to a purely deterministic process.
CookGAN: Causality Based Text-to-Image Synthesis
Particularly, a cooking simulator sub-network is proposed to incrementally make changes to food images based on the interaction between ingredients and cooking methods over a series of steps.
CPM R-CNN: Calibrating Point-guided Misalignment in Object Detection
In object detection, offset-guided and point-guided regression dominate anchor-based and anchor-free method separately.
An Empirical Bayes Approach to Frequency Estimation
In this paper we show that the classical problem of frequency estimation can be formulated and solved efficiently in an empirical Bayesian framework by assigning a uniform a priori probability distribution to the unknown frequency.
Automatic Group Cohesiveness Detection With Multi-modal Features
The task is to predict the cohesive level for a group of people in images.
Graph Neural Networks for Image Understanding Based on Multiple Cues: Group Emotion Recognition and Event Recognition as Use Cases
A graph neural network (GNN) for image understanding based on multiple cues is proposed in this paper.
R2GAN: Cross-Modal Recipe Retrieval With Generative Adversarial Network
Representing procedure text such as recipe for crossmodal retrieval is inherently a difficult problem, not mentioning to generate image from recipe for visualization.
