Search Results for author:
Found 95 papers, 53 papers with code
Self-supervised Motion Representation via Scattering Local Motion Cues
Motion representation is key to many computer vision problems but has never been well studied in the literature.
PuzzleBench: A Fully Dynamic Evaluation Framework for Large Multimodal Models on Puzzle Solving
Large Multimodal Models (LMMs) have demonstrated impressive capabilities across a wide range of multimodal tasks, achieving ever-increasing performance on various evaluation benchmarks.
The Tenth NTIRE 2025 Efficient Super-Resolution Challenge Report
This paper presents a comprehensive review of the NTIRE 2025 Challenge on Single-Image Efficient Super-Resolution (ESR).
Information Density Principle for MLLM Benchmarks
Therefore, we propose a critical principle of Information Density, which examines how much insight a benchmark can provide for the development of MLLMs.
Image Quality Assessment: From Human to Machine Preference
Considering the huge gap between human and machine visual systems, this paper proposes the topic: Image Quality Assessment for Machine Vision for the first time.
Towards All-in-One Medical Image Re-Identification
ComPA condenses medical content into a continuous modality representation and dynamically adjusts the modality-agnostic model with modality-specific parameters at runtime.
Text-to-SQL Domain Adaptation via Human-LLM Collaborative Data Annotation
Text-to-SQL models, which parse natural language (NL) questions to executable SQL queries, are increasingly adopted in real-world applications.
Interactive Symbolic Regression through Offline Reinforcement Learning: A Co-Design Framework
Symbolic Regression (SR) holds great potential for uncovering underlying mathematical and physical relationships from observed data.
ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMs
Creating data reports is time-consuming, as it requires iterative exploration and understanding of data, followed by summarizing the insights.
CSF-Net: Cross-Modal Spatiotemporal Fusion Network for Pulmonary Nodule Malignancy Predicting
These findings indicate that our multimodal spatiotemporal fusion approach, which combines follow-up data with clinical information, surpasses existing methods, underscoring its effectiveness in predicting nodule malignancy.
Medical Manifestation-Aware De-Identification
Face de-identification (DeID) has been widely studied for common scenes, but remains under-researched for medical scenes, mostly due to the lack of large-scale patient face datasets.
How Much Can Time-related Features Enhance Time Series Forecasting?
Recent advancements in long-term time series forecasting (LTSF) have primarily focused on capturing cross-time and cross-variate (channel) dependencies within historical data.
CP-UNet: Contour-based Probabilistic Model for Medical Ultrasound Images Segmentation
We design a novel down-sampling module to enable the contour probability distribution modeling and encoding stages to acquire global-local features.
InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translation
To achieve this goal, we introduce InterTrans, an LLM-based automated code translation approach that, in contrast to existing approaches, leverages intermediate translations across PLs to bridge the syntactic and semantic gaps between source and target PLs.
Rethinking Data Selection at Scale: Random Selection is Almost All You Need
Supervised fine-tuning (SFT) is crucial for aligning Large Language Models (LLMs) with human instructions.
R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?
However, various corruptions in the real world mean that images will not be as ideal as in simulations, presenting significant challenges for the practical application of LMMs.
A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models
Our framework works with any library that provides API documentation and focuses on two unit tasks: API recommendation and code example generation, along with four metrics to evaluate the generated APIs and code examples, such as the proportion of incorrect API recommendations for Task 1, and the proportion of code examples where no specific API is invoked and uncompilable/unexecutable code examples for Task 2.
Free-VSC: Free Semantics from Visual Foundation Models for Unsupervised Video Semantic Compression
Unsupervised video semantic compression (UVSC), i. e., compressing videos to better support various analysis tasks, has recently garnered attention.
EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
In this work, we narrow this gap by conducting the first study on the privacy risks of generalist web agents in adversarial environments.
SQLucid: Grounding Natural Language Database Queries with Interactive Explanations
Though recent advances in machine learning have led to significant improvements in natural language interfaces for databases, the accuracy and reliability of these systems remain limited, especially in high-stakes domains.
Selective Prompt Anchoring for Code Generation
To mitigate this issue, we propose Selective Prompt Anchoring (SPA) to guide code LLMs to pay more attention to user intent when generating code.
BadMerging: Backdoor Attacks Against Model Merging
Considering that a merged model may incorporate tasks from different domains, BadMerging can jointly compromise the tasks provided by the adversary (on-task attack) and other contributors (off-task attack) and solve the corresponding unique challenges with novel attack designs.
Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models with Adaptive Expert Placement
The cost of even a single failure is significant, as all GPUs need to wait idle until the failure is resolved, potentially losing considerable training progress as training has to restart from checkpoints.
Solutions to Deepfakes: Can Camera Hardware, Cryptography, and Deep Learning Verify Real Images?
The exponential progress in generative AI poses serious implications for the credibility of all real images and videos.
GAIA: Rethinking Action Quality Assessment for AI-Generated Videos
Assessing action quality is both imperative and challenging due to its significant impact on the quality of AI-generated videos, further complicated by the inherently ambiguous nature of actions within AI-generated video (AIGV).
SMC++: Masked Learning of Unsupervised Video Semantic Compression
In this paper, we propose a Masked Video Modeling (MVM)-powered compression framework that particularly preserves video semantics, by jointly mining and compressing the semantics in a self-supervised manner.
Deep Space Separable Distillation for Lightweight Acoustic Scene Classification
Acoustic scene classification (ASC) is highly important in the real world.
Explainable Fake News Detection With Large Language Model via Defense Among Competing Wisdom
To detect fake news from a sea of diverse, crowded and even competing narratives, in this paper, we propose a novel defense-based explainable fake news detection framework.
Exploring the Impact of the Output Format on the Evaluation of Large Language Models for Code Translation
Code translation between programming languages is a long-existing and critical task in software engineering, facilitating the modernization of legacy systems, ensuring cross-platform compatibility, and enhancing software performance.
CarbonNet: How Computer Vision Plays a Role in Climate Change? Application: Learning Geomechanics from Subsurface Geometry of CCS to Mitigate Global Warming
We introduce a new approach using computer vision to predict the land surface displacement from subsurface geometry images for Carbon Capture and Sequestration (CCS).
PE-MVCNet: Multi-view and Cross-modal Fusion Network for Pulmonary Embolism Prediction
In a clinical setting, physicians tend to rely on the contextual information provided by Electronic Medical Records (EMR) to interpret medical imaging.
Sym-Q: Adaptive Symbolic Regression via Sequential Decision-Making
Our results demonstrate that Sym-Q excels not only in recovering underlying mathematical structures but also uniquely learns to efficiently refine the output expression based on reward signals, thereby discovering underlying expressions.
PeaTMOSS: A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Software
Our analysis of this dataset provides the first summary statistics for the PTM supply chain, showing the trend of PTM development and common shortcomings of PTM package documentation.
ZS4C: Zero-Shot Synthesis of Compilable Code for Incomplete Code Snippets using LLMs
On average, ZS4C can infer more accurate import statements (with an F1 score of 0. 98) than SnR, with an improvement of 8. 5% in the F1.
Studying and Recommending Information Highlighting in Stack Overflow Answers
Conclusion: Our findings suggest that it is possible to develop recommendation models for highlighting information for answers with different formatting styles on Stack Overflow.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Visual Question Answering (VQA)
on InfographicVQA
Where have you been? A Study of Privacy Risk for Point-of-Interest Recommendation
To better understand and quantify the privacy leakage in mobility data-based ML models, we design a privacy attack suite containing data extraction and membership inference attacks tailored for point-of-interest (POI) recommendation models, one of the most widely used mobility data-based ML models.
SoK: Pitfalls in Evaluating Black-Box Attacks
However, these works make different assumptions on the adversary's knowledge and current literature lacks a cohesive organization centered around the threat model.
Ultrasound Image Segmentation of Thyroid Nodule via Latent Semantic Feature Co-Registration
Segmentation of nodules in thyroid ultrasound imaging plays a crucial role in the detection and treatment of thyroid cancer.
PeaTMOSS: Mining Pre-Trained Models in Open-Source Software
Developing and training deep learning models is expensive, so software engineers have begun to reuse pre-trained deep learning models (PTMs) and fine-tune them for downstream tasks.
DHBE: Data-free Holistic Backdoor Erasing in Deep Neural Networks via Restricted Adversarial Distillation
Instead of the staged pipeline, the DHBE treats the backdoor erasing task as a unified adversarial procedure, which seeks equilibrium between two different competing processes: distillation and backdoor regularization.
Towards Better Entity Linking with Multi-View Enhanced Distillation
Aiming at learning entity representations that can match divergent mentions, this paper proposes a Multi-View Enhanced Distillation (MVD) framework, which can effectively transfer knowledge of multiple fine-grained and mention-relevant parts within entities from cross-encoders to dual-encoders.
Interactive Text-to-SQL Generation via Editable Step-by-Step Explanations
Many techniques have been proposed to automatically generate SQL from natural language, but they suffer from two issues: (1) they still make many mistakes, particularly for complex queries, and (2) they do not provide a flexible way for non-expert users to validate and refine incorrect queries.
Non-Semantics Suppressed Mask Learning for Unsupervised Video Semantic Compression
Most video compression methods aim to improve the decoded video visual quality, instead of particularly guaranteeing the semantic-completeness, which deteriorates downstream video analysis tasks, e. g., action recognition.
PLUE: Language Understanding Evaluation Benchmark for Privacy Policies in English
Privacy policies provide individuals with information about their rights and how their personal information is handled.
FastClass: A Time-Efficient Approach to Weakly-Supervised Text Classification
Weakly-supervised text classification aims to train a classifier using only class descriptions and unlabeled data.
Weakly Supervised Online Action Detection for Infant General Movements
Although general movements assessment(GMA) has shown promising results in early CP detection, it is laborious.
Conditional Supervised Contrastive Learning for Fair Text Classification
Contrastive representation learning has gained much attention due to its superior performance in learning representations from both image and sequential data.
FisheyeDistill: Self-Supervised Monocular Depth Estimation with Ordinal Distillation for Fisheye Cameras
In this paper, we deal with the problem of monocular depth estimation for fisheye cameras in a self-supervised manner.
CNN-Augmented Visual-Inertial SLAM with Planar Constraints
The CNN depth effectively bootstraps the back-end optimization of SLAM and meanwhile the CNN uncertainty adaptively weighs the contribution of each feature point to the back-end optimization.
Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies
Prior studies in privacy policies frame the question answering (QA) task as identifying the most relevant text segment or a list of sentences from a policy document given a user query.
Multi-agent Actor-Critic with Time Dynamical Opponent Model
In this paper, we propose to exploit the fact that the agents seek to improve their expected cumulative reward and introduce a novel \textit{Time Dynamical Opponent Model} (TDOM) to encode the knowledge that the opponent policies tend to improve over time.
A Coding Framework and Benchmark towards Low-Bitrate Video Understanding
The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved w. r. t.
A Prescriptive Dirichlet Power Allocation Policy with Deep Reinforcement Learning
Moreover, we demonstrate the applicability of the proposed algorithm on a prescriptive operation case, where we propose the Dirichlet power allocation policy and evaluate the performance on a case study of a set of multiple lithium-ion (Li-I) battery systems.
Towards Return Parity in Markov Decision Processes
We first provide a decomposition theorem for return disparity, which decomposes the return disparity of any two MDPs sharing the same state and action spaces into the distance between group-wise reward functions, the discrepancy of group policies, and the discrepancy between state visitation distributions induced by the group policies.
Automated Detection of GDPR Disclosure Requirements in Privacy Policies using Deep Active Learning
Since GDPR came into force in May 2018, companies have worked on their data practices to comply with this privacy law.
AnANet: Modeling Association and Alignment for Cross-modal Correlation Classification
Thus researchers study the definition of cross-modal correlation category and construct various classification systems and predictive models.
Self-Conditioned Probabilistic Learning of Video Rescaling
After optimization, the downscaled video by our framework preserves more meaningful information, which is beneficial for both the upscaling step and the downstream tasks, e. g., video action recognition task.
EAN: Event Adaptive Network for Enhanced Action Recognition
In this paper, we propose a unified action recognition framework to investigate the dynamic nature of video content by introducing the following designs.
Ranked #15 on
Action Recognition
on Something-Something V1
What and How long: Prediction of Mobile App Engagement
Based on the analysis, we further investigate a novel mobile app engagement prediction problem -- can we predict simultaneously what app the user will use next and how long he/she will stay on that app?
Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report
To address this problem, we introduce the first Mobile AI challenge, where the target is to develop quantized deep learning-based camera scene classification solutions that can demonstrate a real-time performance on smartphones and IoT platforms.
CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU
We then identify a sequence of "GPU-friendly" cryptographic protocols to enable privacy-preserving evaluation of both linear and non-linear operations on the GPU.
Understanding and Mitigating Accuracy Disparity in Regression
With the widespread deployment of large-scale prediction systems in high-stakes domains, e. g., face recognition, criminal justice, etc., disparity in prediction accuracy between different demographic subgroups has called for fundamental understanding on the source of such disparity and algorithmic intervention to mitigate it.
Using Prior Knowledge to Guide BERT's Attention in Semantic Textual Matching Tasks
We study the problem of incorporating prior knowledge into a deep Transformer-based model, i. e., Bidirectional Encoder Representations from Transformers (BERT), to enhance its performance on semantic textual matching tasks.
Meta Federated Learning
Due to its distributed methodology alongside its privacy-preserving features, Federated Learning (FL) is vulnerable to training time adversarial attacks.
SkillBot: Identifying Risky Content for Children in Alexa Skills
Finally, we identify a novel risk in the VPA ecosystem: confounding utterances, or voice commands shared by multiple apps that may cause a user to interact with a different app than intended.
Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
Data heterogeneity has been identified as one of the key features in federated learning but often overlooked in the lens of robustness to adversarial attacks.
Intent Classification and Slot Filling for Privacy Policies
We refer to predicting the privacy practice explained in a sentence as intent classification and identifying the text spans sharing specific information as slot filling.
Battery Model Calibration with Deep Reinforcement Learning
In this paper, we implement a Reinforcement Learning-based framework for reliably and efficiently inferring calibration parameters of battery models.
ToHRE: A Top-Down Classification Strategy with Hierarchical Bag Representation for Distantly Supervised Relation Extraction
Distantly Supervised Relation Extraction (DSRE) has proven to be effective to find relational facts from texts, but it still suffers from two main problems: the wrong labeling problem and the long-tail problem.
Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee
In comparison with the existing RL algorithms, the proposed method can achieve superior performance in terms of maintaining safety.
PolicyQA: A Reading Comprehension Dataset for Privacy Policies
Prior studies in this domain frame the QA task as retrieving the most relevant text segment or a list of sentences from the policy document given a question.
Dual-path CNN with Max Gated block for Text-Based Person Re-identification
With that in mind, a novel Dual-path CNN with Max Gated block (DCMG) is proposed to extract discriminative word embeddings and make visual-textual association concern more on remarkable features of both modalities.
Object Detection in the Context of Mobile Augmented Reality
Ordinary object detection approaches process information from the images only, and they are oblivious to the camera pose with regard to the environment and the scale of the environment.
Perceptron Synthesis Network: Rethinking the Action Scale Variances in Videos
More specifically, an \textit{action perceptron synthesizer} is proposed to generate the kernels from a bag of fixed-size kernels that are interacted by dense routing paths.
Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search
In this paper, we introduce a new reinforcement learning (RL) based neural architecture search (NAS) methodology for effective and efficient generative adversarial network (GAN) architecture search.
Ranked #19 on
Image Generation
on STL-10
Model-Targeted Poisoning Attacks with Provable Convergence
Our attack is the first model-targeted poisoning attack that provides provable convergence for convex models, and in our experiments, it either exceeds or matches state-of-the-art attacks in terms of attack success rate and distance to the target model.
Real-Time Model Calibration with Deep Reinforcement Learning
The dynamic, real-time, and accurate inference of model parameters from empirical data is of great importance in many scientific and engineering disciplines that use computational models (such as a digital twin) for the analysis and prediction of complex physical processes.
GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks
In this paper, we propose GraphLIME, a local interpretable model explanation for graphs using the Hilbert-Schmidt Independence Criterion (HSIC) Lasso, which is a nonlinear feature selection method.
$H_\infty$ Model-free Reinforcement Learning with Robust Stability Guarantee
In this paper, we introduce and extend the idea of robust stability and $H_\infty$ control to design policies with both stability and robustness guarantee.
Convex Reconstruction of Structured Matrix Signals from Linear Measurements (I): Theoretical Results
The first part is about conditions for stability and robustness in signal reconstruction via solving the convex programming from noise-free or noisy measurements. We establish uniform sufficient conditions which are very close to necessary conditions and non-uniform conditions are also discussed.
Model-free Learning Control of Nonlinear Stochastic Systems with Stability Guarantee
Reinforcement learning (RL) offers a principled way to achieve the optimal cumulative performance index in discrete-time nonlinear stochastic systems, which are modeled as Markov decision processes.
Variational Constrained Reinforcement Learning with Application to Planning at Roundabout
In this paper, we combine variational learning and constrained reinforcement learning to simultaneously learn a Conditional Representation Model (CRM) to encode the states into safe and unsafe distributions respectively as well as to learn the corresponding safe policy.
Adversarial Privacy Preservation under Attribute Inference Attack
With the prevalence of machine learning services, crowdsourced data containing sensitive information poses substantial privacy challenges.
A Novel Cascade Binary Tagging Framework for Relational Triple Extraction
Extracting relational triples from unstructured text is crucial for large-scale knowledge graph construction.
Ranked #5 on
Relation Extraction
on NYT11-HRL
Jointly Modeling Hierarchical and Horizontal Features for Relational Triple Extraction
In this work, we first introduce the hierarchical dependency and horizontal commonality between the two levels, and then propose an entity-enhanced dual tagging framework that enables the triple extraction (TE) task to utilize such interactions with self-learned entity features through an auxiliary entity extraction (EE) task, without breaking the joint decoding of relational triples.
Hybrid Batch Attacks: Finding Black-box Adversarial Examples with Limited Queries
In a black-box setting, the adversary only has API access to the target model and each query is expensive.
Cryptography and Security
Trade-offs and Guarantees of Adversarial Representation Learning for Information Obfuscation
Meanwhile, it is clear that in general there is a tension between minimizing information leakage and maximizing task accuracy.
PatchNet: A Tool for Deep Patch Classification
This work proposes PatchNet, an automated tool based on hierarchical deep learning for classifying patches by extracting features from commit messages and code changes.
Privacy Partitioning: Protecting User Data During the Deep Learning Inference Phase
We present a practical method for protecting data during the inference phase of deep learning based on bipartite topology threat modeling and an interactive adversarial deep network construction.
Understanding and Mitigating the Security Risks of Voice-Controlled Third-Party Skills on Amazon Alexa and Google Home
The significance of our findings have already been acknowledged by Amazon and Google, and further evidenced by the risky skills discovered on Alexa and Google markets by the new detection systems we built.
Cryptography and Security
Query-limited Black-box Attacks to Classifiers
Specifically, we consider the problem of attacking machine learning classifiers subject to a budget of feature modification cost while minimizing the number of queries, where each query returns only a class and confidence score.
WebAPIRec: Recommending Web APIs to Software Projects via Personalized Ranking
In this light, we propose a new, automated approach called WebAPIRec that takes as input a project profile and outputs a ranked list of {web} APIs that can be used to implement the project.
Understanding IoT Security Through the Data Crystal Ball: Where We Are Now and Where We Are Going to Be
We systemize this process, by proposing a taxonomy for the IoT ecosystem and organizing IoT security into five problem areas.
Cryptography and Security
Background Subtraction via Generalized Fused Lasso Foreground Modeling
Background Subtraction (BS) is one of the key steps in video analysis.
