Search Results for author:
Found 99 papers, 47 papers with code
DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning
The capacity for complex mathematical reasoning is a key benchmark for artificial intelligence.
Image Difference Grounding with Natural Language
We introduce DiffGround, a large-scale and high-quality dataset for IDG, containing image pairs with diverse visual variations along with instructions querying fine-grained differences.
Towards Unified Referring Expression Segmentation Across Omni-Level Visual Target Granularities
Referring expression segmentation (RES) aims at segmenting the entities' masks that match the descriptive language expression.
STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Models
Large Language Models (LLMs) have become increasingly vulnerable to jailbreak attacks that circumvent their safety mechanisms.
TimeZero: Temporal Video Grounding with Reasoning-Guided LVLM
We introduce TimeZero, a reasoning-guided LVLM designed for the temporal video grounding (TVG) task.
SeCap: Self-Calibrating and Adaptive Prompts for Cross-view Person Re-Identification in Aerial-Ground Networks
To address this issue, previous methods attempt to reduce the differences between viewpoints by critical attributes and decoupling the viewpoints.
VisBias: Measuring Explicit and Implicit Social Biases in Vision Language Models
To analyze explicit bias, we directly pose questions to VLMs related to gender and racial differences: (1) Multiple-choice questions based on a given image (e. g., "What is the education level of the person in the image?")
VisFactor: Benchmarking Fundamental Visual Cognition in Multimodal Large Language Models
Multimodal Large Language Models (MLLMs) have demonstrated remarkable advancements in multimodal understanding; however, their fundamental visual cognitive abilities remain largely underexplored.
Mitigating Data Scarcity in Time Series Analysis: A Foundation Model with Series-Symbol Data Generation
Building on this concept, we introduce a series-symbol (S2) dual-modulity data generation mechanism, enabling the unrestricted creation of high-quality time series data paired with corresponding symbolic representations.
A Survey of LLM-based Agents in Medicine: How far are we from Baymax?
Large Language Models (LLMs) are transforming healthcare through the development of LLM-based agents that can understand, reason about, and assist with medical tasks.
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
Multimodal Large Language Models (MLLMs) have expanded the capabilities of traditional language models by enabling interaction through both text and images.
VLMs as GeoGuessr Masters: Exceptional Performance, Hidden Biases, and Privacy Risks
Evaluating four VLMs, we find that while these models demonstrate the ability to recognize geographic information from images, achieving up to $53. 8\%$ accuracy in city prediction, they exhibit significant regional biases.
Making Them a Malicious Database: Exploiting Query Code to Jailbreak Aligned Large Language Models
By treating LLMs as knowledge databases, we translate malicious queries in natural language into structured non-natural query language to bypass the safety alignment mechanisms of LLMs.
EVEv2: Improved Baselines for Encoder-Free Vision-Language Models
Existing encoder-free vision-language models (VLMs) are rapidly narrowing the performance gap with their encoder-based counterparts, highlighting the promising potential for unified multimodal systems with structural simplicity and efficient deployment.
BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASR
Recently, the Mixture of Expert (MoE) architecture, such as LR-MoE, is often used to alleviate the impact of language confusion on the multilingual ASR (MASR) task.
How Should We Build A Benchmark? Revisiting 274 Code-Related Benchmarks For LLMs
We propose How2Bench, which is comprised of a 55-criteria checklist as a set of guidelines to govern the development of code-related benchmarks comprehensively.
MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs
Our study applies existing methods to the MRWeb problem using a newly curated dataset of 500 websites (300 synthetic, 200 real-world).
Sustainable Self-evolution Adversarial Training
With the wide application of deep neural network models in various computer vision tasks, there has been a proliferation of adversarial example generation strategies aimed at deeply exploring model security.
Medchain: Bridging the Gap Between LLM Agents and Clinical Practice through Interactive Sequential Benchmarking
To address this gap, we present MedChain, a dataset of 12, 163 clinical cases that covers five key stages of clinical workflow.
Improving the Transferability of Adversarial Attacks on Face Recognition with Diverse Parameters Augmentation
Experimental results demonstrate that our proposed attack method can effectively enhance the transferability of the crafted adversarial face examples.
On the Shortcut Learning in Multilingual Neural Machine Translation
By carefully designing experiments on different MNMT scenarios and models, we attribute the off-target issue to the overfitting of the shortcuts of (non-centric, centric) language mappings.
Dynamic Textual Prompt For Rehearsal-free Lifelong Person Re-identification
By leveraging this shared textual space as an anchor, we can prompt the ReID model to embed images from various domains into a unified semantic space, thereby alleviating catastrophic forgetting caused by domain shifts.
Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping
To address these limitations, we propose four enhancement strategies: Interactive Element Highlighting, Failureaware Prompting (FAP), Visual Saliency Enhancement, and Visual-Textual Descriptions Combination, all aiming at improving MLLMs' performance on the Interaction-toCode task.
Insight Over Sight? Exploring the Vision-Knowledge Conflicts in Multimodal LLMs
This paper explores the problem of commonsense-level vision-knowledge conflict in Multimodal Large Language Models (MLLMs), where visual information contradicts model's internal commonsense knowledge (see Figure 1).
Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step
To assess the safety of existing models, we introduce a novel jailbreaking method called Chain-of-Jailbreak (CoJ) attack, which compromises image generation models through a step-by-step editing process.
Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness
Second, for the non-toxicity of LLMs, we introduce two works for red-teaming LLMs.
Learning to Ask: When LLM Agents Meet Unclear Instruction
Equipped with the capability to call functions, modern large language models (LLMs) can leverage external tools for addressing a range of tasks unattainable through language skills alone.
On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents
Large language model-based multi-agent systems have shown great abilities across various tasks due to the collaboration of expert agents, each focusing on a specific domain.
Diffusion Feedback Helps CLIP See Better
We demonstrate that DIVA improves CLIP's performance on the challenging MMVP-VLM benchmark which assesses fine-grained visual abilities to a large extent (e. g., 3-7%), and enhances the performance of MLLMs and vision models on multimodal understanding and segmentation tasks.
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training
DeRTa incorporates two novel components: (1) Maximum Likelihood Estimation (MLE) with Harmful Response Prefix, which trains models to recognize and avoid unsafe content by appending a segment of harmful response to the beginning of a safe response, and (2) Reinforced Transition Optimization (RTO), which equips models with the ability to transition from potential harm to safety refusal consistently throughout the harmful response sequence.
Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach
We conduct extensive testing with a dataset comprised of real-world websites and various MLLMs and demonstrate that DCGen achieves up to a 14% improvement in visual similarity over competing methods.
Enhancing Visible-Infrared Person Re-identification with Modality- and Instance-aware Visual Prompt Learning
In this model, we have designed a series of modality-specific prompts, which could enable our model to adapt to and make use of the specific information inherent in different modality inputs, thereby reducing the interference caused by the modality gap and achieving better identification.
Knowledge-to-Jailbreak: One Knowledge Point Worth One Attack
To bridge this gap, we propose a new task, knowledge-to-jailbreak, which aims to generate jailbreaks from domain knowledge to evaluate the safety of LLMs when applied to those domains.
Diffusion Actor-Critic with Entropy Regulator
This algorithm conceptualizes the reverse process of the diffusion model as a novel policy function and leverages the capability of the diffusion model to fit multimodal distributions, thereby enhancing the representational capacity of the policy.
How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHO
However, existing studies focus on imitating well-known public figures or fictional characters, overlooking the potential for simulating ordinary individuals.
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i. e., Kuaishou/Kwai Platform.
How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments
Researchers have examined LLMs' decision-making through the lens of Game Theory.
Beyond Literal Descriptions: Understanding and Locating Open-World Objects Aligned with Human Intentions
To promote classic VG towards human intention interpretation, we propose a new intention-driven visual grounding (IVG) task and build a large-scale IVG dataset termed IntentionVG with free-form intention expressions.
A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models
The significant breakthroughs of Medical Multi-Modal Large Language Models (Med-MLLMs) renovate modern healthcare with robust information synthesis and medical decision support.
Unveiling Parts Beyond Objects: Towards Finer-Granularity Referring Expression Segmentation
To foster future research into fine-grained visual grounding our benchmark RefCOCOm the MRES-32M dataset and model UniRES will be publicly available at https://github. com/Rubics-Xuan/MRES.
LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models
We introduce LogicAsker, a novel approach for evaluating and enhancing the logical reasoning capabilities of large language models (LLMs) such as ChatGPT and GPT-4.
The Earth is Flat? Unveiling Factual Errors in Large Language Models
Current methods for evaluating LLMs' veracity are limited by test data leakage or the need for extensive human labor, hindering efficient and accurate error detection.
New Job, New Gender? Measuring the Social Bias in Image Generation Models
BiasPainter uses a diverse range of seed images of individuals and prompts the image generation models to edit these images using gender, race, and age-neutral queries.
Unveiling Parts Beyond Objects:Towards Finer-Granularity Referring Expression Segmentation
To foster future research into fine-grained visual grounding, our benchmark RefCOCOm, the MRES-32M dataset and model UniRES will be publicly available at https://github. com/Rubics-Xuan/MRES
Training Multi-layer Neural Networks on Ising Machine
This paper proposes an Ising learning algorithm to train quantized neural network (QNN), by incorporating two essential techinques, namely binary representation of topological network and order reduction of loss function.
Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models
Ideally, an advanced agent should possess the ability to accurately describe a given word using an aggressive description while concurrently maximizing confusion in the conservative description, enhancing its participation in the game.
LFAA: Crafting Transferable Targeted Adversarial Examples with Low-Frequency Perturbations
In this work, we present a novel approach to generate transferable targeted adversarial examples by exploiting the vulnerability of deep neural networks to perturbations on high-frequency components of images.
Med-DANet V2: A Flexible Dynamic Architecture for Efficient Medical Volumetric Segmentation
As a pioneering work, a dynamic architecture network for medical volumetric segmentation (i. e. Med-DANet) has achieved a favorable accuracy and efficiency trade-off by dynamically selecting a suitable 2D candidate model from the pre-defined model bank for different slices.
Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models
This paper identifies a cultural dominance issue within large language models (LLMs) due to the predominant use of English data in model training (e. g., ChatGPT).
Distributional Soft Actor-Critic with Three Refinements
To address this issue, we previously proposed the Distributional Soft Actor-Critic (DSAC or DSACv1), an off-policy RL algorithm that enhances value estimation accuracy by learning a continuous Gaussian value distribution.
Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench
Large Language Models (LLMs) have recently showcased their remarkable capacities, not only in natural language processing tasks but also across diverse domains such as clinical medicine, legal consultation, and education.
All Languages Matter: On the Multilingual Safety of Large Language Models
In this work, we build the first multilingual safety benchmark for LLMs, XSafety, in response to the global deployment of LLMs in practice.
Boosting Adversarial Transferability by Block Shuffle and Rotation
In this work, we observe that existing input transformation based attacks, one of the mainstream transfer-based attacks, result in different attention heatmaps on various models, which might limit the transferability.
An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software
Moreover, through retraining the models with the test cases generated by OASIS, the robustness of the moderation model can be improved without performance degradation.
EAVL: Explicitly Align Vision and Language for Referring Image Segmentation
Our method harnesses the potential of the multi-modal features in the segmentation stage and aligns language features of different emphases with image features to achieve fine-grained text-to-pixel correlation.
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
We propose a novel framework CipherChat to systematically examine the generalizability of safety alignment to non-natural languages -- ciphers.
Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench
Evaluating Large Language Models' (LLMs) anthropomorphic capabilities has become increasingly important in contemporary discourse.
Revisiting the Reliability of Psychological Scales on Large Language Models
Recent research has focused on examining Large Language Models' (LLMs) characteristics from a psychological standpoint, acknowledging the necessity of understanding their behavioral characteristics.
Validating Multimedia Content Moderation Software via Semantic Fusion
To this end, content moderation software has been widely deployed on these platforms to detect and blocks toxic content.
BiasAsker: Measuring the Bias in Conversational AI System
Particularly, it is hard to generate inputs that can comprehensively trigger potential bias due to the lack of data containing both social groups as well as biased properties.
CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation
Referring image segmentation (RIS) is a fundamental vision-language task that intends to segment a desired object from an image based on a given natural language expression.
FreMIM: Fourier Transform Meets Masked Image Modeling for Medical Image Segmentation
The research community has witnessed the powerful potential of self-supervised Masked Image Modeling (MIM), which enables the models capable of learning visual representation from unlabeled data.
Med-Tuning: A New Parameter-Efficient Tuning Framework for Medical Volumetric Segmentation
Thus, it is of increasing importance to fine-tune pre-trained models for medical volumetric segmentation tasks in a both effective and parameter-efficient manner.
ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback
Therefore, we propose ParroT, a framework to enhance and regulate the translation abilities during chat based on open-source LLMs (e. g., LLaMA), human-written translation and feedback data.
Improving the Transferability of Adversarial Samples by Path-Augmented Method
However, such methods selected the image augmentation path heuristically and may augment images that are semantics-inconsistent with the target images, which harms the transferability of the generated adversarial samples.
ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark
ChatGPT is a cutting-edge artificial intelligence language model developed by OpenAI, which has attracted a lot of attention due to its surprisingly strong ability in answering follow-up questions.
MTTM: Metamorphic Testing for Textual Content Moderation Software
In addition, we leverage the test cases generated by MTTM to retrain the model we explored, which largely improves model robustness (0% to 5. 9% EFR) while maintaining the accuracy on the original test set.
Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine
By evaluating on a number of benchmark test sets, we find that ChatGPT performs competitively with commercial translation products (e. g., Google Translate) on high-resource European languages but lags behind significantly on low-resource or distant languages.
Smoothing Policy Iteration for Zero-sum Markov Games
In this paper, we propose the smoothing policy iteration (SPI) algorithm to solve the zero-sum MGs approximately, where the maximum operator is replaced by the weighted LogSumExp (WLSE) function to obtain the nearly optimal equilibrium policies.
Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languages
This paper describes Tencent's multilingual machine translation systems for the WMT22 shared task on Large-Scale Machine Translation Evaluation for African Languages.
QSAN: A Near-term Achievable Quantum Self-Attention Network
Self-Attention Mechanism (SAM) is good at capturing the internal connections of features and greatly improves the performance of machine learning models, espeacially requiring efficient characterization and feature extraction of high-dimensional data.
Positive-Negative Equal Contrastive Loss for Semantic Segmentation
The contextual information is critical for various computer vision tasks, previous works commonly design plug-and-play modules and structural losses to effectively extract and aggregate the global context.
Med-DANet: Dynamic Architecture Network for Efficient Medical Volumetric Segmentation
For 3D medical image (e. g. CT and MRI) segmentation, the difficulty of segmenting each slice in a clinical case varies greatly.
Understanding and Mitigating the Uncertainty in Zero-Shot Translation
Zero-shot translation is a promising direction for building a comprehensive multilingual neural machine translation~(MNMT) system.
AEON: A Method for Automatic Evaluation of NLP Test Cases
However, in practice, many of the generated test cases fail to preserve similar semantic meaning and are unnatural (e. g., grammar errors), which leads to a high false alarm rate and unnatural test cases.
DST: Dynamic Substitute Training for Data-free Black-box Attack
With the wide applications of deep neural network models in various computer vision tasks, more and more works study the model vulnerability to adversarial examples.
Improving Adversarial Transferability via Neuron Attribution-Based Attacks
Deep neural networks (DNNs) are known to be vulnerable to adversarial examples.
Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation
In this paper, we present a substantial step in better understanding the SOTA sequence-to-sequence (Seq2Seq) pretraining for neural machine translation~(NMT).
TransBTSV2: Towards Better and More Efficient Volumetric Segmentation of Medical Images
Different from TransBTS, the proposed TransBTSV2 is not limited to brain tumor segmentation (BTS) but focuses on general medical image segmentation, providing a stronger and more efficient 3D baseline for volumetric segmentation of medical images.
BBS-KWS:The Mandarin Keyword Spotting System Won the Video Keyword Wakeup Challenge
We propose a mandarin keyword spotting system (KWS) with several novel and effective improvements, including a big backbone (B) model, a keyword biasing (B) mechanism and the introduction of syllable modeling units (S).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Optimal Variable Speed Limit Control Strategy on Freeway Segments under Fog Conditions
This study developed optimal VSL control strategy under fog conditions with fully consideration of factors that affect traffic safety risks.
Language Models are Good Translators
Inspired by the recent progress of large-scale pre-trained language models on machine translation in a limited scenario, we firstly demonstrate that a single language model (LM4MT) can achieve comparable performance with strong encoder-decoder NMT models on standard machine translation benchmarks, using the same training data and similar amount of model parameters.
Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition
Deep neural networks, particularly face recognition models, have been shown to be vulnerable to both digital and physical adversarial examples.
Delving into Data: Effectively Substitute Training for Black-box Attack
Previous substitute training approaches focus on stealing the knowledge of the target model based on real training data or synthetic data, without exploring what kind of data can further improve the transferability between the substitute and target models.
TransBTS: Multimodal Brain Tumor Segmentation Using Transformer
To capture the local 3D context information, the encoder first utilizes 3D CNN to extract the volumetric spatial feature maps.
Ranked #1 on
Brain Tumor Segmentation
on BRATS 2019
(Dice Score metric)
Recurrent Model Predictive Control
This paper proposes an off-line algorithm, called Recurrent Model Predictive Control (RMPC), to solve general nonlinear finite-horizon optimal control problems.
Recurrent Model Predictive Control: Learning an Explicit Recurrent Controller for Nonlinear Systems
This paper proposes an offline control algorithm, called Recurrent Model Predictive Control (RMPC), to solve large-scale nonlinear finite-horizon optimal control problems.
Rethinking the Value of Transformer Components
Transformer becomes the state-of-the-art translation model, while it is not well studied how each intermediate component contributes to the model performance, which poses significant challenges for designing optimal architectures.
Ultrasound Liver Fibrosis Diagnosis using Multi-indicator guided Deep Neural Networks
An indicator-guided learning mechanism is further proposed to ease the training of the proposed model.
A New Screening Method for COVID-19 based on Ocular Feature Recognition by Machine Learning Tools
After more than four months study, we found that the confirmed cases of COVID-19 present the consistent ocular pathological symbols; and we propose a new screening method of analyzing the eye-region images, captured by common CCD and CMOS cameras, could reliably make a rapid risk screening of COVID-19 with very high accuracy.
FM2u-Net: Face Morphological Multi-Branch Network for Makeup-Invariant Face Verification
It is challenging in learning a makeup-invariant face verification model, due to (1) insufficient makeup/non-makeup face training pairs, (2) the lack of diverse makeup faces, and (3) the significant appearance changes caused by cosmetics.
Long-Term Cloth-Changing Person Re-identification
Specifically, we consider that under cloth-changes, soft-biometrics such as body shape would be more reliable.
Signal-to-Noise Ratio of Microwave Photonic Filter With an Interferometric Structure Based on an Incoherent Broadband Optical Source
Theoretical analysis shows that the SNR is a function of the center frequency of the passband, the modulation index, the chromatic dispersion, and the shape of the IBOS.
Learning to Augment Expressions for Few-shot Fine-grained Facial Expression Recognition
Considering the phenomenon of uneven data distribution and lack of samples is common in real-world scenarios, we further evaluate several tasks of few-shot expression learning by virtue of our F2ED, which are to recognize the facial expressions given only few training instances.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
DeepEnFM: Deep neural networks with Encoder enhanced Factorization Machine
Instead of learning the cross features directly, DeepEnFM adopts the Transformer encoder as a backbone to align the feature embeddings with the clues of other fields.
A Fine-Grained Facial Expression Database for End-to-End Multi-Pose Facial Expression Recognition
First, we create a new facial expression dataset of more than 200k images with 119 persons, 4 poses and 54 expressions.
Facial Expression Recognition
Facial Expression Recognition (FER)
+2
FPETS : Fully Parallel End-to-End Text-to-Speech System
End-to-end Text-to-speech (TTS) system can greatly improve the quality of synthesised speech.
Pose-Normalized Image Generation for Person Re-identification
Person Re-identification (re-id) faces two major challenges: the lack of cross-view paired training data and learning discriminative identity-sensitive and view-invariant features in the presence of large pose variations.
