Search Results for author:
Found 473 papers, 188 papers with code
《二十四史》古代汉语语义依存图库构建(Construction of Semantic Dependency Graph Bank of Ancient Chinese in twenty four histories)
“语义依存图是NLP处理语义的深层分析方法, 能够对句子中词与词之间的语义进行分析。该文针对古代汉语特点, 在制定古代汉语语义依存图标注规范的基础上, 以《二十四史》为语料来源, 完成标注了规模为3000句的古代汉语语义依存图库, 标注一致性的kappa值为78. 83%。通过与现代汉语语义依存图库的对比, 对依存图库基本情况进行统计, 分析古代汉语的语义特色和规律。统计显示, 古代汉语语义分布宏观上符合齐普夫定律, 在语义事件描述上具有强烈的历史性叙事和正式文体特征, 如以人物纪传为中心, 时间、地点等周边角色描述细致, 叙事语言冷静客观, 缺少描述情态、语气、程度、时间状态等的修饰词语等。 "
Towards Efficient Coarse-to-Fine Networks for Action and Gesture Recognition
State-of-the-art approaches to video-based action and gesture recognition often employ two key concepts: First, they employ multistream processing; second, they use an ensemble of convolutional networks.
Ranked #1 on
Action Classification
on Jester test
SgSum:Transforming Multi-document Summarization into Sub-graph Selection
Comparing with traditional methods, our method has two main advantages: (1) the relations between sentences are captured by modeling both the graph structure of the whole document set and the candidate sub-graphs; (2) directly outputs an integrate summary in the form of sub-graph which is more informative and coherent.
Weight Excitation: Built-in Attention Mechanisms in Convolutional Neural Networks
We propose novel approaches for simultaneously identifying important weights of a convolutional neural network (ConvNet) and providing more attention to the important weights during training.
Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation
Word Segmentation is a fundamental step for understanding Chinese language.
Meta-CQG: A Meta-Learning Framework for Complex Question Generation over Knowledge Bases
Complex question generation over knowledge bases (KB) aims to generate natural language questions involving multiple KB relations or functional constraints.
基于统一模型的藏文新闻摘要(Abstractive Summarization of Tibetan News Based on Hybrid Model)
Seq2seq神经网络模型在中英文文本摘要的研究中取得了良好的效果, 但在低资源语言的文本摘要研究还处于探索阶段, 尤其是在藏语中。此外, 目前还没有大规模的标注语料库进行摘要提取。本文提出了一种生成藏文新闻摘要的统一模型。利用TextRank算法解决了藏语标注训练数据不足的问题。然后, 采用两层双GRU神经网络提取代表原始新闻的句子, 减少冗余信息。最后, 使用基于注意力机制的Seq2Seq来生成理解式摘要。同时, 我们加入了指针网络来处理未登录词的问题。实验结果表明, ROUGE-1评分比传统模型提高了2%。 关键词:文本摘要;藏文;TextRank; 指针网络;Bi-GRU
基于强化学习的古今汉语句子对齐研究(Research on Sentence Alignment of Ancient and Modern Chinese based on Reinforcement Learning)
“基于深度学习的有监督机器翻译取得了良好的效果, 但训练过程中需要大量质量较高的对齐语料。对于中文古今翻译场景, 高质量的平行语料并不多, 而粗对齐的篇章、段语料比较容易获得, 因此语料对齐很有研究价值和研究必要。在传统双语平行语料的句子对齐研究中, 传统方法根据双语文本中的长度、词汇、共现文字等语法信息, 建立一个综合评判标准来衡量两个句对之间相似度。此类方法虽然在单句对齐上取得了较好的效果, 但是对于句子语义匹配的能力有限, 并且在一些多对多的对齐模式上的性能表现不佳。在本文中我们提出尝试利用现在发展迅速且具有强大语义表示能力的预训练语言模型来考虑双语的语义信息, 但是单独使用预训练语言模型只能考虑相对局部的信息, 因此我们提出采用基于动态规划算法的强化学习训练目标来整合段落全局信息, 并且进行无监督训练。实验结果证明我们提出的方法训练得到的模型性能优于此前获得最好表现的基线模型, 尤其相较于传统模型难以处理的多对多对齐模式下, 性能提升较大。”
Ophora: A Large-Scale Data-Driven Text-Guided Ophthalmic Surgical Video Generation Model
In ophthalmic surgery, developing an AI system capable of interpreting surgical videos and predicting subsequent operations requires numerous ophthalmic surgical videos with high-quality annotations, which are difficult to collect due to privacy concerns and labor consumption.
Depth-Sensitive Soft Suppression with RGB-D Inter-Modal Stylization Flow for Domain Generalization Semantic Segmentation
Hence, we propose a novel framework, namely Depth-Sensitive Soft Suppression with RGB-D inter-modal stylization flow (DSSS), focusing on learning domain-invariant features from depth maps for the DG semantic segmentation.
DSPO: Direct Semantic Preference Optimization for Real-World Image Super-Resolution
Recent advances in diffusion models have improved Real-World Image Super-Resolution (Real-ISR), but existing methods lack human feedback integration, risking misalignment with human preference and may leading to artifacts, hallucinations and harmful content generation.
Efficient Spiking Point Mamba for Point Cloud Analysis
Due to the poor performance of simply transferring Mamba to 3D SNNs, SPM is designed to utilize both the sequence modeling capabilities of Mamba and the temporal feature extraction of SNNs.
Spiking Neural Network for Intra-cortical Brain Signal Decoding
Decoding brain signals accurately and efficiently is crucial for intra-cortical brain-computer interfaces.
BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification
In this paper, we present BrainPrompt, an innovative framework that enhances Graph Neural Networks (GNNs) by integrating Large Language Models (LLMs) with knowledge-driven prompts, enabling more effective capture of complex, non-imaging information and external knowledge for neurological disease identification.
Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs
Our exploration demonstrates that Ascend NPUs are capable of efficiently and effectively training dense models with more than 100 billion parameters.
JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization
This paper introduces JavisDiT, a novel Joint Audio-Video Diffusion Transformer designed for synchronized audio-video generation (JAVG).
OpenHuEval: Evaluating Large Language Model on Hungarian Specifics
We introduce OpenHuEval, the first benchmark for LLMs focusing on the Hungarian language and specifics.
Harmonizing Visual Representations for Unified Multimodal Understanding and Generation
Unifying visual understanding and generation within a single multimodal framework remains a significant challenge, as the two inherently heterogeneous tasks require representations at different levels of granularity.
Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistency
We present Free4D, a novel tuning-free framework for 4D scene generation from a single image.
ACVUBench: Audio-Centric Video Understanding Benchmark
Audio often serves as an auxiliary modality in video understanding tasks of audio-visual large language models (LLMs), merely assisting in the comprehension of visual information.
ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistency
RAW-to-sRGB mapping, or the simulation of the traditional camera image signal processor (ISP), aims to generate DSLR-quality sRGB images from raw data captured by smartphone sensors.
CCMusic: An Open and Diverse Database for Chinese Music Information Retrieval Research
Data are crucial in various computer-related fields, including music information retrieval (MIR), an interdisciplinary area bridging computer science and music.
V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning
Human processes video reasoning in a sequential spatio-temporal reasoning logic, we first identify the relevant frames ("when") and then analyse the spatial relationships ("where") between key objects, and finally leverage these relationships to draw inferences ("what").
Spatial Distillation based Distribution Alignment (SDDA) for Cross-Headset EEG Classification
A non-invasive brain-computer interface (BCI) enables direct interaction between the user and external devices, typically via electroencephalogram (EEG) signals.
MagicInfinite: Generating Infinite Talking Videos with Your Words and Voice
We present MagicInfinite, a novel diffusion Transformer (DiT) framework that overcomes traditional portrait animation limitations, delivering high-fidelity results across diverse character types-realistic humans, full-body figures, and stylized anime characters.
Feature Point Extraction for Extra-Affine Image
The issue concerning the significant decline in the stability of feature extraction for images subjected to large-angle affine transformations, where the angle exceeds 50 degrees, still awaits a satisfactory solution.
Identifying Ising and percolation phase transitions based on KAN method
The results demonstrate that the KAN can indeed predict the critical points of percolation models.
LION-FS: Fast & Slow Video-Language Thinker as Online Video Assistant
However, existing online video assistants often sacrifice assistant efficacy for real-time efficiency by processing low-frame-rate videos with coarse-grained visual features. To overcome the trade-off between efficacy and efficiency, we propose "Fast & Slow Video-Language Thinker" as an onLIne videO assistaNt, LION-FS, achieving real-time, proactive, temporally accurate, and contextually precise responses.
DoF-Gaussian: Controllable Depth-of-Field for 3D Gaussian Splatting
We also introduce a synthetic dataset to assess refocusing capabilities and the model's ability to learn precise lens parameters.
MIGE: A Unified Framework for Multimodal Instruction-Based Image Generation and Editing
Experiments show that MIGE excels in both subject-driven generation and instruction-based editing while setting a state-of-the-art in the new task of instruction-based subject-driven editing.
Delta Decompression for MoE-based LLMs Compression
Based on observations of expert diversity, we decompose their weights into a shared base weight and unique delta weights.
Autoregressive Image Generation Guided by Chains of Thought
In the field of autoregressive (AR) image generation, models based on the 'next-token prediction' paradigm of LLMs have shown comparable performance to diffusion models by reducing inductive biases.
MVCNet: Multi-View Contrastive Network for Motor Imagery Classification
Two contrastive modules are further introduced: a cross-view contrastive module that enforces consistency of original and augmented views, and a cross-model contrastive module that aligns features extracted from both branches.
video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model
video-SALMONN-o1 achieves 3-8% accuracy improvements over the LLaVA-OneVision baseline across different video reasoning benchmarks.
CoPEFT: Fast Adaptation Framework for Multi-Agent Collaborative Perception with Parameter-Efficient Fine-Tuning
However, training a robust collaborative perception model requires collecting sufficient training data that covers all possible collaboration scenarios, which is impractical due to intolerable deployment costs.
Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction
In this paper, we propose a novel retrieval method based on natural language grammatical error explanations (GEE) to address this issue.
CoS: Chain-of-Shot Prompting for Long Video Understanding
Multi-modal Large Language Models (MLLMs) struggle with long videos due to the need for excessive visual tokens.
WanJuanSiLu: A High-Quality Open-Source Webtext Dataset for Low-Resource Languages
This paper introduces the open-source dataset WanJuanSiLu, designed to provide high-quality training corpora for low-resource languages, thereby advancing the research and development of multilingual models.
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
Experiments on various mathematical reasoning benchmarks show that O1-Pruner not only significantly reduces inference overhead but also achieves higher accuracy, providing a novel and promising solution to this challenge.
Progressive Document-level Text Simplification via Large Language Models
Research on text simplification has primarily focused on lexical and sentence-level changes.
DarkFarseer: Inductive Spatio-temporal Kriging via Hidden Style Enhancement and Sparsity-Noise Mitigation
Based on graph neural networks (GNNs) extracting the relationships between physical and virtual sensors, ISK can infer the measurements of virtual sensors from physical sensors.
Multi-Aggregator Time-Warping Heterogeneous Graph Neural Network for Personalized Micro-Video Recommendation
Micro-video recommendation is attracting global attention and becoming a popular daily service for people of all ages.
Quantum Cognition-Inspired EEG-based Recommendation via Graph Neural Networks
Current recommendation systems recommend goods by considering users' historical behaviors, social relations, ratings, and other multi-modals.
ZeroFlow: Overcoming Catastrophic Forgetting is Easier than You Think
This work provides essential insights and tools for advancing forward pass methods to overcome forgetting.
Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptation
Capitalizing on the complementary advantages of generative and discriminative models has always been a compelling vision in machine learning, backed by a growing body of research.
AdaCo: Overcoming Visual Foundation Model Noise in 3D Semantic Segmentation via Adaptive Label Correction
Our proposed AdaCo can effectively mitigate the performance limitations of label-free learning networks in 3D semantic segmentation tasks.
Ultra-Low Complexity On-Orbit Compression for Remote Sensing Imagery via Block Modulated Imaging
Efficient compression of remote sensing imagery is a critical solution to alleviate these burdens on satellites.
Generating Unseen Nonlinear Evolution in Sea Surface Temperature Using a Deep Learning-Based Latent Space Data Assimilation Framework
To fuse multi-source data and reconstruct the nonlinear evolution missing from observations, geoscientists are developing future-oriented DA methods.
NPC: Neural Predictive Control for Fuel-Efficient Autonomous Trucks
Fuel efficiency is a crucial aspect of long-distance cargo transportation by oil-powered trucks that economize on costs and decrease carbon emissions.
Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration
However, previous KD methods for image restoration overlook the state of the student during the distillation, adopting a fixed solution space that limits the capability of KD.
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
Recent advancements in multimodal large language models (MLLMs) have made significant strides in open-world understanding.
GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expression
Audio-driven talking head generation necessitates seamless integration of audio and visual data amidst the challenges posed by diverse input portraits and intricate correlations between audio and facial motions.
FD-LLM: Large Language Model for Fault Diagnosis of Machines
We assess the fault diagnosis capabilities of four open-sourced LLMs based on the FD-LLM framework, and evaluate the models' adaptability and generalizability under various operational conditions and machine components, namely for traditional fault diagnosis, cross-operational conditions, and cross-machine component settings.
HiFiVFS: High Fidelity Video Face Swapping
Face swapping aims to generate results that combine the identity from the source with attributes from the target.
MRIFE: A Mask-Recovering and Interactive-Feature-Enhancing Semantic Segmentation Network For Relic Landslide Detection
Relic landslide, formed over a long period, possess the potential for reactivation, making them a hazardous geological phenomenon.
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining
Surgical practice involves complex visual interpretation, procedural skills, and advanced medical knowledge, making surgical vision-language pretraining (VLP) particularly challenging due to this complexity and the limited availability of annotated data.
A Survey on LLM-as-a-Judge
Accurate and consistent evaluation is crucial for decision-making across numerous fields, yet it remains a challenging task due to inherent subjectivity, variability, and scale.
GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI
Despite significant advancements in general AI, its effectiveness in the medical domain is limited by the lack of specialized medical knowledge.
Fact-Level Confidence Calibration and Self-Correction
Confidence calibration in LLMs, i. e., aligning their self-assessed confidence with the actual accuracy of their responses, enabling them to self-evaluate the correctness of their outputs.
UNSCT-HRNet: Modeling Anatomical Uncertainty for Landmark Detection in Total Hip Arthroplasty
Total hip arthroplasty (THA) relies on accurate landmark detection from radiographic images, but unstructured data caused by irregular patient postures or occluded anatomical markers pose significant challenges for existing methods.
A Preview of XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL
On the other hand, we implement the ICL approach with an example selection method based on named entity recognition to prevent overemphasis on entities.
Ranked #1 on
Text-To-SQL
on SQL-Eval
Sub-DM:Subspace Diffusion Model with Orthogonal Decomposition for MRI Reconstruction
To tackle these challenges, we introduce subspace diffusion model with orthogonal decomposition, a method (referred to as Sub-DM) that restrict the diffusion process via projections onto subspace as the k-space data distribution evolves toward noise.
Technical Report for Soccernet 2023 -- Dense Video Captioning
In the task of dense video captioning of Soccernet dataset, we propose to generate a video caption of each soccer action and locate the timestamp of the caption.
Semi-Supervised Self-Learning Enhanced Music Emotion Recognition
Music emotion recognition (MER) aims to identify the emotions conveyed in a given musical piece.
PGDiffSeg: Prior-Guided Denoising Diffusion Model with Parameter-Shared Attention for Breast Cancer Segmentation
Therefore, this paper proposes a novel PGDiffSeg (Prior-Guided Diffusion Denoising Model with Parameter-Shared Attention) that applies diffusion denoising methods to breast cancer medical image segmentation, accurately recovering the affected areas from Gaussian noise.
Hi-Mamba: Hierarchical Mamba for Efficient Image Super-Resolution
State Space Models (SSM), such as Mamba, have shown strong representation ability in modeling long-range dependency with linear complexity, achieving successful applications from high-level to low-level vision tasks.
Can a large language model be a gaslighter?
In contrast, we advanced three safety alignment strategies to strengthen (by 12. 05%) the safety guardrail of LLMs.
HybridBooth: Hybrid Prompt Inversion for Efficient Subject-Driven Generation
Recent advancements in text-to-image diffusion models have shown remarkable creative capabilities with textual prompts, but generating personalized instances based on specific subjects, known as subject-driven generation, remains challenging.
Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics
While humans effortlessly discern intrinsic dynamics and adapt to new scenarios, modern AI systems often struggle.
Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization
To address potential catastrophic forgetting of non-captioning abilities due to mrDPO, we propose rebirth tuning, which finetunes the pre-DPO LLM by using the captions generated by the mrDPO-trained model as supervised labels.
MRF-Net: An Infrared Remote Sensing Image Thin Cloud Removal Method With the Intra-Inter Coherent Constraint
To address this problem, we proposed the multiscale residual fusion network (MRF-Net) to remove thin cloud from infrared remote sensing imagery.
Video Instruction Tuning With Synthetic Data
The development of video large multimodal models (LMMs) has been hindered by the difficulty of curating large amounts of high-quality raw data from the web.
A Generalized Tensor Formulation for Hyperspectral Image Super-Resolution Under General Spatial Blurring
Hyperspectral super-resolution is commonly accomplished by the fusing of a hyperspectral imaging of low spatial resolution with a multispectral image of high spatial resolution, and many tensor-based approaches to this task have been recently proposed.
MinerU: An Open-Source Solution for Precise Document Content Extraction
Document content analysis has been a crucial research area in computer vision.
Contrasformer: A Brain Network Contrastive Transformer for Neurodegenerative Condition Identification
It generates a prior-knowledge-enhanced contrast graph to address the distribution shifts across sub-populations by a two-stream attention mechanism.
BihoT: A Large-Scale Dataset and Benchmark for Hyperspectral Camouflaged Object Tracking
This bias allows the tracker to directly use the visual features obtained from the false-color images generated by hyperspectral images without the need to extract spectral features.
Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic Segmentation
Specifically, the point cloud is initially separated into independent point sets by category to provide initial conditions for the generation of feature subspaces.
Ranked #13 on
Semantic Segmentation
on S3DIS Area5
Deep Code Search with Naming-Agnostic Contrastive Multi-View Learning
However, developers may not follow the same naming conventions and the same variable may have different variable names in different implementations, bringing a challenge to deep learning based code search methods that rely on explicit variable correspondences to understand source code.
Beyond Inter-Item Relations: Dynamic Adaption for Enhancing LLM-Based Sequential Recommendation
Built on top of coarse-grained adaption for capturing inter-item relations, DARec is further enhanced with (1) context masking that models intra-item relations to help LLM better understand token and item semantics in the context of SRS, (2) collaborative knowledge injection that helps LLM incorporate long-term collaborative knowledge, and (3) a dynamic adaption mechanism that uses Bayesian optimization to flexibly choose layer-wise adapter architectures in order to better incorporate different sequential information.
ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area
We benchmark ChemVLM against a range of open-source and proprietary multimodal large language models on various tasks.
DeepInteraction++: Multi-Modality Interaction for Autonomous Driving
To demonstrate the effectiveness of the proposed strategy, we design DeepInteraction++, a multi-modal interaction framework characterized by a multi-modal representational interaction encoder and a multi-modal predictive interaction decoder.
Respiratory Subtraction for Pulmonary Microwave Ablation Evaluation
The experimental results confirm the effectiveness of the respiratory subtraction method and the proposed quantitative evaluation metric in assessing lung tumor treatment.
GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI
Large Vision-Language Models (LVLMs) are capable of handling diverse data types such as imaging, text, and physiological signals, and can be applied in various fields.
MDT-A2G: Exploring Masked Diffusion Transformers for Co-Speech Gesture Generation
In an attempt to bridge this research gap, we introduce a novel Masked Diffusion Transformer for co-speech gesture generation, referred to as MDT-A2G, which directly implements the denoising process on gesture sequences.
Dynamic Object Queries for Transformer-based Incremental Object Detection
These new object queries are aggregated with those from previous phases to adapt both old and new knowledge well.
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #3 on
Question Answering
on PeerQA
Multi-Granularity Semantic Revision for Large Language Model Distillation
At the sequence level, we propose a sequence correction and re-generation (SCRG) strategy.
Generalizable Implicit Motion Modeling for Video Frame Interpolation
Motion modeling is critical in flow-based Video Frame Interpolation (VFI).
Investigating Public Fine-Tuning Datasets: A Complex Review of Current Practices from a Construction Perspective
Construction techniques and methods for public fine-tuning datasets of Large Language Models (LLMs), including data generation and data augmentation among others, are detailed.
LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models
To this end, we introduce LLaVA-NeXT-Interleave, which simultaneously tackles Multi-image, Multi-frame (video), Multi-view (3D), and Multi-patch (single-image) scenarios in LMMs.
Ranked #6 on
Zero-Shot Video Question Answer
on VNBench
Music Era Recognition Using Supervised Contrastive Learning and Artist Information
For the case where the artist information is available, we extend the audio-based model to take multimodal inputs and develop a framework, called MultiModal Contrastive (MMC) learning, to enhance the training.
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output
This long-context capability allows IXC-2. 5 to excel in tasks requiring extensive input and output contexts.
Ranked #4 on
Video Question Answering
on TVBench
WildAvatar: Web-scale In-the-wild Video Dataset for 3D Avatar Creation
However, their annotating requirements are impractical for real-world images or videos, posing challenges toward real-world applications on current avatar creation methods.
Comprehensive Generative Replay for Task-Incremental Segmentation with Concurrent Appearance and Semantic Forgetting
Generalist segmentation models are increasingly favored for diverse tasks involving various objects from different image sources.
A Comprehensive Solution to Connect Speech Encoder and Large Language Model for ASR
Finally, we explore training and inference methods to mitigate high insertion errors.
GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization
The extraordinary ability of generative models emerges as a new trend in image editing and generating realistic images, posing a serious threat to the trustworthiness of multimedia data and driving the research of image manipulation detection and location (IMDL).
video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models
To obtain fine-grained temporal information required by speech understanding, while keeping efficient for other video elements, this paper proposes a novel multi-resolution causal Q-Former (MRC Q-Former) structure to connect pre-trained audio-visual encoders and the backbone large language model.
OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text
In this paper, we introduce OmniCorpus, a 10 billion-scale image-text interleaved dataset.
Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASR
This paper addresses challenges in integrating new languages into a pre-trained multilingual automatic speech recognition (mASR) system, particularly in scenarios where training data for existing languages is limited or unavailable.
HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluation
Leveraging HalluDial, we conduct a comprehensive meta-evaluation of LLMs' hallucination evaluation capabilities in information-seeking dialogues and introduce a specialized judge language model, HalluJudge.
F-LMM: Grounding Frozen Large Multimodal Models
To address this issue, we present F-LMM -- grounding frozen off-the-shelf LMMs in human-AI conversations -- a straightforward yet effective design based on the fact that word-pixel correspondences conducive to visual grounding inherently exist in the attention weights of well-trained LMMs.
PSBD: Prediction Shift Uncertainty Unlocks Backdoor Detection
PSBD is motivated by an intriguing Prediction Shift (PS) phenomenon, where poisoned models' predictions on clean data often shift away from true labels towards certain other labels with dropout applied during inference, while backdoor samples exhibit less PS.
On the Effects of Data Scale on UI Control Agents
Moreover, AndroidControl is the most diverse computer control dataset to date, including 14, 548 unique tasks over 833 Android apps, thus allowing us to conduct in-depth analysis of the model performance in and out of the domain of the training data.
OpenDataLab: Empowering General Artificial Intelligence with Open Datasets
The dispersion of data sources and diversity of data formats often lead to inefficiencies in data retrieval and processing, significantly impeding the progress of AI research and applications.
Organizing Background to Explore Latent Classes for Incremental Few-shot Semantic Segmentation
During incrementally learning novel classes, the data distribution of old classes will be destroyed, leading to catastrophic forgetting.
Lifelong Learning and Selective Forgetting via Contrastive Strategy
The joint task of the two is dubbed Learning with Selective Forgetting (LSF).
Detection-Correction Structure via General Language Model for Grammatical Error Correction
Grammatical error correction (GEC) is a task dedicated to rectifying texts with minimal edits, which can be decoupled into two components: detection and correction.
The Binary Quantized Neural Network for Dense Prediction via Specially Designed Upsampling and Attention
Then we further optimize the attention method which plays an important role in segmentation but has huge computation complexity.
Combining Radiomics and Machine Learning Approaches for Objective ASD Diagnosis: Verifying White Matter Associations with ASD
To address this gap, we develop a computer-aided diagnostic model focusing on white matter regions in brain MRI by employing radiomics and machine learning methods.
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
AndroidWorld and the experiments in this paper are available at github. com/google-research/android_world.
Collaboration of Teachers for Semi-supervised Object Detection
To address these issues, we propose the Collaboration of Teachers Framework (CTF), which consists of multiple pairs of teacher and student models for training.
TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment
Recent advancements in image understanding have benefited from the extensive use of web image-text pairs.
MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo
We present MVSGaussian, a new generalizable 3D Gaussian representation approach derived from Multi-View Stereo (MVS) that can efficiently reconstruct unseen scenes.
Picking watermarks from noise (PWFN): an improved robust watermarking model against intensive distortions
To improve the robustness of the decoder against stronger noise, this paper proposes to introduce a denoise module between the noise layer and the decoder.
ESP: Extro-Spective Prediction for Long-term Behavior Reasoning in Emergency Scenarios
Furthermore, a new metric named clamped temporal error (CTE) is proposed to give a more comprehensive evaluation of prediction performance, especially in time-sensitive emergency events of subseconds.
FoundaBench: Evaluating Chinese Fundamental Knowledge Capabilities of Large Language Models
In the burgeoning field of large language models (LLMs), the assessment of fundamental knowledge remains a critical challenge, particularly for models tailored to Chinese language and culture.
BezierFormer: A Unified Architecture for 2D and 3D Lane Detection
B\'{e}zierFormer formulate queries as B\'{e}zier control points and incorporate a novel B\'{e}zier curve attention mechanism.
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites
Compared to both open-source and proprietary models, InternVL 1. 5 shows competitive performance, achieving state-of-the-art results in 8 of 18 benchmarks.
Ranked #6 on
Multiple-choice
on Neptune-Full
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking.
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD
The Large Vision-Language Model (LVLM) field has seen significant advancements, yet its progression has been hindered by challenges in comprehending fine-grained visual content due to limited resolution.
Ranked #47 on
Visual Question Answering
on MM-Vet
LIPT: Latency-aware Image Processing Transformer
Extensive experiments on multiple image processing tasks (e. g., image super-resolution (SR), JPEG artifact reduction, and image denoising) demonstrate the superiority of LIPT on both latency and PSNR.
Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution
Knowledge distillation (KD) is a promising yet challenging model compression technique that transfers rich learning representations from a well-performing but cumbersome teacher model to a compact student model.
Make Continual Learning Stronger via C-Flat
A general framework of C-Flat applied to all CL categories and a thorough comparison with loss minima optimizer and flat minima based CL approaches is presented in this paper, showing that our method can boost CL performance in almost all cases.
IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions
Recent advances have demonstrated the powerful capability of transformer architecture in image restoration.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
Distilling Semantic Priors from SAM to Efficient Image Restoration Models
SPD leverages a self-distillation manner to distill the fused semantic priors to boost the performance of original IR models.
CodeS: Natural Language to Code Repository via Multi-Layer Sketch
For feedback-based evaluation, we develop a VSCode plugin for CodeS and engage 30 participants in conducting empirical studies.
IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection
HSF applies Point-to-Grid and Grid-to-Region transformers to capture the multimodal scene context at different granularities.
Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations
We introduce CHARM, the first benchmark for comprehensively and in-depth evaluating the commonsense reasoning ability of large language models (LLMs) in Chinese, which covers both globally known and Chinese-specific commonsense.
RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition
Notably, our approach demonstrates a significant improvement in performance on 5 fine-grained visual recognition benchmarks, 11 few-shot image recognition datasets, and the 2 object detection datasets under the zero-shot recognition setting.
Parameter Efficient Reinforcement Learning from Human Feedback
In this work, we empirically evaluate the setup of Parameter Efficient Reinforcement Learning from Human Feedback (PE-RLHF) that leverages LoRA fine-tuning for Reward Modeling, and Reinforcement Learning.
KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction
After instruction tuning, KnowCoder further exhibits strong generalization ability on unseen schemas and achieves up to $\textbf{12. 5%}$ and $\textbf{21. 9%}$, compared to sota baselines, under the zero-shot setting and the low resource setting, respectively.
Ranked #1 on
UIE
on ACE 2005-EAE
SSF-Net: Spatial-Spectral Fusion Network with Spectral Angle Awareness for Hyperspectral Object Tracking
Hyperspectral video (HSV) offers valuable spatial, spectral, and temporal information simultaneously, making it highly suitable for handling challenges such as background clutter and visual similarity in object tracking.
WanJuan-CC: A Safe and High-Quality Open-sourced English Webtext Dataset
To evaluate the quality and utility of the dataset, we trained 1B-parameter and 3B-parameter models using WanJuan-CC and another dataset, RefinedWeb.
Unlocking the Power of Large Language Models for Entity Alignment
To address the constraints of limited input KG data, ChatEA introduces a KG-code translation module that translates KG structures into a format understandable by LLMs, thereby allowing LLMs to utilize their extensive background knowledge to improve EA accuracy.
An Error-Matching Exclusion Method for Accelerating Visual SLAM
In Visual SLAM, achieving accurate feature matching consumes a significant amount of time, severely impacting the real-time performance of the system.
A Feature Matching Method Based on Multi-Level Refinement Strategy
Feature matching is a fundamental and crucial process in visual SLAM, and precision has always been a challenging issue in feature matching.
A Robust Error-Resistant View Selection Method for 3D Reconstruction
To address the issue of increased triangulation uncertainty caused by selecting views with small camera baselines in Structure from Motion (SFM) view selection, this paper proposes a robust error-resistant view selection method.
Region Feature Descriptor Adapted to High Affine Transformations
To address the issue of feature descriptors being ineffective in representing grayscale feature information when images undergo high affine transformations, leading to a rapid decline in feature matching accuracy, this paper proposes a region feature descriptor based on simulating affine transformations using classification.
A Highlight Removal Method for Capsule Endoscopy Images
The images captured by Wireless Capsule Endoscopy (WCE) always exhibit specular reflections, and removing highlights while preserving the color and texture in the region remains a challenge.
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model
We introduce InternLM-XComposer2, a cutting-edge vision-language model excelling in free-form text-image composition and comprehension.
Ranked #18 on
Visual Question Answering
on MM-Vet v2
Improving Natural Language Capability of Code Large Language Model
Code large language models (Code LLMs) have demonstrated remarkable performance in code generation.
UNIMO-G: Unified Image Generation through Multimodal Conditional Diffusion
This paper presents UNIMO-G, a simple multimodal conditional diffusion framework that operates on multimodal prompts with interleaved textual and visual inputs, which demonstrates a unified ability for both text-driven and subject-driven image generation.
A GAN-based data poisoning framework against anomaly detection in vertical federated learning
Specifically, the malicious participant initially employs semi-supervised learning to train a surrogate target model.
Evolutionary Alternating Direction Method of Multipliers for Constrained Multi-Objective Optimization with Unknown Constraints
Constrained multi-objective optimization problems (CMOPs) pervade real-world applications in science, engineering, and design.
A Generalist FaceX via Learning Unified Facial Representation
This work presents FaceX framework, a novel facial generalist model capable of handling diverse facial tasks simultaneously.
DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detection
Specifically, DMA builds a domain-mixing 3D instance bank for the teacher and student models during training, resulting in aligned data representation.
Domain Similarity-Perceived Label Assignment for Domain Generalized Underwater Object Detection
The inherent characteristics and light fluctuations of water bodies give rise to the huge difference between different layers and regions in underwater environments.
UINav: A Practical Approach to Train On-Device Automation Agents
Automation systems that can autonomously drive application user interfaces to complete user tasks are of great benefit, especially when users are situationally or permanently impaired.
CGS-Mask: Making Time Series Predictions Intuitive for All
Artificial intelligence (AI) has immense potential in time series prediction, but most explainable tools have limited capabilities in providing a systematic understanding of important features over time.
CBQ: Cross-Block Quantization for Large Language Models
Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs.
Brain Computer Interface Technology for Future Battlefield
With the development of artificial intelligence and unmanned equipment, human-machine hybrid formations will be the main focus in future combat formations.
GenDet: Towards Good Generalizations for AI-Generated Image Detection
The misuse of AI imagery can have harmful societal effects, prompting the creation of detectors to combat issues like the spread of fake news.
Knowledge Graph Driven Recommendation System Algorithm
We first use a single-layer neural network to merge individual node features in the graph, and then adjust the aggregation weights of neighboring entities by incorporating influence factors.
An Object Fine-Grained Change Detection Method Based on Frequency Decoupling Interaction for High-Resolution Remote Sensing Images
This module utilizes wavelet transform for frequency decoupling, separating features into different components based on their frequency magnitudes.
Identifying percolation phase transitions with unsupervised learning based on largest clusters
This paper suggests that, by inputting the largest cluster rather than the original configuration into the learning model, unsupervised learning can indeed predict the critical point of the percolation model.
FireMatch: A Semi-Supervised Video Fire Detection Network Based on Consistency and Distribution Alignment
Considering the limited quantity of labeled video data, we propose a semi-supervised fire detection model called FireMatch, which is based on consistency regularization and adversarial distribution alignment.
An invariant feature extraction for multi-modal images matching
This paper aims at providing an effective multi-modal images invariant feature extraction and matching algorithm for the application of multi-source data analysis.
Video-Helpful Multimodal Machine Translation
In addition to the extensive training set, EVA contains a video-helpful evaluation set in which subtitles are ambiguous, and videos are guaranteed helpful for disambiguation.
SALMONN: Towards Generic Hearing Abilities for Large Language Models
Hearing is arguably an essential ability of artificial intelligence (AI) agents in the physical world, which refers to the perception and understanding of general auditory information consisting of at least three types of sounds: speech, audio events, and music.
MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning
Recognizing the significance of pitch in capturing the nuances of IPTs and the importance of onset in locating IPT events, we investigate multi-task finetuning with pitch and onset detection as auxiliary tasks.
On the Convergence of Federated Averaging under Partial Participation for Over-parameterized Neural Networks
Federated learning (FL) is a widely employed distributed paradigm for collaboratively training machine learning models from multiple clients without sharing local data.
Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models
Audio-visual large language models (LLM) have drawn significant attention, yet the fine-grained combination of both input streams is rather under-explored, which is challenging but necessary for LLMs to understand general video inputs.
Cross-head mutual Mean-Teaching for semi-supervised medical image segmentation
To address these concerns, we propose a novel Cross-head mutual mean-teaching Network (CMMT-Net) incorporated strong-weak data augmentation, thereby benefitting both self-training and consistency learning.
A Holistic Evaluation of Piano Sound Quality
Unlike previous studies that focused on the effect of piano performance techniques on sound quality, this study evaluates the inherent sound quality of different pianos.
Model2Scene: Learning 3D Scene Representation via Contrastive Language-CAD Models Pre-training
In this paper, we propose Model2Scene, a novel paradigm that learns free 3D scene representation from Computer-Aided Design (CAD) models and languages.
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition
We propose InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition.
Ranked #9 on
Visual Question Answering (VQA)
on InfiMM-Eval
IFT: Image Fusion Transformer for Ghost-free High Dynamic Range Imaging
By matching similar patches between frames, objects with large motion ranges in dynamic scenes can be aligned, which can effectively alleviate the generation of artifacts.
Connecting Speech Encoder and Large Language Model for ASR
Q-Former-based LLMs can generalise well to out-of-domain datasets, where 12% relative WER reductions over the Whisper baseline ASR model were achieved on the Eval2000 test set without using any in-domain training data from Switchboard.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Data Upcycling Knowledge Distillation for Image Super-Resolution
Knowledge distillation (KD) compresses deep neural networks by transferring task-related knowledge from cumbersome pre-trained teacher models to compact student models.
Ranked #32 on
Image Super-Resolution
on Urban100 - 4x upscaling
EMelodyGen: Emotion-Conditioned Melody Generation in ABC Notation with the Musical Feature Template
The EMelodyGen system focuses on emotional melody generation in ABC notation controlled by the musical feature template.
MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation
We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation.
MiChao-HuaFen 1.0: A Specialized Pre-trained Corpus Dataset for Domain-specific Large Models
With the advancement of deep learning technologies, general-purpose large models such as GPT-4 have demonstrated exceptional capabilities across various domains.
SoccerNet 2023 Challenges Results
More information on the tasks, challenges, and leaderboards are available on https://www. soccer-net. org.
VIGC: Visual Instruction Generation and Correction
A practical solution to this problem would be to utilize the available multimodal large language models (MLLMs) to generate instruction data for vision-language tasks.
WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models
The rise in popularity of ChatGPT and GPT-4 has significantly accelerated the development of large models, leading to the creation of numerous impressive large language models(LLMs) and multimodal large language models (MLLMs).
Improving Anomaly Segmentation with Multi-Granularity Cross-Domain Alignment
Anomaly segmentation plays a pivotal role in identifying atypical objects in images, crucial for hazard detection in autonomous driving systems.
Purely Speckled Intensity Images Need for SAR Despeckling with SDS-SAR
To address these challenges, we propose a Self-supervised Despeckling Strategy for SAR images (SDS-SAR) that relies solely on speckled intensity data for training.
A Hybrid CNN-Transformer Architecture with Frequency Domain Contrastive Learning for Image Deraining
Image deraining is a challenging task that involves restoring degraded images affected by rain streaks.
Improving Generalization in Visual Reinforcement Learning via Conflict-aware Gradient Agreement Augmentation
In particular, CG2A develops a Gradient Agreement Solver to adaptively balance the varying gradient magnitudes, and introduces a Soft Gradient Surgery strategy to alleviate the gradient conflicts.
Adaptive Graph Convolution Networks for Traffic Flow Forecasting
The AGC-net is constructed by the Adaptive Graph Convolution (AGC) based on a novel context attention mechanism, which consists of a set of graph wavelets with various learnable scales.
NeMO: Neural Map Growing System for Spatiotemporal Fusion in Bird's-Eye-View and BDD-Map Benchmark
We also provide a new scene-level BEV map evaluation setting along with the corresponding baseline for a more comprehensive comparison.
Balancing Logit Variation for Long-tailed Semantic Segmentation
In this way, we manage to close the gap between the feature areas of different categories, resulting in a more balanced representation.
Contextual Object Detection with Multimodal Large Language Models
Moreover, we present ContextDET, a unified multimodal model that is capable of end-to-end differentiable modeling of visual-language contexts, so as to locate, identify, and associate visual objects with language inputs for human-AI interaction.
On the Value of Myopic Behavior in Policy Reuse
Leveraging learned strategies in unfamiliar scenarios is fundamental to human intelligence.
Language-Guided 3D Object Detection in Point Cloud for Autonomous Driving
In this work, we propose a new multi-modal visual grounding task, termed LiDAR Grounding.
Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentation
To this end, we propose T2S-DA, which we interpret as a form of pulling Target to Source for Domain Adaptation, encouraging the model in learning similar cross-domain features.
Phonetic and Prosody-aware Self-supervised Learning Approach for Non-native Fluency Scoring
Specifically, we first pre-train the model using a reconstruction loss function, by masking phones and their durations jointly on a large amount of unlabeled speech and text prompts.
PaLM 2 Technical Report
Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM.
Ranked #1 on
Question Answering
on StrategyQA
High-Resolution Remote Sensing Bitemporal Image Change Detection Based on Feature Interaction and Multitask Learning
With the development of remote sensing technology, high-resolution (HR) remote sensing optical images have gradually become the main source of change detection data.
Mlinear: Rethink the Linear Model for Time-series Forecasting
In this paper, we carefully examine the opposing properties of CI and CD, and raise a practical question that has not been effectively answered, e. g.,"How to effectively mix the CI and CD properties of time series to achieve better predictive performance?"
TransHP: Image Classification with Hierarchical Prompting
This paper explores a hierarchical prompting mechanism for the hierarchical image classification (HIC) task.
Adaptive Mask Sampling and Manifold to Euclidean Subspace Learning with Distance Covariance Representation for Hyperspectral Image Classification
Subsequently, based on distance covariance descriptor, a dual channel distance covariance representation (DC-DCR) module is proposed for modeling unified spectral-spatial feature representations and exploring spectral-spatial relationships, especially linear and nonlinear interdependence in spectral domain.
Ranked #1 on
Hyperspectral Image Classification
on Indian Pines
(OA@5%perclass metric)
Hyperspectral image analysis
Hyperspectral Image Classification
+1
Siamese DETR
In this work, we present Siamese DETR, a Siamese self-supervised pretraining approach for the Transformer architecture in DETR.
