Search Results for author:
Found 238 papers, 85 papers with code
A Unified Encoding of Structures in Transition Systems
Transition systems usually contain various dynamic structures (e. g., stacks, buffers).
A Novel Framework Based on Medical Concept Driven Attention for Explainable Medical Code Prediction via External Knowledge
Medical code prediction from clinical notes aims at automatically associating medical codes with the clinical notes.
Word Reordering for Zero-shot Cross-lingual Structured Prediction
Adapting word order from one language to another is a key problem in cross-lingual structured prediction.
Task-oriented Domain-specific Meta-Embedding for Text Classification
Meta-embedding learning, which combines complementary information in different word embeddings, have shown superior performances across different Natural Language Processing tasks.
A Knowledge-Informed Large Language Model Framework for U.S. Nuclear Power Plant Shutdown Initiating Event Classification for Probabilistic Risk Assessment
Text vectorization based on the SDIE patterns generates feature vectors that are highly separable by using a simple binary classifier.
DPI-TTS: Directional Patch Interaction for Fast-Converging and Style Temporal Modeling in Text-to-Speech
In recent years, speech diffusion models have advanced rapidly.
Text Prompt is Not Enough: Sound Event Enhanced Prompt Adapter for Target Style Audio Generation
Additionally, the Sound Event Reference Style Transfer Dataset (SERST) is introduced for the proposed target style audio generation task, enabling dual-prompt audio generation using both text and audio references.
LLM-GAN: Construct Generative Adversarial Network Through Large Language Models For Explainable Fake News Detection
Explainable fake news detection predicts the authenticity of news items with annotated explanations.
Multi-modal Adversarial Training for Zero-Shot Voice Cloning
The discriminator is used in a training pipeline that improves both the acoustic and prosodic features of a TTS model.
GrassNet: State Space Model Meets Graph Neural Network
To overcome these issues, in this paper, we propose Graph State Space Network (GrassNet), a novel graph neural network with theoretical support that provides a simple yet effective scheme for designing and learning arbitrary graph spectral filters.
P/D-Serve: Serving Disaggregated Large Language Model at Scale
To overcome previous problems, this paper proposes an end-to-end system P/D-Serve, complying with the paradigm of MLOps (machine learning operations), which models end-to-end (E2E) P/D performance and enables: 1) fine-grained P/D organization, mapping the service with RoCE (RDMA over converged ethernet) as needed, to facilitate similar processing and dynamic adjustments on P/D ratios; 2) on-demand forwarding upon rejections for idle prefill, decoupling the scheduler from regular inaccurate reports and local queues, to avoid timeouts in prefill; and 3) efficient KVCache transfer via optimized D2D access.
CorrAdaptor: Adaptive Local Context Learning for Correspondence Pruning
In the fields of computer vision and robotics, accurate pixel-level correspondences are essential for enabling advanced tasks such as structure-from-motion and simultaneous localization and mapping.
DIffSteISR: Harnessing Diffusion Prior for Superior Real-world Stereo Image Super-Resolution
Extensive experimental results demonstrate that DiffSteISR accurately reconstructs natural and precise textures from low-resolution stereo images while maintaining a high consistency of semantic and texture between the left and right views.
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
For tasks such as text-to-speech (TTS), voice conversion (VC), and automatic speech recognition (ASR), a cross-modal fine-grained (frame-level) sequence representation is desired, emphasizing the semantic content of the text modality while de-emphasizing the paralinguistic information of the speech modality.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model
In this paper, we propose UnifiedMLLM, a comprehensive model designed to represent various tasks using a unified representation.
Towards Automated Data Sciences with Natural Language and SageCopilot: Practices and Lessons Learned
While the field of NL2SQL has made significant advancements in translating natural language instructions into executable SQL scripts for data querying and processing, achieving full automation within the broader data science pipeline - encompassing data querying, analysis, visualization, and reporting - remains a complex challenge.
Serialized Point Mamba: A Serialized Point Cloud Mamba Segmentation Model
Point cloud segmentation is crucial for robotic visual perception and environmental understanding, enabling applications such as robotic navigation and 3D reconstruction.
Ranked #1 on
3D Instance Segmentation
on ScanNet(v2)
(mAP@50 metric)
How Deep is your Guess? A Fresh Perspective on Deep Learning for Medical Time-Series Imputation
By identifying conceptual gaps in the literature and existing reviews, we devise a taxonomy grounded on the inductive bias of neural imputation frameworks, resulting in a classification of existing deep imputation strategies based on their suitability for specific imputation scenarios and data-specific properties.
A Comprehensive Survey on the Security of Smart Grid: Challenges, Mitigations, and Future Research Opportunities
The review particularly includes an extensive analysis of coordinated attacks that incorporate multiple attack strategies and exploit vulnerabilities across various smart grid components to increase their adverse impact, demonstrating the complexity and potential severity of these threats.
ASRRL-TTS: Agile Speaker Representation Reinforcement Learning for Text-to-Speech Speaker Adaptation
The results indicate that the ASRRL method significantly outperforms traditional fine-tuning approaches, achieving higher speaker similarity and better overall speech quality with limited reference speeches.
Looking From the Future: Multi-order Iterations Can Enhance Adversarial Attack Transferability
In this paper, we rethink the optimization process and propose a novel sequence optimization concept, which is named Looking From the Future (LFF).
A Multi-Stage Goal-Driven Network for Pedestrian Trajectory Prediction
The network comprises three main components: a conditional variational autoencoder (CVAE), an attention module, and a multi-stage goal evaluator.
MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation
Despite advancements in AIGC technologies for text and image generation, the foley audio dubbing remains rudimentary due to difficulties in cross-modal scene matching and content correlation.
CorrMAE: Pre-training Correspondence Transformers with Masked Autoencoder
To tackle these challenges, we propose a pre-training method to acquire a generic inliers-consistent representation by reconstructing masked correspondences, providing a strong initial representation for downstream tasks.
DFA-GNN: Forward Learning of Graph Neural Networks by Direct Feedback Alignment
Specifically, DFA-GNN extends the principles of DFA to adapt to graph data and unique architecture of GNNs, which incorporates the information of graph topology into the feedback links to accommodate the non-Euclidean characteristics of graph data.
Mollification Effects of Policy Gradient Methods
Policy gradient methods have enabled deep reinforcement learning (RL) to approach challenging continuous control problems, even when the underlying systems involve highly nonlinear dynamics that generate complex non-smooth optimization landscapes.
Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression
In this paper, we introduce \textbf{DecoQuant}, a novel data-free low-bit quantization technique based on tensor decomposition methods, to effectively compress KV cache.
Dynamic Loss Decay based Robust Oriented Object Detection on Remote Sensing Images with Noisy Labels
The ambiguous appearance, tiny scale, and fine-grained classes of objects in remote sensing imagery inevitably lead to the noisy annotations in category labels of detection dataset.
QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
For negative knowledge, we propose an innovative self-adversarial approach that generates low-quality rationales by sampling previous iterations of smaller language models, embracing the idea that one can learn from one's own weaknesses.
Understanding the Difficulty of Solving Cauchy Problems with PINNs
Physics-Informed Neural Networks (PINNs) have gained popularity in scientific computing in recent years.
Deep Reinforcement Learning-aided Transmission Design for Energy-efficient Link Optimization in Vehicular Communications
This letter presents a deep reinforcement learning (DRL) approach for transmission design to optimize the energy efficiency in vehicle-to-vehicle (V2V) communication links.
Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data
The former indicates that representation collapse in local model will subsequently impact the global model and other local models.
Innovative Quantitative Analysis for Disease Progression Assessment in Familial Cerebral Cavernous Malformations
To alleviate this problem, we propose a quantitative statistical framework for FCCM, comprising an efficient annotation module, an FCCM lesion segmentation module, and an FCCM lesion quantitative statistics module.
OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models
We also observe that the initiation denoising timestep for noise blending is the key to identity preservation and layout.
Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration
Specifically, we utilize sparse self-attention to filter out redundant information and noise, directing the model's attention to focus on the features more relevant to the degraded regions in need of reconstruction.
Zero-Shot Aerial Object Detection with Visual Description Regularization
Instead of directly encoding the descriptions into class embedding space which suffers from the representation gap problem, we propose to infuse the prior inter-class visual similarity conveyed in the descriptions into the embedding learning.
Feynman Diagrams as Computational Graphs
We propose a computational graph representation of high-order Feynman diagrams in Quantum Field Theory (QFT), applicable to any combination of spatial, temporal, momentum, and frequency domains.
A Heterogeneous Dynamic Convolutional Neural Network for Image Super-resolution
The lower network utilizes a symmetric architecture to enhance relations of different layers to mine more structural information, which is complementary with a upper network for image super-resolution.
PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining
We introduce a privacy auditing scheme for ML models that relies on membership inference attacks using generated data as "non-members".
PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal
For the prompt generation, we first propose a prompt pre-training strategy to train a frequency prompt encoder that encodes the ground-truth image into LF and HF prompts.
Anything in Any Scene: Photorealistic Video Object Insertion
Our proposed general framework encompasses three key processes: 1) integrating a realistic object into a given scene video with proper placement to ensure geometric realism; 2) estimating the sky and environmental lighting distribution and simulating realistic shadows to enhance the light realism; 3) employing a style transfer network that refines the final video output to maximize photorealism.
GroundingGPT:Language Enhanced Multi-modal Grounding Model
Beyond capturing global information like other multi-modal models, our proposed model excels at tasks demanding a detailed understanding of local information within the input.
Dual Teacher Knowledge Distillation with Domain Alignment for Face Anti-spoofing
In this paper, we propose a domain adversarial attack (DAA) method to mitigate the training instability problem by adding perturbations to the input images, which makes them indistinguishable across domains and enables domain alignment.
SynSP: Synergy of Smoothness and Precision in Pose Sequences Refinement
To leverage this quality information we propose a motion refinement network termed SynSP to achieve a Synergy of Smoothness and Precision in the sequence refinement tasks.
Knowledge Enhanced Conditional Imputation for Healthcare Time-series
We present an end-to-end architecture for managing complex missingness in multivariate time series derived from hospital electronic health records (EHRs).
Towards Real-World Blind Face Restoration with Generative Diffusion Prior
We propose BFRffusion which is thoughtfully designed to effectively extract features from low-quality face images and could restore realistic and faithful facial details with the generative prior of the pretrained Stable Diffusion.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
Research on Multilingual Natural Scene Text Detection Algorithm
Natural scene text detection is a significant challenge in computer vision, with tremendous potential applications in multilingual, diverse, and complex text scenarios.
VSFormer: Visual-Spatial Fusion Transformer for Correspondence Pruning
Then, we model these visual cues and correspondences by a joint visual-spatial fusion module, simultaneously embedding visual cues into correspondences for pruning.
Learning to Skip for Language Modeling
Overparameterized large-scale language models have impressive generalization performance of in-context few-shot learning.
Boost Adversarial Transferability by Uniform Scale and Mix Mask Method
To address these challenges, we propose a framework called Uniform Scale and Mix Mask Method (US-MM) for adversarial example generation.
Complementary Advantages of ChatGPTs and Human Readers in Reasoning: Evidence from English Text Reading Comprehension
Our study reveals that human readers and ChatGPTs have their respective advantages and disadvantages in drawing inferences from text reading comprehension, unlocking a complementary relationship in text-based reasoning.
Pseudo Label-Guided Data Fusion and Output Consistency for Semi-Supervised Medical Image Segmentation
Inspired by semi-supervised algorithms that use both labeled and unlabeled data for training, we propose the PLGDF framework, which builds upon the mean teacher network for segmenting medical images with less annotation.
Controlled Decoding from Language Models
KL-regularized reinforcement learning (RL) is a popular alignment framework to control the language model responses towards high reward outcomes.
Language Modelling
Multi-Objective Reinforcement Learning
+2
VCL Challenges 2023 at ICCV 2023 Technical Report: Bi-level Adaptation Method for Test-time Adaptive Object Detection
This report outlines our team's participation in VCL Challenges B Continual Test_time Adaptation, focusing on the technical details of our approach.
SCB-Dataset3: A Benchmark for Detecting Student Classroom Behavior
The use of deep learning methods to automatically detect students' classroom behavior is a promising approach for analyzing their class performance and improving teaching effectiveness.
Estimation and Testing of Forecast Rationality with Many Moments
We in this paper utilize P-GMM (Cheng and Liao, 2015) moment selection procedure to select valid and relevant moments for estimating and testing forecast rationality under the flexible loss proposed by Elliott et al. (2005).
Base Station Beamforming Design for Near-field XL-IRS Beam Training
However, this approach may cause degraded beam training performance in practice due to the near-field channel model of the BS-IRS link.
Deep Video Restoration for Under-Display Camera
With the pipeline, we build the first large-scale UDC video restoration dataset called PexelsUDC, which includes two subsets named PexelsUDC-T and PexelsUDC-P corresponding to different displays for UDC.
Learning Speech Representation From Contrastive Token-Acoustic Pretraining
However, existing contrastive learning methods in the audio field focus on extracting global descriptive information for downstream audio classification tasks, making them unsuitable for TTS, VC, and ASR tasks.
Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation for Semi-Supervised Medical Image Segmentation
In this paper, we present a novel semi-supervised learning method, Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation (DCPA), for medical image segmentation.
Semantic-aware Consistency Network for Cloth-changing Person Re-Identification
Despite recent progress in CC-ReID, existing approaches are still hindered by the interference of clothing variations since they lack effective constraints to keep the model consistently focused on clothing-irrelevant regions.
Blind Face Restoration for Under-Display Camera via Dictionary Guided Transformer
However, due to the characteristics of the display, images taken by UDC suffer from significant quality degradation.
Development of a Knowledge Graph Embeddings Model for Pain
This paper describes the construction of such knowledge graph embedding models of pain concepts, extracted from the unstructured text of mental health electronic health records, combined with external knowledge created from relations described in SNOMED CT, and their evaluation on a subject-object link prediction task.
Deep Semantic Graph Matching for Large-scale Outdoor Point Clouds Registration
The adjacent points with the same category labels are then clustered together using the Euclidean clustering algorithm to obtain the semantic instances, which are represented by three kinds of attributes including spatial location information, semantic categorical information, and global geometric shape information.
Fine-grained building roof instance segmentation based on domain adapted pretraining and composite dual-backbone
The diversity of building architecture styles of global cities situated on various landforms, the degraded optical imagery affected by clouds and shadows, and the significant inter-class imbalance of roof types pose challenges for designing a robust and accurate building roof instance segmentor.
HGDNet: A Height-Hierarchy Guided Dual-Decoder Network for Single View Building Extraction and Height Estimation
Unifying the correlative single-view satellite image building extraction and height estimation tasks indicates a promising way to share representations and acquire generalist model for large-scale urban 3D reconstruction.
Few-shot Class-Incremental Semantic Segmentation via Pseudo-Labeling and Knowledge Distillation
Given only one or a few images labeled with the novel classes and a much larger set of unlabeled images, we transfer the knowledge from labeled images to unlabeled images with a coarse-to-fine pseudo-labeling approach in two steps.
Class-Incremental Semantic Segmentation
Knowledge Distillation
Class-Specific Distribution Alignment for Semi-Supervised Medical Image Classification
Despite the success of deep neural networks in medical image classification, the problem remains challenging as data annotation is time-consuming, and the class distribution is imbalanced due to the relative scarcity of diseases.
Image Classification
Semi-supervised Medical Image Classification
Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding
However, existing methods suffer from three problems: the high dimensionality and waveform distortion of discrete speech representations, the prosodic averaging problem caused by the duration prediction model in non-autoregressive frameworks, and the information redundancy and dimension explosion problems of existing semantic encoding methods.
LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement
In this paper, we address this limitation by proposing a degradation-aware learning scheme for LLIE using diffusion models, which effectively integrates degradation and image priors into the diffusion process, resulting in improved image enhancement.
Semi-Supervised Medical Image Segmentation with Co-Distribution Alignment
Medical image segmentation has made significant progress when a large amount of labeled data are available.
An Intelligent Remote Sensing Image Quality Inspection System
Due to the inevitable presence of quality problems, quality inspection of remote sensing images is indeed an indispensable step between the acquisition and the application of them.
Bridging the Gap: Multi-Level Cross-Modality Joint Alignment for Visible-Infrared Person Re-Identification
Visible-Infrared person Re-IDentification (VI-ReID) is a challenging cross-modality image retrieval task that aims to match pedestrians' images across visible and infrared cameras.
Cross-Modality Person Re-identification
Image Classification
+4
BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training
In this study, we systematically analyze and compare various mainstream and cutting-edge automatic metrics from the perspective of their guidance for training machine translation systems.
Seeing is not Believing: An Identity Hider for Human Vision Privacy Protection
Concretely, the identity hider benefits from two specially designed modules: 1) The virtual face generation module generates a virtual face with a new appearance by manipulating the latent space of StyleGAN2.
Improving Text Matching in E-Commerce Search with A Rationalizable, Intervenable and Fast Entity-Based Relevance Model
We also show that pretraining the QE module with auto-generated QE data from user logs can further improve the overall performance.
PCDAL: A Perturbation Consistency-Driven Active Learning Approach for Medical Image Segmentation and Classification
To address this limitation, we propose an AL-based method that can be simultaneously applied to 2D medical image classification, segmentation, and 3D medical image segmentation tasks.
Valley: Video Assistant with Large Language model Enhanced abilitY
Large language models (LLMs), with their remarkable conversational capabilities, have demonstrated impressive performance across various applications and have emerged as formidable AI assistants.
Boosting Fast and High-Quality Speech Synthesis with Linear Diffusion
When decreasing the number of sampling steps (i. e., the number of line segments used to fit the path), the ease of fitting straight lines compared to curves allows us to generate higher quality samples from a random noise with fewer iterations.
GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions
Second, we introduce a residual dense transformer block (RDTB) as the final GridFormer layer.
Improving speech translation by fusing speech and text
To tackle these gaps, we propose \textbf{F}use-\textbf{S}peech-\textbf{T}ext (\textbf{FST}), a cross-modal model which supports three distinct input modalities for translation: speech, text, and fused speech-text.
Graph Propagation Transformer for Graph Representation Learning
The core insight of our method is to fully consider the information propagation among nodes and edges in a graph when building the attention module in the transformer blocks.
Ranked #2 on
Graph Regression
on PCQM4M-LSC
(Validation MAE metric)
PaLM 2 Technical Report
Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM.
Ranked #1 on
Question Answering
on StrategyQA
Student Classroom Behavior Detection based on YOLOv7-BRA and Multi-Model Fusion
We constructed a dataset, which contained 11, 248 labels and 4, 001 images, with an emphasis on the common behavior of raising hands in a classroom setting (Student Classroom Behavior dataset, SCB-Dataset).
RFR-WWANet: Weighted Window Attention-Based Recovery Feature Resolution Network for Unsupervised Image Registration
Furthermore, shifted window partitioning operations are inflexible, indicating that they cannot perceive the semantic information over uncertain distances and automatically bridge the global connections between windows.
A Soft Coordination Method of Heterogeneous Devices in Distribution System Voltage Control
The proposed method achieves the soft coordination by establishing a modified actor-critic algorithm to train a proxy model of inverters.
GRIG: Few-Shot Generative Residual Image Inpainting
Image inpainting is the task of filling in missing or masked region of an image with semantically meaningful contents.
The Cascaded Forward Algorithm for Neural Network Training
Backpropagation algorithm has been widely used as a mainstream learning procedure for neural networks in the past decade, and has played a significant role in the development of deep learning.
Feature Completion Transformer for Occluded Person Re-identification
In this paper, different from most previous works that discard the occluded region, we propose a Feature Completion Transformer (FCFormer) to implicitly complement the semantic information of occluded parts in the feature space.
Noise2Music: Text-conditioned Music Generation with Diffusion Models
We introduce Noise2Music, where a series of diffusion models is trained to generate high-quality 30-second music clips from text prompts.
Ranked #5 on
Text-to-Music Generation
on MusicCaps
Spatio-Temporal Point Process for Multiple Object Tracking
As such, we propose a novel framework that can effectively predict and mask-out the noisy and confusing detection results before associating the objects into trajectories.
Spatio-Temporal Context Modeling for Road Obstacle Detection
The results indicate that our method with spatio-temporal context modeling is superior to existing methods for road obstacle detection.
A Multi-Scale Framework for Out-of-Distribution Detection in Dermoscopic Images
The automatic detection of skin diseases via dermoscopic images can improve the efficiency in diagnosis and help doctors make more accurate judgments.
Mortality Prediction with Adaptive Feature Importance Recalibration for Peritoneal Dialysis Patients: a deep-learning-based study on a real-world longitudinal follow-up dataset
Here, our objective is to develop a deep learning model for a real-time, individualized, and interpretable mortality prediction model - AICare.
