Search Results for author:
Found 115 papers, 38 papers with code
PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification
The other is to classification on feature distribution of visual tokens from vision encoder.
Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition
The past years have witnessed a proliferation of large language models (LLMs).
Prompting Multi-Modal Tokens to Enhance End-to-End Autonomous Driving Imitation Learning with LLMs
The utilization of Large Language Models (LLMs) within the realm of reinforcement learning, particularly as planners, has garnered a significant degree of attention in recent scholarly literature.
Weakly Supervised Deep Hyperspherical Quantization for Image Retrieval
Deep quantization methods have shown high efficiency on large-scale image retrieval.
Multi-perspective Memory Enhanced Network for Identifying Key Nodes in Social Networks
Existing key node identification methods usually consider node influence only from the propagation structure perspective and have insufficient generalization ability to unknown scenarios.
Hierarchical Information Enhancement Network for Cascade Prediction in Social Networks
In this work, we propose a novel Hierarchical Information Enhancement Network (HIENet) for cascade prediction.
Reinforcement Learning with Generalizable Gaussian Splatting
The quality of the environment representation directly influences the achievement of the learning task.
Decode Neural signal as Speech
In this paper, we explore the brain-to-text translation of MEG signals in a speech-decoding formation.
StableMask: Refining Causal Masking in Decoder-only Transformer
The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling.
Diffusion-based graph generative methods
Being the most cutting-edge generative methods, diffusion methods have shown great advances in wide generation tasks.
Scientific Large Language Models: A Survey on Biological & Chemical Domains
Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension, representing a significant stride toward artificial general intelligence.
Study Features via Exploring Distribution Structure
In this paper, we present a novel framework for data redundancy measurement based on probabilistic modeling of datasets, and a new criterion for redundancy detection that is resilient to noise.
A Contrast Based Feature Selection Algorithm for High-dimensional Data set in Machine Learning
Feature selection is an important process in machine learning and knowledge discovery.
Feature Selection via Maximizing Distances between Class Conditional Distributions
Our framework directly explores the discriminative information of features in the sense of distributions for supervised classification.
Deep Blind Super-Resolution for Satellite Video
Therefore, this paper proposes a practical Blind SVSR algorithm (BSVSR) to explore more sharp cues by considering the pixel-wise blur levels in a coarse-to-fine manner.
Exploiting Polarized Material Cues for Robust Car Detection
Car detection is an important task that serves as a crucial prerequisite for many automated driving functions.
From Function to Distribution Modeling: A PAC-Generative Approach to Offline Optimization
This paper considers the problem of offline optimization, where the objective function is unknown except for a collection of ``offline" data examples.
Multiform Evolution for High-Dimensional Problems with Low Effective Dimensionality
In this paper, we scale evolutionary algorithms to high-dimensional optimization problems that deceptively possess a low effective dimensionality (certain dimensions do not significantly affect the objective function).
Mind the Gap Between Conversations for Improved Long-Term Dialogue Generation
Knowing how to end and resume conversations over time is a natural part of communication, allowing for discussions to span weeks, months, or years.
GenKIE: Robust Generative Multimodal Document Key Information Extraction
Key information extraction (KIE) from scanned documents has gained increasing attention because of its applications in various domains.
Fully Spiking Neural Network for Legged Robots
In recent years, legged robots based on deep reinforcement learning have made remarkable progress.
InstructProtein: Aligning Human and Protein Language via Knowledge Instruction
To address this challenge, we propose InstructProtein, an innovative LLM that possesses bidirectional generation capabilities in both human and protein languages: (i) taking a protein sequence as input to predict its textual function description and (ii) using natural language to prompt protein sequence generation.
Scalable Neural Dynamic Equivalence for Power Systems
Traditional grid analytics are model-based, relying strongly on accurate models of power systems, especially the dynamic models of generators, controllers, loads and other dynamic components.
Physics-Aware Neural Dynamic Equivalence of Power Systems
The contributions are threefold: (1) an ODE-Net-enabled NeuDyE formulation to enable a continuous-time, data-driven dynamic equivalence of power systems; (2) a physics-informed NeuDyE learning method (PI-NeuDyE) to actively control the closed-loop accuracy of NeuDyE without an additional verification module; (3) a physics-guided NeuDyE (PG-NeuDyE) to enhance the method's applicability even in the absence of analytical physics models.
Enhancing 6-DoF Object Pose Estimation through Multiple Modality Fusion: A Hybrid CNN Architecture with Cross-Layer and Cross-Modal Integration
To tackle this challenge, we proposed a pioneering two-stage hybrid Convolutional Neural Network (CNN) architecture, which connects segmentation and pose estimation in tandem.
Ranked #1 on
6D Pose Estimation using RGBD
on YCB-Video
Graph Sampling-based Meta-Learning for Molecular Property Prediction
To effectively utilize many-to-many correlations of molecules and properties, we propose a Graph Sampling-based Meta-learning (GS-Meta) framework for few-shot molecular property prediction.
Spiking Denoising Diffusion Probabilistic Models
In our paper, we put forward Spiking Denoising Diffusion Probabilistic Models (SDDPM), a new class of SNN-based generative models that achieve high sample quality.
Knowledge-Enhanced Hierarchical Information Correlation Learning for Multi-Modal Rumor Detection
The explosive growth of rumors with text and images on social media platforms has drawn great attention.
UltraGlove: Hand Pose Estimation with Mems-Ultrasonic Sensors
Hand tracking is an important aspect of human-computer interaction and has a wide range of applications in extended reality devices.
Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models
We propose the "Ask an Expert" framework in which the model is trained with access to an "expert" which it can consult at each turn.
A Survey on Asking Clarification Questions Datasets in Conversational Systems
The ability to understand a user's underlying needs is critical for conversational systems, especially with limited input from users in a conversation.
Newton-Cotes Graph Neural Networks: On the Time Evolution of Dynamic Systems
Reasoning system dynamics is one of the most important analytical approaches for many scientific studies.
GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark
With a fast developing pace of geographic applications, automatable and intelligent models are essential to be designed to handle the large volume of information.
Optimal Investment and Consumption Strategies with General and Linear Transaction Costs under CRRA Utility
Transaction costs play a critical role in asset allocation and consumption strategies in portfolio management.
DUFormer: Solving Power Line Detection Task in Aerial Images using Semantic Segmentation
Unmanned aerial vehicles (UAVs) are frequently used for inspecting power lines and capturing high-resolution aerial images.
The Short Text Matching Model Enhanced with Knowledge via Contrastive Learning
The model uses a generative model to generate corresponding complement sentences and uses the contrastive learning method to guide the model to obtain more semantically meaningful encoding of the original sentence.
Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels
In this paper, we advocate remedying the deficiency of label supervision for the MLR with incomplete labels by deriving a structured semantic prior about the label-to-label correspondence via a semantic prior prompter.
MGeo: Multi-Modal Geographic Pre-Training Method
Single-modal PTMs can barely make use of the important GC and therefore have limited performance.
Efficient RGB-T Tracking via Cross-Modality Distillation
Most current RGB-T trackers adopt a two-stream structure to extract unimodal RGB and thermal features and complex fusion strategies to achieve multi-modal feature fusion, which require a huge number of parameters, thus hindering their real-life applications.
Ranked #5 on
Rgb-T Tracking
on GTOT
Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning
Audio-visual speech recognition (AVSR) has gained remarkable success for ameliorating the noise-robustness of speech recognition.
Predicted Trajectory Guidance Control Framework of Teleoperated Ground Vehicles Compensating for Delays
Maneuverability and drivability of the teleoperated ground vehicle could be seriously degraded by large communication delays if the delays are not properly compensated.
QuadFormer: Quadruple Transformer for Unsupervised Domain Adaptation in Power Line Segmentation of Aerial Images
This process is studied in Unsupervised domain adaptation (UDA).
Causal Deep Reinforcement Learning Using Observational Data
Offline reinforcement learning promises to alleviate this issue by exploiting the vast amount of observational data available in the real world.
Wider and Higher: Intensive Integration and Global Foreground Perception for Image Matting
This paper reviews recent deep-learning-based matting research and conceives our wider and higher motivation for image matting.
Explore Contextual Information for 3D Scene Graph Generation
3D scene graph generation (SGG) has been of high interest in computer vision.
Masked Spiking Transformer
The combination of Spiking Neural Networks (SNNs) and Transformers has attracted significant attention due to their potential for high energy efficiency and high-performance nature.
Trajectory Tracking Control of the Bionic Joint Actuated by Pneumatic Artificial Muscle Based on Robust Modeling
Then, a hybrid model is established based on the two models (the nonlinear model and the LTI model) and corresponding to it, a cascaded controller is developed, the outer loop of which is an H-infinite controller for the angular position tracking designed by loop-shaping design procedure (LSDP) and the inner loop is a nonlinear controller based on the feedback linearization theory for the PAM driving pressure control.
Relay Hindsight Experience Replay: Self-Guided Continual Reinforcement Learning for Sequential Object Manipulation Tasks with Sparse Rewards
To solve these tasks efficiently, we propose a novel self-guided continual RL framework, RelayHER (RHER).
Differentiable Point-Based Radiance Fields for Efficient View Synthesis
We propose a differentiable rendering algorithm for efficient novel view synthesis.
Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problem
We introduce the task of implicit offensive text detection in dialogues, where a statement may have either an offensive or non-offensive interpretation, depending on the listener and context.
StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts
Our experiments demonstrate that state-of-the-art models on the bAbI dataset struggle on the StepGame dataset.
Ranked #1 on
Question Answering
on StepGame
Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking
Existing approaches that have considered such relations generally fall short in: (1) fusing prior slot-domain membership relations and dialogue-aware dynamic slot relations explicitly, and (2) generalizing to unseen domains.
Dialogue State Tracking
Multi-domain Dialogue State Tracking
+1
Hybrid Routing Transformer for Zero-Shot Learning
This design makes the presented transformer model a hybrid of 1) top-down and bottom-up attention pathways and 2) dynamic and static routing pathways.
Deep Reinforcement Learning for Entity Alignment
Embedding-based methods have attracted increasing attention in recent entity alignment (EA) studies.
Unleashing the Power of Transformer for Graphs
Our experiments demonstrate DET has achieved superior performance compared to the respective state-of-the-art methods in dealing with molecules, networks and knowledge graphs with various sizes.
Prompt-Guided Injection of Conformation to Pre-trained Protein Model
To incorporate conformational knowledge to PTPMs, we propose an interaction-conformation prompt (IC prompt) that is learned through back-propagation with the protein-protein interaction task.
OntoProtein: Protein Pretraining With Gene Ontology Embedding
We construct a novel large-scale knowledge graph that consists of GO and its related proteins, and gene annotation texts or protein sequences describe all nodes in the graph.
On Exploring Pose Estimation as an Auxiliary Learning Task for Visible-Infrared Person Re-identification
Visible-infrared person re-identification (VI-ReID) has been challenging due to the existence of large discrepancies between visible and infrared modalities.
Cross-Modality Deep Feature Learning for Brain Tumor Segmentation
Recent advances in machine learning and prevalence of digital medical images have opened up an opportunity to address the challenging brain tumor segmentation (BTS) task by using deep convolutional neural networks.
FMCNet: Feature-Level Modality Compensation for Visible-Infrared Person Re-Identification
Then, a feature-level modality compensation module is present to generate those missing modality-specific features from existing modality-shared ones.
A New Perspective on the Effects of Spectrum in Graph Neural Networks
Many improvements on GNNs can be deemed as operations on the spectrum of the underlying graph matrix, which motivates us to directly study the characteristics of the spectrum and their effects on GNN performance.
Ranked #3 on
Graph Classification
on ENZYMES
Structure-Aware Random Fourier Kernel for Graphs
Gaussian Processes (GPs) define distributions over functions and their generalization capabilities depend heavily on the choice of kernels.
Variational Continual Bayesian Meta-Learning
Conventional meta-learning considers a set of tasks from a stationary distribution.
Molecular Contrastive Learning with Chemical Element Knowledge Graph
To address these issues, we construct a Chemical Element Knowledge Graph (KG) to summarize microscopic associations between elements and propose a novel Knowledge-enhanced Contrastive Learning (KCL) framework for molecular representation learning.
Principled Representation Learning for Entity Alignment
Embedding-based entity alignment (EEA) has recently received great attention.
Dynamic Modeling of Hand-Object Interactions via Tactile Sensing
This work takes a step on dynamics modeling in hand-object interactions from dense tactile sensing, which opens the door for future applications in activity learning, human-computer interactions, and imitation learning for robotics.
ABMDRNet: Adaptive-Weighted Bi-Directional Modality Difference Reduction Network for RGB-T Semantic Segmentation
Semantic segmentation models gain robustness against poor lighting conditions by virtue of complementary information from visible (RGB) and thermal images.
Ranked #27 on
Thermal Image Segmentation
on MFN Dataset
Depth-Aware Mirror Segmentation
To exploit depth information in mirror segmentation, we first construct a large-scale RGB-D mirror segmentation dataset, which we subsequently employ to train a novel depth-aware mirror segmentation framework.
Exploring Modality-shared Appearance Features and Modality-invariant Relation Features for Cross-modality Person Re-Identification
Most existing cross-modality person re-identification works rely on discriminative modality-shared features for reducing cross-modality variations and intra-modality variations.
Cross-Modality Person Re-identification
Person Re-Identification
Middle-level Fusion for Lightweight RGB-D Salient Object Detection
The former one first uses two sub-networks to extract unimodal features from RGB and depth images, respectively, and then fuses them for SOD.
Smart Scribbles for Image Mating
Image matting is an ill-posed problem that usually requires additional user input, such as trimaps or scribbles.
Onfocus Detection: Identifying Individual-Camera Eye Contact from Unconstrained Images
Onfocus detection aims at identifying whether the focus of the individual captured by a camera is on the camera or not.
Tracking Air Pollution in China: Near Real-Time PM2.5 Retrievals from Multiple Data Sources
Accordingly, a full-coverage high-resolution air pollutant dataset with timely updates and historical long-term records is essential to support both research and environmental management.
Single-photon imaging over 200 km
Long-range active imaging has widespread applications in remote sensing and target recognition.
Field-tunable toroidal moment in a chiral-lattice magnet
A toroidal dipole moment appears independent of the electric and magnetic dipole moment in the multipole expansion of electrodynamics.
Strongly Correlated Electrons Materials Science
Densely Nested Top-Down Flows for Salient Object Detection
To this end, this paper revisits the role of top-down modeling in salient object detection and designs a novel densely nested top-down flows (DNTDF)-based framework.
Hierarchical excitations from correlated spin tetrahedra on the breathing pyrochlore lattice
The hierarchy of the coupling strengths in a physical system often engenders an effective model at low energies where the decoupled high-energy modes are integrated out.
Strongly Correlated Electrons
Automatic Comic Generation with Stylistic Multi-page Layouts and Emotion-driven Text Balloon Generation
Finally, as opposed to using the same type of balloon as in previous works, we propose an emotion-aware balloon generation method to create different types of word balloons by analyzing the emotion of subtitles and audios.
Slot Self-Attentive Dialogue State Tracking
Then a stacked slot self-attention is applied on these features to learn the correlations among slots.
Multi-scale Information Assembly for Image Matting
Image matting is a long-standing problem in computer graphics and vision, mostly identified as the accurate estimation of the foreground in input images.
A catastrophic charge density wave in BaFe$_2$Al$_9$
Single crystal x-ray diffraction reveals super-lattice peaks in the low-temperature phase signaling the development of a CDW lattice modulation.
Strongly Correlated Electrons Materials Science
Coherent optical communications using coherence-cloned Kerr soliton microcombs
Dissipative Kerr soliton microcomb has been recognized as a promising on-chip multi-wavelength laser source for fiber optical communications, as its comb lines possess frequency and phase stability far beyond independent lasers.
Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency
We propose \textit{dynamics cycles} that align dynamic robot behavior across two domains using a cycle-consistency constraint.
Replica-Exchange Nos\'e-Hoover Dynamics for Bayesian Learning on Large Datasets
In this paper, we present a new practical method for Bayesian learning that can rapidly draw representative samples from complex posterior distributions with multiple isolated modes in the presence of mini-batch noise.
Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction
We present a novel method for testing the safety of self-driving vehicles in simulation.
SNR-Based Teachers-Student Technique for Speech Enhancement
Specifically, this method consists of multiple teacher models and a student model.
DRFN: Deep Recurrent Fusion Network for Single-Image Super-Resolution with Large Factors
Recently, single-image super-resolution has made great progress owing to the development of deep convolutional neural networks (CNNs).
Self-Attentive Hawkes Processes
The proposed method adapts self-attention to fit the intensity function of Hawkes processes.
Replica-exchange Nosé-Hoover dynamics for Bayesian learning on large datasets
In this paper, we present a new practical method for Bayesian learning that can rapidly draw representative samples from complex posterior distributions with multiple isolated modes in the presence of mini-batch noise.
Modeling Point Clouds with Self-Attention and Gumbel Subset Sampling
Thereby, we for the first time propose an end-to-end learnable and task-agnostic sampling operation, named Gumbel Subset Sampling (GSS), to select a representative subset of input points.
Learning Context Graph for Person Search
Person re-identification has achieved great progress with deep convolutional neural networks.
Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset
Second, to better cover the stochastic distribution of real rain streaks, we propose a novel SPatial Attentive Network (SPANet) to remove rain streaks in a local-to-global manner.
Ranked #3 on
Single Image Deraining
on RainCityscapes
Adversarial Attack and Defense on Point Sets
A set of novel 3D point cloud attack operations are proposed via pointwise gradient perturbation and adversarial point attachment / detachment.
GFCN: A New Graph Convolutional Network Based on Parallel Flows
In view of the huge success of convolution neural networks (CNN) for image classification and object recognition, there have been attempts to generalize the method to general graph-structured data.
Variational Self-attention Model for Sentence Representation
This paper proposes a variational self-attention model (VSAM) that employs variational inference to derive self-attention.
Improving Medical Short Text Classification with Semantic Expansion Using Word-Cluster Embedding
However, in electronic medical records (EMR), the texts containing sentences are shorter than that in general domain, which leads to the lack of semantic features and the ambiguity of semantic.
MedSim: A Novel Semantic Similarity Measure in Bio-medical Knowledge Graphs
We present MedSim, a novel semantic SIMilarity method based on public well-established bio-MEDical knowledge graphs (KGs) and large-scale corpus, to study the therapeutic substitution of antibiotics.
Parallel-tempered Stochastic Gradient Hamiltonian Monte Carlo for Approximate Multimodal Posterior Sampling
We propose a new sampler that integrates the protocol of parallel tempering with the Nos\'e-Hoover (NH) dynamics.
Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series
As an indicator of the level of risk or the degree of variation, volatility is important to analyse the financial market, and it is taken into consideration in various decision-making processes in financial activities.
Hybrid Noise Removal in Hyperspectral Imagery With a Spatial-Spectral Gradient Network
The existence of hybrid noise in hyperspectral images (HSIs) severely degrades the data quality, reduces the interpretation accuracy of HSIs, and restricts the subsequent HSIs applications.
Hyperspectral Image Denoising Employing a Spatial-Spectral Deep Residual Convolutional Neural Network
Hyperspectral image (HSI) denoising is a crucial preprocessing procedure to improve the performance of the subsequent HSI interpretation and applications.
Active Object Reconstruction Using a Guided View Planner
Inspired by the recent advance of image-based object reconstruction using deep learning, we present an active reconstruction model using a guided view planner.
Image Correction via Deep Reciprocating HDR Transformation
Given an input LDR image, we first reconstruct the missing details in the HDR domain.
A Many-Objective Evolutionary Algorithm With Two Interacting Processes: Cascade Clustering and Reference Point Incremental Learning
This paper proposes a many-objective optimization algorithm with two interacting processes: cascade clustering and reference point incremental learning (CLIA).
Missing Data Reconstruction in Remote Sensing image with a Unified Spatial-Temporal-Spectral Deep Convolutional Neural Network
Because of the internal malfunction of satellite sensors and poor atmospheric conditions such as thick cloud, the acquired remote sensing data often suffer from missing information, i. e., the data usability is greatly reduced.
Prognostication of chronic disorders of consciousness using brain functional networks and clinical characteristics
Disorders of consciousness are a heterogeneous mixture of different diseases or injuries.
Neurons and Cognition
Learning a Dilated Residual Network for SAR Image Despeckling
In this paper, to break the limit of the traditional linear models for synthetic aperture radar (SAR) image despeckling, we propose a novel deep learning approach by learning a non-linear end-to-end mapping between the noisy and clean SAR images with a dilated residual network (SAR-DRN).
Strategyproof Mechanisms for Additively Separable Hedonic Games and Fractional Hedonic Games
In this paper we consider strategyproof mechanisms for additively separable hedonic games and fractional hedonic games, that is, partitioning methods without payments such that utility maximizing agents have no incentive to lie about their true preferences.
Sparse Representation based Multi-sensor Image Fusion: A Review
As a result of several successful applications in computer vision and image processing, sparse representation (SR) has attracted significant attention in multi-sensor image fusion.
Unsupervised Video Analysis Based on a Spatiotemporal Saliency Detector
Recently, the spectrum analysis based visual saliency approach has attracted a lot of interest due to its simplicity and good performance, where the phase information of the image is used to construct the saliency map.
Predicting Multiple Attributes via Relative Multi-task Learning
Relative attributes learning aims to learn ranking functions describing the relative strength of attributes.
Relative Hidden Markov Models for Evaluating Motion Skill
The proposed algorithm effectively learns a model from the training data so that the attribute under consideration is linked to the likelihood of the inputs under the learned model.
