Search Results for author:
Found 89 papers, 46 papers with code
Convergence of Spectral Principal Paths: How Deep Networks Distill Linear Representations from Noisy Inputs
High-level representations have become a central focus in enhancing AI transparency and control, shifting attention from individual neurons or circuits to structured semantic directions that align with human-interpretable concepts.
CogniPair: From LLM Chatbots to Conscious AI Agents -- GNWT-Based Multi-Agent Digital Twins for Social Pairing -- Dating & Hiring Applications
Current large language model (LLM) agents lack authentic human psychological processes necessary for genuine digital twins and social AI applications.
Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought
As Large Language Models (LLMs) rapidly advance, we introduce Hunyuan-TurboS, a novel large hybrid Transformer-Mamba Mixture of Experts (MoE) model.
CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs
As post-training techniques evolve, large language models (LLMs) are increasingly augmented with structured multi-step reasoning abilities, often optimized through reinforcement learning.
Object-Shot Enhanced Grounding Network for Egocentric Video
To address these limitations, we propose OSGNet, an Object-Shot enhanced Grounding Network for egocentric video.
Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval
The pioneer ZS-CIR studies focus on converting the CIR task into a standard text-to-image retrieval task by pre-training a textual inversion network that can map a given image into a single pseudo-word token.
Why Representation Engineering Works: A Theoretical and Empirical Study in Vision-Language Models
However, in Vision-Language Models (VLMs), visual input can override factual linguistic knowledge, leading to hallucinated responses that contradict reality.
TIME: Temporal-sensitive Multi-dimensional Instruction Tuning and Benchmarking for Video-LLMs
Video large language models have achieved remarkable performance in tasks such as video question answering, however, their temporal understanding remains suboptimal.
Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understanding
To address these challenges, we propose learning the mapping between exocentric and egocentric domains, leveraging the extensive exocentric knowledge within existing MLLMs to enhance egocentric video understanding.
GenMol: A Drug Discovery Generalist with Discrete Diffusion
Drug discovery is a complex process that involves multiple scenarios and stages, such as fragment-constrained molecule generation, hit generation and lead optimization.
Enhanced then Progressive Fusion with View Graph for Multi-View Clustering
Specifically, we introduce two key innovations: (1) a Feature Channel Attention Encoder (FCAencoder), which adaptively enhances the most discriminative features in each view, and (2) a View Graph-based Progressive Fusion Mechanism, which constructs a view graph using optimal transport (OT) distance to progressively fuse similar views while minimizing inter-view conflicts.
Fair Diagnosis: Leveraging Causal Modeling to Mitigate Medical Bias
In medical image analysis, model predictions can be affected by sensitive attributes, such as race and gender, leading to fairness concerns and potential biases in diagnostic outcomes.
Molecule Generation with Fragment Retrieval Augmentation
Fragment-based drug discovery, in which molecular fragments are assembled into new molecules with desirable biochemical properties, has achieved great success.
BioNeMo Framework: a modular, high-performance library for AI model development in drug discovery
We introduce the BioNeMo Framework to facilitate the training of computational biology and chemistry AI models across hundreds of GPUs.
Vision-guided and Mask-enhanced Adaptive Denoising for Prompt-based Image Editing
Despite their advances, existing methods still encounter three key issues: 1) limited capacity of the text prompt in guiding target image generation, 2) insufficient mining of word-to-patch and patch-to-patch relationships for grounding editing areas, and 3) unified editing strength for all regions during each denoising step.
Ranked #3 on
Text-based Image Editing
on PIE-Bench
Enhancing Performance and Scalability of Large-Scale Recommendation Systems with Jagged Flash Attention
In production models, we observe 10% QPS improvement and 18% memory savings, enabling us to scale our recommendation systems with longer features and more complex architectures.
Pseudo-triplet Guided Few-shot Composed Image Retrieval
In the first stage, we propose an attentive masking and captioning-based pseudo triplet generation method, to construct pseudo triplets from pure image data and use them to fulfill the CIR-task specific pertaining.
ObjectNLQ @ Ego4D Episodic Memory Challenge 2024
In this report, we present our approach for the Natural Language Query track and Goal Step track of the Ego4D Episodic Memory Benchmark at CVPR 2024.
HCQA @ Ego4D EgoSchema Challenge 2024
Then in Inference-guided Answering, HCQA utilizes this hierarchical information to reason and answer given question.
Enhancing Visible-Infrared Person Re-identification with Modality- and Instance-aware Visual Prompt Learning
In this model, we have designed a series of modality-specific prompts, which could enable our model to adapt to and make use of the specific information inherent in different modality inputs, thereby reducing the interference caused by the modality gap and achieving better identification.
An Empirical Study on the Fairness of Foundation Models for Multi-Organ Image Segmentation
We test state-of-the-art foundation models for medical image segmentation, including the original SAM, medical SAM and SAT models, to evaluate segmentation efficacy across different demographic groups and identify disparities.
DualBind: A Dual-Loss Framework for Protein-Ligand Binding Affinity Prediction
Accurate prediction of protein-ligand binding affinities is crucial for drug development.
Async Learned User Embeddings for Ads Delivery Optimization
In recommendation systems, high-quality user embeddings can capture subtle preferences, enable precise similarity calculations, and adapt to changing preferences over time to maintain relevance.
Breaking Through the Noisy Correspondence: A Robust Model for Image-Text Matching
To reduce the negative impact of noisy correspondence, we propose a novel model that first transforms the noisy correspondence filtering problem into a similarity distribution modeling problem by exploiting the powerful capabilities of pre-trained models.
Cross-modal retrieval with noisy correspondence
Image-text matching
+1
Research on geometric figure classification algorithm based on Deep Learning
In recent years, with the rapid development of computer information technology, the development of artificial intelligence has been accelerating.
The state-of-the-art in Cardiac MRI Reconstruction: Results of the CMRxRecon Challenge in MICCAI 2023
To address this issue, we organized the Cardiac MRI Reconstruction Challenge (CMRxRecon) in 2023, in collaboration with the 26th International Conference on MICCAI.
On the Markov Property of Neural Algorithmic Reasoning: Analyses and Methods
To address challenges in training ForgetNet at early stages, we further introduce G-ForgetNet, which uses a gating mechanism to allow for the selective integration of historical embeddings.
Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue
We then develop a module named Joint Cross Attention-based Sentiment Inference (JCA-SI) by extending the multimodal sentiment analysis model JCA to derive the joint sentiment label for each video-audio clip.
SPT: Spectral Transformer for Red Giant Stars Age and Mass Estimation
The age and mass of red giants are essential for understanding the structure and evolution of the Milky Way.
Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog
In response to these challenges, we present an iterative search and reasoning framework, which consists of a textual encoder, a visual encoder, and a generator.
Video Timeline Modeling For News Story Understanding
In this paper, we present a novel problem, namely video timeline modeling.
TMac: Temporal Multi-Modal Graph Learning for Acoustic Event Classification
We observe that these audiovisual data naturally have temporal attributes, such as the time information for each frame in the video.
CMRxRecon: An open cardiac MRI dataset for the competition of accelerated image reconstruction
However, a limitation of CMR is its slow imaging speed, which causes patient discomfort and introduces artifacts in the images.
UniSA: Unified Generative Framework for Sentiment Analysis
Sentiment analysis is a crucial task that aims to understand people's emotional states and predict emotional categories based on multimodal information.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
Reinforcement Graph Clustering with Unknown Cluster Number
To enable the deep graph clustering algorithms to work without the guidance of the predefined cluster number, we propose a new deep graph clustering method termed Reinforcement Graph Clustering (RGC).
Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems
Advances in artificial intelligence (AI) are fueling a new paradigm of discoveries in natural sciences.
Structure Guided Multi-modal Pre-trained Transformer for Knowledge Graph Reasoning
To this end, we propose the graph Structure Guided Multimodal Pretrained Transformer for knowledge graph reasoning, termed SGMPT.
QH9: A Quantum Hamiltonian Prediction Benchmark for QM9 Molecules
Supervised machine learning approaches have been increasingly used in accelerating electronic structure prediction as surrogates of first-principle computational methods, such as density functional theory (DFT).
A Survey on Video Moment Localization
Video moment localization, also known as video moment retrieval, aiming to search a target segment within a video described by a given natural language query.
Graph Mixup with Soft Alignments
We conduct systematic experiments to show that S-Mixup can improve the performance and generalization of graph neural networks (GNNs) on various graph classification tasks.
G$^2$uardFL: Safeguarding Federated Learning Against Backdoor Attacks through Attributed Client Graph Clustering
Through empirical evaluation, comparing G$^2$uardFL with cutting-edge defenses, such as FLAME (USENIX Security 2022) [28] and DeepSight (NDSS 2022) [36], against various backdoor attacks including 3DFed (SP 2023) [20], our results demonstrate its significant effectiveness in mitigating backdoor attacks while having a negligible impact on the aggregated model's performance on benign samples (i. e., the primary task performance).
arXiv4TGC: Large-Scale Datasets for Temporal Graph Clustering
It makes evaluating models for large-scale temporal graph clustering challenging.
Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization
In this work, we propose to simultaneously incorporate label and environment causal independence (LECI) to fully make use of label and environment information, thereby addressing the challenges faced by prior methods on identifying causal and invariant subgraphs.
Message Intercommunication for Inductive Relation Reasoning
However, the uni-directional message-passing mechanism hinders such models from exploiting hidden mutual relations between entities in directed graphs.
Deep Temporal Graph Clustering
To solve the problem, we propose a general framework for deep Temporal Graph Clustering called TGC, which introduces deep clustering techniques to suit the interaction sequence-based batch-processing pattern of temporal graphs.
Ranked #1 on
Node Clustering
on arXivCS
3D Molecular Geometry Analysis with 2D Graphs
Here, we propose a novel deep learning framework to predict 3D geometries from molecular graphs.
Learnable Pillar-based Re-ranking for Image-Text Retrieval
Image-text retrieval aims to bridge the modality gap and retrieve cross-modal content based on semantic similarities.
SARF: Aliasing Relation Assisted Self-Supervised Learning for Few-shot Relation Reasoning
Moreover, most of the existing methods ignore leveraging the beneficial information from aliasing relations (AR), i. e., data-rich relations with similar contextual semantics to the target data-poor relation.
Cross-modal Audio-visual Co-learning for Text-independent Speaker Verification
Visual speech (i. e., lip motion) is highly related to auditory speech due to the co-occurrence and synchronization in speech production.
Self-Supervised Temporal Graph learning with Temporal and Structural Intensity Alignment
To address this issue, we propose a self-supervised method called S2T for temporal graph learning, which extracts both temporal and structural information to learn more informative node representations.
A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal
According to the graph types, existing KGR models can be roughly divided into three categories, i. e., static models, temporal models, and multi-modal models.
COVID-19 Activity Risk Calculator as a Gamified Public Health Intervention Tool
In order to create a simple easy-to-use tool for estimating different individual risks associated with carrying out daily-life activity, we developed COVID-19 Activity Risk Calculator (CovARC).
Coordinating Cross-modal Distillation for Molecular Property Prediction
The existing graph methods have demonstrated that 3D geometric information is significant for better performance in MPP.
DiffBP: Generative Diffusion of 3D Molecules for Target Protein Binding
Previous works usually generate atoms in an auto-regressive way, where element types and 3D coordinates of atoms are generated one by one.
I4U System Description for NIST SRE'20 CTS Challenge
This manuscript describes the I4U submission to the 2020 NIST Speaker Recognition Evaluation (SRE'20) Conversational Telephone Speech (CTS) Challenge.
FedVMR: A New Federated Learning method for Video Moment Retrieval
As the first that is explored in VMR field, the new task is defined as video moment retrieval with distributed data.
Deep Spectro-temporal Artifacts for Detecting Synthesized Speech
The Audio Deep Synthesis Detection (ADD) Challenge has been held to detect generated human-like speech.
Gradient-Guided Importance Sampling for Learning Binary Energy-Based Models
the discrete data space to approximately construct the provably optimal proposal distribution, which is subsequently used by importance sampling to efficiently estimate the original ratio matching objective.
Spoofing-Aware Attention based ASV Back-end with Multiple Enrollment Utterances and a Sampling Strategy for the SASV Challenge 2022
Current state-of-the-art automatic speaker verification (ASV) systems are vulnerable to presentation attacks, and several countermeasures (CMs), which distinguish bona fide trials from spoofing ones, have been explored to protect ASV.
Topological structure of complex predictions
Complex prediction models such as deep learning are the output from fitting machine learning, neural networks, or AI models to a set of training data.
GraphFM: Improving Large-Scale GNN Training via Feature Momentum
GraphFM-IB applies FM to in-batch sampled data, while GraphFM-OB applies FM to out-of-batch data that are 1-hop neighborhood of in-batch data.
Empowering GNNs via Edge-Aware Weisfeiler-Leman Algorithm
Message passing graph neural networks (GNNs) are known to have their expressiveness upper-bounded by 1-dimensional Weisfeiler-Leman (1-WL) algorithm.
Generating 3D Molecules for Target Protein Binding
Second, to preserve the desirable equivariance property, we select a local reference atom according to the designed auxiliary classifiers and then construct a local spherical coordinate system.
Neighbor2Seq: Deep Learning on Massive Graphs by Transforming Neighbors to Sequences
Therefore, our Neighbor2Seq naturally endows GNNs with the efficiency and advantages of deep learning operations on grid-like data by precomputing the Neighbor2Seq transformations.
Inductive Representation Learning in Temporal Networks via Mining Neighborhood and Community Influences
Therefore, we propose a new inductive network representation learning method called MNCI by mining neighborhood and community influences in temporal networks.
Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphs
Here, we propose to predict the ground-state 3D geometries from molecular graphs using machine learning methods.
Ranked #1 on
3D Geometry Prediction
on Molecule3D val
$m$-mix: Generating hard negatives via multiple samples mixing for contrastive learning
Negative pairs are essential in contrastive learning, which plays the role of avoiding degenerate solutions.
Gradient-Guided Importance Sampling for Learning Discrete Energy-Based Models
In this study, we propose ratio matching with gradient-guided importance sampling (RMwGGIS) to alleviate the above limitations.
GraphEBM: Towards Permutation Invariant and Multi-Objective Molecular Graph Generation
In this work, we propose GraphEBM, a molecular graph generation method via energy-based models (EBMs), as an exploratory work to perform permutation invariant and multi-objective molecule generation.
A Novel Patch Convolutional Neural Network for View-based 3D Model Retrieval
Thus, we tackle this problem from the perspective of exploiting the relationships between patch features to capture long-range associations among multi-view images.
Multigranular Visual-Semantic Embedding for Cloth-Changing Person Re-identification
To solve these issues, in this work, a novel multigranular visual-semantic embedding algorithm (MVSE) is proposed for cloth-changing person ReID, where visual semantic information and human attributes are embedded into the network, and the generalized features of human appearance can be well learned to effectively solve the problem of clothing changes.
Dynamic Modality Interaction Modeling for Image-Text Retrieval
To address these issues, we develop a novel modality interaction modeling network based upon the routing mechanism, which is the first unified and dynamic multimodal interaction framework towards image-text retrieval.
Multi-Modal Relational Graph for Cross-Modal Video Moment Retrieval
Toward this end, we contribute a multi-modal relational graph to capture the interactions among objects from the visual and textual content to identify the differences among similar video moment candidates.
Fast Quantum Property Prediction via Deeper 2D and 3D Graph Networks
We employ our methods to participate in the 2021 KDD Cup on OGB Large-Scale Challenge (OGB-LSC), which aims to predict the HOMO-LUMO energy gap of molecules.
Exploring Deep Learning for Joint Audio-Visual Lip Biometrics
Audio-visual (AV) lip biometrics is a promising authentication technique that leverages the benefits of both the audio and visual modalities in speech communication.
DIG: A Turnkey Library for Diving into Graph Deep Learning Research
Although there exist several libraries for deep learning on graphs, they are aiming at implementing basic operations for graph deep learning.
Spherical Message Passing for 3D Graph Networks
Based on such observations, we propose the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning.
Ranked #3 on
Drug Discovery
on QM9
GraphEBM: Molecular Graph Generation with Energy-Based Models
We note that most existing approaches for molecular graph generation fail to guarantee the intrinsic property of permutation invariance, resulting in unexpected bias in generative models.
Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery
Here we develop a suite of comprehensive machine learning methods and tools spanning different computational models, molecular representations, and loss functions for molecular property prediction and drug discovery.
The Diversified Ensemble Neural Network
Ensemble is a general way of improving the accuracy and stability of learning models, especially for the generalization ability on small datasets.
Frame-wise Cross-modal Matching for Video Moment Retrieval
Another contribution is that we propose an additional predictor to utilize the internal frames in the model training to improve the localization accuracy.
Towards Deeper Graph Neural Networks
Based on our theoretical and empirical analysis, we propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
Ranked #4 on
Node Classification
on AMZ Computers
Strongly local p-norm-cut algorithms for semi-supervised learning and local graph clustering
For this problem, we propose a novel generalization of random walk, diffusion, or smooth function methods in the literature to a convex p-norm cut function.
Non-Local Graph Neural Networks
Modern graph neural networks (GNNs) learn node embeddings through multilayer local aggregation and achieve great success in applications on assortative graphs.
NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results
This challenge is based on a newly collected validation and testing image datasets, and hence, named SIDD+.
KGAT: Knowledge Graph Attention Network for Recommendation
To provide more accurate, diverse, and explainable recommendation, it is compulsory to go beyond modeling user-item interactions and take side information into account.
Ranked #2 on
Link Prediction
on Yelp
Cost-Sensitive Feature Selection by Optimizing F-Measures
Feature selection is beneficial for improving the performance of general machine learning tasks by extracting an informative subset from the high-dimensional features.
Assessment of central serous chorioretinopathy (CSC) depicted on color fundus photographs using deep Learning
To investigate whether and to what extent central serous chorioretinopathy (CSC) depicted on color fundus photographs can be assessed using deep learning technology.
Sparsely Grouped Multi-task Generative Adversarial Networks for Facial Attribute Manipulation
To overcome such a key limitation, we propose Sparsely Grouped Generative Adversarial Networks (SG-GAN) as a novel approach that can translate images on sparsely grouped datasets where only a few samples for training are labelled.
