Search Results for author:
Found 227 papers, 143 papers with code
UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer
Furthermore, we adopt a simple concatenation operation to integrate the reference appearance into the model and incorporate the pose information of the reference image for enhanced pose alignment.
DMPT: Decoupled Modality-aware Prompt Tuning for Multi-modal Object Re-identification
Current multi-modal object re-identification approaches based on large-scale pre-trained backbones (i. e., ViT) have displayed remarkable progress and achieved excellent performance.
Taming Consistency Distillation for Accelerated Human Image Animation
Recent advancements in human image animation have been propelled by video diffusion models, yet their reliance on numerous iterative denoising steps results in high inference costs and slow speeds.
SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models
The rapid advancement of multi-modal large reasoning models (MLRMs) -- enhanced versions of multimodal language models (MLLMs) equipped with reasoning capabilities -- has revolutionized diverse applications.
Exploring the Evolution of Physics Cognition in Video Generation: A Survey
Recent advancements in video generation have witnessed significant progress, especially with the rapid advancement of diffusion models.
Towards Unified Latent Space for 3D Molecular Latent Diffusion Modeling
In this work, we propose \textbf{U}nified Variational \textbf{A}uto-\textbf{E}ncoder for \textbf{3D} Molecular Latent Diffusion Modeling (\textbf{UAE-3D}), a multi-modal VAE that compresses 3D molecules into latent sequences from a unified latent space, while maintaining near-zero reconstruction error.
Multimodal Language Modeling for High-Accuracy Single Cell Transcriptomics Analysis and Generation
Pre-trained language models (PLMs) have revolutionized scientific research, yet their application to single-cell analysis remains limited.
Route Sparse Autoencoder to Interpret Large Language Models
In this paper, we introduce Route Sparse Autoencoder (RouteSAE), a new framework that integrates a routing mechanism with a shared SAE to efficiently extract features from multiple layers.
DreamRelation: Relation-Centric Video Customization
First, in Relational Decoupling Learning, we disentangle relations from subject appearances using relation LoRA triplet and hybrid mask training strategy, ensuring better generalization across diverse relationships.
RePO: ReLU-based Preference Optimization
Aligning large language models (LLMs) with human preferences is critical for real-world deployment, yet existing methods like RLHF face computational and stability challenges.
START: Self-taught Reasoner with Tools
In this paper, we introduce START (Self-Taught Reasoner with Tools), a novel tool-integrated long CoT reasoning LLM that significantly enhances reasoning capabilities by leveraging external tools.
NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation
To combine these advantages for 3D molecule generation, we propose a foundation model -- NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation.
Reinforced Lifelong Editing for Language Models
Large language models (LLMs) acquire information from pre-training corpora, but their stored knowledge can become inaccurate or outdated over time.
AnyEdit: Edit Any Knowledge Encoded in Language Models
Additionally, AnyEdit serves as a plug-and-play framework, enabling current editing methods to update knowledge with arbitrary length and format, significantly advancing the scope and practicality of LLM knowledge editing.
Multi-agent Architecture Search via Agentic Supernet
Large Language Model (LLM)-empowered multi-agent systems extend the cognitive boundaries of individual agents through disciplined collaboration and interaction, while constructing these systems often requires labor-intensive manual designs.
DAMO: Data- and Model-aware Alignment of Multi-modal LLMs
Direct Preference Optimization (DPO) has shown effectiveness in aligning multi-modal large language models (MLLM) with human preferences.
Addressing Delayed Feedback in Conversion Rate Prediction via Influence Functions
In the realm of online digital advertising, conversion rate (CVR) prediction plays a pivotal role in maximizing revenue under cost-per-conversion (CPA) models, where advertisers are charged only when users complete specific actions, such as making a purchase.
PatchRec: Multi-Grained Patching for Efficient LLM-based Sequential Recommendation
The framework consists of two stages: (1) Patch Pre-training, which familiarizes LLMs with item-level compression patterns, and (2) Patch Fine-tuning, which teaches LLMs to model sequences at multiple granularities.
A Unified Invariant Learning Framework for Graph Classification
It provides a unified perspective on invariant graph learning, emphasizing both structural and semantic invariance principles to identify more robust stable features.
CookingDiffusion: Cooking Procedural Image Generation with Stable Diffusion
These prompts encompass text prompts (representing cooking steps), image prompts (corresponding to cooking images), and multi-modal prompts (mixing cooking steps and images), ensuring the consistent generation of cooking procedural images.
DVM: Towards Controllable LLM Agents in Social Deduction Games
This control not only ensures that NPCs can adapt to varying difficulty levels during gameplay, but also provides insights into the safety and fairness of LLM agents.
Position-aware Graph Transformer for Recommendation
Collaborative recommendation fundamentally involves learning high-quality user and item representations from interaction data.
FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusion
Visual diffusion models achieve remarkable progress, yet they are typically trained at limited resolutions due to the lack of high-resolution data and constrained computation resources, hampering their ability to generate high-fidelity images or videos at higher resolutions.
Intelligent System for Automated Molecular Patent Infringement Assessment
Automated drug discovery offers significant potential for accelerating the development of novel therapeutics by substituting labor-intensive human workflows with machine-driven processes.
Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matters
Recent success of text-to-image (T2I) generation and its increasing practical applications, enabled by diffusion models, require urgent consideration of erasing unwanted concepts, e. g., copyrighted, offensive, and unsafe ones, from the pre-trained models in a precise, timely, and low-cost manner.
Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity
For efficient anomaly detection in long videos, we propose the Anomaly-focused Temporal Sampler (ATS).
Unified Parameter-Efficient Unlearning for LLMs
The advent of Large Language Models (LLMs) has revolutionized natural language processing, enabling advanced understanding and reasoning capabilities across a variety of tasks.
PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation
The key challenge lies in maintaining high ID fidelity consistently while preserving the original motion dynamic and semantic following after the identity injection.
Feature-interactive Siamese graph encoder-based image analysis to predict STAS from histopathology images in lung cancer
Spread through air spaces (STAS) is a distinct invasion pattern in lung cancer, crucial for prognosis assessment and guiding surgical decisions.
DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Control
In this paper, we present DreamVideo-2, a zero-shot video customization framework capable of generating videos with a specific subject and motion trajectory, guided by a single image and a bounding box sequence, respectively, and without the need for test-time fine-tuning.
Generate and Instantiate What You Prefer: Text-Guided Diffusion for Sequential Recommendation
Recent advancements in generative recommendation systems, particularly in the realm of sequential recommendation tasks, have shown promise in enhancing generalization to new items.
RosePO: Aligning LLM-based Recommenders with Human Values
Recently, there has been a growing interest in leveraging Large Language Models (LLMs) for recommendation systems, which usually adapt a pre-trained LLM to the recommendation scenario through supervised fine-tuning (SFT).
Animate-X: Universal Character Image Animation with Enhanced Motion Representation
Our in-depth analysis suggests to attribute this limitation to their insufficient modeling of motion, which is unable to comprehend the movement pattern of the driving video, thus imposing a pose sequence rigidly onto the target character.
$α$-DPO: Adaptive Reward Margin is What Direct Preference Optimization Needs
Aligning large language models (LLMs) with human values and intentions is crucial for their utility, honesty, and safety.
DiffGAD: A Diffusion-based Unsupervised Graph Anomaly Detector
Graph Anomaly Detection (GAD) is crucial for identifying abnormal entities within networks, garnering significant attention across various fields.
EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models
Our experiments with extensive data indicate that the model trained on generated data of the advanced model can approximate its generation capability.
Neuron-Level Sequential Editing for Large Language Models
This work explores sequential model editing in large language models (LLMs), a critical task that involves modifying internal knowledge within LLMs continuously through multi-round editing, each incorporating updates or corrections to adjust the model outputs without the need for costly retraining.
Text-guided Diffusion Model for 3D Molecule Generation
The de novo generation of molecules with targeted properties is crucial in biology, chemistry, and drug discovery.
AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
To address this, we introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters.
Replace Anyone in Videos
Recent advancements in controllable human-centric video generation, particularly with the rise of diffusion models, have demonstrated considerable progress.
SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding
To develop an LLM specialized in scientific literature understanding, we propose a hybrid strategy that integrates continual pre-training (CPT) and supervised fine-tuning (SFT), to simultaneously infuse scientific domain knowledge and enhance instruction-following capabilities for domain-specific tasks. cIn this process, we identify two key challenges: (1) constructing high-quality CPT corpora, and (2) generating diverse SFT instructions.
Customizing Language Models with Instance-wise LoRA for Sequential Recommendation
Sequential recommendation systems predict the next interaction item based on users' past interactions, aligning recommendations with individual preferences.
Invariant Graph Learning Meets Information Bottleneck for Out-of-Distribution Generalization
In this work, we propose a novel framework, called Invariant Graph Learning based on Information bottleneck theory (InfoIGL), to extract the invariant features of graphs and enhance models' generalization ability to unseen distributions.
Adaptive Self-supervised Robust Clustering for Unstructured Data with Unknown Cluster Number
ASRC obtains the final clustering results by applying RCC to the learned feature representations with their consistent graph structure and edge weights.
Reinforced Prompt Personalization for Recommendation with Large Language Models
In this work, we introduce the concept of instance-wise prompting, aiming at personalizing discrete prompts for individual users.
Rethinking Visual Content Refinement in Low-Shot CLIP Adaptation
However, these adaptation methods are usually operated on the global view of an input image, and thus biased perception of partial local details of the image.
BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillation
Spiking neural networks (SNNs), which mimic biological neural system to convey information via discrete spikes, are well known as brain-inspired models with excellent computing efficiency.
$β$-DPO: Direct Preference Optimization with Dynamic $β$
Direct Preference Optimization (DPO) has emerged as a compelling approach for training Large Language Models (LLMs) to adhere to human preferences.
Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
We categorize noise into pointwise noise, which includes low-quality data points, and pairwise noise, which encompasses erroneous data pair associations that affect preference rankings.
Disentangling Masked Autoencoders for Unsupervised Domain Generalization
Domain Generalization (DG), designed to enhance out-of-distribution (OOD) generalization, is all about learning invariance against domain shifts utilizing sufficient supervision signals.
Language Representations Can be What Recommenders Need: Findings and Potentials
Contrary to prevailing understanding that LMs and traditional recommenders learn two distinct representation spaces due to the huge gap in language and behavior modeling objectives, this work re-examines such understanding and explores extracting a recommendation space directly from the language representation space.
DotaMath: Decomposition of Thought with Code Assistance and Self-correction for Mathematical Reasoning
In this paper, we introduce a series of LLMs that employs the Decomposition of thought with code assistance and self-correction for mathematical reasoning, dubbed as DotaMath.
Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM
We train a lightweight temporal sampler to select frames with high anomaly response and fine-tune a multimodal large language model (LLM) to generate explanatory content.
On Softmax Direct Preference Optimization for Recommendation
Specifically, we incorporate multiple negatives in user preference data and devise an alternative version of DPO loss tailored for LM-based recommenders, which is extended from the traditional full-ranking Plackett-Luce (PL) model to partial rankings and connected to softmax sampling strategies.
Hello Again! LLM-powered Personalized Agent for Long-term Dialogue
Crucial to addressing this real-world need are event summary and persona management, which enable reasoning for appropriate long-term dialogue responses.
SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN
Spiking neural network (SNN) has attracted great attention due to its characteristic of high efficiency and accuracy.
UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation
First, to reduce the optimization difficulty and ensure temporal coherence, we map the reference image along with the posture guidance and noise video into a common feature space by incorporating a unified video diffusion model.
Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning
Through the integration of LLMs, ToolRec enables conventional recommender systems to become external tools with a natural language interface.
ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretraining
To resolve the challenges above, we propose a new pretraining method, ReactXT, for reaction-text modeling, and a new dataset, OpenExp, for experimental procedure prediction.
ProtT3: Protein-to-Text Generation for Text-based Protein Understanding
ProtT3 empowers an LM to understand protein sequences of amino acids by incorporating a PLM as its protein understanding module, enabling effective protein-to-text generation.
AttacKG+:Boosting Attack Knowledge Graph Construction with Large Language Models
Thus, we propose a fully automatic LLM-based framework to construct attack knowledge graphs named: AttacKG+.
BotDGT: Dynamicity-aware Social Bot Detection with Dynamic Graph Transformers
To tackle these challenges, we propose BotDGT, a novel framework that not only considers the topological structure, but also effectively incorporates dynamic nature of social network.
CIRP: Cross-Item Relational Pre-training for Multimodal Product Bundling
However, the cross-item relations have been under-explored in the current multimodal pre-train models.
Enhance Image Classification via Inter-Class Image Mixup with Diffusion Model
Text-to-image (T2I) generative models have recently emerged as a powerful tool, enabling the creation of photo-realistic images and giving rise to a multitude of applications.
SELECTOR: Heterogeneous graph network with convolutional masked autoencoder for multimodal robust prediction of cancer survival
To mitigate the impact of missing features within the modality on prediction accuracy, we devised a convolutional masked autoencoder (CMAE) to process the heterogeneous graph post-feature reconstruction.
PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency
Then, in the first stage, question-SQL pairs are retrieved as few-shot demonstrations, prompting the LLM to generate a preliminary SQL (PreSQL).
Ranked #2 on
Text-To-SQL
on spider
GlanceVAD: Exploring Glance Supervision for Label-efficient Video Anomaly Detection
In recent years, video anomaly detection has been extensively investigated in both unsupervised and weakly supervised settings to alleviate costly temporal labeling.
Conformal Shield: A Novel Adversarial Attack Detection Framework for Automatic Modulation Classification
Deep learning algorithms have become an essential component in the field of cognitive radio, especially playing a pivotal role in automatic modulation classification.
General Debiasing for Graph-based Collaborative Filtering via Adversarial Graph Dropout
However, we have discovered that this aggregation mechanism comes with a drawback, which amplifies biases present in the interaction graph.
MolTC: Towards Molecular Relational Modeling In Language Models
Molecular Relational Learning (MRL), aiming to understand interactions between molecular pairs, plays a pivotal role in advancing biochemical research.
EXGC: Bridging Efficiency and Explainability in Graph Condensation
Graph representation learning on vast datasets, like web data, has made significant strides.
DiffsFormer: A Diffusion Transformer on Stock Factor Augmentation
Our research demonstrates the efficacy of leveraging AIGS and the DiffsFormer architecture to mitigate data scarcity in stock forecasting tasks.
Towards 3D Molecule-Text Interpretation in Language Models
Through 3D molecule-text alignment and 3D molecule-centric instruction tuning, 3D-MoLM establishes an integration of 3D molecular encoder and LM.
Alleviating Structural Distribution Shift in Graph Anomaly Detection
Graph anomaly detection (GAD) is a challenging binary classification problem due to its different structural distribution between anomalies and normal nodes -- abnormal nodes are a minority, therefore holding high heterophily and low homophily compared to normal nodes.
Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security
Next, we discuss several key challenges to achieve intelligent, efficient and secure Personal LLM Agents, followed by a comprehensive survey of representative solutions to address these challenges.
LASO: Language-guided Affordance Segmentation on 3D Object
With this regard we propose a novel task Language-guided Affordance Segmentation on 3D Object (LASO) which challenges a model to segment a 3D object's part relevant to a given affordance question.
A Recipe for Scaling up Text-to-Video Generation with Text-free Videos
Following such a pipeline, we study the effect of doubling the scale of training set (i. e., video-only WebVid10M) with some randomly collected text-free videos and are encouraged to observe the performance improvement (FID from 9. 67 to 8. 19 and FVD from 484 to 441), demonstrating the scalability of our approach.
Ranked #7 on
Text-to-Video Generation
on MSR-VTT
BSL: Understanding and Improving Softmax Loss for Recommendation
Loss functions steer the optimization direction of recommendation models and are critical to model performance, but have received relatively little attention in recent recommendation research.
InstructVideo: Instructing Video Diffusion Models with Human Feedback
To tackle this problem, we propose InstructVideo to instruct text-to-video diffusion models with human feedback by reward fine-tuning.
DreamTalk: When Emotional Talking Head Generation Meets Diffusion Probabilistic Models
To more conveniently specify personalized emotions, a diffusion-based style predictor is utilized to predict the personalized emotion directly from the audio, eliminating the need for extra emotion reference.
VideoLCM: Video Latent Consistency Model
Consistency models have demonstrated powerful capability in efficient image generation and allowed synthesis within a few sampling steps, alleviating the high computational cost in diffusion models.
Hierarchical Spatio-temporal Decoupling for Text-to-Video Generation
At the structure level, we decompose the T2V task into two steps, including spatial reasoning and temporal reasoning, using a unified denoiser.
Ranked #6 on
Text-to-Video Generation
on MSR-VTT
LLaRA: Large Language-Recommendation Assistant
Treating the "sequential behaviors of users" as a distinct modality beyond texts, we employ a projector to align the traditional recommender's ID embeddings with the LLM's input space.
SCTc-TE: A Comprehensive Formulation and Benchmark for Temporal Event Forecasting
Temporal complex event forecasting aims to predict the future events given the observed events from history.
MultiCBR: Multi-view Contrastive Learning for Bundle Recommendation
It does, however, have two limitations: 1) the two-view formulation does not fully exploit all the heterogeneous relations among users, bundles and items; and 2) the "early contrast and late fusion" framework is less effective in capturing user preference and difficult to generalize to multiple views.
I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models
By this means, I2VGen-XL can simultaneously enhance the semantic accuracy, continuity of details and clarity of generated videos.
On the Opportunities of Green Computing: A Survey
To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic.
Large Language Model Can Interpret Latent Space of Sequential Recommender
Sequential recommendation is to predict the next item of interest for a user, based on her/his interaction history with previous items.
Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion
Scrutinizing previous studies, we can summarize a common learning-to-classify paradigm -- given a positive item, a recommender model performs negative sampling to add negative items and learns to classify whether the user prefers them or not, based on his/her historical interaction sequence.
Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss
To bridge the gap, we delve into the reasons underpinning the success of contrastive loss in CF, and propose a principled Adversarial InfoNCE loss (AdvInfoNCE), which is a variant of InfoNCE, specially tailored for CF methods.
Leveraging Multimodal Features and Item-level User Feedback for Bundle Construction
Specifically, we use self-attention modules to combine the multimodal and multi-item features, and then leverage both item- and bundle-level contrastive learning to enhance the representation learning, thus to counter the modality missing, noise, and sparsity problems.
Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendation
Therefore, the data sparsity issue of reward signals and state transitions is very severe, while it has long been overlooked by existing RL recommenders. Worse still, RL methods learn through the trial-and-error mode, but negative feedback cannot be obtained in implicit feedback recommendation tasks, which aggravates the overestimation problem of offline RL recommender.
Rethinking Tokenizer and Decoder in Masked Graph Modeling for Molecules
Our results show that a subgraph-level tokenizer and a sufficiently expressive decoder with remask decoding have a large impact on the encoder's representation learning.
ReLM: Leveraging Language Models for Enhanced Chemical Reaction Prediction
Predicting chemical reactions, a fundamental challenge in chemistry, involves forecasting the resulting products from a given reaction process.
MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
MolCA enables an LM (e. g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector.
Ranked #7 on
Molecule Captioning
on ChEBI-20
A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction
Large language models (LLMs) have demonstrated great potential for domain-specific applications, such as the law domain.
Few-shot Action Recognition with Captioning Foundation Models
In this paper, we develop an effective plug-and-play framework called CapFSAR to exploit the knowledge of multimodal models without manually annotating text.
On Generative Agents in Recommendation
Recommender systems are the cornerstone of today's information dissemination, yet a disconnect between offline metrics and online performance greatly hinders their development.
Robust Collaborative Filtering to Popularity Distribution Shift
The popularity shortcut tricks are good for in-distribution (ID) performance but poorly generalized to out-of-distribution (OOD) data, i. e., when popularity distribution of test data shifts w. r. t.
MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning
In this paper, we conduct an investigation for such data augmentation in math reasoning and are intended to answer: (1) What strategies of data augmentation are more effective; (2) What is the scaling relationship between the amount of augmented data and model performance; and (3) Can data augmentation incentivize generalization to out-of-domain mathematical reasoning tasks?
Ranked #60 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Text-to-Image Generation for Abstract Concepts
Recent years have witnessed the substantial progress of large-scale models across various domains, such as natural language processing and computer vision, facilitating the expression of concrete concepts.
HR-Pro: Point-supervised Temporal Action Localization via Hierarchical Reliability Propagation
For snippet-level learning, we introduce an online-updated memory to store reliable snippet prototypes for each class.
Ranked #1 on
Weakly Supervised Action Localization
on BEOID
RLIPv2: Fast Scaling of Relational Language-Image Pre-training
In this paper, we propose RLIPv2, a fast converging model that enables the scaling of relational pre-training to large-scale pseudo-labelled scene graph data.
Ranked #1 on
Zero-Shot Human-Object Interaction Detection
on HICO-DET
(using extra training data)
Context-aware Event Forecasting via Graph Disentanglement
The task of event forecasting aims to model the relational and temporal patterns based on historical events and makes forecasting to what will happen in the future.
ModelScope Text-to-Video Technical Report
This paper introduces ModelScopeT2V, a text-to-video synthesis model that evolves from a text-to-image synthesis model (i. e., Stable Diffusion).
Ranked #9 on
Text-to-Video Generation
on MSR-VTT
Online Distillation-enhanced Multi-modal Transformer for Sequential Recommendation
To be specific, we first introduce an ID-aware Multi-modal Transformer module in the item representation learning stage to facilitate information interaction among different features.
Redundancy-aware Transformer for Video Question Answering
This paper identifies two kinds of redundancy in the current VideoQA paradigm.
Adaptive Semantic Consistency for Cross-domain Few-shot Classification
In this way, the proposed ASC enables explicit transfer of source domain knowledge to prevent the model from overfitting the target domain.
Discovering Spatio-Temporal Rationales for Video Question Answering
We then conduct extensive studies to verify the importance of STR as well as the proposed answer interaction mechanism.
LightGT: A Light Graph Transformer for Multimedia Recommendation
Considering its challenges in effectiveness and efficiency, we propose a novel Transformer-based recommendation model, termed as Light Graph Transformer model (LightGT).
Ranked #1 on
Multi-Media Recommendation
on Kwai
(Recall@10 metric)
DEDUCE: Multi-head attention decoupled contrastive learning to discover cancer subtypes based on multi-omics data
Methods: We propose a model, named DEDUCE, based on a symmetric multi-head attention encoders (SMAE), for unsupervised contrastive learning to analyze multi-omics cancer data, with the aim of identifying and characterizing cancer subtypes.
Discovering Dynamic Causal Space for DAG Structure Learning
Discovering causal structure from purely observational data (i. e., causal discovery), aiming to identify causal relationships among variables, is a fundamental task in machine learning.
VideoComposer: Compositional Video Synthesis with Motion Controllability
The pursuit of controllability as a higher standard of visual content creation has yielded remarkable progress in customizable image synthesis.
Ranked #5 on
Text-to-Video Generation
on EvalCrafter Text-to-Video (ECTV) Dataset
(using extra training data)
GIF: A General Graph Unlearning Strategy via Influence Function
Then, we recognize the crux to the inability of traditional influence function for graph unlearning, and devise Graph Influence Function (GIF), a model-agnostic unlearning method that can efficiently and accurately estimate parameter changes in response to a $\epsilon$-mass perturbation in deleted data.
MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition
To address these issues, we develop a Motion-augmented Long-short Contrastive Learning (MoLo) method that contains two crucial components, including a long-short contrastive objective and a motion autodecoder.
Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition
In this paper, we begin with analyzing the feature representation behavior in the open-set action recognition (OSAR) problem based on the information bottleneck (IB) theory, and propose to enlarge the instance-specific (IS) and class-specific (CS) information contained in the feature for better performance.
Multi-task Adversarial Learning for Semi-supervised Trajectory-User Linking
We first project users and trajectories into the common latent feature space through learning a projection function (generator) to minimize the distance between the user distribution and the trajectory distribution.
CLIP-guided Prototype Modulating for Few-shot Action Recognition
Learning from large-scale contrastive language-image pre-training like CLIP has shown remarkable success in a wide range of downstream tasks recently, but it is still under-explored on the challenging few-shot action recognition (FSAR) task.
Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting
Despite great success in low-dimensional linear systems, it has been observed that these approaches overly exploit easier-to-fit samples, thus inevitably learning spurious edges.
Invariant Collaborative Filtering to Popularity Distribution Shift
Collaborative Filtering (CF) models, despite their great success, suffer from severe performance drops due to popularity distribution shifts, where these changes are ubiquitous and inevitable in real-world scenarios.
HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition
To be specific, HyRSM++ consists of two key components, a hybrid relation module and a temporal set matching metric.
Space-time Prompting for Video Class-incremental Learning
In this paper, we will fill this gap by learning multiple prompts based on a powerful image-language pre-trained model, i. e., CLIP, making it fit for video class-incremental learning (VCIL).
Causal Inference for Knowledge Graph based Recommendation
Knowledge Graph (KG), as a side-information, tends to be utilized to supplement the collaborative filtering (CF) based recommendation model.
MIAD: A Maintenance Inspection Dataset for Unsupervised Anomaly Detection
However, existing datasets for unsupervised anomaly detection are biased towards manufacturing inspection, not considering maintenance inspection which is usually conducted under outdoor uncontrolled environment such as varying camera viewpoints, messy background and degradation of object surface after long-term working.
Real-World Image Super Resolution via Unsupervised Bi-directional Cycle Domain Transfer Learning based Generative Adversarial Network
First, the UBCDTN is able to produce an approximated real-like LR image through transferring the LR image from an artificially degraded domain to the real-world LR image domain.
Semantic Encoder Guided Generative Adversarial Face Ultra-Resolution Network
In this paper, we propose a novel face super-resolution method, namely Semantic Encoder guided Generative Adversarial Face Ultra-Resolution Network (SEGA-FURN) to ultra-resolve an unaligned tiny LR face image to its HR counterpart with multiple ultra-upscaling factors (e. g., 4x and 8x).
Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift
From the perspective of invariant learning and stable learning, a recently well-established paradigm for out-of-distribution generalization, stable features of the graph are assumed to causally determine labels, while environmental features tend to be unstable and can lead to the two primary types of distribution shifts.
Learning a Condensed Frame for Memory-Efficient Video Class-Incremental Learning
The former is to reduce the memory cost by preserving only one condensed frame instead of the whole video, while the latter aims to compensate the lost spatio-temporal details in the Frame Condensing stage.
Provably Learning Diverse Features in Multi-View Data with Midpoint Mixup
Mixup is a data augmentation technique that relies on training using random convex combinations of data points and their labels.
Slippage-robust Gaze Tracking for Near-eye Display
In recent years, head-mounted near-eye display devices have become the key hardware foundation for virtual reality and augmented reality.
Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filtering
Collaborative filtering (CF) models easily suffer from popularity bias, which makes recommendation deviate from users' actual preferences.
Understanding Edge-of-Stability Training Dynamics with a Minimalist Example
Globally we observe that the training dynamics for our example has an interesting bifurcating behavior, which was also observed in the training of neural nets.
Vision-Based Defect Classification and Weight Estimation of Rice Kernels
To compensate for the imbalance of different kernel numbers and classify kernels with multiple flaws accurately, we propose a multi-stage workflow which is able to locate the kernels in the captured image and classify their properties.
Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep Networks
Monotonic linear interpolation (MLI) - on the line connecting a random initialization with the minimizer it converges to, the loss and accuracy are monotonic - is a phenomenon that is commonly observed in the training of neural networks.
Equivariant and Invariant Grounding for Video Question Answering
Specifically, the equivariant grounding encourages the answering to be sensitive to the semantic changes in the causal scene and question; in contrast, the invariant grounding enforces the answering to be insensitive to the changes in the environment scene.
MAR: Masked Autoencoders for Efficient Action Recognition
Inspired by this, we propose propose Masked Action Recognition (MAR), which reduces the redundant computation by discarding a proportion of patches and operating only on a part of the videos.
Ranked #11 on
Action Recognition
on Something-Something V2
MLGOPerf: An ML Guided Inliner to Optimize Performance
This paper presents MLGOPerf; the first end-to-end framework capable of optimizing performance using LLVM's ML-Inliner.
Self-Supervised Learning for Multimedia Recommendation
To capture multi-modal patterns in the data itself, we go beyond the supervised learning paradigm, and incorporate the idea of self-supervised learning (SSL) into multimedia recommendation.
Ranked #3 on
Multi-modal Recommendation
on Amazon Sports
Context-aware Proposal Network for Temporal Action Detection
This technical report presents our first place winning solution for temporal action detection task in CVPR-2022 AcitivityNet Challenge.
Let Invariant Rationale Discovery Inspire Graph Contrastive Learning
Specifically, without supervision signals, RGCL uses a rationale generator to reveal salient features about graph instance-discrimination as the rationale, and then creates rationale-aware views for contrastive learning.
Invariant Grounding for Video Question Answering
At its core is understanding the alignments between visual scenes in video and linguistic semantics in question to yield the answer.
CrossCBR: Cross-view Contrastive Learning for Bundle Recommendation
Recent methods usually take advantage of both user-bundle and user-item interactions information to obtain informative representations for users and bundles, corresponding to bundle view and item view, respectively.
Differentiable Invariant Causal Discovery
This paper proposes Differentiable Invariant Causal Discovery (DICD), utilizing the multi-environment information based on a differentiable framework to avoid learning spurious edges and wrong causal directions.
Benchmarking Unsupervised Anomaly Detection and Localization
From the time the MVTec AD dataset was proposed to the present, new research methods that are constantly being proposed push its precision to saturation.
Copy Motion From One to Another: Fake Motion Video Generation
3) To enhance texture details, we encode facial features with geometric guidance and employ local GANs to refine the face, feet, and hands.
A Novel Speech-Driven Lip-Sync Model with CNN and LSTM
Generating synchronized and natural lip movement with speech is one of the most important tasks in creating realistic virtual characters.
Hybrid Relation Guided Set Matching for Few-shot Action Recognition
To overcome the two limitations, we propose a novel Hybrid Relation guided Set Matching (HyRSM) approach that incorporates two key components: hybrid relation module and set matching metric.
Cross Pairwise Ranking for Unbiased Item Recommendation
In this work, we develop a new learning paradigm named Cross Pairwise Ranking (CPR) that achieves unbiased recommendation without knowing the exposure mechanism.
Reinforced Causal Explainer for Graph Neural Networks
Such reward accounts for the dependency of the newly-added edge and the previously-added edges, thus reflecting whether they collaborate together and form a coalition to pursue better explanations.
Rumor Detection with Self-supervised Learning on Texts and Social Graph
At the core is to model the rumor characteristics inherent in rich information, such as propagation patterns in social network and semantic patterns in post content, and differentiate them from the truth.
Learning from Untrimmed Videos: Self-Supervised Video Representation Learning with Hierarchical Consistency
In this work, we aim to learn representations by leveraging more abundant information in untrimmed videos.
Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation
State-of-the-art methods strive to incorporate additional visual evidences from neighboring frames (supporting frames) to facilitate the pose estimation of the current frame (key frame).
Revisiting Domain Generalized Stereo Matching Networks from a Feature Consistency Perspective
The stereo contrastive feature loss function explicitly constrains the consistency between learned features of matching pixel pairs which are observations of the same 3D points.
Discovering Invariant Rationales for Graph Neural Networks
Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features -- rationale -- which guides the model prediction.
Deconfounding to Explanation Evaluation in Graph Neural Networks
In this work, we propose Deconfounded Subgraph Evaluation (DSE) which assesses the causal effect of an explanatory subgraph on the model prediction.
Training Free Graph Neural Networks for Graph Matching
We present a framework of Training Free Graph Matching (TFGM) to boost the performance of Graph Neural Networks (GNNs) based graph matching, providing a fast promising solution without training (training-free).
On the Effectiveness of Sampled Softmax Loss for Item Recommendation
In this work, we aim to offer a better understanding of SSM for item recommendation.
Causal Attention for Interpretable and Generalizable Graph Classification
To endow the classifier with better interpretation and generalization, we propose the Causal Attention Learning (CAL) strategy, which discovers the causal patterns and mitigates the confounding effect of shortcuts.
Towards Multi-Grained Explainability for Graph Neural Networks
A performant paradigm towards multi-grained explainability is until-now lacking and thus a focus of our work.
Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence
Artificial intelligence (AI) provides a promising substitution for streamlining COVID-19 diagnoses.
FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows
However, current methods can not effectively map image features to a tractable base distribution and ignore the relationship between local and global features which are important to identify anomalies.
Ranked #28 on
Anomaly Detection
on MVTec LOCO AD
Unsupervised Anomaly Detection
Weakly Supervised Defect Detection
Automated Pulmonary Embolism Detection from CTPA Images Using an End-to-End Convolutional Neural Network
We have evaluated our approach using the 20 CTPA test dataset from the PE challenge, achieving a sensitivity of 78. 9%, 80. 7% and 80. 7% at 2 false positives per volume at 0mm, 2mm and 5mm localization error, which is superior to the state-of-the-art methods.
Towards Understanding the Data Dependency of Mixup-style Training
Despite seeing very few true data points during training, models trained using Mixup seem to still minimize the original empirical risk and exhibit better generalization and robustness on various tasks when compared to standard training.
Towards Demystifying Representation Learning with Non-contrastive Self-supervision
Non-contrastive methods of self-supervised learning (such as BYOL and SimSiam) learn representations by minimizing the distance between two views of the same image.
Focus Your Distribution: Coarse-to-Fine Non-Contrastive Learning for Anomaly Detection and Localization
To facilitate the learning with only normal images, we propose a new pretext task called non-contrastive learning for the fine alignment stage.
Ranked #71 on
Anomaly Detection
on MVTec AD
Inductive Lottery Ticket Learning for Graph Neural Networks
In this work, we propose a simple and effective learning paradigm, Inductive Co-Pruning of GNNs (ICPG), to endow graph lottery tickets with inductive pruning capacity.
Time-aware Path Reasoning on Knowledge Graph for Recommendation
In this work, we propose a novel Time-aware Path reasoning for Recommendation (TPRec for short) method, which leverages the potential of temporal information to offer better recommendation with plausible explanations.
Exploring Lottery Ticket Hypothesis in Media Recommender Systems
We get inspirations from the recently proposed lottery ticket hypothesis (LTH), which argues that the dense and over-parameterized model contains a much smaller and sparser sub-model that can reach comparable performance to the full model.
Visual Boundary Knowledge Translation for Foreground Segmentation
To this end, we propose a Translation Segmentation Network (Trans-Net), which comprises a segmentation network and two boundary discriminators.
Contrastive Learning for Cold-Start Recommendation
It aims to maximize the mutual dependencies between item content and collaborative signals.
Exploring Stronger Feature for Temporal Action Localization
Temporal action localization aims to localize starting and ending time with action category.
OadTR: Online Action Detection with Transformers
Most recent approaches for online action detection tend to apply Recurrent Neural Network (RNN) to capture long-range temporal structure.
Ranked #8 on
Online Action Detection
on THUMOS'14
Proposal Relation Network for Temporal Action Detection
We calculate the detection results by assigning the proposals with corresponding classification results.
Ranked #3 on
Temporal Action Localization
on ActivityNet-1.3
(using extra training data)
Weakly-Supervised Temporal Action Localization Through Local-Global Background Modeling
Then our proposed Local-Global Background Modeling Network (LGBM-Net) is trained to localize instances by using only video-level labels based on Multi-Instance Learning (MIL).
Weakly-supervised Learning
Weakly-supervised Temporal Action Localization
+1
Smart Contract Vulnerability Detection: From Pure Neural Network to Interpretable Graph Feature and Expert Pattern Fusion
In this paper, we explore combining deep learning with expert patterns in an explainable fashion.
Relation Modeling in Spatio-Temporal Action Localization
This paper presents our solution to the AVA-Kinetics Crossover Challenge of ActivityNet workshop at CVPR 2021.
Ranked #4 on
Spatio-Temporal Action Localization
on AVA-Kinetics
(using extra training data)
A Stronger Baseline for Ego-Centric Action Detection
This technical report analyzes an egocentric video action detection method we used in the 2021 EPIC-KITCHENS-100 competition hosted in CVPR2021 Workshop.
Understanding Deflation Process in Over-parametrized Tensor Decomposition
In this paper we study the training dynamics for gradient flow on over-parametrized tensor decomposition problems.
Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition
In this paper, we present empirical results for training a stronger video vision transformer on the EPIC-KITCHENS-100 Action Recognition dataset.
Deconfounded Recommendation for Alleviating Bias Amplification
In this work, we scrutinize the cause-effect factors for bias amplification, identifying the main reason lies in the confounder effect of imbalanced item distribution on user representation and prediction score.
A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation
Influenced by the great success of deep learning in computer vision and language understanding, research in recommendation has shifted to inventing new recommender models based on neural networks.
A-FMI: Learning Attributions from Deep Networks via Feature Map Importance
Gradient-based attribution methods can aid in the understanding of convolutional neural networks (CNNs).
Self-Supervised Learning for Semi-Supervised Temporal Action Proposal
In this paper, we focus on applying the power of self-supervised methods to improve semi-supervised action proposal generation.
Ranked #2 on
Semi-Supervised Action Detection
on ActivityNet-1.3
Temporal Context Aggregation Network for Temporal Action Proposal Refinement
In this paper, we propose Temporal Context Aggregation Network (TCANet) to generate high-quality action proposals through "local and global" temporal context aggregation and complementary as well as progressive boundary refinement.
Ranked #10 on
Temporal Action Localization
on ActivityNet-1.3
A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks
Real-world data usually have high dimensionality and it is important to mitigate the curse of dimensionality.
Learning Intents behind Interactions with Knowledge Graph for Recommendation
In this study, we explore intents behind a user-item interaction by using auxiliary item knowledge, and propose a new model, Knowledge Graph-based Intent Network (KGIN).
Causal Screening to Interpret Graph Neural Networks
In this work, we focus on the causal interpretability in GNNs and propose a method, Causal Screening, from the perspective of cause-effect.
Beyond Lazy Training for Over-parameterized Tensor Decomposition
We show that in a lazy training regime (similar to the NTK regime for neural networks) one needs at least $m = \Omega(d^{l-1})$, while a variant of gradient descent can find an approximate tensor when $m = O^*(r^{2. 5l}\log d)$.
Self-supervised Graph Learning for Recommendation
In this work, we explore self-supervised learning on user-item graph, so as to improve the accuracy and robustness of GCNs for recommendation.
Ranked #6 on
Collaborative Filtering
on Yelp2018
Bias and Debias in Recommender System: A Survey and Future Directions
This motivates us to provide a systematic survey of existing work on RS biases.
Multi-Level Temporal Pyramid Network for Action Detection
By this means, the proposed MLTPN can learn rich and discriminative features for different action instances with different durations.
Disentangled Graph Collaborative Filtering
Such uniform approach to model user interests easily results in suboptimal representations, failing to model diverse relationships and disentangle user intents in representations.
Interactive Path Reasoning on Graph for Conversational Recommendation
Traditional recommendation systems estimate user preference on items from past interaction history, thus suffering from the limitations of obtaining fine-grained and dynamic user preference.
Guarantees for Tuning the Step Size using a Learning-to-Learn Approach
Solving this problem using a learning-to-learn approach -- using meta-gradient descent on a meta-objective based on the trajectory that the optimizer generates -- was recently shown to be effective.
CBR-Net: Cascade Boundary Refinement Network for Action Detection: Submission to ActivityNet Challenge 2020 (Task 1)
In this report, we present our solution for the task of temporal action localization (detection) (task 1) in ActivityNet Challenge 2020.
Temporal Fusion Network for Temporal Action Localization:Submission to ActivityNet Challenge 2020 (Task E)
This technical report analyzes a temporal action localization method we used in the HACS competition which is hosted in Activitynet Challenge 2020. The goal of our task is to locate the start time and end time of the action in the untrimmed video, and predict action category. Firstly, we utilize the video-level feature information to train multiple video-level action classification models.
Hierarchical Fashion Graph Network for Personalized Outfit Recommendation
Fashion outfit recommendation has attracted increasing attentions from online shopping services and fashion communities. Distinct from other scenarios (e. g., social networking or content sharing) which recommend a single item (e. g., a friend or picture) to a user, outfit recommendation predicts user preference on a set of well-matched fashion items. Hence, performing high-quality personalized outfit recommendation should satisfy two requirements -- 1) the nice compatibility of fashion items and 2) the consistence with user preference.
Visual Perception Model for Rapid and Adaptive Low-light Image Enhancement
The core of VP model is to decompose the light source into light intensity and light spatial distribution to describe the perception process of HVS, offering refinement estimation of illumination and reflectance.
XTDrone: A Customizable Multi-Rotor UAVs Simulation Platform
A customizable multi-rotor UAVs simulation platform based on ROS, Gazebo and PX4 is presented.
Robotics
Reinforced Negative Sampling over Knowledge Graph for Recommendation
Properly handling missing data is a fundamental challenge in recommendation.
