Search Results for author:
Found 99 papers, 26 papers with code
ReCamMaster: Camera-Controlled Generative Rendering from A Single Video
However, altering camera trajectories of a given video remains under-explored, despite its importance in the field of video creation.
TIME: Temporal-sensitive Multi-dimensional Instruction Tuning and Benchmarking for Video-LLMs
Video large language models have achieved remarkable performance in tasks such as video question answering, however, their temporal understanding remains suboptimal.
Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understanding
To address these challenges, we propose learning the mapping between exocentric and egocentric domains, leveraging the extensive exocentric knowledge within existing MLLMs to enhance egocentric video understanding.
ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis
Audio-driven human gesture synthesis is a crucial task with broad applications in virtual avatars, human-computer interaction, and creative content generation.
RectifiedHR: Enable Efficient High-Resolution Image Generation via Energy Rectification
In this paper, we propose RectifiedHR, an straightforward and efficient solution for training-free high-resolution image generation.
HAIC: Improving Human Action Understanding and Generation with Better Captions for Multi-modal Large Language Models
Recent Multi-modal Large Language Models (MLLMs) have made great progress in video understanding.
Be a Multitude to Itself: A Prompt Evolution Framework for Red Teaming
Large Language Models (LLMs) have gained increasing attention for their remarkable capacity, alongside concerns about safety arising from their potential to produce harmful content.
SPPD: Self-training with Process Preference Learning Using Dynamic Value Margin
Recently, enhancing the numerical and logical reasoning capability of Large Language Models (LLMs) has emerged as a research hotspot.
FlexDuo: A Pluggable System for Enabling Full-Duplex Capabilities in Speech Dialogue Systems
Full-Duplex Speech Dialogue Systems (Full-Duplex SDS) have significantly enhanced the naturalness of human-machine interaction by enabling real-time bidirectional communication.
Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
Moreover, we find that Finedeep achieves optimal results when balancing depth and width, specifically by adjusting the number of expert sub-layers and the number of experts per sub-layer.
DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
As large language models continue to scale, computational costs and resource consumption have emerged as significant challenges.
iMOVE: Instance-Motion-Aware Video Understanding
Enhancing the fine-grained instance spatiotemporal motion perception capabilities of Video Large Language Models is crucial for improving their temporal and general video understanding.
CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation
In the first stage, we design an interactive workflow that allows users to intuitively construct 3D-aware conditional signals by positioning object bounding boxes and defining camera movements within the 3D space.
Efficient Triangular Arbitrage Detection via Graph Neural Networks
This work highlights the potential of using GNNs for solving optimization problems in finance and provides a promising approach for real-time arbitrage detection in dynamic financial markets.
Visualizing the Local Atomic Environment Features of Machine Learning Interatomic Potential
This paper addresses the challenges of creating efficient and high-quality datasets for machine learning potential functions.
Improving Video Generation with Human Feedback
Video generation has achieved significant advances through rectified flow techniques, but issues like unsmooth motion and misalignment between videos and prompts persist.
GameFactory: Creating New Games with Generative Interactive Videos
In this paper, we present GameFactory, a framework focused on exploring scene generalization in game video generation.
ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning
To address these challenges, we introduce ConceptMaster, an innovative framework that effectively tackles the critical issues of identity decoupling while maintaining concept fidelity in customized videos.
Biology Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models
Large language models have already demonstrated their formidable capabilities in general domains, ushering in a revolutionary transformation.
Owl-1: Omni World Model for Consistent Long Video Generation
As videos are observations of the underlying evolving world, we propose to model the long-term developments in a latent space and use VGMs to film them into videos.
Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models
In this paper, we introduce a novel single-stage continual pretraining method, Head-Adaptive Rotary Position Encoding (HARPE), to equip LLMs with long context modeling capabilities while simplifying the training process.
3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation
Previous methods on controllable video generation primarily leverage 2D control signals to manipulate object motions and have achieved remarkable synthesis results.
SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints
Recent advancements in video diffusion models have shown exceptional abilities in simulating real-world dynamics and maintaining 3D consistency.
StyleMaster: Stylize Your Video with Artistic Generation and Translation
Style control has been popular in video generation models.
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning
In this approach, the Reasoner generates reasoning responses according to text prompts, which can evolve iteratively as a policy based on feedback from the Critic.
Towards Precise Scaling Laws for Video Diffusion Transformers
Achieving optimal performance of video diffusion transformers within given data and compute budget is crucial due to their high training costs.
Video-Text Dataset Construction from Multi-AI Feedback: Promoting Weak-to-Strong Preference Learning for Video Large Language Models
To address these issues, we propose a high-quality VQA preference dataset, called \textit{\textbf{M}ultiple \textbf{M}ultimodal \textbf{A}rtificial \textbf{I}ntelligence \textbf{P}reference Datasets in \textbf{V}QA} (\textbf{MMAIP-V}), which is constructed by sampling from the response distribution set and using an external scoring function for response evaluation.
VIVID-10M: A Dataset and Baseline for Versatile and Interactive Video Local Editing
VIVID-10M is the first large-scale hybrid image-video local editing dataset aimed at reducing data construction and model training costs, which comprises 9. 7M samples that encompass a wide range of video editing tasks.
MolReFlect: Towards Fine-grained In-Context Alignment between Molecules and Texts
Previous endeavours often treat the molecule as a general SMILES string or molecular graph, neglecting the fine-grained alignments between the molecular sub-structures and the descriptive textual phrases, which are crucial for accurate and explainable predictions.
Ranked #2 on
Text-based de novo Molecule Generation
on ChEBI-20
MolReFlect: Towards In-Context Fine-grained Alignments between Molecules and Texts
Previous endeavours often treat the molecule as a general SMILES string or molecular graph, neglecting the fine-grained alignments between the molecular sub-structures and the descriptive textual phrases, which are crucial for accurate and explainable predictions.
Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulation
Realistic simulation of dynamic scenes requires accurately capturing diverse material properties and modeling complex object interactions grounded in physical principles.
DMQR-RAG: Diverse Multi-Query Rewriting for RAG
In this paper, we introduce DMQR-RAG, a Diverse Multi-Query Rewriting framework designed to improve the performance of both document retrieval and final responses in RAG.
Kwai-STaR: Transform LLMs into State-Transition Reasoners
In this work, we define mathematical problem-solving as a process of transiting from an initial unsolved state to the final resolved state, and propose Kwai-STaR framework, which transforms LLMs into State-Transition Reasoners to improve their intuitive reasoning capabilities.
Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content
As visual generation technologies continue to advance, the scale of video datasets has expanded rapidly, and the quality of these datasets is critical to the performance of video generation models.
LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning
This paper presents an advanced mathematical problem-solving framework, LLaMA-Berry, for enhancing the mathematical reasoning ability of Large Language Models (LLMs).
Focus On What Matters: Separated Models For Visual-Based RL Generalization
Perceiving the pre-eminence of image reconstruction in representation learning, we propose SMG (Separated Models for Generalization), a novel approach that exploits image reconstruction for generalization.
Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding
For iterative refinement, we first leverage a video-language model to generate synthetic annotations, resulting in a refined dataset.
Towards Unified 3D Hair Reconstruction from Single-View Portraits
Our experiments demonstrate that reconstructing braided and un-braided 3D hair from single-view images via a unified approach is possible and our method achieves the state-of-the-art performance in recovering complex hairstyles.
ERABAL: Enhancing Role-Playing Agents through Boundary-Aware Learning
Role-playing is an emerging application in the field of Human-Computer Interaction (HCI), primarily implemented through the alignment training of a large language model (LLM) with assigned characters.
SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs
Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable perceptual and reasoning abilities, typically comprising a Vision Encoder, an Adapter, and a Large Language Model (LLM).
Ranked #71 on
Visual Question Answering
on MM-Vet
Asymmetric Graph Error Control with Low Complexity in Causal Bandits
In this paper, the causal bandit problem is investigated, in which the objective is to select an optimal sequence of interventions on nodes in a causal graph.
ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area
We benchmark ChemVLM against a range of open-source and proprietary multimodal large language models on various tasks.
ViMo: Generating Motions from Casual Videos
Although humans have the innate ability to imagine multiple possible actions from videos, it remains an extraordinary challenge for computers due to the intricate camera movements and montages.
The Economic Analysis of the Common Pool Method through the HARA Utility Functions
Water market is a contemporary marketplace for water trading and is deemed to one of the most efficient instruments to improve the social welfare.
The Stochastic Conjugate Subgradient Algorithm For Kernel Support Vector Machines
Moreover, it significantly enhances both speed and accuracy of the optimization process.
EVLM: An Efficient Vision-Language Model for Visual Understanding
However, when dealing with long sequences of visual signals or inputs such as videos, the self-attention mechanism of language models can lead to significant computational overhead.
PlacidDreamer: Advancing Harmony in Text-to-3D Generation
Previous methods utilize end-to-end 3D generation models to initialize 3D Gaussians, multi-view diffusion models to enforce multi-view consistency, and text-to-image diffusion models to refine details with score distillation algorithms.
A Comprehensive Survey on Kolmogorov Arnold Networks (KAN)
Through this comprehensive survey of Kolmogorov-Arnold Networks(KAN), we have gained a thorough understanding of its theoretical foundation, architectural design, application scenarios, and current research progress.
LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control
Instead of following mainstream diffusion-based methods, we explore and extend the potential of the implicit-keypoint-based framework, which effectively balances computational efficiency and controllability.
Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model
The Mixture-of-Experts (MoE) has gained increasing attention in studying Large Vision-Language Models (LVLMs).
Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector
Hallucination detection is a challenging task for large language models (LLMs), and existing studies heavily rely on powerful closed-source LLMs such as GPT-4.
Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B
This paper introduces the MCT Self-Refine (MCTSr) algorithm, an innovative integration of Large Language Models (LLMs) with Monte Carlo Tree Search (MCTS), designed to enhance performance in complex mathematical reasoning tasks.
VideoTetris: Towards Compositional Text-to-Video Generation
Diffusion models have demonstrated great success in text-to-video (T2V) generation.
A-SDM: Accelerating Stable Diffusion through Model Assembly and Feature Inheritance Strategies
1) For the tuning method, we design a model assembly strategy to reconstruct a lightweight model while preserving performance through distillation.
Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs
The core of our approach is the observation that a pre-trained language model can confidently predict multiple contiguous tokens, forming the basis for a \textit{lexical unit}, in which these contiguous tokens could be decoded in parallel.
Learning Multi-dimensional Human Preference for Text-to-Image Generation
Current metrics for text-to-image models typically rely on statistical metrics which inadequately represent the real preference of humans.
Evaluating Readability and Faithfulness of Concept-based Explanations
Based on this, we quantify the faithfulness of a concept explanation via perturbation.
Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues
In this paper, we propose to explicitly capture the complex rules to help the user simulator pose diverse and in-depth instruction.
UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark
To further evaluate the IAA capability of MLLMs, we construct the UNIAA-Bench, which consists of three aesthetic levels: Perception, Description, and Assessment.
End-to-end training of Multimodal Model and ranking Model
In this paper, we propose an industrial multimodal recommendation framework named EM3: End-to-end training of Multimodal Model and ranking Model, which sufficiently utilizes multimodal information and allows personalized ranking tasks to directly train the core modules in the multimodal model to obtain more task-oriented content features, without overburdening resource consumption.
Motion Inversion for Video Customization
In this work, we present a novel approach for motion customization in video generation, addressing the widespread gap in the exploration of motion representation within video generative models.
DouRN: Improving DouZero by Residual Neural Networks
Deep reinforcement learning has made significant progress in games with imperfect information, but its performance in the card game Doudizhu (Chinese Poker/Fight the Landlord) remains unsatisfactory.
DragAnything: Motion Control for Anything using Entity Representation
We introduce DragAnything, which utilizes a entity representation to achieve motion control for any object in controllable video generation.
A Survey on Applications of Reinforcement Learning in Spatial Resource Allocation
Given these advancements, there has been a surge in novel methods employing reinforcement learning to tackle spatial resource allocation problems.
ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors
The safety of Large Language Models (LLMs) has gained increasing attention in recent years, but there still lacks a comprehensive approach for detecting safety issues within LLMs' responses in an aligned, customizable and explainable manner.
Enhancing Role-playing Systems through Aggressive Queries: Evaluation and Improvement
Experiments on models improved by RoleAD indicate that our adversarial dataset ameliorates this deficiency, with the improvements demonstrating a degree of generalizability in ordinary scenarios.
ChemLLM: A Chemical Large Language Model
However, the community lacks an LLM specifically designed for chemistry.
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization
In light of recent advances in multimodal Large Language Models (LLMs), there is increasing attention to scaling them from image-text data to more informative real-world videos.
Ranked #3 on
Text-to-Video Generation
on MSR-VTT
Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion
In practice, users often desire the ability to control object motion and camera movement independently for customized video creation.
Rethinking Cross-Attention for Infrared and Visible Image Fusion
The DIIM is designed by modifying the vanilla cross-attention mechanism, which can promote the extraction of the discrepancy information of the source images.
Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint
To address it, we propose a new RL method named RLMEC that incorporates a generative model as the reward model, which is trained by the erroneous solution rewriting task under the minimum editing constraint, and can produce token-level rewards for RL training.
I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models
I2V-Adapter adeptly propagates the unnoised input image to subsequent noised frames through a cross-frame attention mechanism, maintaining the identity of the input image without any changes to the pretrained T2V model.
Paragraph-to-Image Generation with Information-Enriched Diffusion Model
In this paper, we introduce an information-enriched diffusion model for paragraph-to-image generation task, termed ParaDiffusion, which delves into the transference of the extensive semantic comprehension capabilities of large language models to the task of image generation.
Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarios
Although chain-of-thought (CoT) prompting combined with language models has achieved encouraging results on complex reasoning tasks, the naive greedy decoding used in CoT prompting usually causes the repetitiveness and local optimality.
DialogBench: Evaluating LLMs as Human-like Dialogue Systems
In this paper, we propose DialogBench, a dialogue evaluation benchmark that contains 12 dialogue tasks to probe the capabilities of LLMs as human-like dialogue systems should have.
USDC: Unified Static and Dynamic Compression for Visual Transformer
To alleviate two disadvantages of two categories of methods, we propose to unify the static compression and dynamic compression techniques jointly to obtain an input-adaptive compressed model, which can further better balance the total compression ratios and the model performances.
ASP: Automatic Selection of Proxy dataset for efficient AutoML
Deep neural networks have gained great success due to the increasing amounts of data, and diverse effective neural network designs.
Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models
In this paper, we introduce Parrot, a solution aiming to enhance multi-turn instruction following for LLMs.
KwaiYiiMath: Technical Report
Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on mathematical tasks requiring multi-step reasoning.
Ranked #95 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization
Specifically, we introduce a well-designed visual tokenizer to translate the non-linguistic image into a sequence of discrete tokens like a foreign language that LLM can read.
Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks
We propose an improved end-to-end Minimax optimization method for this sparse learning problem to better balance the model performance and the computation efficiency.
Safe Reinforcement Learning with Dead-Ends Avoidance and Recovery
In this paper, we propose a method to construct a boundary that discriminates safe and unsafe states.
Cyclic Delay-Doppler Shift: A Simple Transmit Diversity Technique for Delay-Doppler Waveforms in Doubly Selective Channels
Delay-Doppler waveform design has been considered as a promising solution to achieve reliable communication under high-mobility channels for the space-air-ground-integrated networks (SAGIN).
ClusterLog: Clustering Logs for Effective Log-based Anomaly Detection
With the increasing prevalence of scalable file systems in the context of High Performance Computing (HPC), the importance of accurate anomaly detection on runtime logs is increasing.
Optimal Settings for Cryptocurrency Trading Pairs
2) It satisfies the connectivity constraint, that is, all currencies are guaranteed to be tradable.
Modeling Randomly Walking Volatility with Chained Gamma Distributions
2) If the variational inference(VI) is used for state estimation, it runs much faster than Monte Carlo(MC) methods since the calculation of the posterior uses only basic arithmetic operations.
PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems
However, we observe that GPU devices in training recommender systems are underutilized, and they cannot attain an expected throughput improvement as what it has achieved in CV and NLP areas.
AMCAD: Adaptive Mixed-Curvature Representation based Advertisement Retrieval System
Moreover, to deploy AMCAD in Taobao, one of the largest ecommerce platforms with hundreds of million users, we design an efficient two-layer online retrieval framework for the task of graph based advertisement retrieval.
Juvenile state hypothesis: What we can learn from lottery ticket hypothesis researches?
The original lottery ticket hypothesis performs pruning and weight resetting after training convergence, exposing it to the problem of forgotten learning knowledge and potential high cost of training.
M6-T: Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling.
Graph Intention Network for Click-through Rate Prediction in Sponsored Search
Moreover, it is difficult for user to jump out of their specific historical behaviors for possible interest exploration, namely weak generalization problem.
SCMA Codebook Design Based on Uniquely Decomposable Constellation Groups
Thus, BER performance of the proposed codebook design approach outperforms that of the existing codebook design schemes in both uncoded and coded SCMA systems, especially for large-size codebooks.
Tunable ferroelectricity in hBN intercalated twisted double-layer graphene
Van der Waals (vdW) assembly of two-dimensional materials has been long recognized as a powerful tool to create unique systems with properties that cannot be found in natural compounds.
Mesoscale and Nanoscale Physics Materials Science
Singlino-dominated dark matter in $Z_3$-NMSSM
Approximate analytical formulas describing the dark matter abundance and cross section in the scattering with nucleons are used to illustrate a dependence on theoretical parameters in neutralino and Higgs sectors.
High Energy Physics - Phenomenology
Radiative Decays of Charged Leptons in the Seesaw Effective Field Theory with One-loop Matching
For the first time, the Wilson coefficients of all the relevant six-dimensional operators are computed by carrying out the one-loop matching between the effective theory and full seesaw model, and applied to calculate the total rates of radiative decays of charged leptons.
High Energy Physics - Phenomenology High Energy Physics - Experiment
RLScheduler: An Automated HPC Batch Job Scheduler Using Reinforcement Learning
Today high-performance computing (HPC) platforms are still dominated by batch jobs.
A bi-diffusion based layer-wise sampling method for deep learning in large graphs
The Graph Convolutional Network (GCN) and its variants are powerful models for graph representation learning and have recently achieved great success on many graph-based applications.
Projecting "better than randomly": How to reduce the dimensionality of very large datasets in a way that outperforms random projections
However, the recent development of randomized principal component analysis (RPCA) has opened up the possibility of obtaining approximate principal components on very large datasets.
Gear Training: A new way to implement high-performance model-parallel training
The training of Deep Neural Networks usually needs tremendous computing resources.
