Search Results for author:
Found 22 papers, 13 papers with code
MiniMax-01: Scaling Foundation Models with Lightning Attention
This approach enables us to conduct efficient training and inference on models with hundreds of billions of parameters across contexts spanning millions of tokens.
Scaling Diffusion Language Models via Adaptation from Autoregressive Models
Diffusion Language Models (DLMs) have emerged as a promising new paradigm for text generative modeling, potentially addressing limitations of autoregressive (AR) models.
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
Our work highlights the potential of diffusion-based approaches in advancing AI capabilities for sophisticated language understanding and problem-solving tasks.
SubgoalXL: Subgoal-based Expert Learning for Theorem Proving
This paper introduces SubgoalXL, a novel approach that synergizes subgoal-based proofs with expert learning to enhance LLMs' capabilities in formal theorem proving within the Isabelle environment.
Ranked #3 on
Automated Theorem Proving
on miniF2F-test
(using extra training data)
Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models
Recently, diffusion models have garnered significant interest in the field of text processing due to their many potential advantages compared to conventional autoregressive models.
Linear Attention via Orthogonal Memory
Given that orthogonal memory compresses global information, we further dissect the context to amplify fine-grained local information.
Automated interpretation of congenital heart disease from multi-view echocardiograms
Our video-based model can diagnose with an accuracy of 93. 9\% (binary classification), and 92. 1\% (3-class classification) in a collected 2D video testing set, which does not need key-frame selection and view annotation in testing.
Self-Infilling Code Generation
This work introduces self-infilling code generation, a general framework that incorporates infilling operations into auto-regressive decoding.
Attentive Multi-Layer Perceptron for Non-autoregressive Generation
Furthermore, we marry AMLP with popular NAR models, deriving a highly efficient NAR-AMLP architecture with linear time and space complexity.
SI-SD: Sleep Interpreter through awake-guided cross-subject Semantic Decoding
Understanding semantic content from brain activity during sleep represents a major goal in neuroscience.
K-means Clustering Based Feature Consistency Alignment for Label-free Model Evaluation
The label-free model evaluation aims to predict the model performance on various test sets without relying on ground truths.
Retrieved Sequence Augmentation for Protein Representation Learning
RSA links query protein sequences to a set of sequences with similar structures or properties in the database and combines these sequences for downstream prediction.
A Reparameterized Discrete Diffusion Model for Text Generation
This work studies discrete diffusion probabilistic models with applications to natural language generation.
Efficient Attention via Control Variates
Built upon previous progress of RFA, we characterize this gap through the lens of control variates and show that RFA can be decomposed into a sum of multiple control variate estimators for each element in the sequence.
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling
In this paper, we propose Comprehensive Attention Benchmark (CAB) under a fine-grained attention taxonomy with four distinguishable attention patterns, namely, noncausal self, causal self, noncausal cross, and causal cross attentions.
Learning Feature Disentanglement and Dynamic Fusion for Recaptured Image Forensic
Image recapture seriously breaks the fairness of artificial intelligent (AI) systems, which deceives the system by recapturing others' images.
Poincaré Heterogeneous Graph Neural Networks for Sequential Recommendation
In this paper, we propose a Poincar\'{e}-based heterogeneous graph neural network named PHGR to model the sequential pattern information as well as hierarchical information contained in the data of SR scenarios simultaneously.
Linear Complexity Randomized Self-attention Mechanism
By combining the expressiveness in RA and the efficiency in RFA, we develop a novel linear complexity self-attention mechanism called linear randomized attention (LARA).
Ripple Attention for Visual Perception with Sub-quadratic Complexity
Transformer architectures are now central to sequence modeling tasks.
HCGR: Hyperbolic Contrastive Graph Representation Learning for Session-based Recommendation
Session-based recommendation (SBR) learns users' preferences by capturing the short-term and sequential patterns from the evolution of user behaviors.
Cascaded Head-colliding Attention
Transformers have advanced the field of natural language processing (NLP) on a variety of important tasks.
Generative Semantic Hashing Enhanced via Boltzmann Machines
Generative semantic hashing is a promising technique for large-scale information retrieval thanks to its fast retrieval speed and small memory footprint.
