Search Results for author:
Found 17 papers, 11 papers with code
Kimi-VL Technical Report
We present Kimi-VL, an efficient open-source Mixture-of-Experts (MoE) vision-language model (VLM) that offers advanced multimodal reasoning, long-context understanding, and strong agent capabilities - all while activating only 2. 8B parameters in its language decoder (Kimi-VL-A3B).
Muon is Scalable for LLM Training
Recently, the Muon optimizer based on matrix orthogonalization has demonstrated strong results in training small-scale language models, but the scalability to larger models has not been proven.
MoBA: Mixture of Block Attention for Long-Context LLMs
Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI).
Kimi k1.5: Scaling Reinforcement Learning with LLMs
Moreover, we present effective long2short methods that use long-CoT techniques to improve short-CoT models, yielding state-of-the-art short-CoT reasoning results -- e. g., 60. 8 on AIME, 94. 6 on MATH500, 47. 3 on LiveCodeBench -- outperforming existing short-CoT models such as GPT-4o and Claude Sonnet 3. 5 by a large margin (up to +550%).
A Self-enhancement Approach for Domain-specific Chatbot Training via Knowledge Mining and Digest
This paper introduces a novel approach to enhance LLMs by effectively extracting the relevant knowledge from domain-specific textual sources, and the adaptive training of a chatbot with domain-specific inquiries.
Unsupervised Parallel Corpus Mining on Web Data
In contrast, there is a large-scale of parallel corpus created by humans on the Internet.
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
With the success of language pretraining, it is highly desirable to develop more efficient architectures of good scalability that can exploit the abundant unlabeled data at a lower cost.
Ranked #6 on
Reading Comprehension
on RACE
Correlation-aware Unsupervised Change-point Detection via Graph Neural Networks
Keywords: Multivariate Time Series, Change-point Detection, Graph Neural Networks
Bridging the domain gap in cross-lingual document classification
We consider the setting of semi-supervised cross-lingual understanding, where labeled data is available in a source language (English), but only unlabeled data is available in the target language.
Re-examination of the Role of Latent Variables in Sequence Modeling
With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence.
Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
In this paper, we combine the ideas from both stochastic latent variables and dilated convolutions, and propose a new architecture to model sequential data, termed as Stochastic WaveNet, where stochastic latent variables are injected into the WaveNet structure.
Large-scale Cloze Test Dataset Designed by Teachers
Cloze test is widely adopted in language exams to evaluate students' language proficiency.
Learning Graph Convolution Filters from Data Manifold
Convolution Neural Network (CNN) has gained tremendous success in computer vision tasks with its outstanding ability to capture the local latent features.
Large-scale Cloze Test Dataset Created by Teachers
Cloze tests are widely adopted in language exams to evaluate students' language proficiency.
Learning Depthwise Separable Graph Convolution from Data Manifold
Convolution Neural Network (CNN) has gained tremendous success in computer vision tasks with its outstanding ability to capture the local latent features.
RACE: Large-scale ReAding Comprehension Dataset From Examinations
We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task.
Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
Multivariate time series forecasting is an important machine learning problem across many domains, including predictions of solar plant energy output, electricity consumption, and traffic jam situation.
Ranked #1 on
Univariate Time Series Forecasting
on Solar-Power
