Search Results for author:
Found 13 papers, 5 papers with code
CoRT: Code-integrated Reasoning within Thinking
Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable progress in natural language reasoning with long chain-of-thought (CoT), yet they remain inefficient or inaccurate when handling complex mathematical operations.
RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques
In this work, we introduce a new benchmark designed to assess the critique capabilities of LLMs.
Enabling Scalable Oversight via Self-Evolving Critic
Despite their remarkable performance, the development of Large Language Models (LLMs) faces a critical challenge in scalable oversight: providing effective feedback for tasks where human evaluation is difficult or where LLMs outperform humans.
An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling
Unlike the standard learning-to-optimize framework that requires optimization solutions generated by solvers, our unsupervised method adjusts the network weights directly from the evaluation of the primal-dual gap.
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks.
Adam-mini: Use Fewer Learning Rates To Gain More
Adam-mini reduces memory by cutting down the learning rate resources in Adam (i. e., $1/\sqrt{v}$).
PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming
In this work, we propose an FOM-unrolled neural network (NN) called PDHG-Net, and propose a two-stage L2O method to solve large-scale LP problems.
Why Transformers Need Adam: A Hessian Perspective
SGD performs worse than Adam by a significant margin on Transformers, but the reason remains unclear.
Federated Learning with Lossy Distributed Source Coding: Analysis and Optimization
We then conduct an FL convergence analysis to connect the aggregation distortion and the FL convergence performance.
The Global Landscape of Neural Networks: An Overview
Second, we discuss a few rigorous results on the geometric properties of wide networks such as "no bad basin", and some modifications that eliminate sub-optimal local minima and/or decreasing paths to infinity.
Sub-Optimal Local Minima Exist for Neural Networks with Almost All Non-Linear Activations
More specifically, we prove that for any multi-layer network with generic input data and non-linear activation functions, sub-optimal local minima can exist, no matter how wide the network is (as long as the last hidden layer has at least two neurons).
On the Benefit of Width for Neural Networks: Disappearance of Bad Basins
Wide networks are often believed to have a nice optimization landscape, but what rigorous results can we prove?
Sparsity Learning Based Multiuser Detection in Grant-Free Massive-Device Multiple Access
In this work, we study the multiuser detection (MUD) problem for a grant-free massive-device multiple access (MaDMA) system, where a large number of single-antenna user devices transmit sporadic data to a multi-antenna base station (BS).
