1 code implementation • 31 Jan 2025 • Pu Yang, Yunzhen Feng, ZiYuan Chen, Yuhang Wu, Zhuoyuan Li
Modern foundation models often undergo iterative ``bootstrapping'' in their post-training phase: a model generates synthetic data, an external verifier filters out low-quality samples, and the high-quality subset is used for further fine-tuning.
no code implementations • 3 Jan 2025 • Pu Yang, Bin Dong
Image captioning is a critical task at the intersection of computer vision and natural language processing, with wide-ranging applications across various domains.
no code implementations • 11 Jun 2024 • Yunzhen Feng, Elvis Dohmatob, Pu Yang, Francois Charton, Julia Kempe
Large Language Models (LLM) are increasingly trained on data generated by other LLM, either because generated text and images become part of the pre-training corpus, or because synthetized data is used as a replacement for expensive human-annotation.
no code implementations • 10 Feb 2024 • Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, Julia Kempe
We discover a wide range of decay phenomena, analyzing loss of scaling, shifted scaling with number of generations, the ''un-learning" of skills, and grokking when mixing human and synthesized data.
1 code implementation • 18 Apr 2023 • Christopher J. Smith, Alaa Al Khourdajie, Pu Yang, Doris Folini
CO2 emissions trajectories resulting from the optimal level of emissions abatement in all pathways are also sensitive to climate uncertainty, with 2050 emissions ranging from -12 to +14 GtCO2/yr in the 1. 5C scenario.
1 code implementation • 5 Dec 2022 • Pu Yang, Bin Dong
In this paper, we propose an alternating training framework for jointly learning a good pair of samplers and reconstructors via deep reinforcement learning (RL).
no code implementations • 1 Jan 2021 • Bochen Lv, Pu Yang, Zehao Wang, Zhanxing Zhu
And the log-spectrum difference of the adversarial examples and clean image is more concentrated in the high-frequency part than the low-frequency part.
no code implementations • 3 Mar 2020 • Austin Reiter, Menglin Jia, Pu Yang, Ser-Nam Lim
Most deep learning-based methods rely on a late fusion technique whereby multiple feature types are encoded and concatenated and then a multi layer perceptron (MLP) combines the fused embedding to make predictions.