Denoising Cosine Similarity: A Theory-Driven Approach for Efficient Representation Learning

no code implementations • 19 Apr 2023 • Takumi Nakagawa, Yutaro Sanada, Hiroki Waida, Yuhui Zhang, Yuichiro Wada, Kōsaku Takanashi, Tomonori Yamada, Takafumi Kanamori

To this end, inspired by recent works on denoising and the success of the cosine-similarity-based objective functions in representation learning, we propose the denoising Cosine-Similarity (dCS) loss.

Denoising Representation Learning

Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis

no code implementations • 1 Apr 2023 • Hiroki Waida, Yuichiro Wada, Léo Andéol, Takumi Nakagawa, Yuhui Zhang, Takafumi Kanamori

We first prove that the formulation characterizes the structure of representations learned with the kernel-based contrastive learning framework.

Contrastive Learning Representation Learning

Deep Clustering with a Constraint for Topological Invariance based on Symmetric InfoNCE

no code implementations • 6 Mar 2023 • Yuhui Zhang, Yuichiro Wada, Hiroki Waida, Kaito Goto, Yusaku Hino, Takafumi Kanamori

To address the problem, we propose a constraint utilizing symmetric InfoNCE, which helps an objective of deep clustering method in the scenario train the model so as to be efficient for not only non-complex topology but also complex topology datasets.

Clustering Deep Clustering