Beyond Embeddings: The Promise of Visual Table in Multi-Modal Models

1 code implementation • 27 Mar 2024 • Yiwu Zhong, Zi-Yuan Hu, Michael R. Lyu, LiWei Wang

When visual tables serve as standalone visual representations, our model can closely match or even beat the SOTA MLLMs that are built on CLIP visual embeddings.

Representation Learning Visual Question Answering +2

VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Control

1 code implementation • ICCV 2023 • Zi-Yuan Hu, Yanyang Li, Michael R. Lyu, LiWei Wang

In particular, our VL-PET-large with lightweight PET module designs significantly outperforms VL-Adapter by 2. 92% (3. 41%) and LoRA by 3. 37% (7. 03%) with BART-base (T5-base) on image-text tasks.

Image Captioning Text Generation +4

CLEVA: Chinese Language Models EVAluation Platform

1 code implementation • 9 Aug 2023 • Yanyang Li, Jianqiao Zhao, Duo Zheng, Zi-Yuan Hu, Zhi Chen, Xiaohui Su, Yongfeng Huang, Shijia Huang, Dahua Lin, Michael R. Lyu, LiWei Wang

With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model's capabilities has become an increasingly significant issue.

Prototypical Graph Contrastive Learning

1 code implementation • 17 Jun 2021 • Shuai Lin, Pan Zhou, Zi-Yuan Hu, Shuojia Wang, Ruihui Zhao, Yefeng Zheng, Liang Lin, Eric Xing, Xiaodan Liang

However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer from the critical sampling bias issue, i. e., the negatives likely having the same semantic structure with the query, leading to performance degradation.

Clustering Contrastive Learning +1

BCFNet: A Balanced Collaborative Filtering Network with Attention Mechanism

1 code implementation • 10 Mar 2021 • Zi-Yuan Hu, Jin Huang, Zhi-Hong Deng, Chang-Dong Wang, Ling Huang, Jian-Huang Lai, Philip S. Yu

Representation learning tries to learn a common low dimensional space for the representations of users and items.

Collaborative Filtering Representation Learning