Search Results for author: Jiajun Song

Found 12 papers, 11 papers with code

Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?

1 code implementation2 Apr 2025 Yi-Long Lu, Chunhui Zhang, Jiajun Song, Lifeng Fan, Wei Wang

Theory of Mind (ToM), the ability to attribute mental states to others, is fundamental for human social intelligence and a critical capability for advanced Artificial Intelligence.

Attribute Reinforcement Learning (RL)

Proposing and solving olympiad geometry with guided tree search

no code implementations14 Dec 2024 Chi Zhang, Jiajun Song, Siyu Li, Yitao Liang, Yuxi Ma, Wei Wang, Yixin Zhu, Song-Chun Zhu

Mathematics olympiads are prestigious competitions, with problem proposing and solving highly honored.

Out-of-distribution generalization via composition: a lens through induction heads in Transformers

1 code implementation18 Aug 2024 Jiajun Song, Zhuoyan Xu, Yiqiao Zhong

We empirically examined the training dynamics of Transformers on a synthetic example and conducted extensive experiments on a variety of pretrained LLMs, focusing on a type of components known as induction heads.

In-Context Learning Out-of-Distribution Generalization

A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning

1 code implementation6 Jul 2024 Jiajun Song, Jiajun Luo, Rongwei Lu, Shuzhao Xie, Bin Chen, Zhi Wang

Asynchronous Federated Learning (AFL) confronts inherent challenges arising from the heterogeneity of devices (e. g., their computation capacities) and low-bandwidth environments, both potentially causing stale model updates (e. g., local gradients) for global aggregation.

Federated Learning image-classification +1

Dataset and Benchmark for Urdu Natural Scenes Text Detection, Recognition and Visual Question Answering

1 code implementation21 May 2024 Hiba Maryam, Ling Fu, Jiajun Song, Tajrian ABM Shafayet, Qidi Luo, Xiang Bai, Yuliang Liu

The development of Urdu scene text detection, recognition, and Visual Question Answering (VQA) technologies is crucial for advancing accessibility, information retrieval, and linguistic diversity in digital content, facilitating better understanding and interaction with Urdu-language visual data.

Diversity Information Retrieval +5

Synthesizing Knowledge-enhanced Features for Real-world Zero-shot Food Detection

1 code implementation14 Feb 2024 Pengfei Zhou, Weiqing Min, Jiajun Song, Yang Zhang, Shuqiang Jiang

The complexity of food semantic attributes further makes it more difficult for current ZSD methods to distinguish various food categories.

Attribute Generalized Zero-Shot Object Detection +2

Uncovering hidden geometry in Transformers via disentangling position and context

1 code implementation7 Oct 2023 Jiajun Song, Yiqiao Zhong

Given embedding vector $\boldsymbol{h}_{c, t} \in \mathbb{R}^d$ at sequence position $t \le T$ in a sequence (or context) $c \le C$, extracting the mean effects yields the decomposition \[ \boldsymbol{h}_{c, t} = \boldsymbol{\mu} + \mathbf{pos}_t + \mathbf{ctx}_c + \mathbf{resid}_{c, t} \] where $\boldsymbol{\mu}$ is the global mean vector, $\mathbf{pos}_t$ and $\mathbf{ctx}_c$ are the mean vectors across contexts and across positions respectively, and $\mathbf{resid}_{c, t}$ is the residual vector.

Dictionary Learning POS +1

Cannot find the paper you are looking for? You can Submit a new open access paper.