no code implementations • NAACL (TextGraphs) 2021 • Jinman Zhao, Gerald Penn, Huan Ling
In this paper, we define an abstract task called structural realization that generates words given a prefix of words and a partial representation of a parse tree.
no code implementations • 5 Mar 2024 • Jinman Zhao, Xueyan Zhang
We present a comprehensive evaluation of large language models(LLMs)' ability to reason about composition relations through a benchmark encompassing 1, 500 test cases in English, designed to cover six distinct types of composition relations: Positional, Comparative, Personal, Mathematical, Identity, and Other.
no code implementations • 1 Mar 2024 • Jinman Zhao, Yitian Ding, Chen Jia, Yining Wang, Zifan Qian
We investigate the outputs of the GPT series of LLMs in various languages using our three measurement methods.
1 code implementation • NeurIPS 2023 • Tuan Dinh, Jinman Zhao, Samson Tan, Renato Negrinho, Leonard Lausen, Sheng Zha, George Karypis
We find that the presence of potential bugs significantly degrades the generation performance of the high-performing Code-LLMs.
no code implementations • 1 Jun 2023 • Hengzhi Pei, Jinman Zhao, Leonard Lausen, Sheng Zha, George Karypis
To better solve this task, we query a program analyzer for information relevant to a given function call, and consider ways to provide the analyzer results to different code completion models during inference and training.
1 code implementation • 30 Mar 2020 • Seohyun Kim, Jinman Zhao, Yuchi Tian, Satish Chandra
We provide comprehensive experimental evaluation of our proposal, along with alternative design choices, on a standard Python dataset, as well as on a Python corpus internal to Facebook.
Ranked #1 on Type prediction on Py150
Type prediction Value prediction Software Engineering
1 code implementation • EMNLP 2018 • Jinman Zhao, Sidharth Mudgal, YIngyu Liang
We approach the problem of generalizing pre-trained word embeddings beyond fixed-size vocabularies without using additional contextual information.
no code implementations • 11 Sep 2018 • Jinman Zhao, Aws Albarghouthi, Vaibhav Rastogi, Somesh Jha, Damien Octeau
We address the problem of discovering communication links between applications in the popular Android mobile operating system, an important problem for security and privacy in Android.
no code implementations • NeurIPS 2018 • Lingjiao Chen, Hongyi Wang, Jinman Zhao, Dimitris Papailiopoulos, Paraschos Koutris
Distributed implementations of mini-batch stochastic gradient descent (SGD) suffer from communication overheads, attributed to the high frequency of gradient updates inherent in small-batch training.