Search Results for author: Toby Jia-Jun Li

Found 16 papers, 7 papers with code

EyeTrans: Merging Human and Machine Attention for Neural Code Summarization

1 code implementation21 Feb 2024 Yifan Zhang, Jiliang Li, Zachary Karas, Aakash Bansal, Toby Jia-Jun Li, Collin McMillan, Kevin Leach, Yu Huang

Neural code summarization leverages deep learning models to automatically generate brief natural language summaries of code snippets.

Code Summarization

Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Tasks

no code implementations16 Nov 2023 Yuxuan Lu, Bingsheng Yao, Shao Zhang, Yun Wang, Peng Zhang, Tun Lu, Toby Jia-Jun Li, Dakuo Wang

Large Language Models (LLMs) have demonstrated considerable advances, and several claims have been made about their exceeding human performance.

Active Learning

UI Layout Generation with LLMs Guided by UI Grammar

no code implementations24 Oct 2023 Yuwen Lu, Ziang Tong, Qinyi Zhao, Chengzhi Zhang, Toby Jia-Jun Li

The recent advances in Large Language Models (LLMs) have stimulated interest among researchers and industry professionals, particularly in their application to tasks concerning mobile user interfaces (UIs).

In-Context Learning Position

Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation

no code implementations19 Oct 2023 Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, Haijun Xia

To address this limitation, we propose a framework that facilitates the structured generation of design space in which users can seamlessly explore, evaluate, and synthesize a multitude of responses.

AutoDroid: LLM-powered Task Automation in Android

no code implementations29 Aug 2023 Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, Yunxin Liu

Mobile task automation is an attractive technique that aims to enable voice-based hands-free user interaction with smartphones.

Language Modelling

Interactive Text-to-SQL Generation via Editable Step-by-Step Explanations

1 code implementation12 May 2023 Yuan Tian, Zheng Zhang, Zheng Ning, Toby Jia-Jun Li, Jonathan K. Kummerfeld, Tianyi Zhang

Many techniques have been proposed to automatically generate SQL from natural language, but they suffer from two issues: (1) they still make many mistakes, particularly for complex queries, and (2) they do not provide a flexible way for non-expert users to validate and refine incorrect queries.

Text-To-SQL

VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping

no code implementations16 Apr 2023 Zheng Zhang, Jie Gao, Ranjodh Singh Dhaliwal, Toby Jia-Jun Li

In argumentative writing, writers must brainstorm hierarchical writing goals, ensure the persuasiveness of their arguments, and revise and organize their plans through drafting.

Persuasiveness Text Generation

KnowledgeShovel: An AI-in-the-Loop Document Annotation System for Scientific Knowledge Base Construction

1 code implementation6 Oct 2022 Shao Zhang, Yuting Jia, Hui Xu, Dakuo Wang, Toby Jia-Jun Li, Ying Wen, Xinbing Wang, Chenghu Zhou

Constructing a comprehensive, accurate, and useful scientific knowledge base is crucial for human researchers synthesizing scientific knowledge and for enabling Al-driven scientific discovery.

A Bottom-Up End-User Intelligent Assistant Approach to Empower Gig Workers against AI Inequality

no code implementations29 Apr 2022 Toby Jia-Jun Li, Yuwen Lu, Jaylexia Clark, Meng Chen, Victor Cox, Meng Jiang, Yang Yang, Tamara Kay, Danielle Wood, Jay Brockman

The AI inequality is caused by (1) the technology divide in who has access to AI technologies in gig work; and (2) the data divide in who owns the data in gig work leads to unfair working conditions, growing pay gap, neglect of workers' diverse preferences, and workers' lack of trust in the platforms.

Position

StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement

1 code implementation13 Feb 2022 Zheng Zhang, Ying Xu, Yanhao Wang, Bingsheng Yao, Daniel Ritchie, Tongshuang Wu, Mo Yu, Dakuo Wang, Toby Jia-Jun Li

Despite its benefits for children's skill development and parent-child bonding, many parents do not often engage in interactive storytelling by having story-related dialogues with their child due to limited availability or challenges in coming up with appropriate questions.

Chatbot

Interactive Task Learning from GUI-Grounded Natural Language Instructions and Demonstrations

1 code implementation ACL 2020 Toby Jia-Jun Li, Tom Mitchell, Brad Myers

We show SUGILITE, an intelligent task automation agent that can learn new tasks and relevant associated concepts interactively from the user{'}s natural language instructions and demonstrations, using the graphical user interfaces (GUIs) of third-party mobile apps.

Privacy-Preserving Script Sharing in GUI-based Programming-by-Demonstration Systems

no code implementations17 Apr 2020 Toby Jia-Jun Li, Jingya Chen, Brandon Canfield, Brad A. Myers

An important concern in end user development (EUD) is accidentally embedding personal information in program artifacts when sharing them.

Privacy Preserving

Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrations

no code implementations30 Aug 2019 Toby Jia-Jun Li, Marissa Radensky, Justin Jia, Kirielle Singarajah, Tom M. Mitchell, Brad A. Myers

In this paper, we describe a new multi-modal domain-independent approach that combines natural language programming and programming-by-demonstration to allow users to first naturally describe tasks and associated conditions at a high level, and then collaborate with the agent to recursively resolve any ambiguities or vagueness through conversations and demonstrations.

Human-Computer Interaction

Cannot find the paper you are looking for? You can Submit a new open access paper.