Search Results for author: Hongxia Jin

Found 61 papers, 9 papers with code

Conditional Image Repainting via Semantic Bridge and Piecewise Value Function

no code implementations ECCV 2020 Shuchen Weng, Wenbo Li, Dawei Li, Hongxia Jin, Boxin Shi

We study conditional image repainting where a model is trained to generate visual content conditioned on user inputs, and composite the generated content seamlessly onto a user provided image while preserving the semantics of users' inputs.

Compositional Generalization in Spoken Language Understanding

no code implementations25 Dec 2023 Avik Ray, Yilin Shen, Hongxia Jin

State-of-the-art spoken language understanding (SLU) models have shown tremendous success in benchmark SLU datasets, yet they still fail in many practical scenario due to the lack of model compositionality when trained on limited training data.

Spoken Language Understanding

Token Fusion: Bridging the Gap between Token Pruning and Token Merging

no code implementations2 Dec 2023 Minchul Kim, Shangqian Gao, Yen-Chang Hsu, Yilin Shen, Hongxia Jin

In this paper, we introduce "Token Fusion" (ToFu), a method that amalgamates the benefits of both token pruning and token merging.

Computational Efficiency Image Generation

Continual Diffusion with STAMINA: STack-And-Mask INcremental Adapters

no code implementations30 Nov 2023 James Seale Smith, Yen-Chang Hsu, Zsolt Kira, Yilin Shen, Hongxia Jin

We show that STAMINA outperforms the prior SOTA for the setting of text-to-image continual customization on a 50-concept benchmark composed of landmarks and human faces, with no stored replay data.

Continual Learning Hard Attention +1

Explainable and Accurate Natural Language Understanding for Voice Assistants and Beyond

no code implementations25 Sep 2023 Kalpa Gunaratna, Vijay Srinivasan, Hongxia Jin

Therefore to bridge this gap, we transform the full joint NLU model to be `inherently' explainable at granular levels without compromising on accuracy.

General Classification Intent Detection +6

Instruction-following Evaluation through Verbalizer Manipulation

no code implementations20 Jul 2023 Shiyang Li, Jun Yan, Hai Wang, Zheng Tang, Xiang Ren, Vijay Srinivasan, Hongxia Jin

We conduct a comprehensive evaluation of four major model families across nine datasets, employing twelve sets of verbalizers for each of them.

Instruction Following

AlpaGasus: Training A Better Alpaca with Fewer Data

3 code implementations17 Jul 2023 Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, Hongxia Jin

Large language models (LLMs) strengthen instruction-following capability through instruction-finetuning (IFT) on supervised instruction/response data.

Instruction Following

Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA

no code implementations12 Apr 2023 James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, Hongxia Jin

We show that C-LoRA not only outperforms several baselines for our proposed setting of text-to-image continual customization, which we refer to as Continual Diffusion, but that we achieve a new state-of-the-art in the well-established rehearsal-free continual learning setting for image classification.

Continual Learning Image Classification

To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinement

no code implementations6 Apr 2023 Yashas Malur Saidutta, Rakshith Sharma Srinivasa, Ching-Hua Lee, Chouchang Yang, Yilin Shen, Hongxia Jin

We show that existing deep keyword spotting mechanisms can be improved by Successive Refinement, where the system first classifies whether the input audio is speech or not, followed by whether the input is keyword-like or not, and finally classifies which keyword was uttered.

Keyword Spotting

ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation

no code implementations30 Jan 2023 Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor, Xin Eric Wang

Such object navigation tasks usually require large-scale training in visual environments with labeled objects, which generalizes poorly to novel objects in unknown environments.

Efficient Exploration Language Modelling +2

GOHSP: A Unified Framework of Graph and Optimization-based Heterogeneous Structured Pruning for Vision Transformer

no code implementations13 Jan 2023 Miao Yin, Burak Uzkent, Yilin Shen, Hongxia Jin, Bo Yuan

We first develop a graph-based ranking for measuring the importance of attention heads, and the extracted importance information is further integrated to an optimization-based procedure to impose the heterogeneous structured sparsity patterns on the ViT models.

Hybrid Rule-Neural Coreference Resolution System based on Actor-Critic Learning

no code implementations20 Dec 2022 Yu Wang, Hongxia Jin

A coreference resolution system is to cluster all mentions that refer to the same entity in a given context.

coreference-resolution

A Robust Semantic Frame Parsing Pipeline on a New Complex Twitter Dataset

no code implementations18 Dec 2022 Yu Wang, Hongxia Jin

In this paper, we introduce a robust semantic frame parsing pipeline that can handle both \emph{OOD} patterns and \emph{OOV} tokens in conjunction with a new complex Twitter dataset that contains long tweets with more \emph{OOD} patterns and \emph{OOV} tokens.

Semantic Frame Parsing Spoken Language Understanding

Neural Coreference Resolution based on Reinforcement Learning

no code implementations18 Dec 2022 Yu Wang, Hongxia Jin

The target of a coreference resolution system is to cluster all mentions that refer to the same entity in a given context.

Clustering coreference-resolution +2

Numerical Optimizations for Weighted Low-rank Estimation on Language Model

no code implementations2 Nov 2022 Ting Hua, Yen-Chang Hsu, Felicity Wang, Qian Lou, Yilin Shen, Hongxia Jin

However, standard SVD treats the parameters within the matrix with equal importance, which is a simple but unrealistic assumption.

Language Modelling

Explainable Slot Type Attentions to Improve Joint Intent Detection and Slot Filling

no code implementations19 Oct 2022 Kalpa Gunaratna, Vijay Srinivasan, Akhila Yerukola, Hongxia Jin

In this work, we propose a novel approach that: (i) learns to generate additional slot type specific features in order to improve accuracy and (ii) provides explanations for slot filling decisions for the first time in a joint NLU model.

Intent Detection Natural Language Understanding +2

A Closer Look at Knowledge Distillation with Features, Logits, and Gradients

no code implementations18 Mar 2022 Yen-Chang Hsu, James Smith, Yilin Shen, Zsolt Kira, Hongxia Jin

Knowledge distillation (KD) is a substantial strategy for transferring learned knowledge from one neural network model to another.

Incremental Learning Knowledge Distillation +2

MGA-VQA: Multi-Granularity Alignment for Visual Question Answering

no code implementations25 Jan 2022 Peixi Xiong, Yilin Shen, Hongxia Jin

In contrast to previous works, our model splits alignment into different levels to achieve learning better correlations without needing additional data and annotations.

Question Answering Visual Question Answering

Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding

no code implementations NAACL 2021 Ting Hua, Yilin Shen, Changsheng Zhao, Yen-Chang Hsu, Hongxia Jin

Most existing continual learning approaches suffer from low accuracy and performance fluctuation, especially when the distributions of old and new data are significantly different.

Continual Learning domain classification +1

Lite-MDETR: A Lightweight Multi-Modal Detector

no code implementations CVPR 2022 Qian Lou, Yen-Chang Hsu, Burak Uzkent, Ting Hua, Yilin Shen, Hongxia Jin

The key primitive is that Dictionary-Lookup-Transformormations (DLT) is proposed to replace Linear Transformation (LT) in multi-modal detectors where each weight in Linear Transformation (LT) is approximately factorized into a smaller dictionary, index, and coefficient.

object-detection Object Detection +3

Automatic Mixed-Precision Quantization Search of BERT

no code implementations30 Dec 2021 Changsheng Zhao, Ting Hua, Yilin Shen, Qian Lou, Hongxia Jin

Knowledge distillation, Weight pruning, and Quantization are known to be the main directions in model compression.

Knowledge Distillation Model Compression +2

ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphs

1 code implementation13 Dec 2021 Manas Gaur, Kalpa Gunaratna, Vijay Srinivasan, Hongxia Jin

To address this open problem, we propose Information SEEking Question generator (ISEEQ), a novel approach for generating ISQs from just a short user query, given a large text corpus relevant to the user query.

Information Retrieval Knowledge Graphs +3

Exploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data

no code implementations28 Oct 2021 Junjiao Tian, Yen-Change Hsu, Yilin Shen, Hongxia Jin, Zsolt Kira

We are the first to propose a method that works well across both OOD detection and calibration and under different types of shifts.

Out of Distribution (OOD) Detection

DictFormer: Tiny Transformer with Shared Dictionary

no code implementations ICLR 2022 Qian Lou, Ting Hua, Yen-Chang Hsu, Yilin Shen, Hongxia Jin

DictFormer significantly reduces the redundancy in the transformer's parameters by replacing the prior transformer's parameters with compact, shared dictionary, a few unshared coefficients, and indices.

Abstractive Text Summarization Language Modelling +2

Exploring Covariate and Concept Shift for Detection and Confidence Calibration of Out-of-Distribution Data

no code implementations29 Sep 2021 Junjiao Tian, Yen-Chang Hsu, Yilin Shen, Hongxia Jin, Zsolt Kira

To this end, we theoretically derive two score functions for OOD detection, the covariate shift score and concept shift score, based on the decomposition of KL-divergence for both scores, and propose a geometrically-inspired method (Geometric ODIN) to improve OOD detection under both shifts with only in-distribution data.

Out of Distribution (OOD) Detection

An Adversarial Learning based Multi-Step Spoken Language Understanding System through Human-Computer Interaction

no code implementations6 Jun 2021 Yu Wang, Yilin Shen, Hongxia Jin

In this paper, we introduce a novel multi-step spoken language understanding system based on adversarial learning that can leverage the multiround user's feedback to update slot values.

Dialogue State Tracking Semantic Frame Parsing +2

A Coarse to Fine Question Answering System based on Reinforcement Learning

no code implementations1 Jun 2021 Yu Wang, Hongxia Jin

In this paper, we present a coarse to fine question answering (CFQA) system based on reinforcement learning which can efficiently processes documents with different lengths by choosing appropriate actions.

Question Answering reinforcement-learning +1

Data Augmentation for Voice-Assistant NLU using BERT-based Interchangeable Rephrase

no code implementations EACL 2021 Akhila Yerukola, Mason Bretan, Hongxia Jin

We introduce a data augmentation technique based on byte pair encoding and a BERT-like self-attention model to boost performance on spoken language understanding tasks.

Data Augmentation intent-classification +5

Negative Data Augmentation

2 code implementations ICLR 2021 Abhishek Sinha, Kumar Ayush, Jiaming Song, Burak Uzkent, Hongxia Jin, Stefano Ermon

Empirically, models trained with our method achieve improved conditional/unconditional image generation along with improved anomaly detection capabilities.

Action Recognition Anomaly Detection +9

SAFENet: A Secure, Accurate and Fast Neural Network Inference

no code implementations ICLR 2021 Qian Lou, Yilin Shen, Hongxia Jin, Lei Jiang

A cryptographic neural network inference service is an efficient way to allow two parties to execute neural network inference without revealing either party’s data or model.

Modeling Token-level Uncertainty to Learn Unknown Concepts in SLU via Calibrated Dirichlet Prior RNN

no code implementations16 Oct 2020 Yilin Shen, Wenhu Chen, Hongxia Jin

We design a Dirichlet Prior RNN to model high-order uncertainty by degenerating as softmax layer for RNN model training.

slot-filling Slot Filling +1

Generating Dialogue Responses from a Semantic Latent Space

no code implementations EMNLP 2020 Wei-Jen Ko, Avik Ray, Yilin Shen, Hongxia Jin

Existing open-domain dialogue generation models are usually trained to mimic the gold response in the training set using cross-entropy loss on the vocabulary.

Dialogue Generation valid

PGLP: Customizable and Rigorous Location Privacy through Policy Graph

3 code implementations4 May 2020 Yang Cao, Yonghui Xiao, Shun Takagi, Li Xiong, Masatoshi Yoshikawa, Yilin Shen, Jinfei Liu, Hongxia Jin, Xiaofeng Xu

Third, we design a private location trace release framework that pipelines the detection of location exposure, policy graph repair, and private trajectory release with customizable and rigorous location privacy.

Cryptography and Security Computers and Society

Reward Constrained Interactive Recommendation with Natural Language Feedback

no code implementations4 May 2020 Ruiyi Zhang, Tong Yu, Yilin Shen, Hongxia Jin, Changyou Chen, Lawrence Carin

Text-based interactive recommendation provides richer user feedback and has demonstrated advantages over traditional interactive recommender systems.

Recommendation Systems reinforcement-learning +2

Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data

2 code implementations CVPR 2020 Yen-Chang Hsu, Yilin Shen, Hongxia Jin, Zsolt Kira

Deep neural networks have attained remarkable performance when applied to data that comes from the same distribution as that of the training set, but can significantly degrade otherwise.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

Text-Based Interactive Recommendation via Constraint-Augmented Reinforcement Learning

no code implementations NeurIPS 2019 Ruiyi Zhang, Tong Yu, Yilin Shen, Hongxia Jin, Changyou Chen

Text-based interactive recommendation provides richer user preferences and has demonstrated advantages over traditional interactive recommender systems.

Recommendation Systems reinforcement-learning +2

A Progressive Model to Enable Continual Learning for Semantic Slot Filling

no code implementations IJCNLP 2019 Yilin Shen, Xiangyu Zeng, Hongxia Jin

ProgModel consists of a novel context gate that transfers previously learned knowledge to a small size expanded component; and meanwhile enables this new component to be fast trained to learn from new data.

Continual Learning slot-filling +2

Fast Domain Adaptation of Semantic Parsers via Paraphrase Attention

no code implementations WS 2019 Avik Ray, Yilin Shen, Hongxia Jin

However, state-of-the art attention based neural parsers are slow to retrain which inhibits real time domain adaptation.

Domain Adaptation

Iterative Delexicalization for Improved Spoken Language Understanding

no code implementations15 Oct 2019 Avik Ray, Yilin Shen, Hongxia Jin

Recurrent neural network (RNN) based joint intent classification and slot tagging models have achieved tremendous success in recent years for building spoken language understanding and dialog systems.

intent-classification Intent Classification +1

SkillBot: Towards Automatic Skill Development via User Demonstration

no code implementations NAACL 2019 Yilin Shen, Avik Ray, Hongxia Jin, S Nama, eep

We present SkillBot that takes the first step to enable end users to teach new skills in personal assistants (PA).

Natural Language Understanding

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

no code implementations ICCV 2019 Ramprasaath R. Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck, Dhruv Batra, Devi Parikh

Many vision and language models suffer from poor visual grounding - often falling back on easy-to-learn language priors rather than basing their decisions on visual concepts in the image.

Image Captioning Question Answering +2

A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling

1 code implementation NAACL 2018 Yu Wang, Yilin Shen, Hongxia Jin

The most effective algorithms are based on the structures of sequence to sequence models (or "encoder-decoder" models), and generate the intents and semantic tags either using separate models or a joint model.

Intent Detection +4

A Variational Dirichlet Framework for Out-of-Distribution Detection

no code implementations ICLR 2019 Wenhu Chen, Yilin Shen, Hongxia Jin, William Wang

With the recently rapid development in deep learning, deep neural networks have been widely adopted in many real-life applications.

Out-of-Distribution Detection Variational Inference

User Information Augmented Semantic Frame Parsing using Coarse-to-Fine Neural Networks

no code implementations18 Sep 2018 Yilin Shen, Xiangyu Zeng, Yu Wang, Hongxia Jin

The results show that our approach leverages such simple user information to outperform state-of-the-art approaches by 0. 25% for intent detection and 0. 31% for slot filling using standard training data.

Intent Detection Semantic Frame Parsing +3

Robust Spoken Language Understanding via Paraphrasing

no code implementations17 Sep 2018 Avik Ray, Yilin Shen, Hongxia Jin

Learning intents and slot labels from user utterances is a fundamental step in all spoken language understanding (SLU) and dialog systems.

Spoken Language Understanding

CRUISE: Cold-Start New Skill Development via Iterative Utterance Generation

no code implementations ACL 2018 Yilin Shen, Avik Ray, Abhishek Patel, Hongxia Jin

We present a system, CRUISE, that guides ordinary software developers to build a high quality natural language understanding (NLU) engine from scratch.

Natural Language Understanding

Beyond Word Embeddings: Learning Entity and Concept Representations from Large Scale Knowledge Bases

no code implementations1 Jan 2018 Walid Shalaby, Wlodek Zadrozny, Hongxia Jin

We evaluate our concept embedding models on two tasks: (1) analogical reasoning, where we achieve a state-of-the-art performance of 91% on semantic analogies, (2) concept categorization, where we achieve a state-of-the-art performance on two benchmark datasets achieving categorization accuracy of 100% on one and 98% on the other.

Semantic Parsing Word Embeddings

Deep Neural Network Approximation using Tensor Sketching

no code implementations21 Oct 2017 Shiva Prasad Kasiviswanathan, Nina Narodytska, Hongxia Jin

Deep neural networks are powerful learning models that achieve state-of-the-art performance on many computer vision, speech, and language processing tasks.

Private Incremental Regression

no code implementations4 Jan 2017 Shiva Prasad Kasiviswanathan, Kobbi Nissim, Hongxia Jin

Our first contribution is a generic transformation of private batch ERM mechanisms into private incremental ERM mechanisms, based on a simple idea of invoking the private batch ERM procedure at some regular time intervals.

BIG-bench Machine Learning regression

Cannot find the paper you are looking for? You can Submit a new open access paper.