Search Results for author:
Found 75 papers, 39 papers with code
Singlish Message Paraphrasing: A Joint Task of Creole Translation and Text Normalization
In this work, we introduce a joint paraphrasing task of creole translation and text normalization of Singlish messages, which can shed light on how to process other language varieties and dialects.
MoWE-Audio: Multitask AudioLLMs with Mixture of Weak Encoders
The rapid advancements in large language models (LLMs) have significantly enhanced natural language processing capabilities, facilitating the development of AudioLLMs that process and understand speech and audio inputs alongside text.
MEDSAGE: Enhancing Robustness of Medical Dialogue Summarization to ASR Errors with LLM-generated Synthetic Dialogues
Experimental results show that LLMs can effectively model ASR noise, and incorporating this noisy data into the training process significantly improves the robustness and accuracy of medical dialogue summarization systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs
We present the first systematic evaluation examining format bias in performance of large language models (LLMs).
PRESENT: Zero-Shot Text-to-Prosody Control
We attain 25. 3% hanzi CER and 13. 0% pinyin CER with the JETS model.
TTSlow: Slow Down Text-to-Speech with Efficiency Robustness Evaluations
In this paper, we propose TTSlow, a novel adversarial approach specifically tailored to slow down the speech generation process in TTS systems.
AudioBench: A Universal Benchmark for Audio Large Language Models
We introduce AudioBench, a universal benchmark designed to evaluate Audio Large Language Models (AudioLLMs).
Decompose and Aggregate: A Step-by-Step Interpretable Evaluation Framework
The acceleration of Large Language Models (LLMs) research has opened up new possibilities for evaluating generated texts.
CRAFT: Extracting and Tuning Cultural Instructions from the Wild
Large language models (LLMs) have rapidly evolved as the foundation of various natural language processing (NLP) applications.
CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment
This performance discrepancy mainly stems from the imbalanced distribution of training data across languages during pre-training and instruction tuning stages.
Resilience of Large Language Models for Noisy Instructions
As the rapidly advancing domain of natural language processing (NLP), large language models (LLMs) have emerged as powerful tools for interpreting human commands and generating text across various tasks.
Automatic Speech Recognition
Optical Character Recognition
+3
Personality-aware Student Simulation for Conversational Intelligent Tutoring Systems
Intelligent Tutoring Systems (ITSs) can provide personalized and self-paced learning experience.
Scaffolding Language Learning via Multi-modal Tutoring Systems with Pedagogical Instructions
Intelligent tutoring systems (ITSs) that imitate human tutors and aim to provide immediate and customized instructions or feedback to learners have shown their effectiveness in education.
Granular Change Accuracy: A More Accurate Performance Metric for Dialogue State Tracking
Current metrics for evaluating Dialogue State Tracking (DST) systems exhibit three primary limitations.
Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing
Large Language Models (LLMs) have demonstrated significant potential in handling complex reasoning tasks through step-by-step rationale generation.
LOCOST: State-Space Models for Long Document Abstractive Summarization
State-space models are a low-complexity alternative to transformers for encoding long sequences and capturing long-term dependencies.
Picking the Underused Heads: A Network Pruning Perspective of Attention Head Selection for Fusing Dialogue Coreference Information
In this work, we investigate the attention head selection and manipulation strategy for feature injection from a network pruning perspective, and conduct a case study on dialogue summarization.
Prompt Optimization via Adversarial In-Context Learning
We propose a new method, Adversarial In-Context Learning (adv-ICL), to optimize prompt for in-context learning (ICL) by employing one LLM as a generator, another as a discriminator, and a third as a prompt modifier.
ChOiRe: Characterizing and Predicting Human Opinions with Chain of Opinion Reasoning
Aligning language models (LMs) with human opinion is challenging yet vital to enhance their grasp of human values, preferences, and beliefs.
VerityMath: Advancing Mathematical Reasoning by Self-Verification Through Unit Consistency
Our findings indicate that our approach, which incorporates unit consistency, currently slightly underperforms compared to an approach that does not.
Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection
Data collection from manual labeling provides domain-specific and task-aligned supervision for data-driven approaches, and a critical mass of well-annotated resources is required to achieve reasonable performance in natural language processing tasks.
CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation
Annotated data plays a critical role in Natural Language Processing (NLP) in training models and evaluating their performance.
Finite-context Indexing of Restricted Output Space for NLP Models Facing Noisy Input
NLP models excel on tasks with clean inputs, but are less accurate with noisy inputs.
Instructive Dialogue Summarization with Query Aggregations
With the advancement of instruction-finetuned language models, we introduce instruction-tuning to dialogues to expand the capability set of dialogue summarization models.
Ranked #1 on
Text Summarization
on DialogSum
On Context Utilization in Summarization with Large Language Models
In this paper, we conduct the first comprehensive study on context utilization and position bias in summarization.
LLM Based Multi-Document Summarization Exploiting Main-Event Biased Monotone Submodular Content Extraction
Multi-document summarization is a challenging task due to its inherent subjective bias, highlighted by the low inter-annotator ROUGE-1 score of 0. 4 among DUC-2004 reference summaries.
SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning
We present SeaEval, a benchmark for multilingual foundation models.
Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation
A challenge in the Dialogue State Tracking (DST) field is adapting models to new domains without using any supervised data, zero-shot domain adaptation.
Multiple output samples per input in a single-output Gaussian process
The standard Gaussian Process (GP) only considers a single output sample per input in the training set.
Guiding Computational Stance Detection with Expanded Stance Triangle Framework
Stance detection determines whether the author of a piece of text is in favor of, against, or neutral towards a specified target, and can be used to gain valuable insights into social media.
Exploring Self-supervised Logic-enhanced Training for Large Language Models
Existing efforts to improve logical reasoning ability of language models have predominantly relied on supervised fine-tuning, hindering generalization to new domains and/or tasks.
Decomposed Prompting for Machine Translation Between Related Languages using Large Language Models
This study investigates machine translation between related languages i. e., languages within the same family that share linguistic characteristics such as word order and lexical similarity.
Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer
Adapting a large language model for multiple-attribute text style transfer via fine-tuning can be challenging due to the significant amount of computational resources and labeled data required for the specific task.
Modeling What-to-ask and How-to-ask for Answer-unaware Conversational Question Generation
In addition, we propose Conv-Distinct, a novel evaluation metric for CQG, to evaluate the diversity of the generated conversation from a context.
SNIPER Training: Single-Shot Sparse Training for Text-to-Speech
Text-to-speech (TTS) models have achieved remarkable naturalness in recent years, yet like most deep neural models, they have more parameters than necessary.
Are Current Task-oriented Dialogue Systems Able to Satisfy Impolite Users?
We hope by releasing the impolite dialogue corpus and establishing the benchmark evaluations, more researchers are encouraged to investigate this new challenging research task.
Towards Summary Candidates Fusion
To bypass this limitation, we propose a new paradigm in second-stage abstractive summarization called SummaFusion that fuses several summary candidates to produce a novel abstractive second-stage summary.
EPIC TTS Models: Empirical Pruning Investigations Characterizing Text-To-Speech Models
In this work, we seek to answer the question: what are the characteristics of selected sparse techniques on the performance and model complexity?
CoHS-CQG: Context and History Selection for Conversational Question Generation
While previous studies mainly focus on how to model the flow and alignment of the conversation, there has been no thorough study to date on which parts of the context and history are necessary for the model.
Multi-Document Summarization with Centroid-Based Pretraining
In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary.
A Transformer-based Neural Language Model that Synthesizes Brain Activation Maps from Free-Form Text Queries
In this paper, we present Text2Brain, an easy to use tool for synthesizing brain activation maps from open-ended text queries.
Multimodal Dialogue State Tracking
Specifically, we introduce a novel dialogue state tracking task to track the information of visual objects that are mentioned in video-grounded dialogues.
Domain-specific Language Pre-training for Dialogue Comprehension on Clinical Inquiry-Answering Conversations
There is growing interest in the automated extraction of relevant information from clinical dialogues.
Learning from Bootstrapping and Stepwise Reinforcement Reward: A Semi-Supervised Framework for Text Style Transfer
To take advantage of both supervised and unsupervised paradigms and tackle the challenges, in this work, we propose a semi-supervised framework for text style transfer.
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization
Sequence-to-sequence neural networks have recently achieved great success in abstractive summarization, especially through fine-tuning large pre-trained language models on the downstream dataset.
Ranked #2 on
Document Summarization
on CNN / Daily Mail
Large-Scale Acoustic Characterization of Singaporean Children's English Pronunciation
In tense and lax vowel pairs, we also consistently observe that the distinction is less conspicuous for Singaporean children compared to the other speaker groups.
Progressive Continual Learning for Spoken Keyword Spotting
Catastrophic forgetting is a thorny challenge when updating keyword spotting (KWS) models after deployment.
DMRST: A Joint Framework for Document-Level Multilingual RST Discourse Segmentation and Parsing
While previous work significantly improves the performance of RST discourse parsing, they are not readily applicable to practical use cases: (1) EDU segmentation is not integrated into most existing tree parsing frameworks, thus it is not straightforward to apply such models on newly-coming data.
Improving Multi-Party Dialogue Discourse Parsing via Domain Integration
While multi-party conversations are often less structured than monologues and documents, they are implicitly organized by semantic level correlations across the interactive turns, and dialogue discourse analysis can be applied to predict the dependency structure and relations between the elementary discourse units, and provide feature-rich structural information for downstream tasks.
Ranked #4 on
Discourse Parsing
on STAC
Text2Brain: Synthesis of Brain Activation Maps from Free-form Text Query
In this work, we propose Text2Brain, a neural network approach for coordinate-based meta-analysis of neuroimaging studies to synthesize brain activation maps from open-ended text queries.
Controllable Neural Dialogue Summarization with Personal Named Entity Planning
The conditional sequences are modulated to decide what types of information or what perspective to focus on when forming summaries to tackle the under-constrained problem in summarization tasks.
Dynamic Sliding Window for Meeting Summarization
In this work, we first analyze the linguistic characteristics of meeting transcripts on a representative corpus, and find that the sentences comprising the summary correlate with the meeting agenda.
Multilingual Speech Evaluation: Case Studies on English, Malay and Tamil
While speech evaluation on English has been popular, automatic speech scoring on low resource languages remains challenging.
$C^3$: Compositional Counterfactual Contrastive Learning for Video-grounded Dialogues
Video-grounded dialogue systems aim to integrate video understanding and dialogue understanding to generate responses that are relevant to both the dialogue and video context.
Coreference-Aware Dialogue Summarization
Summarizing conversations via neural approaches has been gaining research traction lately, yet it is still challenging to obtain practical solutions.
VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Tasks
Neural module networks (NMN) have achieved success in image-grounded tasks such as Visual Question Answering (VQA) on synthetic images.
Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues
PDC model then learns to predict reasoning paths over this semantic graph.
N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking
Augmentation of task-oriented dialogues has followed standard methods used for plain-text such as back-translation, word-level manipulation, and paraphrasing despite its richly annotated structure.
Senone-aware Adversarial Multi-task Training for Unsupervised Child to Adult Speech Adaptation
Acoustic modeling for child speech is challenging due to the high acoustic variability caused by physiological differences in the vocal tract.
VilNMN: A Neural Module Network approach to Video-Grounded Language Tasks
Neural module networks (NMN) have achieved success in image-grounded tasks such as question answering (QA) on synthetic images.
An End-to-End Document-Level Neural Discourse Parser Exploiting Multi-Granularity Representations
Document-level discourse parsing, in accordance with the Rhetorical Structure Theory (RST), remains notoriously challenging.
Multilingual Neural RST Discourse Parsing
Text discourse parsing plays an important role in understanding information flow and argumentative structure in natural language.
BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues
Video-grounded dialogues are very challenging due to (i) the complexity of videos which contain both spatial and temporal variations, and (ii) the complexity of user utterances which query different segments and/or different objects in videos over multiple dialogue turns.
Domain-shift Conditioning using Adaptable Filtering via Hierarchical Embeddings for Robust Chinese Spell Check
Spell check is a useful application which processes noisy human-generated text.
Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization
In this paper, we propose a neural framework that can flexibly control summary generation by introducing a set of sub-aspect functions (i. e. importance, diversity, position).
UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues
Building an end-to-end conversational agent for multi-domain task-oriented dialogues has been an open challenge for two main reasons.
Multimodal Transformer with Pointer Network for the DSTC8 AVSD Challenge
Audio-Visual Scene-Aware Dialog (AVSD) is an extension from Video Question Answering (QA) whereby the dialogue agent is required to generate natural language responses to address user queries and carry on conversations.
Hierarchical Character Embeddings: Learning Phonological and Semantic Representations in Languages of Logographic Origin using Recursive Neural Networks
Using recursive neural network imposes a prior on the mapping from logographs to embeddings since the network must read in the sub-units in logographs according to the order specified by the recursive structures.
Joint Learning of Word and Label Embeddings for Sequence Labelling in Spoken Language Understanding
We propose an architecture to jointly learn word and label embeddings for slot filling in spoken language understanding.
Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversations
Such linguistic characteristics of dialogue topics make sentence-level extractive summarization approaches used in spoken documents ill-suited for summarizing conversations.
Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems
Developing Video-Grounded Dialogue Systems (VGDS), where a dialogue is conducted based on visual and audio aspects of a given video, is significantly more challenging than traditional image or text-grounded dialogue systems because (1) feature space of videos span across multiple picture frames, making it difficult to obtain semantic information; and (2) a dialogue agent must perceive and process information from different modalities (audio, video, caption, etc.)
Ranked #4 on
Response Generation
on SIMMC2.0
Fast Prototyping a Dialogue Comprehension System for Nurse-Patient Conversations on Symptom Monitoring
Data for human-human spoken dialogues for research and development are currently very limited in quantity, variety, and sources; such data are even scarcer in healthcare.
Phonology-Augmented Statistical Framework for Machine Transliteration using Limited Linguistic Resources
The problem is compounded by the limited linguistic resources available when converting foreign words to transliterated words in the target language.
Multimodal neural pronunciation modeling for spoken languages with logographic origin
Graphemes of most languages encode pronunciation, though some are more explicit than others.
