Search Results for author:
Found 85 papers, 30 papers with code
Are Pre-trained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification.
CORI: CJKV Benchmark with Romanization Integration -- A step towards Cross-lingual Transfer Beyond Textual Scripts
Naively assuming English as a source language may hinder cross-lingual transfer for many languages by failing to consider the importance of language contact.
Bullion: A Column Store for Machine Learning
The past two decades have witnessed columnar storage revolutionizing data warehousing and analytics.
R^2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding
Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries.
$R^2$-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding
Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries.
Ranked #2 on
Highlight Detection
on QVHighlights
Cooperative Classification and Rationalization for Graph Generalization
To address these challenges, in this paper, we propose a Cooperative Classification and Rationalization (C2R) method, consisting of the classification and the rationalization module.
kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning
Task-Oriented Parsing (TOP) enables conversational assistants to interpret user commands expressed in natural language, transforming them into structured outputs that combine elements of both natural language and intent/slot tags.
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text
Moreover, a significant gap in the current landscape is the absence of a realistic benchmark for evaluating the effectiveness of grounding LLMs on heterogeneous knowledge sources (e. g., knowledge base and text).
CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks
While Chain-of-Thought prompting is popular in reasoning tasks, its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored.
VKIE: The Application of Key Information Extraction on Video Text
Extracting structured information from videos is critical for numerous downstream applications in the industry.
L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models
Recently, large language models (LLMs), especially those that are pretrained on code, have demonstrated strong capabilities in generating programs from natural language inputs in a few-shot or even zero-shot manner.
Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagation
In this work, we propose the first framework for restoring videos from all adverse weather conditions by developing a video adverse-weather-component suppression network (ViWS-Net).
Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tables
Furthermore, TAG-QA outperforms the end-to-end model T5 by 16% and 12% on BLEU-4 and PARENT F-score, respectively.
Investigating Answerability of LLMs for Long-Form Question Answering
As we embark on a new era of LLMs, it becomes increasingly crucial to understand their capabilities, limitations, and differences.
FedJudge: Federated Legal Large Language Model
To this end, in this paper, we propose the first Federated Legal Large Language Model (FedJudge) framework, which fine-tunes Legal LLMs efficiently and effectively.
XGen-7B Technical Report
Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context.
Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?
The multimedia community has shown a significant interest in perceiving and representing the physical world with multimodal pretrained neural network models, and among them, the visual-language pertaining (VLP) is, currently, the most captivating topic.
Modeling Uncertainty and Using Post-fusion as Fallback Improves Retrieval Augmented Generation with LLMs
This paper aims to fill this gap by investigating different methods of combining retrieved passages with LLMs to enhance answer generation.
Rethinking Data Perturbation and Model Stabilization for Semi-supervised Medical Image Segmentation
Studies on semi-supervised medical image segmentation (SSMIS) have seen fast progress recently.
Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning
Recent advanced methods in Natural Language Understanding for Task-oriented Dialogue (TOD) Systems (e. g., intent detection and slot filling) require a large amount of annotated data to achieve competitive performance.
System-Initiated Transitions from Chit-Chat to Task-Oriented Dialogues with Transition Info Extractor and Transition Sentence Generator
In this work, we study dialogue scenarios that start from chit-chat but eventually switch to task-related services, and investigate how a unified dialogue model, which can engage in both chit-chat and task-oriented dialogues, takes the initiative during the dialogue mode transition from chit-chat to task-oriented in a coherent and cooperative manner.
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI
Despite advancements in conversational AI, language models encounter challenges to handle diverse conversational tasks, and existing dialogue dataset collections often lack diversity and comprehensiveness.
Choice Models and Permutation Invariance: Demand Estimation in Differentiated Products Markets
We demonstrate how non-parametric estimators like neural nets can easily approximate such functionals and overcome the curse of dimensionality that is inherent in the non-parametric estimation of choice functions.
Unified Conversational Models with System-Initiated Transitions between Chit-Chat and Task-Oriented Dialogues
We contribute two efficient prompt models which can proactively generate a transition sentence to trigger system-initiated transitions in a unified dialogue model.
LMs: Understanding Code Syntax and Semantics for Code Analysis
We break down the abilities needed for artificial intelligence~(AI) models to address SE tasks related to code analysis into three categories: 1) syntax understanding, 2) static behavior understanding, and 3) dynamic behavior understanding.
HPE:Answering Complex Questions over Text by Hybrid Question Parsing and Execution
It comprises two central pillars: (1) We parse the question of varying complexity into an intermediate representation, named H-expression, which is composed of simple questions as the primitives and symbolic operations representing the relationships among them; (2) To execute the resulting H-expressions, we design a hybrid executor, which integrates the deterministic rules to translate the symbolic operations with a drop-in neural reader network to answer each decomposed simple question.
MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning
The first Multimodal Emotion Recognition Challenge (MER 2023) was successfully held at ACM Multimedia.
Timestamps as Prompts for Geography-Aware Location Recommendation
In this paper, we revisit the problem of location recommendation and point out that explicitly modeling temporal information is a great help when the model needs to predict not only the next location but also further locations.
End-to-End Personalized Next Location Recommendation via Contrastive User Preference Modeling
History encoder is designed to model mobility patterns from historical check-in sequences, while query generator explicitly learns user preferences to generate user-specific intention queries.
Just Noticeable Visual Redundancy Forecasting: A Deep Multimodal-driven Approach
Just noticeable difference (JND) refers to the maximum visual change that human eyes cannot perceive, and it has a wide range of applications in multimedia systems.
Locating the Sources of Sub-synchronous Oscillations Induced by the Control of Voltage Source Converters Based on Energy Structure and Nonlinearity Detection
The oscillation phenomena associated with the control of voltage source converters (VSCs) are widely concerning, and locating the source of these oscillations is crucial to suppressing them; therefore, this paper presents a locating scheme, based on the energy structure and nonlinearity detection.
Deep Hashing With Minimal-Distance-Separated Hash Centers
This paper presents an optimization method that finds hash centers with a constraint on the minimal distance between any pair of hash centers, which is non-trivial due to the non-convex nature of the problem.
OSAN: A One-Stage Alignment Network To Unify Multimodal Alignment and Unsupervised Domain Adaptation
An intuitive way to handle these two problems is to fulfill these tasks in two separate stages: aligning modalities followed by domain adaptation, or vice versa.
Discrete Point-wise Attack Is Not Enough: Generalized Manifold Adversarial Attack for Face Recognition
Classical adversarial attacks for Face Recognition (FR) models typically generate discrete examples for target identity with a single state image.
AugTriever: Unsupervised Dense Retrieval by Scalable Data Augmentation
Dense retrievers have made significant strides in text retrieval and open-domain question answering, even though most achievements were made possible only with large amounts of human supervision.
Equivalent Inertia Provided by Droop Control of Fast Frequency Regulation Resources
First, an equivalent-scenario-based method is proposed to evaluate the equivalent inertia provided by the droop control, which shows that the droop control with a constant droop coefficient provides time-variant equivalent inertia.
A Dynamic Equivalent Method for PMSG-WTG Based Wind Farms Considering wind Speeds and Fault Severities
In order to reduce the complexity of simulation of power systems including large-scale wind farms, it is critical to develop dynamic equivalent methods for wind farms which are applicable to the expected contingency analysis.
Grafting Pre-trained Models for Multimodal Headline Generation
In this paper, we propose a novel approach to graft the video encoder from the pre-trained video-language model on the generative pre-trained language model.
Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database
Parsing natural language questions into executable logical forms is a useful and interpretable way to perform question answering on structured data such as knowledge bases (KB) or databases (DB).
Unsupervised Extractive Summarization with Heterogeneous Graph Embeddings for Chinese Document
In the scenario of unsupervised extractive summarization, learning high-quality sentence representations is essential to select salient sentences from the input document.
Data-driven Emergency Frequency Control for Multi-Infeed Hybrid AC-DC System
This paper develops a fully data-driven linear quadratic regulator (LQR) for the HVDC to provide temporal frequency support.
Distributed Emergency Frequency Control Considering Transient Stability Constraints in Multi-Infeed Hybrid AC-DC System
Both of these two control laws can guarantee transient stability constraints, restore system frequency and achieve the defined optimal control objective.
ConceptNet infused DialoGPT for Underlying Commonsense Understanding and Reasoning in Dialogue Response Generation
The pre-trained conversational models still fail to capture the implicit commonsense (CS) knowledge hidden in the dialogue interaction, even though they were pre-trained with an enormous dataset.
OS-MSL: One Stage Multimodal Sequential Link Framework for Scene Segmentation and Classification
In this paper, from an alternate perspective to overcome the above challenges, we unite these two tasks into one task by a new form of predicting shots link: a link connects two adjacent shots, indicating that they belong to the same scene or category.
Contrastive Graph Multimodal Model for Text Classification in Videos
However, to our knowledge, there is no existing work focused on the second step of video text classification, which will limit the guidance to downstream tasks such as video indexing and browsing.
Incentive Mechanism Design for Emergency Frequency Control in Multi-Infeed Hybrid AC-DC System
In multi-infeed hybrid AC-DC (MIDC) systems, the emergency frequency control (EFC) with LCC-HVDC systems participating is of vital importance for system frequency stability.
UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection
Finding relevant moments and highlights in videos according to natural language queries is a natural and highly valuable common need in the current video content explosion era.
Ranked #3 on
Highlight Detection
on YouTube Highlights
Practical Evaluation of Adversarial Robustness via Adaptive Auto Attack
Specifically, by observing that adversarial examples to a specific defense model follow some regularities in their starting points, we design an Adaptive Direction Initialization strategy to speed up the evaluation.
Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks
Despite the usefulness of this service, we consider that recommending courses to users directly may neglect their varying degrees of expertise.
Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots
Few-shot table-to-text generation is a task of composing fluent and faithful sentences to convey table content using limited data.
Context Matters in Semantically Controlled Language Generation for Task-oriented Dialogue Systems
We utilize the pre-trained multi-context ConveRT model for context representation in a model trained from scratch; and leverage the immediate preceding user utterance for context generation in a model adapted from the pre-trained GPT-2.
Multi-Modality Microscopy Image Style Transfer for Nuclei Segmentation
Annotating microscopy images for nuclei segmentation is laborious and time-consuming.
Learning to Aggregate Multi-Scale Context for Instance Segmentation in Remote Sensing Images
The proposed model exploits three lightweight plug-and-play modules, namely dense feature pyramid network (DenseFPN), spatial context pyramid (SCP), and hierarchical region of interest extractor (HRoIE), to aggregate global visual context at feature, spatial, and instance domains, respectively.
A Probit Tensor Factorization Model For Relational Learning
A large number of models and algorithms have been proposed to perform link prediction, among which tensor factorization method has proven to achieve state-of-the-art performance in terms of computation efficiency and prediction accuracy.
Dense Hierarchical Retrieval for Open-Domain Question Answering
In this work, we propose Dense Hierarchical Retrieval (DHR), a hierarchical framework that can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic semantics specific to each passage.
Adversarial Attacks on ML Defense Models Competition
Due to the vulnerability of deep neural networks (DNNs) to adversarial examples, a large number of defense techniques have been proposed to alleviate this problem in recent years.
HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization
To capture the semantic graph structure from raw text, most existing summarization approaches are built on GNNs with a pre-trained model.
Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues using BERT
This paper presents an automatic method to evaluate the naturalness of natural language generation in dialogue systems.
Empathetic Dialogue Generation with Pre-trained RoBERTa-GPT2 and External Knowledge
One challenge for dialogue agents is to recognize feelings of the conversation partner and respond accordingly.
From Synthetic to Real: Image Dehazing Collaborating with Unlabeled Real Data
Our DID-Net predicts the three component maps by progressively integrating features across scales, and refines each map by passing an independent refinement network.
Ranked #6 on
Image Dehazing
on Haze4k
Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification.
Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation
In this paper, we claim that the syntactic and semantic structures among natural language are critical for non-autoregressive machine translation and can further improve the performance.
Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translation
In this paper, we claim that the syntactic and semantic structures among natural language are critical for non-autoregressive machine translation and can further improve the performance.
Real-Time Vanishing Point Detector Integrating Under-Parameterized RANSAC and Hough Transform
We propose a novel approach that integrates under-parameterized RANSAC (UPRANSAC) with Hough Transform to detect vanishing points (VPs) from un-calibrated monocular images.
Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning
Effectively filtering out noisy articles as well as bad answers is the key to improving extraction accuracy.
KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning
To promote the ability of commonsense reasoning for text generation, we propose a novel knowledge graph augmented pre-trained language generation model KG-BART, which encompasses the complex relations of concepts through the knowledge graph and produces more logical and natural sentences as output.
ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection
We consider the problem of Human-Object Interaction (HOI) Detection, which aims to locate and recognize HOI instances in the form of <human, action, object> in images.
Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering
Unlike most other QA tasks that focus on linguistic understanding, HeadQA requires deeper reasoning involving not only knowledge extraction, but also complex reasoning with healthcare knowledge.
Commonsense Evidence Generation and Injection in Reading Comprehension
Human tackle reading comprehension not only based on the given context itself but often rely on the commonsense beyond.
Revisiting Convolutional Neural Networks for Citywide Crowd Flow Analytics
This framework consists of three parts: 1) a local feature extraction module to learn representations for each region; 2) a global context module to extract global contextual priors and upsample them to generate the global features; and 3) a region-specific predictor based on tensor decomposition to provide customized predictions for each region, which is very parameter-efficient compared to previous methods.
Fine-Grained Urban Flow Inference
To tackle these issues, we develop a model entitled UrbanFM which consists of two major parts: 1) an inference network to generate fine-grained flow distributions from coarse-grained inputs that uses a feature extraction module and a novel distributional upsampling module; 2) a general fusion subnet to further boost the performance by considering the influence of different external factors.
Orthogonal Nonnegative Tucker Decomposition
In this paper, we study the nonnegative tensor data and propose an orthogonal nonnegative Tucker decomposition (ONTD).
Generative Question Refinement with Deep Reinforcement Learning in Retrieval-based QA System
To improve the quality and retrieval performance of the generated questions, we make two major improvements: 1) To better encode the semantics of ill-formed questions, we enrich the representation of questions with character embedding and the recent proposed contextual word embedding such as BERT, besides the traditional context-free word embeddings; 2) To make it capable to generate desired questions, we train the model with deep reinforcement learning techniques that considers an appropriate wording of the generation as an immediate reward and the correlation between generated question and answer as time-delayed long-term rewards.
Tucker Decomposition Network: Expressive Power and Comparison
Deep neural networks have achieved a great success in solving many machine learning and computer vision problems.
MMKG: Multi-Modal Knowledge Graphs
We present MMKG, a collection of three knowledge graphs that contain both numerical features and (links to) images for all entities as well as entity alignments between pairs of KGs.
UrbanFM: Inferring Fine-Grained Urban Flows
In this paper, we aim to infer the real-time and fine-grained crowd flows throughout a city based on coarse-grained observations.
Ranked #2 on
Fine-Grained Urban Flow Inference
on TaxiBJ-P4
Automatic trajectory measurement of large numbers of crowded objects
Complex motion patterns of natural systems, such as fish schools, bird flocks, and cell groups, have attracted great attention from scientists for years.
Product Title Refinement via Multi-Modal Generative Adversarial Learning
Nowadays, an increasing number of customers are in favor of using E-commerce Apps to browse and purchase products.
Data-driven Blockbuster Planning on Online Movie Knowledge Library
After a thorough investigation of an online movie knowledge library, a novel movie planning framework "Blockbuster Planning with Maximized Movie Configuration Acquaintance" (BigMovie) is introduced in this paper.
ContractFuzzer: Fuzzing Smart Contracts for Vulnerability Detection
Decentralized cryptocurrencies feature the use of blockchain to transfer values among peers on networks without central agency.
Software Engineering Cryptography and Security
Multi-View Multi-Graph Embedding for Brain Network Clustering Analysis
Network analysis of human brain connectivity is critically important for understanding brain function and disease states.
Beyond Binary Labels: Political Ideology Prediction of Twitter Users
Automatic political orientation prediction from social media posts has to date proven successful only in distinguishing between publicly declared liberals and conservatives in the US.
Action2Activity: Recognizing Complex Activities from Sensor Data
As compared to simple actions, activities are much more complex, but semantically consistent with a human's real life.
Simultaneous Feature Learning and Hash Coding with Deep Neural Networks
Similarity-preserving hashing is a widely-used method for nearest neighbour search in large-scale image retrieval tasks.
