Search Results for author:
Found 128 papers, 52 papers with code
Automate Knowledge Concept Tagging on Math Questions with LLMs
Knowledge concept tagging for questions plays a crucial role in contemporary intelligent educational applications, including learning progress diagnosis, practice question recommendations, and course content organization.
Large Language Models for Education: A Survey and Outlook
The advent of Large Language Models (LLMs) has brought in a new era of possibilities in the realm of education.
RadioGAT: A Joint Model-based and Data-driven Framework for Multi-band Radiomap Reconstruction via Graph Attention Networks
Multi-band radiomap reconstruction (MB-RMR) is a key component in wireless communications for tasks such as spectrum management and network planning.
Content Knowledge Identification with Multi-Agent Large Language Models (LLMs)
Teachers' mathematical content knowledge (CK) is of vital importance and need in teacher professional development (PD) programs.
Driving Animatronic Robot Facial Expression From Speech
The proposed approach is capable of generating highly realistic, real-time facial expressions from speech on an animatronic face, significantly advancing robots' ability to replicate nuanced human expressions for natural interaction.
Finding the Missing Data: A BERT-inspired Approach Against Package Loss in Wireless Sensing
Despite the development of various deep learning methods for Wi-Fi sensing, package loss often results in noncontinuous estimation of the Channel State Information (CSI), which negatively impacts the performance of the learning models.
Rethinking Machine Unlearning for Large Language Models
We explore machine unlearning (MU) in the domain of large language models (LLMs), referred to as LLM unlearning.
Large Language Models as Agents in Two-Player Games
By formally defining the training processes of large language models (LLMs), which usually encompasses pre-training, supervised fine-tuning, and reinforcement learning with human feedback, within a single and unified machine learning paradigm, we can glean pivotal insights for advancing LLM technologies.
Boximator: Generating Rich and Controllable Motions for Video Synthesis
Its robust motion controllability is validated by drastic increases in the bounding box alignment metric.
Bringing Generative AI to Adaptive Learning in Education
The recent surge in generative AI technologies, such as large language models and diffusion models, have boosted the development of AI applications in various domains, including science, finance, and education.
TPRF: A Transformer-based Pseudo-Relevance Feedback Model for Efficient and Effective Retrieval
This paper considers Pseudo-Relevance Feedback (PRF) methods for dense retrievers in a resource constrained environment such as that of cheap cloud instances or embedded systems (e. g., smartphones and smartwatches), where memory and CPU are limited and GPUs are not present.
ReFT: Reasoning with Reinforced Fine-Tuning
ReFT first warmups the model with SFT, and then employs on-line reinforcement learning, specifically the PPO algorithm in this paper, to further fine-tune the model, where an abundance of reasoning paths are automatically sampled given the question and the rewards are naturally derived from the ground-truth answers.
Speech Translation with Large Language Models: An Industrial Practice
Given the great success of large language models (LLMs) across various tasks, in this paper, we introduce LLM-ST, a novel and effective speech translation model constructed upon a pre-trained LLM.
Feasibility Conditions for Mobile LiFi
Then, we derive the expression of LiFi transmission rate based on the m-pulse-amplitude-modulation (M-PAM).
Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation
In this paper, we extend the scope of this effectiveness by showing that visual robot manipulation can significantly benefit from large-scale video generative pre-training.
Ranked #2 on
Zero-shot Generalization
on CALVIN
(using extra training data)
Evaluation of Infrastructure-based Warning System on Driving Behaviors-A Roundabout Study
A real-world roundabout in Ann Arbor, Michigan was built in the co-simulation platform as the study area, and the merging scenarios were investigated.
Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation
A risk with these models is the potential generation of inappropriate content, such as biased or harmful images.
Make Pixels Dance: High-Dynamic Video Generation
Creating high-dynamic videos such as motion-rich actions and sophisticated visual effects poses a significant challenge in the field of artificial intelligence.
Ranked #3 on
Text-to-Video Generation
on UCF-101
Enhancing Multimodal Compositional Reasoning of Visual Language Models with Generative Negative Mining
Leveraging these generative hard negative samples, we significantly enhance VLMs' performance in tasks involving multimodal compositional reasoning.
Vision-Language Foundation Models as Effective Robot Imitators
We believe RoboFlamingo has the potential to be a cost-effective and easy-to-use solution for robotics manipulation, empowering everyone with the ability to fine-tune their own robotics policy.
Deep Concept Removal
We address the problem of concept removal in deep neural networks, aiming to learn representations that do not encode certain specified concepts (e. g., gender etc.)
Graph-level Representation Learning with Joint-Embedding Predictive Architectures
Joint-Embedding Predictive Architectures (JEPAs) have recently emerged as a novel and powerful technique for self-supervised representation learning.
Ranked #11 on
Graph Classification
on REDDIT-B
Design of Chain-of-Thought in Math Problem Solving
We also find that Python is a better choice of language than Wolfram for program CoTs.
Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment
However, a major challenge faced by practitioners is the lack of clear guidance on evaluating whether LLM outputs align with social norms, values, and regulations.
Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs
The most popular pipeline for learning on graphs with textual node attributes primarily relies on Graph Neural Networks (GNNs), and utilizes shallow text embedding as initial node representations, which has limitations in general knowledge and profound semantic understanding.
Sensing Aided Uplink Transmission in OTFS ISAC with Joint Parameter Association, Channel Estimation and Signal Detection
In this work, we study sensing-aided uplink transmission in an integrated sensing and communication (ISAC) vehicular network with the use of orthogonal time frequency space (OTFS) modulation.
Features Disentangled Semantic Broadcast Communication Networks
Instead of broadcasting all extracted features, the semantic encoder extracts the disentangled semantic features, and then only the users' intended semantic features are selected for broadcasting, which can further improve the transmission efficiency.
Task-oriented Explainable Semantic Communications
Furthermore, based on the $\beta $-variational autoencoder ($\beta $-VAE), we propose a practical explainable semantic communication system design, which simultaneously achieves semantic features selection and is robust against semantic channel noise.
Generative Diffusion Models on Graphs: Methods and Applications
Given the great success of diffusion models in image generation, increasing efforts have been made to leverage these techniques to advance graph generation in recent years.
Kainate receptor modulation by NETO2
Glutamate-gated kainate receptors (KARs) are ubiquitous in the central nervous system of vertebrates, mediate synaptic transmission on post-synapse, and modulate transmitter release on pre-synapse.
Disentangled Representation for Diversified Recommendations
This calls for a fine-grained understanding of a user's preferences over items, where one needs to recognize the user's choice is driven by the quality of the item itself, or the pre-selected attributes of the item.
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks
X-FM has one language encoder, one vision encoder, and one fusion encoder, as well as a new training method.
Ranked #3 on
Visual Grounding
on RefCOCO+ test B
Do DALL-E and Flamingo Understand Each Other?
To study this question, we propose a reconstruction task where Flamingo generates a description for a given image and DALL-E uses this description as input to synthesize a new image.
Joint Beamforming and PD Orientation Design for Mobile Visible Light Communications
To address this challenge, we propose an alternating optimization algorithm to obtain the transmit beamforming and the PD orientation.
AgAsk: An Agent to Help Answer Farmer's Questions From Scientific Documents
On the basis of these needs we release an information retrieval test collection comprising real questions, a large collection of scientific documents split in passages, and ground truth relevance assessments indicating which passages are relevant to each question.
MeSH Suggester: A Library and System for MeSH Term Suggestion for Systematic Review Boolean Query Construction
One challenge to creating an effective systematic review Boolean query is the selection of effective MeSH Terms to include in the query.
X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks
Vision language pre-training aims to learn alignments between vision and language from a large amount of data.
Ranked #1 on
Cross-Modal Retrieval
on Flickr30k
(using extra training data)
Learning to Counterfactually Explain Recommendations
The key idea is to train a surrogate model to learn the effect of removing a subset of user history on the recommendation.
Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributes
Our theoretical analyses show that directly using proxy models can give a false sense of (un)fairness.
Forgetting Fast in Recommender Systems
In this paper, we investigate fast machine unlearning techniques for recommender systems that can remove the effect of a small amount of training data from the recommendation model without incurring the full cost of retraining.
Biologically Inspired Neural Path Finding
The human brain can be considered to be a graphical structure comprising of tens of billions of biological neurons connected by synapses.
On Calibration of Graph Neural Networks for Node Classification
We investigate the calibration of graph neural networks for node classification, study the effect of existing post-processing calibration methods, and analyze the influence of model capacity, graph density, and a new loss function on calibration.
Directed Acyclic Transformer for Non-Autoregressive Machine Translation
Non-autoregressive Transformers (NATs) significantly reduce the decoding latency by generating all tokens in parallel.
How does Feedback Signal Quality Impact Effectiveness of Pseudo Relevance Feedback for Passage Retrieval?
Pseudo-Relevance Feedback (PRF) assumes that the top results retrieved by a first-stage ranker are relevant to the original query and uses them to improve the query representation for a second round of retrieval.
To Interpolate or not to Interpolate: PRF, Dense and Sparse Retrievers
In this paper we consider the problem of combining the relevance signals from sparse and dense retrievers in the context of Pseudo Relevance Feedback (PRF).
Self-Supervised Audio-and-Text Pre-training with Extremely Low-Resource Parallel Data
Our pre-training framework consists of the following components: (1) Intra-modal Denoising Auto-Encoding (IDAE), which is able to reconstruct input text (audio) representations from a noisy version of itself.
Implicit Feedback for Dense Passage Retrieval: A Counterfactual Approach
We then exploit such historic implicit interactions to improve the effectiveness of a DR. A key challenge that we study is the effect that biases in the click signal, such as position bias, have on the DRs.
A Neural-Symbolic Approach to Natural Language Understanding
Deep neural networks, empowered by pre-trained language models, have achieved remarkable results in natural language understanding (NLU) tasks.
Counterfactually Evaluating Explanations in Recommender Systems
Modern recommender systems face an increasing need to explain their recommendations.
Bridge the Gap between Supervised and Unsupervised Learning for Fine-Grained Classification
Unsupervised learning technology has caught up with or even surpassed supervised learning technology in general object classification (GOC) and person re-identification (re-ID).
Multimodal Entity Tagging with Multimodal Knowledge Base
We also develop a dataset for the problem using an existing MKB.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback: A Reproducibility Study
Finally, we contribute a study of the generalisability of the ANCE-PRF method when dense retrievers other than ANCE are used for the first round of retrieval and for encoding the PRF signal.
Disentangled Contrastive Learning on Graphs
Then we propose a novel factor-wise discrimination objective in a contrastive learning manner, which can force the factorized representations to independently reflect the expressive information from different latent factors.
Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts
Most existing methods in vision language pre-training rely on object-centric features extracted through object detection and make fine-grained alignments between the extracted features and texts.
Ranked #1 on
Image Retrieval
on Flickr30K 1K test
(using extra training data)
Self-Supervised Learning by Estimating Twin Class Distributions
To solve this problem, we propose to maximize the mutual information between the input and the class predictions.
Ranked #1 on
Image Classification
on Oxford-IIIT Pet Dataset
Fine-Grained Image Classification
Representation Learning
+5
The Tensor Brain: A Unified Theory of Perception, Memory and Semantic Decoding
Although memory appears to be about the past, its main purpose is to support the agent in the present and the future.
Text-to-Table: A New Way of Information Extraction
We first employ a seq2seq model fine-tuned from a pre-trained language model to perform the task.
CTAL: Pre-training Cross-modal Transformer for Audio-and-Language Representations
Existing audio-language task-specific predictive approaches focus on building complicated late-fusion mechanisms.
Secoco: Self-Correcting Encoding for Neural Machine Translation
This paper presents Self-correcting Encoding (Secoco), a framework that effectively deals with input noise for robust neural machine translation by introducing self-correcting predictors.
Pseudo Relevance Feedback with Deep Language Models and Dense Retrievers: Successes and Pitfalls
Text-based PRF results show that the use of PRF had a mixed effect on deep rerankers across different datasets.
Multi-Task Learning based Online Dialogic Instruction Detection with Pre-trained Language Models
In this work, we study computational approaches to detect online dialogic instructions, which are widely used to help students understand learning materials, and build effective study habits.
An Educational System for Personalized Teacher Recommendation in K-12 Online Classrooms
In this paper, we propose a simple yet effective solution to build practical teacher recommender systems for online one-on-one classes.
A Multimodal Machine Learning Framework for Teacher Vocal Delivery Evaluation
The quality of vocal delivery is one of the key indicators for evaluating teacher enthusiasm, which has been widely accepted to be connected to the overall course qualities.
Graphhopper: Multi-Hop Scene Graph Reasoning for Visual Question Answering
We conduct an experimental study on the challenging dataset GQA, based on both manually curated and automatically generated scene graphs.
Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification
With the contribution of the CCD and CRP, our CRCKD algorithm can distill the relational knowledge more comprehensively.
Evaluating Document Coherence Modelling
While pretrained language models ("LM") have driven impressive gains over morpho-syntactic and semantic tasks, their ability to model discourse and pragmatic phenomena is less clear.
Deep Convolutional Neural Networks to Predict Mutual Coupling Effects in Metasurfaces
Metasurfaces have provided a novel and promising platform for the realization of compact and large-scale optical devices.
A Sequence-to-Sequence Approach to Dialogue State Tracking
This paper is concerned with dialogue state tracking (DST) in a task-oriented dialogue system.
Ranked #1 on
Multi-domain Dialogue State Tracking
on SGD
Mathematical Word Problem Generation from Commonsense Knowledge Graph and Equations
There is an increasing interest in the use of mathematical word problem (MWP) generation in educational assessment.
AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization
In fact, both fine-grained and coarse-grained tokenizations have advantages and disadvantages for learning of pre-trained language models.
Superpixel-Guided Label Softening for Medical Image Segmentation
If a superpixel intersects with the annotation boundary, we consider a high probability of uncertain labeling within this area.
Fact-based Text Editing
We propose a novel text editing task, referred to as \textit{fact-based text editing}, in which the goal is to revise a given document to better describe the facts in a knowledge base (e. g., several triples).
Ranked #1 on
Fact-based Text Editing
on WebEdit
Scene Graph Reasoning for Visual Question Answering
We propose a novel method that approaches the task by performing context-driven, sequential reasoning based on the objects and their semantic and spatial relationships present in the scene.
Feature Statistics Guided Efficient Filter Pruning
Building compact convolutional neural networks (CNNs) with reliable performance is a critical but challenging task, especially when deploying them in real-world applications.
Siamese Neural Networks for Class Activity Detection
Classroom activity detection (CAD) aims at accurately recognizing speaker roles (either teacher or student) in classrooms.
Spelling Error Correction with Soft-Masked BERT
A state-of-the-art method for the task selects a character from a list of candidates for correction (including non-correction) at each position of the sentence on the basis of BERT, the language representation model.
Identifying At-Risk K-12 Students in Multimodal Online Environments: A Machine Learning Approach
We conduct a wide range of offline and online experiments to demonstrate the effectiveness of our approach.
A Freeform Dielectric Metasurface Modeling Approach Based on Deep Neural Networks
Metasurfaces have shown promising potentials in shaping optical wavefronts while remaining compact compared to bulky geometric optics devices.
Multimodal Learning For Classroom Activity Detection
The experimental results demonstrate the benefits of our approach on learning attention based neural network from classroom data with different modalities, and show our approach is able to outperform state-of-the-art baselines in terms of various evaluation metrics.
A Multimodal Alerting System for Online Class Quality Assurance
To warn the unqualified instructors and ensure the overall education quality, we build a monitoring and alerting system by utilizing multimodal information from the online environment.
Multifunctional Metasurface Design with a Generative Adversarial Network
Metasurfaces have enabled precise electromagnetic wave manipulation with strong potential to obtain unprecedented functionalities and multifunctional behavior in flat optical devices.
A Novel Modeling Approach for All-Dielectric Metasurfaces Using Deep Neural Networks
Metasurfaces have become a promising means for manipulating optical wavefronts in flat and high-performance optical devices.
Conversational Contextual Bandit: Algorithm and Application
Here, a key-term can relate to a subset of arms, for example, a category of articles in news recommendation.
Signal Demodulation with Machine Learning Methods for Physical Layer Visible Light Communications: Prototype Platform, Open Dataset and Algorithms
The dataset is available online, which contains eight types of modulated signals.
Word Embedding based Edit Distance
Text similarity calculation is a fundamental problem in natural language processing and related fields.
Unbiased LambdaMART: An Unbiased Pairwise Learning-to-Rank Algorithm
Although click data is widely used in search systems in practice, so far the inherent bias, most notably position bias, has prevented it from being used in training of a ranker for search, i. e., learning-to-rank.
Supervised and Semi-Supervised Deep Neural Networks for CSI-Based Authentication
In addition, we propose to use the convolutional recurrent neural network (CRNN)---a combination of the CNN and the RNN---to learn local and contextual information in CSI for user authentication.
Paraphrase Generation with Deep Reinforcement Learning
The generator, built as a sequence-to-sequence learning model, can produce paraphrases given a sentence.
Cascaded Attention based Unsupervised Information Distillation for Compressive Summarization
The attention weights are learned automatically by an unsupervised data reconstruction framework which can capture the sentence salience.
Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
In contrast, meta-learning learns from many related tasks a meta-learner that can learn a new task more accurately and faster with fewer examples, where the choice of meta-learners is crucial.
Chunk-Based Bi-Scale Decoder for Neural Machine Translation
In typical neural machine translation~(NMT), the decoder generates a sentence word by word, packing all linguistic granularities in the same time-scale of RNN.
Deep Active Learning for Dialogue Generation
We propose an online, end-to-end, neural generative conversational model for open-domain dialogue.
Coupling Distributed and Symbolic Execution for Natural Language Queries
Building neural networks to query a knowledge base (a table) with natural language is an emerging research topic in deep learning.
Neural Machine Translation with Reconstruction
Although end-to-end Neural Machine Translation (NMT) has achieved remarkable progress in the past two years, it suffers from a major drawback: translations generated by NMT systems often lack of adequacy.
Interactive Attention for Neural Machine Translation
Conventional attention-based Neural Machine Translation (NMT) conducts dynamic alignment in generating the target sentence.
Neural Machine Translation Advised by Statistical Machine Translation
Neural Machine Translation (NMT) is a new approach to machine translation that has made great progress in recent years.
Context Gates for Neural Machine Translation
In neural machine translation (NMT), generation of a target word depends on both source and target contexts.
Memory-enhanced Decoder for Neural Machine Translation
We propose to enhance the RNN decoder in a neural machine translator (NMT) with external memory, as a natural but powerful extension to the state in the decoding RNN.
Neural Machine Translation with External Phrase Memory
In this paper, we propose phraseNet, a neural machine translator with a phrase memory which stores phrase pairs in symbolic form, mined from corpus or specified by human experts.
A Novel Approach to Dropped Pronoun Translation
Finally, we integrate the above outputs into our translation system to recall missing pronouns by both extracting rules from the DP-labelled training data and translating the DP-generated input sentences.
Incorporating Copying Mechanism in Sequence-to-Sequence Learning
CopyNet can nicely integrate the regular way of word generation in the decoder with the new copying mechanism which can choose sub-sequences in the input sequence and put them at proper places in the output sequence.
Modeling Coverage for Neural Machine Translation
Attention mechanism has enhanced state-of-the-art Neural Machine Translation (NMT) by jointly learning to align and translate.
Neural Generative Question Answering
Empirical study shows the proposed model can effectively deal with the variations of questions and answers, and generate right and natural answers by referring to the facts in the knowledge-base.
Neural Enquirer: Learning to Query Tables with Natural Language
Neural Enquirer can be trained with gradient descent, with which not only the parameters of the controlling components and semantic parsing component, but also the embeddings of the tables and query words can be learned from scratch.
Towards Neural Network-based Reasoning
For example, it improves the accuracy on Path Finding(10K) from 33. 4% [6] to over 98%.
A Deep Memory-based Architecture for Sequence-to-Sequence Learning
We propose DEEPMEMORY, a novel deep architecture for sequence-to-sequence learning, which performs the task through a series of nonlinear transformations from the representation of the input sequence (e. g., a Chinese sentence) to the final output sequence (e. g., translation to English).
Learning to Answer Questions From Image Using Convolutional Neural Network
We demonstrate the efficacy of our proposed model on the DAQUAR and COCO-QA datasets, which are two benchmark datasets for the image QA, with the performances significantly outperforming the state-of-the-art.
Reader-Aware Multi-Document Summarization via Sparse Coding
We propose a new MDS paradigm called reader-aware multi-document summarization (RA-MDS).
Multimodal Convolutional Neural Networks for Matching Image and Sentence
In this paper, we propose multimodal convolutional neural networks (m-CNNs) for matching image and sentence.
Ranked #16 on
Image Retrieval
on Flickr30K 1K test
$gen$CNN: A Convolutional Architecture for Word Sequence Prediction
Different from previous work on neural network-based language modeling and generation (e. g., RNN or LSTM), we choose not to greedily summarize the history of words as a fixed length vector.
Convolutional Neural Network Architectures for Matching Natural Language Sentences
Semantic matching is of central importance to many natural language tasks \cite{bordes2014semantic, RetrievalQA}.
Ranked #3 on
Question Answering
on SemEvalCQA
Neural Responding Machine for Short-Text Conversation
We propose Neural Responding Machine (NRM), a neural network-based response generator for Short-Text Conversation.
Context-Dependent Translation Selection Using Convolutional Neural Network
We propose a novel method for translation selection in statistical machine translation, in which a convolutional neural network is employed to judge the similarity between a phrase pair in two languages.
Syntax-based Deep Matching of Short Texts
Many tasks in natural language processing, ranging from machine translation to question answering, can be reduced to the problem of matching two sentences or more generally two short texts.
Encoding Source Language with Convolutional Neural Network for Machine Translation
The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language model with a heuristically chosen source context window, achieving state-of-the-art performance in SMT.
A Parallel and Efficient Algorithm for Learning to Match
In this paper, we tackle this challenge with a novel parallel and efficient algorithm for feature-based matrix factorization.
An Information Retrieval Approach to Short Text Conversation
Human computer conversation is regarded as one of the most difficult problems in artificial intelligence.
A Deep Architecture for Matching Short Texts
Many machine learning problems can be interpreted as learning for matching two types of objects (e. g., images and captions, users and products, queries and documents).
Statistical Consistency of Top-k Ranking
This paper aims to analyze whether existing listwise ranking methods are statistically consistent in the top-k setting.
Ranking Measures and Loss Functions in Learning to Rank
We show that these loss functions are upper bounds of the measure-based ranking errors.
Global Ranking Using Continuous Conditional Random Fields
It can naturally represent the content information of objects as well as the relation information between objects, necessary for global ranking.
