Search Results for author:
Found 65 papers, 17 papers with code
Pseudo-Masked Language Models for Unified Language Model Pre-Training
We propose to pre-train a unified language model for both autoencoding and partially autoregressive language modeling tasks using a novel training procedure, referred to as a pseudo-masked language model (PMLM).
FedMAE: Federated Self-Supervised Learning with One-Block Masked Auto-Encoder
Latest federated learning (FL) methods started to focus on how to use unlabeled data in clients for training due to users' privacy concerns, high labeling costs, or lack of expertise.
Query2doc: Query Expansion with Large Language Models
This paper introduces a simple yet effective query expansion approach, denoted as query2doc, to improve both sparse and dense retrieval systems.
Real-time scheduling of renewable power systems through planning-based reinforcement learning
The well-trained scheduling agent significantly reduces renewable curtailment and load shedding, which are issues arising from traditional scheduling's reliance on inaccurate day-ahead forecasts.
FedIL: Federated Incremental Learning from Decentralized Unlabeled Data with Convergence Analysis
Most existing federated learning methods assume that clients have fully labeled data to train on, while in reality, it is hard for the clients to get task-specific labels due to users' privacy concerns, high labeling costs, or lack of expertise.
Random Padding Data Augmentation
Random Padding is parameter-free, simple to construct, and compatible with the majority of CNN-based recognition models.
Behind the Scenes: Density Fields for Single View Reconstruction
Currently, neural radiance fields (NeRFs) can capture true 3D including color but are too complex to be generated from a single image.
4Seasons: Benchmarking Visual SLAM and Long-Term Localization for Autonomous Driving in Challenging Conditions
In this paper, we present a novel visual SLAM and long-term localization benchmark for autonomous driving in challenging conditions based on the large-scale 4Seasons dataset.
Text Embeddings by Weakly-Supervised Contrastive Pre-training
This paper presents E5, a family of state-of-the-art text embeddings that transfer well to a wide range of tasks.
Title2Event: Benchmarking Open Event Extraction with a Large-scale Chinese Title Dataset
To the best of our knowledge, it is currently the largest manually-annotated Chinese dataset for open event extraction.
Terahertz Communications for Massive Connectivity and Security in 6G and Beyond Era
Terahertz (THz) communications (THzCom) has experienced a meteoric rise of interest, due to its benefits for ultra-high data rate transmission in the sixth generation (6G) and beyond era.
Adversarial Transformer for Repairing Human Airway Segmentation
Discontinuity in the delineation of peripheral bronchioles hinders the potential clinical application of automated airway segmentation models.
CCR: Facial Image Editing with Continuity, Consistency and Reversibility
In this work, we put forward three concepts and corresponding definitions: editing continuity, consistency, and reversibility.
Learning Diverse Document Representations with Deep Query Interactions for Dense Retrieval
In this paper, we propose a new dense retrieval model which learns diverse document representations with deep query interactions.
An Unsupervised Learning Approach for Spectrum Allocation in Terahertz Communication Systems
Numerical results demonstrate that comparing to existing approaches, our proposed unsupervised learning-based approach achieves a higher data rate, especially when the molecular absorption coefficient within the spectrum of interest varies in a highly non-linear manner.
Terahertz Communications for 6G and Beyond Wireless Networks: Challenges, Key Advancements, and Opportunities
The unprecedented increase in wireless data traffic, predicted to occur within the next decade, is motivating academia and industries to look beyond contemporary wireless standards and conceptualize the sixth-generation (6G) wireless networks.
Novel Spectrum Allocation Among Multiple Transmission Windows for Terahertz Communication Systems
We also show that a further data rate gain can be obtained by optimally determining the unused spectra at the edges of TWs, as compared to avoiding using pre-defined spectra at the edges of TWs.
SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval
In this paper, we propose SimLM (Similarity matching with Language Model pre-training), a simple yet effective pre-training method for dense passage retrieval.
Confidence Estimation Transformer for Long-term Renewable Energy Forecasting in Reinforcement Learning-based Power Grid Dispatching
The expansion of renewable energy could help realizing the goals of peaking carbon dioxide emissions and carbon neutralization.
FIRe: Fast Inverse Rendering using Directional and Signed Distance Functions
To this end, we learn a signed distance function (SDF) along with our DDF model to represent a class of shapes.
TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo
In this paper, we present TANDEM a real-time monocular tracking and dense mapping framework.
Spectrum Allocation with Adaptive Sub-band Bandwidth for Terahertz Communication Systems
Aided by numerical results, we show that by enabling and optimizing ASB, significantly higher throughput can be achieved as compared to adopting equal sub-band bandwidth, and this throughput gain is most profound when the power budget constraint is more stringent.
s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning
Pretrained bidirectional Transformers, such as BERT, have achieved significant improvements in a wide variety of language understanding tasks, while it is not straightforward to directly apply them for natural language generation.
SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysis
Specifically, in terms of usability, it is demanding for non-experts to implement deep learning models, obtain the right settings for the entire machine learning pipeline, manage models and datasets, and exploit external data sources all together.
xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering
Dense passage retrieval has been shown to be an effective approach for information retrieval tasks such as open domain question answering.
Coverage Analysis for 3D Terahertz Communication Systems
We conduct novel coverage probability analysis of downlink transmission in a three-dimensional (3D) terahertz (THz) communication (THzCom) system.
MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera
Unlike other multi-view stereo methods, MonoRec is able to reconstruct both static and moving objects by leveraging the predicted masks.
LM-Reloc: Levenberg-Marquardt Based Direct Visual Relocalization
The learned features significantly improve the robustness of direct image alignment, especially for relocalization across different conditions.
4Seasons: A Cross-Season Dataset for Multi-Weather SLAM in Autonomous Driving
We present a novel dataset covering seasonal and challenging perceptual conditions for autonomous driving.
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
In this work, we present an information-theoretic framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.
Ranked #16 on
Zero-Shot Cross-Lingual Transfer
on XTREME
Directional Modulation-Enabled Secure Transmission with Intelligent Reflecting Surface
We propose a new secure transmission scheme which uses directional modulation (DM) with artificial noise and is aided by the intelligent reflecting surface (IRS).
Coverage Analysis for 3D Terahertz Communication Systems with Blockage and Directional Antennas
We further show that the coverage performance improvement brought by increasing the antenna directivity at APs is higher than that brought by increasing the antenna directivity at UEs.
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
We propose D3VO as a novel framework for monocular visual odometry that exploits deep networks on three levels -- deep depth, pose and uncertainty estimation.
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
We propose to pre-train a unified language model for both autoencoding and partially autoregressive language modeling tasks using a novel training procedure, referred to as a pseudo-masked language model (PMLM).
Ranked #4 on
Question Generation
on SQuAD1.1
(using extra training data)
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
The small model (student) is trained by deeply mimicking the self-attention module, which plays a vital role in Transformer networks, of the large model (teacher).
Ranked #8 on
Zero-shot Text Search
on BEIR
Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension
Most machine reading comprehension (MRC) models separately handle encoding and matching with different network architectures.
Multi-Frame GAN: Image Enhancement for Stereo Visual Odometry in Low Light
We propose the concept of a multi-frame GAN (MFGAN) and demonstrate its potential as an image sequence enhancement for stereo visual odometry in low light conditions.
Hybrid Beamforming for Terahertz Multi-Carrier Systems over Frequency Selective Fading
We consider a three-dimensional wideband THz channel by incorporating the joint effect of molecular absorption, high sparsity, and multi-path fading, and consider the carrier frequency offset in multi-carrier systems.
Unified Language Model Pre-training for Natural Language Understanding and Generation
This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks.
Ranked #2 on
Generative Question Answering
on CoQA
(using extra training data)
DirectShape: Direct Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation
Scene understanding from images is a challenging problem encountered in autonomous driving.
Expected Density of Cooperative Bacteria in a 2D Quorum Sensing Based Molecular Communication System
The number of molecules observed at each randomly-distributed bacterium is first derived by characterizing the diffusion and degradation of signaling molecules within the population.
A Disease Diagnosis and Treatment Recommendation System Based on Big Data Mining and Cloud Computing
It is crucial to provide compatible treatment schemes for a disease according to various symptoms at different stages.
Attention-Guided Answer Distillation for Machine Reading Comprehension
Despite that current reading comprehension systems have achieved significant advancements, their promising performances are often obtained at the cost of making an ensemble of numerous models.
Read + Verify: Machine Reading Comprehension with Unanswerable Questions
Machine reading comprehension with unanswerable questions aims to abstain from answering when no answer can be inferred.
Ranked #11 on
Question Answering
on SQuAD2.0 dev
Sequential Copying Networks
Copying mechanism shows effectiveness in sequence-to-sequence based neural network models for text generation tasks, such as abstractive sentence summarization and question generation.
Neural Document Summarization by Jointly Learning to Score and Select Sentences
In this paper, we present a novel end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences.
Ranked #9 on
Extractive Text Summarization
on CNN / Daily Mail
Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry
To this end, we incorporate deep depth predictions into Direct Sparse Odometry (DSO) as direct virtual stereo measurements.
Gated Self-Matching Networks for Reading Comprehension and Question Answering
We first match the question and passage with gated attention-based recurrent networks to obtain the question-aware passage representation.
Ranked #35 on
Question Answering
on SQuAD1.1 dev
Sequence-to-Dependency Neural Machine Translation
Nowadays a typical Neural Machine Translation (NMT) model generates translations from left to right as a linear sequence, during which latent syntactic structures of the target sentences are not explicitly concerned.
S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension
We build the answer extraction model with state-of-the-art neural networks for single passage reading comprehension, and propose an additional task of passage ranking to help answer extraction in multiple passages.
Ambiguity set and learning via Bregman and Wasserstein
Construction of ambiguity set in robust optimization relies on the choice of divergences between probability distributions.
Relaxed Wasserstein with Applications to GANs
Wasserstein Generative Adversarial Networks (WGANs) provide a versatile class of models, which have attracted great attention in various applications.
Challenges in Monocular Visual Odometry: Photometric Calibration, Motion Bias and Rolling Shutter Effect
Monocular visual odometry (VO) and simultaneous localization and mapping (SLAM) have seen tremendous improvements in accuracy, robustness and efficiency, and have gained increasing popularity over recent years.
Monocular Visual Odometry
Simultaneous Localization and Mapping
Selective Encoding for Abstractive Sentence Summarization
We propose a selective encoding model to extend the sequence-to-sequence framework for abstractive sentence summarization.
Ranked #8 on
Text Summarization
on DUC 2004 Task 1
Neural Question Generation from Text: A Preliminary Study
Automatic question generation aims to generate questions from a text passage where the generated questions can be answered by certain sub-spans of the given passage.
Ranked #13 on
Question Generation
on SQuAD1.1
Improving Attention Modeling with Implicit Distortion and Fertility for Machine Translation
In neural machine translation, the attention mechanism facilitates the translation process by producing a soft alignment between the source sentence and the target sentence.
Jointly Modeling Topics and Intents with Global Order Structure
Modeling document structure is of great importance for discourse analysis and related applications.
Radical-Enhanced Chinese Character Embedding
We present a method to leverage radical for learning Chinese character embedding.
Brains and pseudorandom generators
In a pioneering classic, Warren McCulloch and Walter Pitts proposed a model of the central nervous system; motivated by EEG recordings of normal brain activity, Chv\' atal and Goldsmith asked whether or not this model can be engineered to provide pseudorandom number generators.
