Search Results for author: Nikolaos Pappas

Found 29 papers, 15 papers with code

On-Demand AoI Minimization in Resource-Constrained Cache-Enabled IoT Networks with Energy Harvesting Sensors

no code implementations28 Jan 2022 Mohammad Hatami, Markus Leinonen, Zheng Chen, Nikolaos Pappas, Marian Codreanu

We consider a resource-constrained IoT network, where multiple users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an energy harvesting sensor.

Sentence Bottleneck Autoencoders from Transformer Language Models

1 code implementation EMNLP 2021 Ivan Montero, Nikolaos Pappas, Noah A. Smith

Representation learning for text via pretraining a language model on a large corpus has become a standard starting point for building NLP systems.

Denoising Language Modelling +5

Finetuning Pretrained Transformers into RNNs

1 code implementation EMNLP 2021 Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith

Specifically, we propose a swap-then-finetune procedure: in an off-the-shelf pretrained transformer, we replace the softmax attention with its linear-complexity recurrent alternative and then finetune.

Language Modelling Machine Translation +1

Random Feature Attention

no code implementations ICLR 2021 Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, Lingpeng Kong

RFA can be used as a drop-in replacement for conventional softmax attention and offers a straightforward way of learning with recency bias through an optional gating mechanism.

Language Modelling Machine Translation +2

Autonomous Maintenance in IoT Networks via AoI-driven Deep Reinforcement Learning

no code implementations31 Dec 2020 George Stamatakis, Nikolaos Pappas, Alexandros Fragkiadakis, Apostolos Traganitis

Internet of Things (IoT) with its growing number of deployed devices and applications raises significant challenges for network maintenance procedures.

reinforcement-learning

Robot Trajectory Planning With QoS Constrained IRS-assisted Millimeter-Wave Communications

no code implementations27 Oct 2020 Cristian Tatino, Nikolaos Pappas, Di Yuan

We present a solution to the optimization problem that exploits mm-wave channel characteristics to decouple beamforming and trajectory optimizations.

Trajectory Planning Networking and Internet Architecture

Multilevel Text Alignment with Cross-Document Attention

1 code implementation EMNLP 2020 Xuhui Zhou, Nikolaos Pappas, Noah A. Smith

Text alignment finds application in tasks such as citation recommendation and plagiarism detection.

Citation Recommendation

Grounded Compositional Outputs for Adaptive Language Modeling

1 code implementation EMNLP 2020 Nikolaos Pappas, Phoebe Mulcaire, Noah A. Smith

To our knowledge, the result is the first word-level language model with a size that does not depend on the training vocabulary.

Language Modelling

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

5 code implementations ICML 2020 Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, François Fleuret

Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long sequences.

Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation

2 code implementations ICLR 2021 Jungo Kasai, Nikolaos Pappas, Hao Peng, James Cross, Noah A. Smith

We show that the speed disadvantage for autoregressive baselines compared to non-autoregressive methods has been overestimated in three aspects: suboptimal layer allocation, insufficient speed measurement, and lack of knowledge distillation.

Knowledge Distillation Machine Translation +1

Learning-Based Link Scheduling in Millimeter-wave Multi-connectivity Scenarios

no code implementations2 Mar 2020 Cristian Tatino, Nikolaos Pappas, Ilaria Malanchini, Lutz Ewe, Di Yuan

Multi-connectivity is emerging as a promising solution to provide reliable communications and seamless connectivity for the millimeter-wave frequency range.

Optimizing Information Freshness in a Multiple Access Channel with Heterogeneous Devices

no code implementations10 Oct 2019 Zheng Chen, Nikolaos Pappas, Emil Björnson, Erik G. Larsson

We formulate an optimization problem that aims at minimizing the average age of information (AoI) of the EH node subject to the queue stability condition of the grid-connected node.

Information Theory Networking and Internet Architecture Information Theory

Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains

no code implementations29 May 2019 Navid Rekabsaz, Nikolaos Pappas, James Henderson, Banriskhem K. Khonglah, Srikanth Madikeri

In this study, we propose a multilingual neural language model architecture, trained jointly on the domain-specific data of several low-resource languages.

Automatic Speech Recognition Transfer Learning +1

Deep Residual Output Layers for Neural Language Generation

1 code implementation14 May 2019 Nikolaos Pappas, James Henderson

Many tasks, including language generation, benefit from learning the structure of the output space, particularly when the space of output labels is large and the data is sparse.

Language Modelling Machine Translation +1

Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation

1 code implementation TACL 2018 Xiao Pu, Nikolaos Pappas, James Henderson, Andrei Popescu-Belis

We show that the concatenation of these vectors, and the use of a sense selection mechanism based on the weighted average of sense vectors, outperforms several baselines including sense-aware ones.

Machine Translation Translation +1

Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation

1 code implementation WS 2018 Nikolaos Pappas, Lesly Miculicich Werlen, James Henderson

The model is a generalized form of weight tying which shares parameters but allows learning a more flexible relationship with input word embeddings and allows the effective capacity of the output layer to be controlled.

Machine Translation Translation +1

GILE: A Generalized Input-Label Embedding for Text Classification

1 code implementation TACL 2019 Nikolaos Pappas, James Henderson

This forces their parametrization to be dependent on the label set size, and, hence, they are unable to scale to large label sets and generalize to unseen ones.

Classification General Classification +3

Multilingual Hierarchical Attention Networks for Document Classification

2 code implementations IJCNLP 2017 Nikolaos Pappas, Andrei Popescu-Belis

Hierarchical attention networks have recently achieved remarkable performance for document classification in a given language.

Classification Document Classification +2

Multilingual Visual Sentiment Concept Matching

no code implementations7 Jun 2016 Nikolaos Pappas, Miriam Redi, Mercan Topkara, Brendan Jou, Hongyi Liu, Tao Chen, Shih-Fu Chang

The impact of culture in visual emotion perception has recently captured the attention of multimedia research.

Word Embeddings

Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology

no code implementations16 Aug 2015 Brendan Jou, Tao Chen, Nikolaos Pappas, Miriam Redi, Mercan Topkara, Shih-Fu Chang

Our work expressly focuses on the uniqueness of culture and language in relation to human affect, specifically sentiment and emotion semantics, and how they manifest in social multimedia.

Cannot find the paper you are looking for? You can Submit a new open access paper.