Search Results for author: Nikolaos Pappas

Found 48 papers, 17 papers with code

Semantic-Aware Remote Estimation of Multiple Markov Sources Under Constraints

no code implementations25 Mar 2024 Jiping Luo, Nikolaos Pappas

This paper studies semantic-aware communication for remote estimation of multiple Markov sources over a lossy and rate-constrained channel.

Q-Learning Scheduling

Adaptive Federated Learning Over the Air

no code implementations11 Mar 2024 Chenhao Wang, Zihan Chen, Nikolaos Pappas, Howard H. Yang, Tony Q. S. Quek, H. Vincent Poor

In contrast, an Adam-like algorithm converges at the $\mathcal{O}( 1/T )$ rate, demonstrating its advantage in expediting the model training process.

Federated Learning

MAGID: An Automated Pipeline for Generating Synthetic Multi-modal Datasets

no code implementations5 Mar 2024 Hossein Aboutalebi, Hwanjun Song, Yusheng Xie, Arshit Gupta, Justin Sun, Hang Su, Igor Shalyminov, Nikolaos Pappas, Siffi Singh, Saab Mansour

Development of multimodal interactive systems is hindered by the lack of rich, multimodal (text, images) conversational data, which is needed in large quantities for LLMs.

Image-text matching Retrieval +1

Semantic Text Transmission via Prediction with Small Language Models: Cost-Similarity Trade-off

no code implementations1 Mar 2024 Bhavani A Madhabhavi, Gangadhar Karevvanavar, Rajshekhar V Bhat, Nikolaos Pappas

We obtain $(\bar{c}, \bar{s})$ pairs for neural language and first-order Markov chain-based small language models (SLM) for prediction, using both a threshold policy that transmits a word if its cosine similarity with that predicted/completed at the destination is below a threshold, and a periodic policy, which transmits words after a specific interval and predicts/completes the words in between, at the destination.

Scheduling Semantic Similarity +1

DeAL: Decoding-time Alignment for Large Language Models

no code implementations5 Feb 2024 James Y. Huang, Sailik Sengupta, Daniele Bonadiman, Yi-An Lai, Arshit Gupta, Nikolaos Pappas, Saab Mansour, Katrin Kirchhoff, Dan Roth

Current work focuses on alignment at model training time, through techniques such as Reinforcement Learning with Human Feedback (RLHF).

Goal-Oriented Communication, Estimation, and Control over Bidirectional Wireless Links

no code implementations26 Dec 2023 Jie Cao, Ernest Kurniawan, Amnart Boonkajay, Nikolaos Pappas, Sumei Sun, Petar Popovski

This status information, such as dynamic plant state and Markov Process-based context information, is then received/estimated by the controller for remote control.

Scheduling

Analysis of Age of Information in Non-terrestrial Networks

no code implementations6 Oct 2023 Yanwu Lu, Howard Yang, Nikolaos Pappas, Giovanni Geraci, Chuan Ma, Tony Q. S. Quek

Our work fills a gap in the literature by providing a comprehensive analysis of AoI in NTN and offers new insights into the performance of LEO satellite networks.

Experimental Study of Transport Layer Protocols for Wireless Networked Control Systems

no code implementations18 Jun 2023 Polina Kutsevol, Onur Ayan, Nikolaos Pappas, Wolfgang Kellerer

In contrast, in WNCSs, the efficiency of the network is measured by its ability to facilitate control applications, and the data transmission rate should be limited to avoid network congestion.

Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing

no code implementations24 May 2023 Shufan Wang, Sebastien Jean, Sailik Sengupta, James Gung, Nikolaos Pappas, Yi Zhang

In executable task-oriented semantic parsing, the system aims to translate users' utterances in natural language to machine-interpretable programs (API calls) that can be executed according to pre-defined API specifications.

In-Context Learning Retrieval +2

Conversation Style Transfer using Few-Shot Learning

no code implementations16 Feb 2023 Shamik Roy, Raphael Shu, Nikolaos Pappas, Elman Mansimov, Yi Zhang, Saab Mansour, Dan Roth

Conventional text style transfer approaches focus on sentence-level style transfer without considering contextual information, and the style is described with attributes (e. g., formality).

Few-Shot Learning In-Context Learning +5

Backward Compatibility During Data Updates by Weight Interpolation

no code implementations25 Jan 2023 Raphael Schumann, Elman Mansimov, Yi-An Lai, Nikolaos Pappas, Xibin Gao, Yi Zhang

This method interpolates between the weights of the old and new model and we show in extensive experiments that it reduces negative flips without sacrificing the improved accuracy of the new model.

regression

Task-Oriented and Semantics-Aware 6G Networks

no code implementations17 Oct 2022 Hui Zhou, Xiaonan Liu, Yansha Deng, Nikolaos Pappas, Arumugam Nallanathan

In this article, we propose a generic task-oriented and semantics-aware (TOSA) communication framework for various tasks with diverse data types, which incorporates both semantic level information and effectiveness-aware performance metrics.

Modeling Context With Linear Attention for Scalable Document-Level Translation

1 code implementation16 Oct 2022 Zhaofeng Wu, Hao Peng, Nikolaos Pappas, Noah A. Smith

Document-level machine translation leverages inter-sentence dependencies to produce more coherent and consistent translations.

Document Level Machine Translation Document Translation +4

On-Demand AoI Minimization in Resource-Constrained Cache-Enabled IoT Networks with Energy Harvesting Sensors

no code implementations28 Jan 2022 Mohammad Hatami, Markus Leinonen, Zheng Chen, Nikolaos Pappas, Marian Codreanu

We consider a resource-constrained IoT network, where multiple users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an energy harvesting sensor.

Sentence Bottleneck Autoencoders from Transformer Language Models

1 code implementation EMNLP 2021 Ivan Montero, Nikolaos Pappas, Noah A. Smith

Representation learning for text via pretraining a language model on a large corpus has become a standard starting point for building NLP systems.

Denoising Language Modelling +6

Finetuning Pretrained Transformers into RNNs

1 code implementation EMNLP 2021 Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith

Specifically, we propose a swap-then-finetune procedure: in an off-the-shelf pretrained transformer, we replace the softmax attention with its linear-complexity recurrent alternative and then finetune.

Language Modelling Machine Translation +1

Random Feature Attention

no code implementations ICLR 2021 Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, Lingpeng Kong

RFA can be used as a drop-in replacement for conventional softmax attention and offers a straightforward way of learning with recency bias through an optional gating mechanism.

Language Modelling Machine Translation +3

Autonomous Maintenance in IoT Networks via AoI-driven Deep Reinforcement Learning

no code implementations31 Dec 2020 George Stamatakis, Nikolaos Pappas, Alexandros Fragkiadakis, Apostolos Traganitis

Internet of Things (IoT) with its growing number of deployed devices and applications raises significant challenges for network maintenance procedures.

reinforcement-learning Reinforcement Learning (RL)

Robot Trajectory Planning With QoS Constrained IRS-assisted Millimeter-Wave Communications

no code implementations27 Oct 2020 Cristian Tatino, Nikolaos Pappas, Di Yuan

We present a solution to the optimization problem that exploits mm-wave channel characteristics to decouple beamforming and trajectory optimizations.

Trajectory Planning Networking and Internet Architecture

Grounded Compositional Outputs for Adaptive Language Modeling

1 code implementation EMNLP 2020 Nikolaos Pappas, Phoebe Mulcaire, Noah A. Smith

To our knowledge, the result is the first word-level language model with a size that does not depend on the training vocabulary.

Language Modelling

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

5 code implementations ICML 2020 Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, François Fleuret

Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long sequences.

D4RL Language Modelling +1

Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation

2 code implementations ICLR 2021 Jungo Kasai, Nikolaos Pappas, Hao Peng, James Cross, Noah A. Smith

We show that the speed disadvantage for autoregressive baselines compared to non-autoregressive methods has been overestimated in three aspects: suboptimal layer allocation, insufficient speed measurement, and lack of knowledge distillation.

Knowledge Distillation Machine Translation +1

Learning-Based Link Scheduling in Millimeter-wave Multi-connectivity Scenarios

no code implementations2 Mar 2020 Cristian Tatino, Nikolaos Pappas, Ilaria Malanchini, Lutz Ewe, Di Yuan

Multi-connectivity is emerging as a promising solution to provide reliable communications and seamless connectivity for the millimeter-wave frequency range.

Scheduling

Optimizing Information Freshness in a Multiple Access Channel with Heterogeneous Devices

no code implementations10 Oct 2019 Zheng Chen, Nikolaos Pappas, Emil Björnson, Erik G. Larsson

We formulate an optimization problem that aims at minimizing the average age of information (AoI) of the EH node subject to the queue stability condition of the grid-connected node.

Information Theory Networking and Internet Architecture Information Theory

Deep Residual Output Layers for Neural Language Generation

1 code implementation14 May 2019 Nikolaos Pappas, James Henderson

Many tasks, including language generation, benefit from learning the structure of the output space, particularly when the space of output labels is large and the data is sparse.

Language Modelling Machine Translation +1

Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation

1 code implementation TACL 2018 Xiao Pu, Nikolaos Pappas, James Henderson, Andrei Popescu-Belis

We show that the concatenation of these vectors, and the use of a sense selection mechanism based on the weighted average of sense vectors, outperforms several baselines including sense-aware ones.

Clustering Machine Translation +3

Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation

1 code implementation WS 2018 Nikolaos Pappas, Lesly Miculicich Werlen, James Henderson

The model is a generalized form of weight tying which shares parameters but allows learning a more flexible relationship with input word embeddings and allows the effective capacity of the output layer to be controlled.

Machine Translation Translation +1

GILE: A Generalized Input-Label Embedding for Text Classification

1 code implementation TACL 2019 Nikolaos Pappas, James Henderson

This forces their parametrization to be dependent on the label set size, and, hence, they are unable to scale to large label sets and generalize to unseen ones.

General Classification Multi-Task Learning +3

Multilingual Hierarchical Attention Networks for Document Classification

2 code implementations IJCNLP 2017 Nikolaos Pappas, Andrei Popescu-Belis

Hierarchical attention networks have recently achieved remarkable performance for document classification in a given language.

Classification Computational Efficiency +3

Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology

no code implementations16 Aug 2015 Brendan Jou, Tao Chen, Nikolaos Pappas, Miriam Redi, Mercan Topkara, Shih-Fu Chang

Our work expressly focuses on the uniqueness of culture and language in relation to human affect, specifically sentiment and emotion semantics, and how they manifest in social multimedia.

Cultural Vocal Bursts Intensity Prediction

Cannot find the paper you are looking for? You can Submit a new open access paper.