no code implementations • 21 Mar 2024 • Maxime Peyrard, Martin Josifoski, Robert West
We refer to these orchestrated interactions among semantic processors, optimizing and searching in semantic space, as semantic decoding algorithms.
no code implementations • 16 Feb 2024 • Mohammad Hossein Amani, Nicolas Mario Baldwin, Amin Mansouri, Martin Josifoski, Maxime Peyrard, Robert West
Traditional language models, adept at next-token prediction in text sequences, often struggle with transduction tasks between distinct symbolic systems, particularly when parallel data is scarce.
no code implementations • 18 Jan 2024 • Saibo Geng, Berkay Döner, Chris Wendler, Martin Josifoski, Robert West
This paper introduces sketch-guided constrained decoding (SGCD), a novel approach to constrained decoding for blackbox LLMs, which operates without access to the logits of the blackbox LLM.
1 code implementation • 9 Jan 2024 • Tim R. Davidson, Veniamin Veselovsky, Martin Josifoski, Maxime Peyrard, Antoine Bosselut, Michal Kosinski, Robert West
We introduce an approach to evaluate language model (LM) agency using negotiation games.
1 code implementation • 4 Dec 2023 • Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kiciman, Hamid Palangi, Barun Patra, Robert West
Yet the mechanisms underlying this contextual grounding remain unknown, especially in situations where contextual information contradicts factual knowledge stored in the parameters, which LLMs also excel at recalling.
2 code implementations • 2 Aug 2023 • Martin Josifoski, Lars Klein, Maxime Peyrard, Nicolas Baldwin, Yifei Li, Saibo Geng, Julian Paul Schnitzler, Yuxing Yao, Jiheng Wei, Debjit Paul, Robert West
To support rapid and rigorous research, we introduce the aiFlows library embodying Flows.
no code implementations • 24 May 2023 • Veniamin Veselovsky, Manoel Horta Ribeiro, Akhil Arora, Martin Josifoski, Ashton Anderson, Robert West
Large Language Models (LLMs) have democratized synthetic data generation, which in turn has the potential to simplify and broaden a wide gamut of NLP tasks.
2 code implementations • 23 May 2023 • Saibo Geng, Martin Josifoski, Maxime Peyrard, Robert West
In this work, we demonstrate that formal grammars can describe the output space for a much wider range of tasks and argue that GCD can serve as a unified framework for structured NLP tasks in general.
1 code implementation • 7 Mar 2023 • Martin Josifoski, Marija Sakota, Maxime Peyrard, Robert West
This work shows that useful data can be synthetically generated even for tasks that cannot be solved directly by LLMs: for problems with structured outputs, it is possible to prompt an LLM to perform the task in the reverse direction, by generating plausible input text for a target output structure.
no code implementations • 14 Nov 2022 • Jonas Rothfuss, Martin Josifoski, Vincent Fortuin, Andreas Krause
Meta-Learning aims to speed up the learning process on new tasks by acquiring useful inductive biases from datasets of related learning tasks.
1 code implementation • 13 Oct 2022 • Martin Josifoski, Maxime Peyrard, Frano Rajic, Jiheng Wei, Debjit Paul, Valentin Hartmann, Barun Patra, Vishrav Chaudhary, Emre Kiciman, Boi Faltings, Robert West
Specifically, by analyzing the correlation between the likelihood and the utility of predictions across a diverse set of tasks, we provide empirical evidence supporting the proposed taxonomy and a set of principles to structure reasoning when choosing a decoding algorithm.
1 code implementation • NAACL 2022 • Martin Josifoski, Nicola De Cao, Maxime Peyrard, Fabio Petroni, Robert West
Structured and grounded representation of text is typically formalized by closed information extraction, the problem of extracting an exhaustive set of (subject, relation, object) triplets that are consistent with a predefined set of entities and relations from a knowledge base schema.
1 code implementation • 16 Oct 2021 • Maxime Peyrard, Sarvjeet Singh Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Robert West
In particular, we adapt a game-theoretic formulation of IRM (IRM-games) to language models, where the invariance emerges from a specific training schedule in which all the environments compete to optimize their own environment-specific loss by updating subsets of the model in a round-robin fashion.
no code implementations • 1 Jan 2021 • Jonas Rothfuss, Martin Josifoski, Andreas Krause
Bayesian deep learning is a promising approach towards improved uncertainty quantification and sample efficiency.
3 code implementations • ICML Workshop LifelongML 2020 • Jonas Rothfuss, Vincent Fortuin, Martin Josifoski, Andreas Krause
Meta-learning can successfully acquire useful inductive biases from data.
3 code implementations • EMNLP 2020 • Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, Luke Zettlemoyer
This paper introduces a conceptually simple, scalable, and highly effective BERT-based entity linking model, along with an extensive evaluation of its accuracy-speed trade-off.
1 code implementation • 8 Apr 2019 • Martin Josifoski, Ivan S. Paskov, Hristo S. Paskov, Martin Jaggi, Robert West
Finally, although not trained for embedding sentences and words, it also achieves competitive performance on crosslingual sentence and word retrieval tasks.