Search Results for author: Michael Hassid

Found 8 papers, 3 papers with code

The Larger the Better? Improved LLM Code-Generation via Budget Reallocation

no code implementations31 Mar 2024 Michael Hassid, Tal Remez, Jonas Gehring, Roy Schwartz, Yossi Adi

On the other hand, in scenarios where unit-tests are unavailable, a ranking-based selection of candidates from the smaller model falls short of the performance of a single output from larger ones.

Code Generation

Transformers are Multi-State RNNs

1 code implementation11 Jan 2024 Matanel Oren, Michael Hassid, Nir Yarden, Yossi Adi, Roy Schwartz

Our results shed light on the connection between transformers and RNNs, and help mitigate one of LLMs' most painful computational bottlenecks - the size of their key-value cache.

Decoder

EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis

no code implementations10 Aug 2023 Tu Anh Nguyen, Wei-Ning Hsu, Antony D'Avirro, Bowen Shi, Itai Gat, Maryam Fazel-Zarani, Tal Remez, Jade Copet, Gabriel Synnaeve, Michael Hassid, Felix Kreuk, Yossi Adi, Emmanuel Dupoux

Recent work has shown that it is possible to resynthesize high-quality speech based, not on text, but on low bitrate discrete units that have been learned in a self-supervised fashion and can therefore capture expressive aspects of speech that are hard to transcribe (prosody, voice styles, non-verbal vocalization).

Resynthesis Speech Synthesis

How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers

1 code implementation7 Nov 2022 Michael Hassid, Hao Peng, Daniel Rotem, Jungo Kasai, Ivan Montero, Noah A. Smith, Roy Schwartz

Our results motivate research on simpler alternatives to input-dependent attention, as well as on methods for better utilization of this mechanism in the Transformer architecture.

Cannot find the paper you are looking for? You can Submit a new open access paper.