Yossi Adi

My name is Yossi Adi, I'm an Assistant Professor (Senior Lecturer) at the School of Computer Science and Engineering at the Hebrew University and a Research Scientist (part time) at Meta AI Research, FAIR. I'm also a professional drummer at Lucille Crew. I completed my Ph.D in computer science at Bar-Ilan University under the supervision of Prof. Joseph Keshet.

My research interests are focused on developing and analyzing machine learning and deep learning algorithms for speech and language applications. In general, my research usually follows the KISS philosophy. My complete CV can be found here.

As a musician, I'm the drummer and one of the band members of Lucille Crew, an international Groove, Hip Hop and Soul collective based in Tel Aviv.

yossi.adi at mail.huji.ac.il



Israel is currently undergoing a judicial revolution threatening our democracy.




News

Publications

Conferences and Workshops Proceedings

  1. Wei-Ning Hsu, Tal Remez, Bowen Shi, Jacob Donley, Yossi Adi. ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement. The Conference on Computer Vision and Pattern Recognition (CVPR), 2023, [PDF, Samples].
  2. Moshe Mandel, Or Tal, Yossi Adi. AERO: Audio Super Resolution in the Spectral Domain. The 48th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2023, [PDF, Code, Samples].
  3. Roy Sheffer, Yossi Adi. I Hear Your True Colors: Image Guided Audio Generation. The 48th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2023, [PDF, Code, Samples].
  4. Amitay Sicherman, Yossi Adi. Analyzing Discrete Self Supervised Speech Representation For Spoken Language Modeling. The 48th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2023, [PDF, Code, Tool].
  5. Wen-Chin Huang, Benjamin Peloquin, Justine Kao, Changhan Wang, Hongyu Gong, Elizabeth Salesky, Yossi Adi, Ann Lee, Peng-Jen Chen. A Holistic Cascade System, Benchmark, and Human Evaluation Protocol For Expressive Speech-To-Speech Translation. The 48th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2023, [PDF, Demo].
  6. Ali Elkahky, Wei-Ning Hsu, Paden Tomasello, Tu-Anh Nguyen, Robin Algayres, Yossi Adi, Jade Copet, Emmanuel Dupoux, Abdelrahman Mohamed. Effect of Acoustic Unit Granularity on Seq2Seq Representation Learning. The 48th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2023, [PDF].
  7. Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Defossez, Jade Copet, Devi Parikh, Yaniv Taigman, Yossi Adi. AudioGen: Textually Guided Audio Generation. International Conference on Learning Representations (ICLR), 2023, [PDF, Samples].
  8. Felix Kreuk, Adam Polyak, Jade Copet, Eugene Kharitonov, Tu-Anh Nguyen, Morgane Rivière, Wei-Ning Hsu, Abdelrahman Mohamed Emmanuel Dupoux, Yossi Adi. Textless Speech Emotion Conversion using Decomposed and Discrete Representations. Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022, [PDF, Samples, Blog].
  9. Itai Gat, Yossi Adi, Alexander Schwing, Tamir Hazan. On the Importance of Gradient Norm in PAC-Bayesian Bounds. The 36th Annual Conference on Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, 2022, [PDF].
  10. Paden Tomasello, Akshat Shrivastava, Daniel Lazar, Po-Chun Hsu, Duc Le, Adithya Sagar, Ali Elkahky, Jade Copet, Wei-Ning Hsu, Yossi Adi, Robin Algayres, Tu Ahn Nguyen, Emmanuel Dupoux, Luke Zettlemoyer, Abdelrahman Mohamed. STOP: A dataset for Spoken Task Oriented Semantic Parsing. IEEE Spoken Language Technology Workshop (SLT), 2022, [PDF].
  11. Arnon Turetzky, Tzvi Michelson, Yossi Adi, Shmuel Peleg. Deep Audio Waveform Prior. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
  12. Or Tal, Moshe Mandel, Felix Kreuk, Yossi Adi. A Systematic Comparison of Phonetic Aware Techniques for Speech Enhancement. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF, Code].
  13. Shahaf Bassan, Yossi Adi, Jeffrey S. Rosenschein. Unsupervised Symbolic Music Segmentation using Ensemble Temporal Prediction Errors. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
  14. Sravya Popuri, Peng-Jen Chen, Changhan Wang, Juan Pino, Yossi Adi, Jiatao Gu, Wei-Ning Hsu, Ann Lee. Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
  15. Maureen de Seyssel, Marvin Lavechin, Yossi Adi, Emmanuel Dupoux, Guillaume Wisniewski. Probing Phoneme, Language and Speaker Information in Unsupervised Speech Representations. The 23rd Annual Conference of the International Speech Communication Association (Interspeech), 2022, [PDF].
  16. Ann Lee, Hongyu Gong, Paul-Ambroise Duquenne, Holger Schwenk, Peng-Jen Chen, Changhan Wang, Sravya Popuri, Yossi Adi, Juan Pino, Jiatao Gu, Wei-Ning Hsu. Textless Speech-to-Speech Translation on Real Data. Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2022, [PDF, Code, Samples].
  17. Eugene Kharitonov, Jade Copet, Kushal Lakhotia, Tu-Anh Nguyen, Paden Tomasello, Ann Lee, Ali Elkahky, Wei-Ning Hsu, Abdelrahman Mohamed Emmanuel Dupoux, Yossi Adi. textless-lib: a Library for Textless Spoken Language Processing. Conference of the North American Chapter of the Association for Computational Linguistics (NAACL): System Demonstrations, 2022, [PDF, Code].
  18. Eugene Kharitonov∗, Ann Lee∗, Adam Polyak, Yossi Adi, Jade Copet, Kushal Lakhotia, Tu-Anh Nguyen, Morgane Rivière, Abdelrahman Mohamed, Emmanuel Dupoux, Wei-Ning Hsu. Text-Free Prosody-Aware Generative Spoken Language Modeling. The 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022, [PDF].
  19. Ann Lee, Peng-Jen Chen, Changhan Wang, Jiatao Gu, Xutai Ma, Adam Polyak, Yossi Adi, Qing He, Yun Tang, Juan Pino, Wei-Ning Hsu. Direct Speech-to-Speech Translation with Discrete Units. The 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022, [PDF].
  20. Alon Berliner, Guy Rotman, Yossi Adi, Roi Reichart, Tamir Hazan. Learning Discrete Structured Variational Auto-Encoder Using Natural Evolution Strategies. International Conference on Learning Representations (ICLR), 2022, [PDF, Code].
  21. Efthymios Tzinis, Yossi Adi, Vamsi K. Ithapu, Buye Xu, Anurag Kumar. Continual Self-Training with Bootstrapped Remixing For Speech Enhancement. The 47th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2022, [PDF].
  22. Changhan Wang*, Wei-Ning Hsu*, Yossi Adi, Adam Polyak, Ann Lee, Peng-Jen Chen, Jiatao Gu, Juan Pino. FAIRSEQ S^2: A Scalable and Integrable Speech Synthesis Toolkit. Conference on Empirical Methods in Natural Language Processing (EMNLP): System Demonstrations, 2021, [PDF, Code].
  23. *Ori Kabeli, *Yossi Adi, Zhenyu Tang, Buye Xu, Anurag Kumar. Online Self-Attentive Gated RNNs for Real-Time Speaker Separation. Proceedings of Machine Learning in Speech and Language Processing Workshop, 2021, [PDF, Samples].
  24. Adam Polyak, Yossi Adi, Jade Copet, Eugene Kharitonov, Kushal Lakhotia, Wei-Ning Hsu, Adelrahman Mohamed, Emmanuel Dupoux. Speech Resynthesis from Discrete Disentangled Self-Supervised Representations. The 22nd Annual Conference of the International Speech Communication Association (Interspeech), 2021, [PDF, Samples, Code].
  25. Shahar Segal, Yossi Adi, Benny Pinkas, Carsten Baum, Chaya Ganesh, Joseph Keshet. Fairness in the Eyes of the Data: Certifying Machine-Learning Models. The Forth AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2021, [PDF].
  26. Adam Polyak, Lior Wolf, Yossi Adi, Ori Kabeli, Yaniv Taigman. High Fidelity Speech Regeneration with Application to Speech Enhancement. The 46th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2021, [PDF, Samples].
  27. Shlomo E. Chazan, Lior Wolf, Eliya Nachmani, Yossi Adi. Single Channel Voice Separation for Unknown Number of Speakers Under Reverberant and Noisy Settings. The 46th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), 2021, [PDF, Samples].
  28. Felix Kreuk, Joseph Keshet, Yossi Adi. Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation. The 21st Annual Conference of the International Speech Communication Association (Interspeech), 2020, [PDF, Code].
  29. Adam Polyak, Lior Wolf, Yossi Adi, Yaniv Taigman. Unsupervised Cross-Domain Singing Voice Conversion. The 21st Annual Conference of the International Speech Communication Association (Interspeech), 2020, [PDF, Samples].
  30. Alexandre Défossez, Gabriel Synnaeve, Yossi Adi. Real Time Speech Enhancement in the Waveform Domain. The 21st Annual Conference of the International Speech Communication Association (Interspeech), 2020, [PDF, Samples, Code].
  31. Felix Kreuk, Yossi Adi, Bhiksha Raj, Rita Singh, Joseph Keshet. Hide and Speak: Towards Deep Neural Networks for Speech Steganography. The 21st Annual Conference of the International Speech Communication Association (Interspeech), 2020, [PDF, Samples, Code].
  32. Eliya Nachmani, Yossi Adi, Lior Wolf. Voice Separation with an Unknown Number of Multiple Speakers. The 37th International Conference on Machine Learning (ICML), 2020, [PDF, Samples, Code, Blog].
  33. Ben Goldberger, Yossi Adi, Joseph Keshet, Guy Katz. Minimal Modifications of Deep Neural Networks using Verification. The 23rd International Conference on Logic for Programming, Artificial Intelligence and Reasoning (LPAR), Alicante, Spain, 2020, [PDF, Code].
  34. Felix Kreuk, Yaniv Sheena, Joseph Keshet, Yossi Adi. Phoneme Boundary Detection using Learnable Segmental Features. The 45th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020, [PDF, Code].
  35. Yossi Adi, Neil Zeghidour, Ronan Collobert, Nicolas Usunier, Vitaliy Liptchinsky and Gabriel Synnaeve. To Reverse The Gradient or Not: An Empirical Comparison of Adversarial and Multi-Task Learning in Speech Recognition. The 44th IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), Brighton, UK, 2019, [PDF].
  36. Gabi Shalev, Yossi Adi, and Joseph Keshet. Out-of-Distribution Detection using Multiple Semantic Label Representations. The 32nd Annual Conference on Neural Information Processing Systems (NeurIPS), Montréal, Canada, 2018, [PDF].
  37. Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring. Usenix Security, 2018, [PDF, Blog].
  38. Felix Kreuk, Yossi Adi, Moustapha Cisse, and Joseph Keshet. Fooling End-to-End Speaker Verification With Adversarial Examples. The 43nd IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), Alberta, Calgary, Canada, 2018, [PDF].
  39. Moustapha Cisse, Yossi Adi, Natalia Neverova, and Joseph Keshet. Houdini: Fooling Deep Structured Visual and Speech Recognition Models with Adversarial Examples. The 31st Annual Conference on Neural Information Processing Systems (NeurIPS), Long Beach, CA, USA, 2017, [PDF].
  40. Einat Naaman, Yossi Adi, and Joseph Keshet. Learning Similarity Function for Pronunciation Variations. The 18th Annual Conference of the International Speech Communication Association (Interspeech), Stockholm, Sweden, 2017, [PDF].
  41. Yaniv Sheena, Misha Hejna, Yossi Adi, Joseph Keshet. Automatic Measurement of Pre-aspiration. The 18th Annual Conference of the International Speech Communication Association (Interspeech), Stockholm, Sweden, 2017, [PDF].
  42. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi and Yoav Goldberg. Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks. International Conference on Learning Representations (ICLR), Toulon, France, 2017, [PDF].
  43. Yossi Adi, Joseph Keshet, Emily Cibelli, and Matt Goldrick. Sequence Segmentation Using Joint RNN and Structured Prediction Models. The 42nd IEEE International Conference in Acoustic, Speech and Signal Processing (ICASSP), New Orleans, LA, USA, 2017, [PDF].
  44. Yossi Adi, Joseph Keshet, Olga Dmitrieva and Matt Goldrick. Automatic Measurement of Voice Onset Time and Prevoicing using Recurrent Neural Networks. The 17th Annual Conference of the International Speech Communication Association (Interspeech), San Francisco, CA, USA, 2016, [PDF, Code].
  45. Yossi Adi, Joseph Keshet and Matt Goldrick. Vowel Duration Measurement Using Deep Neural Networks. The 25th IEEE International Workshop on Machine Learning for Signal Processing (MLSP), Boston, MA, USA, 2015, [PDF].

Journals

  1. Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoit Sagot, Abdelrahman Mohamed, Emmanuel Dupoux. Generative Spoken Dialogue Language Modeling. Transactions of the Association for Computational Linguistics (TACL), 2022, [PDF, Samples].
  2. *Alexandre Défossez, *Yossi Adi, Gabriel Synnaeve. Differentiable Model Compression via Pseudo Quantization Noise. Transactions on Machine Learning Research (TMLR), 2022, [PDF, Code].
  3. Efthymios Tzinis, Yossi Adi, Vamsi Krishna Ithapu, Buye Xu, Paris Smaragdis, Anurag Kumar. RemixIT: Continual Self-Training of Speech Enhancement Models via Bootstrapped Remixing. IEEE Journal of Selected Topics in Signal Processing, 2022, [PDF].
  4. Kushal Lakhotia*, Eugene Kharitonov*, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu-Anh Nguyen, Jade Copet, Alexei Baevski, Michael Auli, Alexis Conneau, Adelrahman Mohamed, Emmanuel Dupoux. On Generative Spoken Language Modeling from Raw Audio. Transactions of the Association for Computational Linguistics (TACL), 2021, [PDF, Samples, Code, Blog].
  5. Ke Tan, Buye Xu, Anurag Kumar, Eliya Nachmani, Yossi Adi. SAGRNN: Self-Attentive Gated RNN for Binaural Speaker Separation with Interaural Cue Preservation. IEEE Signal Processing Letters, 2020, [PDF, Samples].
  6. Jacob T. Cohen, Alma Cohen, Limor Benyamini, Yossi Adi, Joseph Keshet. Predicting Glottal Closure Insufficiency using Fundamental Frequency Contour Analysis. Head & Neck, Journal of the Sciences and Specialities of the Head and Neck, 2019, [PDF].
  7. Matthew Goldrick, Rhonda McClain, Emily Cibelli, Yossi Adi, Erin Gustafson, Cornelia Moers, and Joseph Keshet. The Influence of Lexical Selection Disruptions on Articulation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 2018, [PDF].
  8. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi and Yoav Goldberg. Analysis of Sentence Embedding Models using Prediction Tasks in Natural Language Processing. IBM Journal of Research and Development, 2017, [PDF].
  9. Yossi Adi, Joseph Keshet, Emily Cibelli, Erin Gustafson, Cynthia Clopper and Matt Goldrick. Automatic Measurement of Vowel Duration via Structured Prediction. Journal of the Acoustical Society of America, 2016, [PDF, Code].
  10. Yossi Adi and Joseph Keshet. StructED : Risk Minimization in Structured Prediction. Journal of Machine Learning Research, 2016, [PDF, Website, Code].

Preprints

  1. Gallil Maimon, Yossi Adi. Speaking Style Conversion With Discrete Self-Supervised Units. arXiv preprint arXiv:2212.09730 (2022), [PDF, Samples & Code].
  2. Felix Kreuk, Yaniv Taigman, Adam Polyak, Jade Copet, Gabriel Synnaeve, Alexandre Defossez, Yossi Adi. Audio Language Modeling using Perceptually-Guided Discrete Representations. arXiv preprint arXiv:2211.01223 (2022), [PDF, Samples].
  3. *Alexandre Défossez, *Jade Copet, ^Gabriel Synnaeve, ^Yossi Adi. High Fidelity Neural Audio Compression. arXiv preprint arXiv:2210.13438 (2022), [PDF, Code].
  4. Itai Gat, Felix Kreuk, Ann Lee, Jade Copet, Gabriel Synnaeve, Emmanuel Dupoux, Yossi Adi. On The Robustness of Self-Supervised Representations For Spoken Language Modeling. arXiv preprint arXiv:2209.15483 (2022), [PDF].
  5. Yossi Adi, Yaniv Nemcovsky, Alex Schwing, Tamir Hazan. On The Generalization of Bayesian Deep Nets for Multi-class Classification. arXiv preprint arXiv:2002.09866 (2020), [PDF].

From the press

  • Our model AudioGen, which generates audio from textual descriptions got attention from the media: [New Scientist].
  • I had a pleasure to chat about my research at the F2F podcast (in Hebrew): [podcast].
  • An article about my research on speech enhancement and source separation was published in The Marker magazine: [article].
  • I am humbled to be part of TheMarker Magazine's 40 Under 40 list (in Herbrew): [The Marker].
  • An article about my music, AI research, and everything in between (in Hebrew): [Mako].
  • Our work on single channel speaker source separation using deep nets got some attention from the media: [Globes (Hebrew), Venture Beat, Digital Trends].
  • A medium blog post about our joint work with Carsten Baum, Moustapha Cissé, Benny Pinkas and Joseph Keshet about watermarking deep neural networks: [Medium].
  • Our joint work with Moustapha Cissé, Natalia Neverova, and Joseph Keshet about fooling structured deep learning models got attention from the media: [New Scientist, MIT Technology Review].

Music

Spotify

Apple Music

Deezer

YouTube

SoundCloud

Bandcamp

Contact

School of Computer Science and Engineering.

The Hebrew University, Jerusalem, Israel, Edmond Safra Campus, Givat Ram.

Rothberg Building C, Room C432.

yossi.adi at mail.huji.ac.il.