Part of what drew me to research was my love for working at the frontier of what can be done with technology. Having a problem, chewing on it, trying something - it fails, that’s ok. Trying again: slightly better! Improving your solution, until you’re satisfied.

When this happens, you look back: if you zoom out, what you see is a discontinuous gap. You now have the ability to do something that could not be done before. You’ve invented something. For most research projects, it’s a very modest result. The new ability is somewhat niche, and does not get traction 1. For some others, it can be huge: recent years have seen the rise of a specific line of works, Large Language Models. These are built on the transformer architecture 2 as well as other techniques such as chain-of-thought reasoning 3, and boy do they deliver.

It turns out that language models, when trained on huge amounts of data, unlock emergent capabilities 4 that range from being amazing coding assistants 5 to accompanying researchers in moving the needle for mathematical research 6. I’ll be honest, I did not see it coming. The fact that you can build crazy good AI agents on top of autoregressive base models still amazes me. In hindsight, the stochastic parrots framing didn’t quite anticipate how far this would go 7.

Looking back, it does feel like magic. These recent times, I could really feel that something has changed. And it’s almost funny to remember how it started: who has never mocked an LLM for not knowing how many ‘r’ there were in ‘strawberries’ 8? A lot of effort went into perfecting these models, to the point that they are here to stay and that the investments required to run them are deeply transforming our economic landscape 9.

Now what is the next frontier ? My bet is on Large Tabular Models. When you think about it, this is about one of the most common types of data encountered deep in the storage facilities of the enterprise world. And it’s shockingly hard to extract value from it. For a while, it was mostly tree-based models, and it was resisting the deep learning revolution. Spend a few months on a use case, create features, tune an xgboost model. You need serious work to pull this off, and it does not exactly scale well: get a new project, start again from scratch.

What has changed ? The promise that by using in context learning, you can leverage attention mechanisms pre-trained on A LOT of tables to almost zero-shot the task that you’re trying to solve 10. And it works quite well, even if we’re still in its infancy as 2026 marks only the second ICML workshop on the topic 11. So when I got the opportunity to explore the field, I jumped on it.

That’s why I’m now ML engineer at Fundamental. The bet is simple: the next discontinuous gap is in enterprise tables. Soon we’ll look back at what’s possible right now as a thing of the past. The forecasting abilities of Large Tabular Models are going to unlock the value that lies in proprietary data and push forward the adoption of ambitious use cases. And I’m crazy excited to be a part of this.

  1. Case in point, one of my own: Functional Output Regression with Infimal Convolution: Exploring the Huber and ε-insensitive Losses, ICML 2022. ↩︎

  2. Vaswani et al., “Attention Is All You Need,” NeurIPS 2017. https://arxiv.org/abs/1706.03762 ↩︎

  3. Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” NeurIPS 2022. https://arxiv.org/abs/2201.11903 ↩︎

  4. Wei et al., “Emergent Abilities of Large Language Models,” TMLR 2022. https://arxiv.org/abs/2206.07682 ↩︎

  5. Chen et al., “Evaluating Large Language Models Trained on Code” (Codex), 2021. https://arxiv.org/abs/2107.03374 ↩︎

  6. Romera-Paredes et al., “Mathematical discoveries from program search with large language models” (FunSearch), Nature 2024. https://www.nature.com/articles/s41586-023-06924-6 ↩︎

  7. Bender, Gebru, McMillan-Major & Shmitchell, “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” FAccT 2021. https://dl.acm.org/doi/10.1145/3442188.3445922 ↩︎

  8. Talmor, Elazar, Goldberg & Berant, “oLMpics: On What Language Model Pre-training Captures,” TACL 2020. https://arxiv.org/abs/1912.13283 ↩︎

  9. Goldman Sachs, “The Potentially Large Effects of AI on Economic Growth” (Briggs & Kodnani, 2023). https://www.gspublishing.com/content/research/en/reports/2023/03/27/d64e052b-0f6e-45d7-967b-d7be35fabd16.pdf ↩︎

  10. van Breugel & van der Schaar, “Position: Why Tabular Foundation Models Should Be a Research Priority,” ICML 2024. https://arxiv.org/abs/2405.01147 ↩︎

  11. ICML 2026 Workshop on Structured Foundation Models. https://icml-structured-fm-workshop.github.io/ ↩︎