John Wang’s Post

So happy we got to host AI21 Labs (one of a handful of foundation model developers in the world) and the AI Tinkerers at our office yesterday and speak on the panel. AI21 is doing really well in the enterprise in part because they understand the trade off between model cost, quality, and speed: many enterprises care much more about cost and speed when you’re pushing tens or hundreds of millions of LLM calls in production. Particularly fascinating was AI21’s model architecture built on a hybrid combination of Transformers and the Mamba architecture: which is really built for speed in long contexts (something enterprises tend to need much more than consumer applications). At the AI Tinkerers group there are people who are really pushing the boundaries of what is possible (from LLMs powering game engines to using LLMs with reinforcement learning)