AI research company Pathway has closed a $30 million seed round at a $500 million valuation. The company develops models based on what it calls a Post-Transformer architecture named BDH.

Why anyone wants to move past the Transformer

Almost every large language model in use today is built on the Transformer architecture. Its power comes from being able to weigh the relationship between every piece of a text and every other piece at once.

That power is also its limit. The number of relationships grows with the square of the text length. Double the text and the computation quadruples. This is where the cost of long context comes from, and why quality falls off beyond a certain point.

The shared goal of alternative architectures is exactly this: obtain the same capability at a cost that grows linearly with length rather than quadratically.

What the valuation actually tells you

A $500 million valuation at seed stage is unusual. Numbers at that level are normally discussed for companies with a product and revenue.

What is being priced here is not revenue but a possibility. The investor's bet runs like this: if the current architecture is approaching a ceiling, owning the architecture that clears it would pay enormously. It is a low-probability, very-high-return wager.

Why this path is hard

Most attempts at new architectures hit the same wall: a design that works well at laboratory scale does not behave as expected when it is scaled up.

Part of why the Transformer won was not theoretical superiority. The architecture suited parallel computation, so it ran efficiently on the hardware that existed. That brought a decade of accumulated work with it: libraries, training methods, optimisation techniques, all shaped around the Transformer.

For a new architecture to compete, being better is not enough; it has to be better by enough to make up for the absence of all that accumulated work.

The claim is not yet verified

We have nothing beyond the company's own description. The BDH name and the Post-Transformer label belong to Pathway; no independent assessment has been published.

There is a known way to evaluate claims of this kind:

  • Is the architecture described in a paper, or only narrated?
  • Has it been compared against a same-size Transformer on the same data?
  • Have the results been reproduced by third parties?

Until all three hold, the claim remains a thesis.

The value of diversity

Even so, funding work of this kind is healthy for the field. AI research has concentrated around a single architecture to an extraordinary degree, and that concentration carries a risk: because alternative paths are not explored, a better path would go undiscovered even if it existed.

The Transformer itself displaced the recurrent networks that once dominated, and in that period too there were researchers working outside the established approach. Capital allocated to those working outside today's established approach is what keeps that possibility open.