It was pretty obvious that you didn't need massive proprietary hardware to run LLMs.
It was only a matter of time before the markets realised this.
Likely another two orders of magnitude left in the bag from number encodings and dropping the use of dense matrices. Not something you can do with gemm.
I did wish for a while that I had taken a job offer at Nvidia as the hype curve would have been good for my bank balance!
I always like Jim's posts. He has amazing experience.
I am not the only one who thinks software optimisation is what the market fears. See this semianalysis post(https://semianalysis.com/2025/03/19/nvidia-gtc-2025-built-for-reasoning-vera-rubin-kyber-cpo-dynamo-inference-jensen-math-feynman/), which says "The concern is that DeepSeek-style software optimization and increasing Nvidia-driven hardware improvements are leading to too much savings, meaning that the demand for AI hardware decreases and the market will be in a token glut. "
It was pretty obvious that you didn't need massive proprietary hardware to run LLMs.
It was only a matter of time before the markets realised this.
Likely another two orders of magnitude left in the bag from number encodings and dropping the use of dense matrices. Not something you can do with gemm.
I did wish for a while that I had taken a job offer at Nvidia as the hype curve would have been good for my bank balance!
I always like Jim's posts. He has amazing experience.