Understanding How LLMs Work

[Cristiano]

LLMs and AI Aren't the Same. Everything You Should Know About What's Behind Chatbots

Wonder what is really powering your ChatGPT or Gemini chatbots? This is everything you need to know about large language models.

www.cnet.com

 

[Bartholomew Gallacher]

Well this barely scratches the surface. Stephen Wolfram, creator of Mathematica, wrote a while ago his own blog post about how LLMs do work, where its borders are and what it cannot do.

Wolfram's post is much, much deeper while still barely skimming the surface, but Wolfram really manages to explain all the major moving parts involved. For me the most interesting take away from it is that scientists at the moment don't understand why ChatGPT works, because we don't understand how neural networks really work internally.

So if you want to have an in depth not to deep but deep enough look at LLM, go with Wolfram. He doesn't disappoint.

What Is ChatGPT Doing … and Why Does It Work?

Stephen Wolfram explores the broader picture of what's going on inside ChatGPT and why it produces meaningful text. Discusses models, training neural nets, embeddings, tokens, transformers, language syntax.

writings.stephenwolfram.com