I've been wondering about this a lot, because I'm not wholly sure how that distinguishes a large language model from people.
I mean, much of what I know, I know because I've read it in, or been told it by, sources that seem to me reliable. I trust vaccinations in general and Covid vaccinations in particular because I'm assured by various people whose professional judgment I trust, like the doctors at my local practice, and by the NHS website, that any vaccinations they offer me have been thoroughly tested and are far more likely to do me good than to harm me. I'm obviously unable to replicate their studies myself, so I just have to take it on trust.
Here's a great explanation about how the underlying fundamental piece of neural networks, the perceptron, works. Note that this was created already back in 1957 by Frank Rosenblatt.
In short terms: LLMs are pattern recognizers, great at recognizing patterns in written text. The difference between them and us is that a human being is not just recognizing patterns, but learns stuff by developing concepts. For example a tree has green leaves, brown trunk and certain form. Which identifying memorising these few rules of thumbs our brain is able to identify trees with ease. A LLM on the other hand will never be able to understand what parts a tree is made of. And in order to recognize trees in a reliable ways you've got to train it with thousands of different photos of trees.
LLMs are putting out the words with highest probability in sentences of appearing in given context. One main consequence of this is that LLMs will never be able to become creative and write unexpected pieces of texts like well versed authors might do. They do not have any fantasy, they are not trying out anything, they are just reproducing stuff and are always slaves to the probabilities. So they have no creatitivity.
Also us humans are constantly learning every minute of every day. LLMs don't do that: they are instead a snapshot of the whole content of the internet at a given time. This data is then used to create and calculate their ridiculously big matrices required to run an LLM. Modern LLMs now have internet search as well, meaning if information is not in the LLM they can crawl the internet. But this little piece of updated info gets lost very soon afterwards, it will be discarded, because the LLM itself will not be updated.
Training a LLM takes several weeks to some months. Updating a LLM would mean that the whole shebang needs to be updated every time. So of course that's not being done.
What the chatbots are using is the result of a training cycle set in stone.
Also big part of the secret sauce and taking quite some time is getting the model ready for public use: since it contains the whole wisdom of the internet, a LLM such GPT has no problems explaining to you how to build explosives, run a drug business, recite song lyrics and other stuff.
So there's a lot of information deep inside every LLM their creators don't want to be exposed to the public. So models are restricted to not answer to such questions. If you can jailbreak them, it will still be exposed.
There are also open source models around like Deepseek or Lama, meaning you get the end result of a training and can do whatever you want to do with it. And there are some hackers and researches around who are developing processes to uncensor such models, sharing their methods and results.
For example a normal LLM model would never create a text where Donald Trump is bragging about his body count and how great each number was. An uncensored model would just comply.
