Language Models

Language Models

Learning objectives

• Understand how language models work
• Know the difference between traditional programming and language models (see the previous section for the foundation)
• Understand what a context window is and why it matters
• Know the most important limitations

In the previous section we talked about generative AI – now we zoom in on the most discussed type: language models (e.g. ChatGPT, Claude, Gemini). How do they work – and what do you need to know to use them effectively?

What is a language model?

Simply put: a language model is trained on an enormous amount of text. It predicts which word is most likely to come next, based on all the words that came before – patterns it learned from training data.

When you write “What is the capital of” the model leans toward the next word being a country name, followed by a response format with a city.

Modern models can answer complex questions, write code, summarize, translate, and reason – but they’re still generating text based on probability and patterns, not human “understanding” in the full sense.

How is a language model trained?

1. Data collection – Enormous amounts of text from books, articles, the web, Wikipedia, forums, and more.

2. Training – The model learns patterns through neural networks: which words follow each other, how sentences are structured, how different text types differ. Requires massive computation; millions of parameters are adjusted.

3. Fine-tuning – Humans rate responses so the model becomes more helpful, relevant, accurate, and safe.

Result: a model that can generate fluent text on almost any topic – within the scope of its training and knowledge cutoff.

What is a context window?

The context window is the total amount of text the model can work with at once – like short-term memory.

Everything must fit there: your question, previous messages, attached text, and the model’s own responses. It’s often measured in tokens (small text pieces – words, syllables, or characters).

Why does it matter?

1. Long documents – An entire book rarely fits; you must split it or use tools like RAG.
2. Long conversations – When the window is full the model drops the oldest parts (in Intric you may get an error if the context is full).
3. Context quality – More relevant context in the right order usually gives better answers.

RAG – giving the model current and comprehensive knowledge

RAG stands for Retrieval-Augmented Generation. Retrieval means exactly what it says: the system looks up the passages that appear to answer your question and sends only those to the model. That lets the model avoid cramming an entire knowledge base into a prompt every time.

A chunk is one of those passages — a paragraph or section of a document, small enough to fit in the context window but large enough to make sense on its own.

Here is the chain, from uploaded document to answer:

1. Documents are uploaded to the knowledge base.
2. Chunking — the document is split into smaller passages. Each chunk keeps metadata about which document, page and section it came from.
3. Embeddings — each chunk is turned into a numeric representation of what the text means, not which words it contains. That is why a search for “holiday” can find a passage that only says “leave”.
4. Vector index — those numeric representations are stored in a searchable register.
5. Retrieval — your question is converted the same way, and the system fetches the chunks closest to it in meaning.
6. The selected chunks and your question go into the context window together.
7. The language model writes the answer from those passages.

It’s like a librarian who fetches the right chapter for you instead of dumping the whole shelf on the table.

Limitations of language models

1. Knowledge cutoff – Training ends at a certain date; events after that don’t exist in the “base model” (connections to the web are a separate layer).

2. Hallucinations – The model can sound convincing but be wrong – especially when filling gaps without adequate sources.

3. No real understanding – Strong on text patterns; weak on anything requiring real experience, sensation, and shared worldview.

4. Context limit – You’re constrained by the context window size and how well the right information actually gets included.

5. Inconsistency – The same question can give slightly different answers on different occasions. The model does not always pick the single most likely next word; it samples among the most likely candidates. How much randomness is allowed is governed by a setting called temperature: low temperature gives more predictable, more uniform answers, high temperature gives more varied and creative ones. Small differences in how you phrase the question also affect the answer.

Summary

• Language models predict the next word based on patterns in training data.
• The context window limits how much can be included at once.
• RAG fetches relevant excerpts so you don’t have to fill entire documents into the prompt.
• Key risks: knowledge cutoff, hallucinations, lack of “real” understanding, and variation in responses.

Test your knowledge

1Question 1 of 32Question 2 of 33Question 3 of 3

Question 1 of 3

Question 1 of 3

What is the most fundamental explanation of how a language model works?

What is the most fundamental explanation of how a language model works?

• It analyzes grammar and applies language rules
• It predicts which word is most likely to come next based on previous words
• It translates everything into code and computes mathematical formulas
• It searches the internet and compiles information from different sources

Back Next