Large Language Model (LLM)

A language model estimates the probability of a token, or a sequence of tokens, occurring within a longer sequence, where a token can be a word, part of a word or a single character^[1]. A large language model predicts sequences of tokens with far more parameters and far more context than the models before it^[2]. It writes by autocompleting: given a question, it generates probabilities for the response that follows^[2].

The architecture behind most of them is the Transformer, which its authors proposed in 2017 as a network based solely on attention mechanisms, with no recurrence and no convolutions^[3]. Google’s course describes it as the most widely used architecture for building LLMs, and self-attention as the way it weighs how much each token in the input affects the reading of every other^[2].

“Large” is about size. A 2020 paper trained a model with 175 billion parameters, ten times more than any earlier non-sparse language model, and showed that scaling up language models greatly improves performance on tasks given only a few examples^[4]. A survey of the field says that the research community coined the term for pre-trained language models of significant size, and notes that beyond a certain scale they show abilities that small models do not^[5].

For search, the point is that generative engines use these models to write their answers. The research that named GEO describes generative engines as satisfying queries by gathering information from several sources and summarizing it with LLMs^[6]. So what an answer says about a brand comes from two places: what the model learned in training, and the pages the engine retrieves at the moment of the question. Pages are the part a site can change.

The text a model writes is not reliable on its own. Google’s course says LLMs hallucinate, meaning their predictions often contain mistakes, and that like all machine-learning models they can show bias^[2]. Access also matters to the text a model learns from. In QueryRev’s study, the names it counts as training crawlers are blocked on 7.9% of the top sites read on average, against 4.8% for the names it counts as search crawlers^[7].

A team does not measure a model. It measures what the model writes about the brand, by asking fixed prompts on each AI surface and recording the answers. Where an assistant offers both, ask each prompt with its web search off, which shows what the model has learned, and with it on, which shows what retrieval adds. The two readings fail in different ways and call for different work.

Repeat every prompt. A model chooses among likely next tokens, and its prediction can be the most probable one or a random pick from those above a threshold^[1]. The same prompt can therefore return different wording and different brands from one run to the next, and a rate over many runs is the figure.

Check every claim the answer makes about the brand against the page that should state it: the price, the policy, the place, the product. Count the answers that are wrong, out of date or silent on a fact the site states plainly, and keep each with its prompt and its date.

Sample data: an invented site, to show the method.

A sample bed shop asks one question about itself, “What is the return window at this shop?”, on one AI surface, three times. With web search off, the first answer names the shop and states a 14-day window, the policy it had two years ago. The second does not name the shop at all. With web search on, the answer names the shop, gives the current 30-day window and links the shop’s returns page.

The shop learns two things. The model’s memory is old and uneven, which the shop cannot edit. The retrieved answer is right because the returns page states the policy in its first sentence and the crawler for that surface can read it. The shop keeps that page current and readable, and repeats the three runs next month.