The evolution of AI looks, from the outside, like a string of unrelated miracles — a chess engine, then face recognition, then a chatbot that writes code. It isn't. It's one ladder. Every generation of AI exists to fix the specific limit that broke the one before it: expert systems gave way to machine learning, machine learning to neural networks, neural networks to transformers, and transformers to the generative models and AI agents we deploy today. Learn the limit each rung removed and the whole history of AI snaps into a single, learnable shape.
I deploy several of these rungs in production as a forward deployed engineer, so this isn't a museum tour — it's the mental model I use to decide which rung a problem actually needs. Reaching for the newest rung when an older one would do is the most expensive mistake in applied AI. One caveat before we climb: this is a conceptual progression, not a strict timeline — the eras overlap, and each rung builds on the one below it rather than cleanly replacing it.
It started by mimicking the human: expert systems
The first useful AI didn't learn anything. Alan Turing's framing was that a machine doesn't have to be a perfect mind — it only has to mimic a human convincingly enough. So the earliest production AI did exactly that: it encoded a human expert's judgment as a set of explicit rules.
Picture a bank deciding a loan. A senior officer's logic might be: if the credit score is above 750 and income is above ₹12 lakh, approve; if existing debt is above ₹10 lakh, reject. Write those rules into software and you have an expert system — artificial intelligence in the most literal sense: a program artificially mimicking a human's decision.
The wall shows up the moment you scale. Real lending has thousands of edge cases — thin credit files, irregular income, regional risk — and a human has to anticipate and hand-write every one. The system is only ever as complete as the rule-writer's imagination. Expert systems are still the right tool more often than people admit: if a problem is fully describable by rules, rules are cheaper, faster, and easier to debug than any model — which is exactly when a hard-coded rule beats reaching for an LLM. But when the rules outrun what a human can enumerate, you need the machine to find them for you.
Machine learning: stop writing the rules, learn them
Machine learning turns the expert system inside out. Instead of a human writing the rule, you show the machine examples and let it infer the rule itself. The textbook version is linear regression. Say you sell ice cream and notice sales track temperature: 10°C brings in ₹10,000, 20°C brings ₹20,000, 30°C brings ₹30,000. The relationship is roughly y = mx + c, and once the machine learns that m is about 1,000 it can predict sales at a temperature it has never seen. Nobody wrote "if 35 degrees then ₹35,000" — the machine derived it.
Geometrically, learning is just finding the best line through a cloud of dots. You can't touch every point, so you settle for the line that's least wrong overall — the optimal function. Real problems have more than one input: a property's price depends on its area and its number of bedrooms, so the model grows to y = m₁x₁ + m₂x₂. Training is the unglamorous loop of nudging those m's — the parameters — again and again until predictions stop improving. An early guess might be 60% off, the next 20%, the next 2%. That iterative correction is the whole game.
The limit here is expressive power. A handful of parameters can only bend into simple shapes. Ask a near-straight function to read a photograph or parse a sentence and it has nothing to say.
Neural networks: stacking parameters to approximate anything
If a few parameters fit simple patterns, the obvious move is to use far more of them and arrange them in layers. That's a neural network. The parameters stop being two numbers and become thousands, then millions, wired into stacked layers where each layer's output feeds the next. The more parameters, the more intricate the patterns the network can capture.
This isn't hand-waving — it's the universal approximation theorem, proven in 1989: a large enough neural network can approximate essentially any continuous function. Add a probability function at the end and the network stops predicting a number and starts predicting classes. Feed it a property's features and it outputs "70% luxury, 30% mediocre"; add more outputs and it spreads across "luxury / average / mid" with a probability on each. The same machinery took over computer vision: a tiny image is just a grid of numbers, a real photo is millions of them, and a deep enough network learns to map those pixels to "cat," "dog," or "donkey."
Then language exposes the next wall. A sentence is a sequence whose meaning depends on word order and on words far apart — "the loan the bank rejected after the manager left." The older sequence models (RNNs and LSTMs) kept forgetting the earlier words by the time they reached the end. They couldn't hold a long thought.
Transformers: attention is all you need
In 2017 a Google team published a paper with an unusually confident title — “Attention Is All You Need” — and it reset the field. Their architecture, the transformer, threw out the sequential machinery and replaced it with attention: a mechanism that weighs the relationship between every word and every other word in the input at once. “It” can reach back and decide it means “the loan,” however many words away that is.
Attention solved the memory problem and — just as important — it was massively parallelizable, so you could train it on far more data far faster. That scalability is the quiet reason almost everything after 2017 happened. The transformer is the engine under nearly every large language model you've heard of since.
Generative AI: predict the next token, then do it again
Train an enormous transformer on an enormous amount of text and a deceptively simple objective produces something startling: just predict the next word. Given "I am going to the," the model assigns a probability to every word in its vocabulary — "office" 0.6, "gym" 0.3, "moon" 0.001 — picks a likely one, appends it, and repeats. Run that loop token after token and the machine writes fluent paragraphs it was never explicitly taught to produce. That's generative AI: a next-token predictor in a loop.
The timeline here carries a lesson I keep returning to. Google had this capability early — it demoed LaMDA, a strikingly human conversational model, at its developer conference in May 2021 — but chose not to release it to the public. OpenAI shipped ChatGPT on 30 November 2022, and it reached a million users in five days. Same underlying breakthrough; one company put it in people's hands. Possessing a technology and deploying it are very different things — which, as a forward deployed engineer, is the whole of my job.
The wall this time isn't capability — it's knowledge. A generative model only knows what was in its training data: the public internet, frozen at a moment in time. It knows nothing about your company's policies, your customer's account, or what happened yesterday. Ask it about your private data and it will confidently invent an answer.
RAG and agents: giving the model your world
The first fix is retrieval-augmented generation (RAG), introduced in a 2020 paper from Facebook AI. The idea is plain: before the model answers, fetch the relevant document — a policy PDF, a knowledge-base article — and hand it over with the question. Instead of “answer from memory,” it becomes “here is the relevant policy, plus the prompt, plus the question — now answer.” The fluency stays; the facts now come from your data.
Agents are the rung we're standing on now. A RAG system still only talks; an agent acts. You give the model a set of tools — functions it can call — and let it decide which to use for a given request. Ask “what's the status of my home loan?” and the agent might call a credit-check tool, then a document-fetch tool, then a browse tool, chaining them to actually finish the task. The model becomes the controller, not just the writer. Deciding which tools an agent should hold, and how tightly to scope it, is its own discipline — I wrote a playbook on scoping an AI agent for that.
What the ladder taught me as a forward deployed engineer
Knowing this history changes how you build, because the newest rung is not automatically the right one. It's tempting to jump straight to the top — stuff everything into a prompt, wire in an LLM, reach for RAG — and skip the unglamorous work below. I've watched that shortcut backfire more than once: when the data foundation underneath isn't solid, the LLM layer has nothing real to stand on, and the impressive demo quietly falls apart in production.
The fix is almost always to climb back down — get the data scraped, processed, and into a proper store first, and only then put a model or an agent on top. The lesson is worth burning in: the LLM amplifies a foundation you've earned; it doesn't replace one. Every rung on this ladder still exists for a reason, and the job isn't to use the fanciest one — it's to match the rung to the problem. That instinct — knowing the whole stack well enough to pick the right layer — is a big part of what separates a developer from a forward deployed engineer.
The ladder isn't finished. Agents are just the highest rung we've built so far, and something will eventually expose their limit the way every rung before them was exposed. But the pattern won't change: each step trades a specific constraint for a new capability, and the engineers who understand why each rung exists will always deploy them better than the ones chasing whatever is newest.
FAQ
What is the difference between an expert system and machine learning?
An expert system follows rules a human writes by hand, so it can only handle the cases its author anticipated. Machine learning infers the rules from examples instead, so it can generalise to situations no one explicitly coded — at the cost of needing data and being harder to inspect.
When were transformers invented, and why do they matter?
The transformer was introduced in the 2017 paper "Attention Is All You Need" by a Google team. It replaced sequential processing with an attention mechanism that relates every word to every other word, solving older models' memory problem and making large-scale training practical. Almost every modern AI model is built on it.
What is the difference between generative AI and an AI agent?
Generative AI produces content — text, code, images — by repeatedly predicting the next token. An AI agent wraps a generative model with tools it can call and lets it decide which to use, so it can take actions like fetching data, running a check, or browsing, instead of only producing text.
What is RAG and why do LLMs need it?
Retrieval-augmented generation (RAG), introduced by Facebook AI in 2020, fetches relevant documents from your own data and gives them to the model before it answers. LLMs only know their training data — the public internet frozen in time — so RAG is how you ground them in private, current, or company-specific information.
Did Google have an AI like ChatGPT before OpenAI?
Yes. Google demoed LaMDA, a human-like conversational model, in May 2021 — over a year before ChatGPT — but chose not to release it publicly. OpenAI shipped ChatGPT in November 2022. It is a clean example of how deploying a technology, not just possessing it, is what creates impact.