Artificial Intelligence

Architecture of
Synthetic Cognition

A functional deconstruction of machine intelligence. Unlike classical programming, which requires explicit rules for every scenario, AI systems build their own rules by finding patterns in data.

This guide traces the lineage of intelligence from its broadest definition down to the specific mechanics of language models. It is a journey from logic (AI) to statistics (ML), to representation (DL), and finally to communication (NLP).

The Hierarchy of Intelligence

The field is a series of nested subsets. Generative AI (like LLMs) is a specific application of Deep Learning, which is a technique of Machine Learning, which is a branch of AI.

Evolution of Thought

Four Approaches to AI

Artificial Intelligence isn't just one goal; it's defined by how we measure success. Traditionally, the field is split along two axes: Human-like vs Rational and Thinking vs Acting.

The Turing Test (1950)

Inference Engines: Forward vs Backward Chaining

Before neural networks, AI relied on Inference Engines—the logical "thinking" component of an Expert System. This is how machines reason using IF-THEN rules to derive conclusions from data.

What is Machine Learning?

Machine Learning (ML) is the science of getting computers to act without being explicitly programmed. Instead of hard-coded rules, the machine learns from patterns in data to make decisions or predictions.

The 4 Types of Learning

Training a Model

The process of teaching a machine. Training is fundamentally an optimization problem: we want to find the configuration of weights that results in the lowest possible error.

import torch
import torch.nn as nn
import torch.optim as optim

# 1. Define Model, Loss Function, and Optimizer
model = nn.Sequential(nn.Linear(10, 5), nn.ReLU(), nn.Linear(5, 1))
criterion = nn.MSELoss() # Measures Mean Squared Error
optimizer = optim.SGD(model.parameters(), lr=0.01) # Gradient Descent

epochs = 100
for epoch in range(epochs):
    # --- 1. FORWARD PASS ---
    predictions = model(training_data)
    
    # --- 2. CALCULATE LOSS ---
    loss = criterion(predictions, actual_labels)
    
    # --- 3. BACKWARD PASS ---
    optimizer.zero_grad() # Clear old gradients
    loss.backward()       # Compute new gradients (Backpropagation)
    optimizer.step()      # Update weights based on gradients
    
    if epoch % 20 == 0:
        print(f"Epoch {epoch} | Loss: {loss.item():.4f}")

The Artificial Neuron

The fundamental atomic unit of learning. It is essentially a linear classifier.

A biological neuron fires when it receives enough stimulation. An artificial neuron mimics this mathematically. It takes inputs (x), multiplies them by learnable weights (w), adds a bias (b), and pushes the result through an activation function (like Sigmoid or ReLU) to introduce non-linearity.

If the weighted sum exceeds a threshold, the neuron "activates." By adjusting the weights, we change what stimulates the neuron—training it to recognize specific patterns.

Gradient Descent

The engine of learning. An iterative algorithm for finding the lowest error.

Imagine being blindfolded on a mountain and trying to find the bottom of the valley. You feel the ground to see which way is "down" (the gradient) and take a small step in that direction. In ML, the "mountain" is the Loss Function (total error), and your position is defined by the model's weights. Gradient Descent updates the weights to move iteratively towards the point of minimal error.

Deep Neural Networks

Feature Abstraction. Why do we stack layers? To create a hierarchy of understanding.

A single layer can only solve simple, linear problems. By stacking layers (Deep Learning), the network learns progressively complex features. In image recognition, the first layer might detect edges. The second layer combines edges to detect shapes (circles, squares). The third layer combines shapes to detect objects (eyes, ears). This hierarchical representation is what layer combines shapes to detect objects (eyes, ears). This hierarchical representation is what makes Deep Learning so powerful.

Convolutional Neural Networks (CNNs)

How computers "see". Using filters to detect spatial patterns.

Images are grids of pixels. A CNN slides "filters" (or kernels) over the image to detect features like edges or curves. The multiplication of the filter values with the pixel values creates a "Feature Map".

Vector Embeddings

Mapping meaning to geometry. Transforming words into coordinates.

To a computer, "Apple" and "Orange" are just different strings. To make them useful, we convert them into lists of numbers (vectors) such that semantically similar words are close together in mathematical space.

This allows for Semantic Arithmetic. We can subtract the "Maleness" vector from "King", add "Femaleness", and the resulting vector points to "Queen".

Self-Attention

Contextual weighting. How models understand the relevance of words.

Before Transformers, models read sentences left-to-right, often forgetting the start of a sentence by the time they reached the end. Self-Attention allows the model to look at every word in a sentence simultaneously and calculate how much each word relates to every other word. In the example below, "it" is ambiguous to a computer. Attention resolves this by linking "it" strongly to "animal" because the animal is tired.

Large Language Models (GenAI)

From "Mining" patterns to generating answers.

Diffusion Models

Generating content via iterative denoising.

How does AI generate images? It doesn't paint like an artist. Instead, it acts like a sculptor revealing a statue from a block of marble, but the "marble" is static noise.

The model is trained to reverse the process of adding noise to an image. To generate a new image, we give it pure random noise and ask it: "What part of this looks like a cat?" It slightly adjusts the pixels. We repeat this thousands of times until the noise becomes a clear image.

Reinforcement Learning: Learning from Experience

Teaching machines to make sequences of decisions by maximizing numerical rewards.

Interactive Case Study: Wumpus World

In this classic AI problem, the agent must find the Gold without falling into a Pit or being eaten by the Wumpus. The agent doesn't know the map; it only perceives sensors: a Breeze (🌬️) indicates a Pit, and a Stench (🤢) indicates the Wumpus. Through trial and error, it learns a Policy to reach the goal.

Artificial General Intelligence (AGI)

Beyond specialization. The theoretical leap to human-level versatility.

Artificial Intelligence today is powerful but narrow. AGI represents the point where a machine matches human cognitive abilities across any domain. It is not just about doing one thing better; it is about the ability to learn everything.

Model Hallucinations

Why statistical models confidently present falsehoods as facts.

Retrieval-Augmented Generation (RAG)

Grounding models in external facts to eliminate hallucinations.

To solve the problem of hallucinations and outdated knowledge, researchers developed Retrieval-Augmented Generation (RAG). Instead of relying purely on the model's static internal weights (its "memory"), a RAG system retrieves relevant documents from an external source (like a database or wiki) and attaches them to the user's prompt as context before generating the answer.