Artificial Intelligence Roadmap — Fundamentals to Advanced

1. Prerequisites

1.1 Programming fundamentals

  • Python syntax
  • Python functions
  • Python classes
  • Python modules
  • Python packages
  • Virtual environments
  • Dependency management
  • Type hints
  • Error handling
  • Iterators
  • Generators
  • Decorators
  • Context managers
  • Async basics
  • File I/O
  • JSON handling
  • CSV handling
  • Logging
  • Unit testing
  • Debugging

1.2 Software engineering fundamentals

  • Git
  • GitHub
  • Code review
  • Clean code
  • Modular design
  • API design
  • CLI design
  • Configuration management
  • Dependency injection
  • Packaging
  • Semantic versioning
  • CI/CD
  • Docker
  • Linux basics
  • Shell scripting
  • Environment variables
  • Secrets management

1.3 Data fundamentals

  • Data types
  • Structured data
  • Semi-structured data
  • Unstructured data
  • Tabular data
  • Text data
  • Image data
  • Audio data
  • Video data
  • Time-series data
  • Graph data
  • Metadata
  • Data quality
  • Data lineage
  • Data governance
  • Data privacy
  • Data versioning

1.4 Computer science fundamentals

  • Data structures
  • Algorithms
  • Complexity analysis
  • Hashing
  • Trees
  • Graphs
  • Sorting
  • Searching
  • Dynamic programming basics
  • Probability algorithms
  • Distributed systems basics
  • Operating system basics
  • Networking basics
  • Database basics

1.5 Hardware fundamentals

  • CPU
  • GPU
  • TPU
  • NPU
  • VRAM
  • RAM
  • Memory bandwidth
  • PCIe
  • NVLink
  • CUDA
  • ROCm
  • Tensor cores
  • Mixed precision
  • Distributed compute
  • Storage throughput
  • Network bandwidth
  • Inference latency

2. AI Landscape

2.1 Artificial Intelligence fundamentals

  • Artificial Intelligence
  • Narrow AI
  • General-purpose AI
  • Symbolic AI
  • Statistical AI
  • Machine learning
  • Deep learning
  • Reinforcement learning
  • Generative AI
  • Agentic AI
  • Multimodal AI
  • Embodied AI
  • Human-in-the-loop AI
  • AI-assisted software engineering

2.2 Core AI disciplines

  • Machine learning
  • Natural language processing
  • Computer vision
  • Speech processing
  • Audio processing
  • Recommendation systems
  • Reinforcement learning
  • Robotics
  • Planning
  • Knowledge representation
  • Search
  • Optimization
  • Generative modeling
  • AI safety
  • AI governance

2.3 AI system types

  • Classifier
  • Regressor
  • Ranker
  • Recommender
  • Detector
  • Segmenter
  • Generator
  • Summarizer
  • Translator
  • Agent
  • Planner
  • Retriever
  • Reasoner
  • Tutor
  • Copilot
  • Autonomous workflow system

2.4 AI development lifecycle

  • Problem definition
  • Success metric definition
  • Data collection
  • Data cleaning
  • Data labeling
  • Data exploration
  • Baseline model
  • Model training
  • Model evaluation
  • Error analysis
  • Model deployment
  • Monitoring
  • Feedback collection
  • Iteration
  • Governance review
  • Retirement

2.5 AI roles

  • Machine Learning Engineer
  • AI Engineer
  • Data Scientist
  • Data Engineer
  • Research Engineer
  • ML Platform Engineer
  • MLOps Engineer
  • LLMOps Engineer
  • AI Product Engineer
  • AI Safety Engineer
  • AI Security Engineer
  • Prompt Engineer
  • Evaluation Engineer
  • Applied Scientist

3. Mathematics for AI

3.1 Linear algebra

  • Scalar
  • Vector
  • Matrix
  • Tensor
  • Dot product
  • Matrix multiplication
  • Transpose
  • Inverse
  • Determinant
  • Rank
  • Norms
  • Eigenvalues
  • Eigenvectors
  • Singular Value Decomposition
  • Principal Component Analysis
  • Orthogonality
  • Projection
  • Basis
  • Linear transformation

3.2 Calculus

  • Function
  • Limit
  • Derivative
  • Partial derivative
  • Gradient
  • Jacobian
  • Hessian
  • Chain rule
  • Multivariable calculus
  • Optimization
  • Convexity
  • Local minimum
  • Global minimum
  • Saddle point
  • Taylor approximation

3.3 Probability

  • Random variable
  • Probability distribution
  • Conditional probability
  • Bayes theorem
  • Expectation
  • Variance
  • Covariance
  • Correlation
  • Independence
  • Joint distribution
  • Marginal distribution
  • Gaussian distribution
  • Bernoulli distribution
  • Binomial distribution
  • Poisson distribution
  • Multinomial distribution

3.4 Statistics

  • Population
  • Sample
  • Estimator
  • Bias
  • Variance
  • Confidence interval
  • Hypothesis testing
  • p-value
  • Statistical significance
  • A/B testing
  • Regression analysis
  • Maximum likelihood estimation
  • Bayesian estimation
  • Bootstrap
  • Cross-validation

3.5 Optimization

  • Objective function
  • Loss function
  • Cost function
  • Gradient descent
  • Stochastic gradient descent
  • Mini-batch gradient descent
  • Momentum
  • RMSProp
  • Adam
  • AdamW
  • Learning rate
  • Learning rate schedule
  • Convex optimization
  • Non-convex optimization
  • Constraint optimization
  • Lagrange multipliers

3.6 Information theory

  • Entropy
  • Cross-entropy
  • KL divergence
  • Mutual information
  • Perplexity
  • Compression
  • Information gain
  • Minimum description length
  • Noise
  • Signal
  • Channel capacity

4. Python AI Ecosystem

4.1 Core libraries

  • NumPy
  • pandas
  • SciPy
  • scikit-learn
  • statsmodels
  • Matplotlib
  • Plotly
  • Seaborn
  • Jupyter
  • Polars
  • PyArrow
  • Dask
  • Ray
  • DuckDB

4.2 Deep learning frameworks

  • PyTorch
  • TensorFlow
  • Keras
  • JAX
  • Flax
  • Haiku
  • Lightning
  • Accelerate
  • DeepSpeed
  • Megatron-LM
  • XLA
  • ONNX
  • TensorRT
  • OpenVINO

4.3 NLP and LLM libraries

  • Hugging Face Transformers
  • Hugging Face Datasets
  • Hugging Face Tokenizers
  • Hugging Face Evaluate
  • Sentence Transformers
  • spaCy
  • NLTK
  • Gensim
  • LangChain
  • LangGraph
  • LlamaIndex
  • Haystack
  • DSPy
  • Instructor
  • Guardrails
  • Outlines

4.4 Computer vision libraries

  • OpenCV
  • PIL/Pillow
  • torchvision
  • timm
  • Albumentations
  • Detectron2
  • MMDetection
  • Ultralytics
  • Kornia
  • Segment Anything tooling
  • ImageIO
  • scikit-image

4.5 MLOps libraries

  • MLflow
  • Weights & Biases
  • Neptune
  • DVC
  • ClearML
  • Feast
  • Great Expectations
  • Evidently
  • BentoML
  • KServe
  • Seldon
  • Kubeflow
  • Airflow
  • Prefect
  • Dagster

4.6 Inference libraries

  • vLLM
  • TensorRT-LLM
  • llama.cpp
  • Ollama
  • TGI
  • SGLang
  • ONNX Runtime
  • OpenVINO Runtime
  • Triton Inference Server
  • TorchServe
  • Ray Serve
  • BentoML
  • KServe
  • MLServer

5. Data Engineering for AI

5.1 Data collection

  • Internal databases
  • Public datasets
  • APIs
  • Web scraping
  • Logs
  • Events
  • Sensors
  • User feedback
  • Documents
  • Images
  • Audio
  • Video
  • Synthetic data
  • Human annotations

5.2 Data storage

  • Relational database
  • Document database
  • Object storage
  • Data lake
  • Data warehouse
  • Lakehouse
  • Vector database
  • Feature store
  • Metadata store
  • Model registry
  • Artifact store
  • Time-series database
  • Search index

5.3 Data formats

  • CSV
  • JSON
  • JSONL
  • Parquet
  • Arrow
  • Avro
  • ORC
  • TFRecord
  • HDF5
  • Zarr
  • Pickle
  • NumPy arrays
  • Image formats
  • Audio formats
  • Video formats

5.4 Data cleaning

  • Missing values
  • Duplicates
  • Outliers
  • Invalid values
  • Type conversion
  • Text normalization
  • Image normalization
  • Label correction
  • Data leakage detection
  • PII detection
  • PII removal
  • Noise reduction
  • Schema validation

5.5 Data labeling

  • Label schema
  • Annotation guidelines
  • Human labeling
  • Expert labeling
  • Weak supervision
  • Programmatic labeling
  • Active learning
  • Label quality
  • Inter-annotator agreement
  • Label noise
  • Label versioning
  • Label audit

5.6 Data pipelines

  • Batch pipeline
  • Streaming pipeline
  • ETL
  • ELT
  • Feature pipeline
  • Training pipeline
  • Inference pipeline
  • Evaluation pipeline
  • Feedback pipeline
  • Re-training pipeline
  • Data validation
  • Pipeline orchestration
  • Pipeline monitoring

6. Exploratory Data Analysis

6.1 Dataset understanding

  • Dataset shape
  • Column types
  • Missing values
  • Duplicate rows
  • Unique values
  • Cardinality
  • Distribution
  • Class balance
  • Correlations
  • Outliers
  • Data ranges
  • Target variable
  • Leakage candidates

6.2 Visualization

  • Histogram
  • Bar chart
  • Box plot
  • Scatter plot
  • Line plot
  • Heatmap
  • Pair plot
  • Confusion matrix plot
  • ROC curve
  • Precision-recall curve
  • Embedding projection
  • t-SNE
  • UMAP
  • PCA visualization

6.3 Statistical analysis

  • Summary statistics
  • Mean
  • Median
  • Mode
  • Variance
  • Standard deviation
  • Percentiles
  • Skewness
  • Kurtosis
  • Correlation
  • Covariance
  • Hypothesis testing
  • Confidence intervals

6.4 Data leakage analysis

  • Target leakage
  • Train-test contamination
  • Time leakage
  • Duplicate leakage
  • Feature leakage
  • Label leakage
  • Post-outcome variable
  • Cross-validation leakage
  • Preprocessing leakage
  • Augmentation leakage

7. Classical Machine Learning

7.1 ML fundamentals

  • Dataset
  • Feature
  • Label
  • Target
  • Model
  • Parameters
  • Hyperparameters
  • Training
  • Validation
  • Testing
  • Generalization
  • Overfitting
  • Underfitting
  • Bias
  • Variance
  • Regularization

7.2 Supervised learning

  • Classification
  • Regression
  • Binary classification
  • Multiclass classification
  • Multilabel classification
  • Linear regression
  • Logistic regression
  • k-nearest neighbors
  • Naive Bayes
  • Decision trees
  • Random forests
  • Gradient boosting
  • Support vector machines
  • Neural networks

7.3 Unsupervised learning

  • Clustering
  • Dimensionality reduction
  • Anomaly detection
  • Density estimation
  • k-means
  • DBSCAN
  • Hierarchical clustering
  • Gaussian mixture models
  • PCA
  • t-SNE
  • UMAP
  • Autoencoders
  • Isolation Forest
  • One-class SVM

7.4 Semi-supervised and weakly supervised learning

  • Semi-supervised learning
  • Weak supervision
  • Pseudo-labeling
  • Self-training
  • Label propagation
  • Consistency regularization
  • Data programming
  • Noisy labels
  • Label model
  • Confidence thresholding

7.5 Feature engineering

  • Numeric features
  • Categorical features
  • Text features
  • Date/time features
  • Interaction features
  • Aggregation features
  • Lag features
  • Rolling features
  • Target encoding
  • One-hot encoding
  • Ordinal encoding
  • Scaling
  • Normalization
  • Standardization
  • Imputation
  • Feature selection

7.6 Model selection

  • Train-test split
  • Validation split
  • Cross-validation
  • Stratified split
  • Time-series split
  • Nested cross-validation
  • Baseline model
  • Hyperparameter search
  • Grid search
  • Random search
  • Bayesian optimization
  • Early stopping
  • Model comparison

7.7 Classical ML evaluation

  • Accuracy
  • Precision
  • Recall
  • F1 score
  • ROC AUC
  • PR AUC
  • Log loss
  • Confusion matrix
  • Mean absolute error
  • Mean squared error
  • Root mean squared error
  • Calibration
  • Lift
  • Gain chart

8. Deep Learning Foundations

8.1 Neural network basics

  • Neuron
  • Layer
  • Weight
  • Bias
  • Activation function
  • Forward pass
  • Backward pass
  • Loss function
  • Gradient
  • Backpropagation
  • Computational graph
  • Automatic differentiation
  • Optimizer
  • Batch
  • Epoch
  • Step

8.2 Activation functions

  • Sigmoid
  • Tanh
  • ReLU
  • Leaky ReLU
  • GELU
  • SiLU
  • Softmax
  • Swish
  • ELU
  • Activation saturation
  • Vanishing gradients
  • Exploding gradients

8.3 Loss functions

  • Mean squared error
  • Mean absolute error
  • Binary cross-entropy
  • Categorical cross-entropy
  • Sparse categorical cross-entropy
  • Hinge loss
  • Contrastive loss
  • Triplet loss
  • KL divergence loss
  • Focal loss
  • Dice loss
  • Perceptual loss

8.4 Optimization in deep learning

  • SGD
  • Momentum
  • Nesterov momentum
  • RMSProp
  • Adam
  • AdamW
  • Adafactor
  • Learning rate schedule
  • Warmup
  • Cosine decay
  • Weight decay
  • Gradient clipping
  • Mixed precision training
  • Gradient accumulation

8.5 Regularization

  • L1 regularization
  • L2 regularization
  • Dropout
  • Weight decay
  • Batch normalization
  • Layer normalization
  • Data augmentation
  • Early stopping
  • Label smoothing
  • Stochastic depth
  • Noise injection
  • Model averaging

8.6 Neural network architectures

  • Multilayer perceptron
  • Convolutional neural network
  • Recurrent neural network
  • LSTM
  • GRU
  • Transformer
  • Autoencoder
  • Variational autoencoder
  • Generative adversarial network
  • Diffusion model
  • Graph neural network
  • Mixture of experts

8.7 Training diagnostics

  • Training loss
  • Validation loss
  • Learning curves
  • Overfitting detection
  • Underfitting detection
  • Gradient norm
  • Activation distribution
  • Weight distribution
  • Dead neurons
  • Data pipeline bottleneck
  • GPU utilization
  • Memory usage
  • Reproducibility

9. Computer Vision

9.1 Image fundamentals

  • Pixel
  • Channel
  • RGB
  • Grayscale
  • Resolution
  • Aspect ratio
  • Color space
  • Histogram
  • Convolution
  • Filter
  • Kernel
  • Edge detection
  • Image normalization
  • Image augmentation

9.2 CNNs

  • Convolution layer
  • Pooling layer
  • Padding
  • Stride
  • Receptive field
  • Feature map
  • Batch normalization
  • Residual connection
  • Depthwise convolution
  • Separable convolution
  • Dilated convolution
  • Transposed convolution
  • Global average pooling

9.3 Vision architectures

  • LeNet
  • AlexNet
  • VGG
  • Inception
  • ResNet
  • DenseNet
  • EfficientNet
  • MobileNet
  • ConvNeXt
  • Vision Transformer
  • Swin Transformer
  • CLIP vision encoder
  • Segment Anything models

9.4 Vision tasks

  • Image classification
  • Object detection
  • Semantic segmentation
  • Instance segmentation
  • Panoptic segmentation
  • Keypoint detection
  • Pose estimation
  • Image retrieval
  • Face recognition
  • OCR
  • Visual question answering
  • Image captioning
  • Image generation
  • Video understanding

9.5 Object detection

  • Bounding box
  • Anchor boxes
  • Non-maximum suppression
  • IoU
  • mAP
  • R-CNN
  • Fast R-CNN
  • Faster R-CNN
  • SSD
  • YOLO
  • DETR
  • RetinaNet
  • Focal loss

9.6 Image segmentation

  • Pixel-wise classification
  • U-Net
  • Mask R-CNN
  • DeepLab
  • FCN
  • SegFormer
  • SAM-style prompting
  • Dice coefficient
  • Jaccard index
  • Boundary metrics
  • Instance masks
  • Panoptic quality

9.7 Vision data engineering

  • Image labeling
  • Bounding box annotation
  • Polygon annotation
  • Keypoint annotation
  • Dataset balancing
  • Image augmentation
  • Synthetic images
  • Domain shift
  • Camera calibration
  • Data leakage in vision
  • Train/validation split by entity

10. Natural Language Processing

10.1 Text fundamentals

  • Token
  • Sentence
  • Document
  • Corpus
  • Vocabulary
  • Lemmatization
  • Stemming
  • Stop words
  • N-grams
  • Bag of words
  • TF-IDF
  • Text normalization
  • Language detection
  • Named entities

10.2 Tokenization

  • Word tokenization
  • Character tokenization
  • Subword tokenization
  • Byte Pair Encoding
  • WordPiece
  • SentencePiece
  • Byte-level tokenization
  • Vocabulary size
  • Special tokens
  • Padding
  • Truncation
  • Attention mask
  • Token budget

10.3 Classical NLP

  • Text classification
  • Sentiment analysis
  • Topic modeling
  • Named entity recognition
  • Part-of-speech tagging
  • Dependency parsing
  • Information extraction
  • Text similarity
  • Document clustering
  • Search ranking
  • Language modeling
  • Machine translation

10.4 Word embeddings

  • One-hot encoding
  • Word2Vec
  • CBOW
  • Skip-gram
  • GloVe
  • FastText
  • Contextual embeddings
  • Sentence embeddings
  • Embedding dimension
  • Cosine similarity
  • Semantic similarity
  • Embedding bias

10.5 Sequence models

  • RNN
  • Bidirectional RNN
  • LSTM
  • GRU
  • Encoder-decoder
  • Attention mechanism
  • Sequence-to-sequence
  • Beam search
  • Teacher forcing
  • Exposure bias
  • Sequence labeling
  • Conditional random fields

10.6 NLP evaluation

  • Accuracy
  • Precision
  • Recall
  • F1
  • BLEU
  • ROUGE
  • METEOR
  • BERTScore
  • Perplexity
  • Exact match
  • Semantic similarity
  • Human evaluation
  • Error taxonomy

11. Transformers

11.1 Transformer fundamentals

  • Encoder
  • Decoder
  • Encoder-decoder architecture
  • Self-attention
  • Cross-attention
  • Multi-head attention
  • Query
  • Key
  • Value
  • Attention score
  • Attention mask
  • Positional encoding
  • Feed-forward network
  • Layer normalization
  • Residual connection

11.2 Attention mechanisms

  • Scaled dot-product attention
  • Causal attention
  • Bidirectional attention
  • Cross-attention
  • Sparse attention
  • Sliding-window attention
  • FlashAttention
  • Multi-query attention
  • Grouped-query attention
  • Rotary positional embeddings
  • ALiBi
  • KV cache

11.3 Transformer model families

  • BERT-style encoder models
  • GPT-style decoder models
  • T5-style encoder-decoder models
  • Vision Transformers
  • Multimodal transformers
  • Sparse transformers
  • Mixture-of-experts transformers
  • Retrieval-augmented transformers
  • Long-context transformers
  • Reasoning-optimized models

11.4 Training transformers

  • Pretraining
  • Fine-tuning
  • Instruction tuning
  • Preference tuning
  • Masked language modeling
  • Causal language modeling
  • Sequence-to-sequence training
  • Next-token prediction
  • Packed sequences
  • Gradient checkpointing
  • Distributed training
  • Mixed precision
  • Token throughput

11.5 Transformer limitations

  • Context window limit
  • Quadratic attention cost
  • Hallucination
  • Positional extrapolation
  • Data contamination
  • Bias
  • Long-range dependency limits
  • Reasoning brittleness
  • Evaluation leakage
  • Interpretability limitations

12. Large Language Models

12.1 LLM fundamentals

  • Foundation model
  • Large language model
  • Parameter count
  • Token
  • Context window
  • Vocabulary
  • Pretraining corpus
  • Next-token prediction
  • Instruction tuning
  • Alignment
  • Chat model
  • Completion model
  • Reasoning model
  • Multimodal model
  • Model card

12.2 LLM capabilities

  • Text generation
  • Summarization
  • Translation
  • Question answering
  • Classification
  • Extraction
  • Code generation
  • Code review
  • Reasoning
  • Planning
  • Tool use
  • Data analysis
  • Multimodal understanding
  • Conversation

12.3 LLM limitations

  • Hallucination
  • Context loss
  • Prompt sensitivity
  • Non-determinism
  • Knowledge cutoff
  • Weak arithmetic
  • Weak long-horizon planning
  • Instruction conflict
  • Jailbreak susceptibility
  • Prompt injection susceptibility
  • Data leakage risk
  • Bias
  • Overconfidence

12.4 LLM model selection

  • Closed model
  • Open-weight model
  • Small language model
  • Large language model
  • Reasoning model
  • Code model
  • Multimodal model
  • Embedding model
  • Reranker model
  • Guardrail model
  • Latency requirement
  • Cost requirement
  • Privacy requirement
  • Hosting requirement
  • License requirement

12.5 LLM APIs

  • Chat completion API
  • Responses API
  • Streaming output
  • Function calling
  • Tool calling
  • Structured output
  • JSON schema output
  • Vision input
  • Audio input
  • Audio output
  • Batch API
  • Embeddings API
  • Moderation API
  • Realtime API

12.6 LLM decoding

  • Greedy decoding
  • Beam search
  • Temperature
  • Top-k sampling
  • Top-p sampling
  • Min-p sampling
  • Repetition penalty
  • Frequency penalty
  • Presence penalty
  • Stop sequences
  • Max tokens
  • Log probabilities
  • Determinism
  • Sampling trade-offs

13. Prompt Engineering

13.1 Prompt fundamentals

  • Instruction
  • Context
  • Input data
  • Output format
  • Constraints
  • Examples
  • Role prompting
  • System prompt
  • Developer prompt
  • User prompt
  • Prompt hierarchy
  • Prompt injection awareness
  • Prompt versioning

13.2 Prompting techniques

  • Zero-shot prompting
  • Few-shot prompting
  • Chain-of-thought prompting
  • Hidden reasoning control
  • Step-by-step prompting
  • Self-consistency
  • ReAct prompting
  • Plan-and-execute prompting
  • Reflection prompting
  • Critique-and-revise
  • Delimiter usage
  • Context compression
  • Structured prompting

13.3 Structured output

  • JSON output
  • JSON schema
  • Pydantic models
  • XML output
  • YAML output
  • Function call output
  • Tool call output
  • Enum constraints
  • Validation
  • Retry on invalid output
  • Output parser
  • Schema evolution

13.4 Prompt evaluation

  • Golden dataset
  • Prompt variants
  • A/B testing
  • Regression testing
  • Human evaluation
  • Automated evaluation
  • Task success metric
  • Format compliance
  • Factuality
  • Safety
  • Robustness
  • Latency
  • Cost
  • Prompt drift

13.5 Prompt security

  • Prompt injection
  • Indirect prompt injection
  • Data exfiltration
  • Tool misuse
  • Instruction hierarchy attack
  • Jailbreak
  • Context poisoning
  • Prompt leakage
  • Output sanitization
  • Tool permissioning
  • Untrusted input isolation

14. Retrieval-Augmented Generation

14.1 RAG fundamentals

  • Retrieval-augmented generation
  • Knowledge base
  • Document ingestion
  • Chunking
  • Embeddings
  • Vector index
  • Similarity search
  • Retriever
  • Context assembly
  • Generator
  • Citation generation
  • Grounded response
  • Answer synthesis
  • Retrieval evaluation

14.2 Document ingestion

  • PDF parsing
  • HTML parsing
  • Markdown parsing
  • DOCX parsing
  • Spreadsheet parsing
  • OCR
  • Table extraction
  • Image extraction
  • Metadata extraction
  • Document normalization
  • Deduplication
  • Access control metadata
  • Incremental ingestion

14.3 Chunking strategies

  • Fixed-size chunking
  • Sentence chunking
  • Paragraph chunking
  • Semantic chunking
  • Recursive chunking
  • Sliding window chunking
  • Parent-child chunking
  • Table-aware chunking
  • Code-aware chunking
  • Chunk overlap
  • Chunk size tuning
  • Metadata enrichment

14.4 Embeddings

  • Dense embeddings
  • Sparse embeddings
  • Hybrid embeddings
  • Sentence embeddings
  • Document embeddings
  • Query embeddings
  • Embedding dimension
  • Similarity metric
  • Cosine similarity
  • Dot product
  • Euclidean distance
  • Embedding normalization
  • Embedding model selection
  • Re-embedding strategy

14.5 Vector databases

  • FAISS
  • pgvector
  • Milvus
  • Qdrant
  • Weaviate
  • Pinecone
  • Chroma
  • Vespa
  • Elasticsearch vector search
  • OpenSearch vector search
  • Redis vector search
  • MongoDB vector search
  • Index tuning
  • Metadata filtering
  • Multi-tenancy

14.6 Retrieval strategies

  • Dense retrieval
  • Sparse retrieval
  • BM25
  • Hybrid retrieval
  • Multi-query retrieval
  • Query expansion
  • Query rewriting
  • HyDE
  • Parent document retrieval
  • Contextual compression
  • MMR
  • Reranking
  • Cross-encoder reranking
  • ColBERT-style retrieval

14.7 RAG evaluation

  • Retrieval precision
  • Retrieval recall
  • Hit rate
  • MRR
  • NDCG
  • Context relevance
  • Faithfulness
  • Answer relevance
  • Groundedness
  • Citation accuracy
  • Hallucination rate
  • Latency
  • Cost
  • Human evaluation

14.8 Advanced RAG

  • Agentic RAG
  • Graph RAG
  • Corrective RAG
  • Adaptive RAG
  • Self-RAG
  • Multi-hop RAG
  • Tool-augmented RAG
  • Multimodal RAG
  • SQL RAG
  • Code RAG
  • Enterprise RAG
  • Access-controlled RAG
  • Federated retrieval

15. AI Agents

15.1 Agent fundamentals

  • Agent
  • Goal
  • Task
  • Environment
  • Observation
  • Action
  • Tool
  • Memory
  • Planning
  • Reflection
  • Execution loop
  • Termination condition
  • Human approval
  • Autonomy level

15.2 Agent architectures

  • ReAct agent
  • Plan-and-execute agent
  • Tool-calling agent
  • Workflow agent
  • Router agent
  • Reflection agent
  • Critic agent
  • Multi-agent system
  • Supervisor-worker architecture
  • Graph-based agent
  • State machine agent
  • Event-driven agent
  • Long-running agent

15.3 Tool use

  • Function calling
  • Tool schema
  • Tool description
  • Tool validation
  • Tool authorization
  • Tool sandboxing
  • Tool timeout
  • Tool retries
  • Tool error handling
  • Tool result summarization
  • Tool result verification
  • Tool audit logging
  • Tool budget
  • Tool registry

15.4 Model Context Protocol

  • MCP client
  • MCP server
  • MCP tools
  • MCP resources
  • MCP prompts
  • Tool discovery
  • Server transport
  • Authorization
  • Consent flow
  • Data source integration
  • Tool execution safety
  • MCP security review
  • MCP observability
  • MCP production readiness

15.5 Agent memory

  • Short-term memory
  • Long-term memory
  • Episodic memory
  • Semantic memory
  • Working memory
  • Conversation memory
  • Vector memory
  • Summary memory
  • Memory retrieval
  • Memory consolidation
  • Memory deletion
  • Memory privacy
  • Memory poisoning prevention

15.6 Agent planning

  • Task decomposition
  • Sequential planning
  • Hierarchical planning
  • Constraint planning
  • Tool planning
  • Plan validation
  • Plan repair
  • Replanning
  • Cost-aware planning
  • Time-aware planning
  • Dependency graph
  • Execution trace
  • Planner-evaluator loop

15.7 Agent evaluation

  • Task success rate
  • Tool call correctness
  • Tool call efficiency
  • Goal completion
  • Step count
  • Latency
  • Cost
  • Error recovery
  • Human intervention rate
  • Safety violation rate
  • Regression tests
  • Simulation environment
  • Trace review
  • Production monitoring

15.8 Agent safety

  • Excessive agency
  • Tool misuse
  • Prompt injection
  • Indirect prompt injection
  • Data exfiltration
  • Unauthorized action
  • Irreversible action
  • Human-in-the-loop gate
  • Permission scoping
  • Sandboxing
  • Transaction preview
  • Rollback capability
  • Rate limiting
  • Audit trail

16. Multimodal AI

16.1 Multimodal fundamentals

  • Text modality
  • Image modality
  • Audio modality
  • Video modality
  • Sensor modality
  • Modality fusion
  • Cross-modal alignment
  • Contrastive learning
  • Multimodal embeddings
  • Vision-language model
  • Audio-language model
  • Video-language model
  • Multimodal reasoning

16.2 Vision-language models

  • Image captioning
  • Visual question answering
  • OCR-aware reasoning
  • Chart understanding
  • Document understanding
  • Object grounding
  • Referring expression comprehension
  • Image-text retrieval
  • Visual instruction tuning
  • Multimodal prompting
  • Visual hallucination
  • Image safety

16.3 Audio AI

  • Speech recognition
  • Speech synthesis
  • Speaker identification
  • Speaker diarization
  • Audio classification
  • Music generation
  • Voice activity detection
  • Wake word detection
  • Speech enhancement
  • Audio embeddings
  • Real-time audio inference
  • Streaming transcription

16.4 Video AI

  • Video classification
  • Action recognition
  • Video captioning
  • Video question answering
  • Object tracking
  • Temporal localization
  • Video segmentation
  • Video generation
  • Frame sampling
  • Temporal attention
  • Long-video understanding
  • Video safety

16.5 Multimodal RAG

  • Text retrieval
  • Image retrieval
  • Audio retrieval
  • Video retrieval
  • Document image retrieval
  • OCR pipeline
  • Table extraction
  • Chart extraction
  • Multimodal embeddings
  • Cross-modal reranking
  • Multimodal answer grounding
  • Citation to source region

17. Generative AI

17.1 Generative modeling fundamentals

  • Generative model
  • Discriminative model
  • Latent space
  • Sampling
  • Likelihood
  • Conditional generation
  • Unconditional generation
  • Autoregressive generation
  • Diffusion generation
  • Adversarial generation
  • Variational generation
  • Controllability
  • Diversity
  • Fidelity

17.2 Diffusion models

  • Forward diffusion
  • Reverse diffusion
  • Noise schedule
  • Denoising model
  • U-Net
  • Latent diffusion
  • Classifier guidance
  • Classifier-free guidance
  • DDPM
  • DDIM
  • Stable Diffusion-style models
  • ControlNet-style conditioning
  • Inpainting
  • Outpainting
  • Image-to-image generation

17.3 Generative adversarial networks

  • Generator
  • Discriminator
  • Adversarial loss
  • Mode collapse
  • Training instability
  • DCGAN
  • StyleGAN
  • Conditional GAN
  • CycleGAN
  • Wasserstein GAN
  • GAN evaluation
  • FID score

17.4 Variational autoencoders

  • Encoder
  • Decoder
  • Latent variable
  • Reparameterization trick
  • ELBO
  • KL divergence
  • Reconstruction loss
  • Latent interpolation
  • Conditional VAE
  • Beta-VAE
  • VAE limitations

17.5 Generative AI product patterns

  • Text generation
  • Image generation
  • Video generation
  • Audio generation
  • Code generation
  • Document generation
  • Synthetic data generation
  • Personalized content
  • Creative assistant
  • Enterprise assistant
  • AI copilot
  • Workflow automation

17.6 Generative AI risks

  • Hallucination
  • Deepfakes
  • Copyright risk
  • Privacy leakage
  • Bias amplification
  • Prompt injection
  • Unsafe content
  • Model misuse
  • Overreliance
  • Attribution failure
  • Watermarking limitations
  • Provenance tracking

18. Fine-Tuning and Adaptation

18.1 Fine-tuning fundamentals

  • Full fine-tuning
  • Feature extraction
  • Transfer learning
  • Domain adaptation
  • Task adaptation
  • Supervised fine-tuning
  • Instruction tuning
  • Preference tuning
  • Continual learning
  • Catastrophic forgetting
  • Data mixture
  • Learning rate selection

18.2 Parameter-efficient fine-tuning

  • LoRA
  • QLoRA
  • AdaLoRA
  • Prefix tuning
  • Prompt tuning
  • P-tuning
  • Adapters
  • IA3
  • BitFit
  • PEFT library
  • Rank selection
  • Target modules
  • Adapter merging

18.3 LLM fine-tuning data

  • Instruction dataset
  • Conversation dataset
  • Preference dataset
  • Domain corpus
  • Synthetic dataset
  • Data filtering
  • Data deduplication
  • Data quality scoring
  • Safety filtering
  • Format standardization
  • Train-validation split
  • Contamination detection

18.4 Alignment methods

  • RLHF
  • RLAIF
  • DPO
  • IPO
  • ORPO
  • KTO
  • Constitutional AI
  • Reward modeling
  • Preference optimization
  • Safety fine-tuning
  • Refusal tuning
  • Harmlessness tuning

18.5 Fine-tuning evaluation

  • Base model comparison
  • Task benchmark
  • Domain benchmark
  • Safety benchmark
  • Overfitting check
  • Regression suite
  • Human evaluation
  • Win rate
  • Loss curves
  • Catastrophic forgetting check
  • Robustness test
  • Deployment A/B test

18.6 Fine-tuning operations

  • Dataset versioning
  • Experiment tracking
  • Checkpointing
  • Hyperparameter logging
  • GPU memory planning
  • Distributed fine-tuning
  • Quantized training
  • Adapter storage
  • Model registry
  • Rollback strategy
  • License compliance
  • Safety review

19. Reinforcement Learning

19.1 RL fundamentals

  • Agent
  • Environment
  • State
  • Observation
  • Action
  • Reward
  • Policy
  • Value function
  • Q-function
  • Episode
  • Trajectory
  • Return
  • Discount factor
  • Exploration
  • Exploitation

19.2 Classical RL

  • Markov Decision Process
  • Bellman equation
  • Dynamic programming
  • Policy iteration
  • Value iteration
  • Monte Carlo methods
  • Temporal difference learning
  • SARSA
  • Q-learning
  • Eligibility traces

19.3 Deep RL

  • Deep Q-Network
  • Experience replay
  • Target network
  • Policy gradient
  • Actor-critic
  • A2C
  • A3C
  • PPO
  • DDPG
  • SAC
  • TD3
  • Distributional RL
  • Offline RL
  • Imitation learning

19.4 RLHF and preference optimization

  • Human feedback
  • Preference dataset
  • Reward model
  • Policy optimization
  • KL penalty
  • PPO for RLHF
  • Direct preference optimization
  • AI feedback
  • Reward hacking
  • Alignment tax
  • Safety constraints
  • Evaluation

19.5 RL applications

  • Game playing
  • Robotics
  • Recommendation optimization
  • Ad bidding
  • Resource allocation
  • Traffic control
  • Operations research
  • Dialogue systems
  • Agent training
  • Tool-use policy learning

20. Recommendation Systems

20.1 Recommender fundamentals

  • User
  • Item
  • Interaction
  • Rating
  • Implicit feedback
  • Explicit feedback
  • Candidate generation
  • Ranking
  • Re-ranking
  • Cold start
  • Exploration
  • Exploitation
  • Personalization
  • Diversity

20.2 Collaborative filtering

  • User-based collaborative filtering
  • Item-based collaborative filtering
  • Matrix factorization
  • Alternating least squares
  • Singular Value Decomposition
  • Implicit matrix factorization
  • Neighborhood methods
  • Latent factors
  • Similarity metrics
  • Sparse interactions

20.3 Content-based recommendation

  • Item features
  • User profile
  • Text embeddings
  • Image embeddings
  • Metadata features
  • Similarity search
  • Feature weighting
  • Hybrid recommendation
  • Cold-start mitigation
  • Explainability

20.4 Deep recommenders

  • Two-tower model
  • Neural collaborative filtering
  • Wide and deep model
  • DeepFM
  • Sequence recommender
  • Transformer recommender
  • Graph recommender
  • Retrieval model
  • Ranking model
  • Negative sampling
  • Hard negatives

20.5 Recommender evaluation

  • Precision@k
  • Recall@k
  • MAP
  • NDCG
  • Hit rate
  • Coverage
  • Diversity
  • Novelty
  • Serendipity
  • Calibration
  • Offline evaluation
  • Online A/B testing
  • Counterfactual evaluation

21. Graph Machine Learning

21.1 Graph fundamentals

  • Node
  • Edge
  • Directed graph
  • Undirected graph
  • Weighted graph
  • Heterogeneous graph
  • Knowledge graph
  • Adjacency matrix
  • Graph traversal
  • Community detection
  • Centrality
  • Graph embeddings

21.2 Graph neural networks

  • Message passing
  • Node embedding
  • Edge embedding
  • Graph embedding
  • Graph convolution
  • GCN
  • GraphSAGE
  • GAT
  • GIN
  • Relational GCN
  • Graph transformer
  • Over-smoothing
  • Neighbor sampling

21.3 Graph ML tasks

  • Node classification
  • Edge prediction
  • Link prediction
  • Graph classification
  • Community detection
  • Recommendation
  • Fraud detection
  • Knowledge graph completion
  • Molecular property prediction
  • Traffic prediction

21.4 Knowledge graphs and AI

  • Entity
  • Relation
  • Triple
  • Ontology
  • RDF
  • SPARQL
  • Entity linking
  • Relation extraction
  • Graph RAG
  • Knowledge graph embeddings
  • Hybrid symbolic-neural reasoning

22. Time-Series AI

22.1 Time-series fundamentals

  • Timestamp
  • Sampling rate
  • Trend
  • Seasonality
  • Noise
  • Stationarity
  • Autocorrelation
  • Lag
  • Rolling window
  • Resampling
  • Missing timestamps
  • Forecast horizon
  • Backtesting

22.2 Classical time-series models

  • Moving average
  • Exponential smoothing
  • AR
  • MA
  • ARMA
  • ARIMA
  • SARIMA
  • VAR
  • Prophet-style models
  • Kalman filter
  • State-space models

22.3 Deep time-series models

  • RNN forecasting
  • LSTM forecasting
  • Temporal CNN
  • Transformer forecasting
  • Temporal Fusion Transformer
  • N-BEATS
  • DeepAR
  • Patch-based models
  • Multivariate forecasting
  • Probabilistic forecasting

22.4 Time-series tasks

  • Forecasting
  • Anomaly detection
  • Classification
  • Segmentation
  • Change point detection
  • Imputation
  • Event prediction
  • Capacity planning
  • Demand forecasting
  • Predictive maintenance

22.5 Time-series evaluation

  • MAE
  • MSE
  • RMSE
  • MAPE
  • SMAPE
  • MASE
  • Pinball loss
  • Prediction interval coverage
  • Rolling backtest
  • Time-series cross-validation
  • Leakage prevention

23. Evaluation and Benchmarking

23.1 Evaluation fundamentals

  • Ground truth
  • Golden dataset
  • Test set
  • Validation set
  • Holdout set
  • Benchmark
  • Metric
  • Baseline
  • Error analysis
  • Regression testing
  • Confidence interval
  • Statistical significance
  • Human evaluation

23.2 ML evaluation

  • Classification metrics
  • Regression metrics
  • Ranking metrics
  • Calibration metrics
  • Fairness metrics
  • Robustness metrics
  • Latency metrics
  • Cost metrics
  • Business metrics
  • Drift metrics
  • Data quality metrics

23.3 LLM evaluation

  • Task success rate
  • Factuality
  • Groundedness
  • Relevance
  • Completeness
  • Instruction following
  • Format compliance
  • Reasoning quality
  • Tool-use correctness
  • Safety compliance
  • Refusal quality
  • Hallucination rate
  • Context utilization
  • Token efficiency

23.4 RAG evaluation

  • Retrieval recall
  • Retrieval precision
  • Hit rate
  • MRR
  • NDCG
  • Context relevance
  • Answer faithfulness
  • Answer relevance
  • Citation accuracy
  • Source coverage
  • Retrieval latency
  • End-to-end latency
  • End-to-end cost

23.5 Agent evaluation

  • Goal completion rate
  • Step accuracy
  • Tool call validity
  • Tool call efficiency
  • Recovery from errors
  • Human intervention rate
  • Unsafe action rate
  • State consistency
  • Trace quality
  • Cost per task
  • Latency per task
  • Long-horizon reliability

23.6 Evaluation methods

  • Exact-match evaluation
  • Rule-based evaluation
  • Model-based evaluation
  • LLM-as-judge
  • Pairwise comparison
  • Rubric-based evaluation
  • Human rating
  • Expert review
  • Adversarial testing
  • Red teaming
  • A/B testing
  • Shadow testing
  • Canary testing

23.7 Evaluation risks

  • Benchmark contamination
  • Overfitting to benchmark
  • Judge bias
  • Metric mismatch
  • Poor ground truth
  • Non-representative test data
  • Distribution shift
  • Hidden confounders
  • Prompt sensitivity
  • Evaluation leakage
  • Human evaluator inconsistency

24. AI Safety and Security

24.1 AI safety fundamentals

  • Safety requirement
  • Harm taxonomy
  • Risk assessment
  • Safety policy
  • Model behavior specification
  • Refusal behavior
  • Safe completion
  • Uncertainty communication
  • Human oversight
  • Fail-safe behavior
  • Monitoring
  • Incident response

24.2 LLM security risks

  • Prompt injection
  • Indirect prompt injection
  • Insecure output handling
  • Training data poisoning
  • Model denial of service
  • Supply chain vulnerability
  • Sensitive information disclosure
  • Insecure plugin design
  • Excessive agency
  • Overreliance
  • Model theft
  • Jailbreak
  • Tool abuse

24.3 Adversarial ML

  • Adversarial examples
  • Evasion attack
  • Poisoning attack
  • Backdoor attack
  • Model inversion
  • Membership inference
  • Model extraction
  • Data exfiltration
  • Robust training
  • Adversarial testing
  • Red team evaluation

24.4 Guardrails

  • Input filtering
  • Output filtering
  • Policy classifier
  • Content moderation
  • PII detection
  • Secrets detection
  • Structured output validation
  • Tool permissioning
  • Action approval
  • Sandboxing
  • Rate limiting
  • Context isolation
  • Response verification

24.5 AI application security

  • Threat modeling
  • Secure prompt design
  • Secure tool design
  • Data boundary design
  • Least privilege
  • Secret isolation
  • Audit logging
  • Dependency scanning
  • Model supply chain scanning
  • Dataset provenance
  • Prompt injection testing
  • Authorization checks
  • Tenant isolation

24.6 Privacy

  • PII handling
  • Data minimization
  • Consent
  • Purpose limitation
  • Data retention
  • Data deletion
  • Anonymization
  • Pseudonymization
  • Differential privacy
  • Federated learning
  • Private inference
  • Confidential computing
  • Privacy impact assessment

24.7 Responsible AI

  • Fairness
  • Bias detection
  • Bias mitigation
  • Transparency
  • Explainability
  • Accountability
  • Human agency
  • Contestability
  • Accessibility
  • Inclusiveness
  • Environmental impact
  • Social impact
  • Documentation

25. Explainability and Interpretability

25.1 Explainability fundamentals

  • Interpretability
  • Explainability
  • Transparency
  • Local explanation
  • Global explanation
  • Feature importance
  • Model-agnostic explanation
  • Model-specific explanation
  • Counterfactual explanation
  • Causal explanation
  • Human-centered explanation

25.2 Classical ML explainability

  • Linear model coefficients
  • Decision tree paths
  • Feature importance
  • Permutation importance
  • Partial dependence plots
  • Individual conditional expectation
  • SHAP
  • LIME
  • Counterfactual examples
  • Calibration curves

25.3 Deep learning interpretability

  • Saliency maps
  • Grad-CAM
  • Integrated gradients
  • Activation visualization
  • Feature visualization
  • Attention visualization
  • Concept activation vectors
  • Layer attribution
  • Neuron analysis
  • Representation probing

25.4 LLM interpretability

  • Attention analysis
  • Logit lens
  • Activation patching
  • Mechanistic interpretability
  • Feature circuits
  • Sparse autoencoders
  • Token attribution
  • Prompt sensitivity analysis
  • Behavior probing
  • Explanation faithfulness

26. MLOps

26.1 MLOps fundamentals

  • Experiment tracking
  • Dataset versioning
  • Feature versioning
  • Model versioning
  • Model registry
  • Pipeline orchestration
  • Reproducibility
  • Environment management
  • Artifact management
  • Model packaging
  • Deployment automation
  • Monitoring
  • Governance

26.2 Experiment management

  • Run tracking
  • Parameters
  • Metrics
  • Artifacts
  • Tags
  • Notes
  • Source code version
  • Dataset version
  • Model checkpoint
  • Comparison dashboard
  • Reproducibility
  • Experiment lineage

26.3 Feature stores

  • Feature definition
  • Feature registry
  • Offline store
  • Online store
  • Feature pipeline
  • Point-in-time correctness
  • Feature freshness
  • Feature reuse
  • Feature monitoring
  • Training-serving skew
  • Entity key
  • Feature view

26.4 Model registry

  • Registered model
  • Model version
  • Model stage
  • Model alias
  • Approval workflow
  • Model card
  • Artifact storage
  • Lineage
  • Rollback
  • Deployment promotion
  • Governance metadata
  • Audit trail

26.5 ML pipelines

  • Data ingestion step
  • Data validation step
  • Feature engineering step
  • Training step
  • Evaluation step
  • Model registration step
  • Deployment step
  • Batch inference step
  • Monitoring step
  • Retraining trigger
  • Pipeline scheduling
  • Pipeline caching

26.6 Model monitoring

  • Data drift
  • Concept drift
  • Prediction drift
  • Feature drift
  • Label drift
  • Model performance
  • Latency
  • Throughput
  • Error rate
  • Resource usage
  • Fairness drift
  • Alerting
  • Incident response

27. LLMOps and AgentOps

27.1 LLMOps fundamentals

  • Prompt management
  • Prompt versioning
  • Prompt registry
  • Model routing
  • Model fallback
  • Model gateway
  • Token tracking
  • Cost tracking
  • Latency tracking
  • Evaluation datasets
  • Regression testing
  • Safety monitoring
  • Trace logging
  • Feedback collection

27.2 Prompt operations

  • Prompt template
  • Prompt variables
  • Prompt deployment
  • Prompt A/B testing
  • Prompt rollback
  • Prompt ownership
  • Prompt approval
  • Prompt security review
  • Prompt drift monitoring
  • Prompt performance tracking

27.3 LLM observability

  • Request trace
  • Response trace
  • Prompt trace
  • Tool call trace
  • Retrieval trace
  • Token usage
  • Cost per request
  • Latency per stage
  • Model error rate
  • Safety event
  • User feedback
  • Session replay
  • Production debugging

27.4 Agent operations

  • Agent trace
  • State transition log
  • Tool call log
  • Memory access log
  • Planner output log
  • Action approval log
  • Error recovery log
  • Human intervention log
  • Policy violation log
  • Task success dashboard
  • Long-running task monitoring
  • Agent rollback
  • Agent kill switch

27.5 Model gateway

  • Provider abstraction
  • Model routing
  • Fallback model
  • Retry policy
  • Rate limiting
  • Budget enforcement
  • Logging
  • Redaction
  • Policy enforcement
  • Caching
  • Load balancing
  • Tenant isolation
  • Audit trail

28. Deployment and Inference

28.1 Deployment patterns

  • Batch inference
  • Online inference
  • Streaming inference
  • Edge inference
  • On-device inference
  • Serverless inference
  • Dedicated endpoint
  • Multi-model endpoint
  • Model gateway
  • Shadow deployment
  • Canary deployment
  • Blue-green deployment

28.2 Model serving

  • REST endpoint
  • gRPC endpoint
  • WebSocket endpoint
  • Streaming responses
  • Request batching
  • Dynamic batching
  • Continuous batching
  • Model warmup
  • Autoscaling
  • Health checks
  • Readiness checks
  • Timeout handling
  • Error handling

28.3 LLM inference optimization

  • KV cache
  • Paged attention
  • Continuous batching
  • Speculative decoding
  • Prefix caching
  • Prompt caching
  • Quantization
  • Tensor parallelism
  • Pipeline parallelism
  • Expert parallelism
  • FlashAttention
  • CUDA graphs
  • Token throughput
  • Time to first token

28.4 Quantization

  • FP32
  • FP16
  • BF16
  • FP8
  • INT8
  • INT4
  • Weight-only quantization
  • Activation quantization
  • Post-training quantization
  • Quantization-aware training
  • GPTQ
  • AWQ
  • GGUF
  • Accuracy-latency trade-off

28.5 Distillation and compression

  • Knowledge distillation
  • Teacher model
  • Student model
  • Logit distillation
  • Feature distillation
  • Pruning
  • Sparsity
  • Low-rank approximation
  • Model merging
  • Weight sharing
  • Compression ratio
  • Accuracy retention

28.6 Edge AI deployment

  • Mobile inference
  • Browser inference
  • Embedded inference
  • Quantized models
  • ONNX Runtime
  • Core ML
  • TensorFlow Lite
  • WebGPU
  • WebNN
  • OpenVINO
  • Latency budget
  • Battery budget
  • Memory budget

29. AI Infrastructure

29.1 Compute infrastructure

  • GPU instance
  • GPU cluster
  • TPU cluster
  • CPU inference
  • Heterogeneous compute
  • GPU scheduling
  • GPU sharing
  • MIG
  • CUDA
  • ROCm
  • Driver compatibility
  • Container runtime
  • Kubernetes GPU operator
  • Resource quotas

29.2 Distributed training

  • Data parallelism
  • Model parallelism
  • Tensor parallelism
  • Pipeline parallelism
  • Expert parallelism
  • ZeRO optimization
  • Gradient accumulation
  • Gradient checkpointing
  • All-reduce
  • NCCL
  • FSDP
  • DeepSpeed
  • Megatron
  • Distributed checkpointing

29.3 Storage infrastructure

  • Dataset storage
  • Object storage
  • Parallel filesystem
  • Local NVMe cache
  • Training checkpoint storage
  • Model artifact storage
  • Vector index storage
  • Feature store storage
  • Data loading throughput
  • Sharded datasets
  • Streaming datasets
  • Storage cost optimization

29.4 Network infrastructure

  • Ethernet
  • InfiniBand
  • RDMA
  • NVLink
  • NCCL communication
  • All-reduce bandwidth
  • Latency
  • Topology awareness
  • Multi-node training
  • Multi-region inference
  • Data transfer cost
  • Network bottleneck analysis

29.5 Kubernetes for AI

  • GPU node pool
  • NVIDIA device plugin
  • GPU Operator
  • Kueue
  • Volcano
  • Ray on Kubernetes
  • Kubeflow
  • KServe
  • Seldon
  • Triton Inference Server
  • vLLM on Kubernetes
  • Autoscaling inference
  • Multi-tenant GPU clusters
  • Cost allocation

29.6 Cost optimization

  • Token cost
  • GPU hour cost
  • Storage cost
  • Network cost
  • Model routing by cost
  • Prompt caching
  • Response caching
  • Batch inference
  • Spot instances
  • Quantization
  • Smaller model selection
  • Autoscaling
  • Idle GPU detection
  • Cost dashboards

30. AI Product Engineering

30.1 Product discovery

  • User problem
  • AI suitability
  • Non-AI baseline
  • User workflow
  • Success metric
  • Failure cost
  • Automation level
  • Human review requirement
  • Trust requirement
  • Latency requirement
  • Privacy requirement
  • Cost requirement
  • Regulatory requirement

30.2 AI UX

  • Conversation design
  • Prompt input UX
  • Output preview
  • Confidence communication
  • Citation display
  • Source traceability
  • Editability
  • Human approval
  • Undo
  • Feedback collection
  • Error recovery
  • Safe refusal UX
  • Progressive disclosure

30.3 Human-in-the-loop workflows

  • Human review
  • Human approval
  • Human correction
  • Escalation
  • Expert validation
  • Feedback loop
  • Active learning loop
  • Quality assurance
  • Audit queue
  • Override mechanism
  • Responsibility boundary

30.4 AI product metrics

  • Task completion rate
  • Time saved
  • User satisfaction
  • Acceptance rate
  • Edit distance
  • Deflection rate
  • Human escalation rate
  • Error rate
  • Hallucination rate
  • Safety event rate
  • Cost per task
  • Latency
  • Retention
  • Business impact

30.5 AI product risks

  • Overautomation
  • User overreliance
  • False confidence
  • Poor explainability
  • Cost explosion
  • Privacy exposure
  • Security exposure
  • Bias harm
  • Regulatory exposure
  • Brand risk
  • Model drift
  • Vendor lock-in

31. AI Governance and Compliance

31.1 Governance fundamentals

  • AI policy
  • AI inventory
  • Model inventory
  • Dataset inventory
  • Risk classification
  • Approval workflow
  • Documentation
  • Audit trail
  • Accountability
  • Ownership
  • Review cadence
  • Incident process
  • Retirement process

31.2 Documentation artifacts

  • Model card
  • Dataset card
  • System card
  • Risk assessment
  • Evaluation report
  • Safety report
  • Data protection impact assessment
  • Security review
  • Architecture decision record
  • Prompt documentation
  • Tool documentation
  • Deployment runbook

31.3 AI risk management

  • Risk identification
  • Risk analysis
  • Risk scoring
  • Risk mitigation
  • Risk monitoring
  • Risk acceptance
  • Harm taxonomy
  • Misuse analysis
  • Failure mode analysis
  • Red teaming
  • Control mapping
  • Residual risk tracking

31.4 Regulatory awareness

  • EU AI Act awareness
  • NIST AI RMF awareness
  • ISO/IEC AI standards awareness
  • GDPR awareness
  • Copyright awareness
  • Data residency awareness
  • Sector-specific regulation
  • High-risk AI classification
  • Transparency obligations
  • Human oversight obligations
  • Record-keeping obligations
  • Vendor governance

31.5 AI audit readiness

  • Model lineage
  • Data lineage
  • Decision logging
  • Access logging
  • Prompt logging
  • Tool call logging
  • Evaluation evidence
  • Safety evidence
  • Approval evidence
  • Incident evidence
  • Change management evidence
  • Retention policy

32. AI Ethics and Societal Impact

32.1 Fairness

  • Protected attributes
  • Sensitive attributes
  • Group fairness
  • Individual fairness
  • Demographic parity
  • Equalized odds
  • Equal opportunity
  • Disparate impact
  • Bias detection
  • Bias mitigation
  • Fairness monitoring
  • Fairness trade-offs

32.2 Transparency

  • User disclosure
  • AI-generated content disclosure
  • Model limitations
  • Data source transparency
  • Citation
  • Explanation
  • Confidence communication
  • Uncertainty communication
  • Auditability
  • Documentation

32.3 Accountability

  • Human owner
  • Model owner
  • Data owner
  • Product owner
  • Review board
  • Escalation path
  • Incident owner
  • User appeal process
  • Correction process
  • Decommissioning process

32.4 Environmental impact

  • Training energy cost
  • Inference energy cost
  • Carbon accounting
  • Model size trade-off
  • Quantization
  • Distillation
  • Efficient inference
  • Hardware utilization
  • Batch efficiency
  • Green AI metrics

33. AI for Software Engineering

33.1 Code intelligence

  • Code completion
  • Code generation
  • Code explanation
  • Code search
  • Code review
  • Bug detection
  • Refactoring
  • Test generation
  • Documentation generation
  • Migration assistance
  • Dependency analysis
  • Security analysis

33.2 AI coding agents

  • Repository understanding
  • Issue analysis
  • Task planning
  • Code editing
  • Patch generation
  • Test execution
  • Error repair
  • Pull request creation
  • Code review response
  • CI failure debugging
  • Agent sandbox
  • Permission control

33.3 Code RAG

  • Repository indexing
  • Symbol extraction
  • AST parsing
  • Chunking code
  • Embedding code
  • Semantic code search
  • Dependency graph
  • Call graph
  • Documentation retrieval
  • Issue retrieval
  • Pull request retrieval
  • Context assembly

33.4 AI-assisted SDLC

  • Requirements analysis
  • Technical design
  • Architecture review
  • Implementation
  • Testing
  • Security review
  • Performance review
  • Documentation
  • Deployment
  • Monitoring
  • Incident response
  • Postmortem generation

33.5 Risks in AI coding

  • Insecure generated code
  • Hallucinated APIs
  • License contamination
  • Dependency risk
  • Secret leakage
  • Incorrect tests
  • Overconfident explanations
  • Production data misuse
  • Unauthorized changes
  • Lack of review
  • Prompt injection through repo content

34. AI Research Literacy

34.1 Reading papers

  • Abstract
  • Introduction
  • Related work
  • Method
  • Experiments
  • Results
  • Ablation study
  • Limitations
  • Appendix
  • Reproducibility
  • Dataset details
  • Evaluation setup
  • Baseline comparison

34.2 Research evaluation

  • Problem significance
  • Novelty
  • Method soundness
  • Experimental rigor
  • Dataset quality
  • Metric appropriateness
  • Statistical significance
  • Compute budget
  • Reproducibility
  • Generalization
  • Failure cases
  • Practical relevance

34.3 Keeping up to date

  • arXiv
  • Papers with Code
  • Hugging Face Papers
  • Conference proceedings
  • NeurIPS
  • ICML
  • ICLR
  • ACL
  • EMNLP
  • CVPR
  • ICCV
  • ECCV
  • MLSys
  • KDD
  • RecSys
  • LWN-style technical analysis for systems AI

34.4 Reproducibility

  • Reproduce baseline
  • Reproduce reported metric
  • Fix random seed
  • Control data split
  • Control preprocessing
  • Match hyperparameters
  • Track environment
  • Track hardware
  • Track dependencies
  • Publish code
  • Publish data
  • Publish model weights

35. Current Ecosystem Awareness

35.1 Core ML ecosystem

  • scikit-learn 1.9 awareness
  • PyTorch 2.12 awareness
  • TensorFlow 2.21 awareness
  • JAX ecosystem awareness
  • NumPy 2.x awareness
  • pandas 3.x awareness
  • Polars awareness
  • DuckDB awareness
  • PyArrow awareness
  • Python version compatibility

35.2 GPU and acceleration ecosystem

  • CUDA 13.x awareness
  • CUDA 12.x compatibility awareness
  • ROCm awareness
  • NVIDIA drivers
  • AMD GPU stack
  • Apple Silicon acceleration
  • TPU stack
  • ONNX Runtime
  • TensorRT
  • OpenVINO
  • WebGPU
  • Hardware-software compatibility matrix

35.3 LLM ecosystem

  • Hugging Face Transformers 5.x awareness
  • vLLM awareness
  • TensorRT-LLM awareness
  • llama.cpp awareness
  • Ollama awareness
  • SGLang awareness
  • TGI awareness
  • LangChain awareness
  • LangGraph awareness
  • LlamaIndex awareness
  • MLflow LLM and agent tracking awareness
  • Model Context Protocol awareness

35.4 Safety and governance ecosystem

  • OWASP Top 10 for LLM Applications awareness
  • NIST AI Risk Management Framework awareness
  • NIST Generative AI Profile awareness
  • AI model cards awareness
  • Dataset cards awareness
  • Red teaming practices awareness
  • Prompt injection testing awareness
  • AI incident response awareness
  • AI audit readiness awareness

35.5 Deployment ecosystem

  • Kubernetes for AI workloads
  • GPU scheduling
  • Ray
  • Kueue
  • Volcano
  • Kubeflow
  • KServe
  • Triton Inference Server
  • Seldon
  • BentoML
  • Cloud AI platforms
  • Edge AI deployment

36. Learning Path

36.1 Phase 1 — Foundations

  • Python
  • Git
  • Linux
  • Math fundamentals
  • NumPy
  • pandas
  • Data cleaning
  • Data visualization
  • Basic statistics
  • Basic SQL
  • Basic software engineering

36.2 Phase 2 — Classical machine learning

  • Supervised learning
  • Unsupervised learning
  • Feature engineering
  • Model selection
  • Cross-validation
  • Classification metrics
  • Regression metrics
  • scikit-learn pipelines
  • Error analysis
  • Model interpretation
  • Baseline building

36.3 Phase 3 — Deep learning

  • Neural networks
  • Backpropagation
  • PyTorch
  • TensorFlow/Keras
  • Optimization
  • Regularization
  • CNNs
  • RNNs
  • Transformers
  • GPU training
  • Experiment tracking
  • Model debugging

36.4 Phase 4 — NLP, vision, and multimodal

  • NLP preprocessing
  • Embeddings
  • Transformer models
  • Computer vision
  • CNN architectures
  • Vision transformers
  • Speech processing
  • Multimodal models
  • Evaluation per modality
  • Domain datasets

36.5 Phase 5 — LLM engineering

  • LLM fundamentals
  • Prompt engineering
  • Structured outputs
  • Function calling
  • Tool calling
  • Embeddings
  • Vector databases
  • RAG
  • LLM evaluation
  • LLM security
  • LLM observability
  • Cost optimization

36.6 Phase 6 — Agents and AI applications

  • Agent architectures
  • Tool use
  • MCP
  • Agent memory
  • Planning
  • Workflow agents
  • Multi-agent systems
  • Agent evaluation
  • Agent safety
  • Human-in-the-loop workflows
  • Production agent monitoring

36.7 Phase 7 — MLOps and production

  • Experiment tracking
  • Model registry
  • Data versioning
  • Feature store
  • Training pipelines
  • Inference serving
  • Model monitoring
  • Drift detection
  • CI/CD for ML
  • Kubernetes deployment
  • GPU infrastructure
  • Incident response

36.8 Phase 8 — Advanced AI engineering

  • Fine-tuning
  • PEFT
  • Alignment
  • Distributed training
  • Inference optimization
  • Quantization
  • Evaluation infrastructure
  • AI security
  • Governance
  • Multimodal RAG
  • AI product design
  • Research literacy

37. Practical Projects

37.1 Project 1 — Classical ML Pipeline

  • Select tabular dataset
  • Clean data
  • Explore data
  • Build baseline
  • Train logistic regression
  • Train random forest
  • Train gradient boosting
  • Evaluate metrics
  • Explain model
  • Package pipeline
  • Write report

37.2 Project 2 — Deep Learning Image Classifier

  • Select image dataset
  • Build data loader
  • Apply augmentations
  • Train CNN baseline
  • Fine-tune pretrained model
  • Track experiments
  • Evaluate accuracy
  • Inspect errors
  • Export model
  • Serve inference endpoint

37.3 Project 3 — NLP Text Classifier

  • Select text dataset
  • Clean text
  • Build TF-IDF baseline
  • Fine-tune transformer
  • Evaluate F1
  • Analyze errors
  • Add explainability
  • Package model
  • Create API
  • Monitor predictions

37.4 Project 4 — RAG Document Assistant

  • Ingest documents
  • Parse PDFs
  • Chunk documents
  • Generate embeddings
  • Store vectors
  • Build retriever
  • Add reranker
  • Build answer generator
  • Add citations
  • Evaluate retrieval
  • Evaluate groundedness
  • Add access control

37.5 Project 5 — Tool-Calling Agent

  • Define tools
  • Define schemas
  • Add tool validation
  • Add tool authorization
  • Implement agent loop
  • Add memory
  • Add human approval
  • Add trace logging
  • Evaluate task success
  • Add safety tests
  • Deploy API

37.6 Project 6 — AI Coding Assistant for a Repository

  • Index repository
  • Parse code symbols
  • Build code embeddings
  • Retrieve relevant files
  • Generate patch suggestions
  • Run tests
  • Analyze test failures
  • Add review checklist
  • Add security checks
  • Add audit logs

37.7 Project 7 — Fine-Tuned Domain Assistant

  • Select base model
  • Build instruction dataset
  • Clean dataset
  • Train LoRA adapter
  • Evaluate baseline
  • Evaluate fine-tuned model
  • Test safety regressions
  • Merge or serve adapter
  • Register model
  • Deploy endpoint

37.8 Project 8 — LLM Evaluation Platform

  • Create evaluation dataset
  • Define metrics
  • Add exact-match evaluator
  • Add LLM-as-judge evaluator
  • Add rubric evaluator
  • Add safety evaluator
  • Add regression dashboard
  • Add prompt comparison
  • Add model comparison
  • Add cost tracking

37.9 Project 9 — AI Observability Stack

  • Log prompts
  • Log responses
  • Log tool calls
  • Log retrieval context
  • Track token usage
  • Track latency
  • Track cost
  • Track safety events
  • Add traces
  • Add dashboards
  • Add alerts
  • Add incident workflow

37.10 Project 10 — Production Inference Service

  • Select model
  • Build container image
  • Add REST API
  • Add streaming
  • Add batching
  • Add caching
  • Add rate limiting
  • Add autoscaling
  • Deploy on Kubernetes
  • Monitor GPU usage
  • Run load test
  • Optimize latency

37.11 Project 11 — Multimodal Assistant

  • Accept text input
  • Accept image input
  • Extract OCR
  • Retrieve document context
  • Analyze image content
  • Generate grounded answer
  • Add citations
  • Add safety filters
  • Evaluate multimodal accuracy
  • Deploy UI

37.12 Project 12 — AI Governance Registry

  • Register models
  • Register datasets
  • Store model cards
  • Store dataset cards
  • Store evaluation reports
  • Store risk assessments
  • Store approval history
  • Track deployments
  • Track incidents
  • Add audit export

38. Competency Checklist

38.1 Junior AI competency

  • Clean and explore datasets
  • Train classical ML models
  • Evaluate classification models
  • Evaluate regression models
  • Use scikit-learn pipelines
  • Use NumPy and pandas
  • Build basic neural networks
  • Use PyTorch or TensorFlow
  • Understand overfitting
  • Understand train-validation-test split
  • Explain model metrics
  • Build basic LLM prompts

38.2 Mid-level AI competency

  • Build end-to-end ML pipeline
  • Engineer features
  • Compare models correctly
  • Debug model errors
  • Fine-tune pretrained models
  • Build RAG systems
  • Use vector databases
  • Build structured-output LLM applications
  • Add evaluation datasets
  • Track experiments
  • Deploy model APIs
  • Monitor model performance
  • Handle data drift

38.3 Senior AI competency

  • Design AI system architecture
  • Select models based on constraints
  • Design evaluation strategy
  • Design RAG architecture
  • Design agent architecture
  • Design safety controls
  • Design LLM observability
  • Optimize inference latency
  • Optimize inference cost
  • Lead fine-tuning projects
  • Manage production incidents
  • Define governance process
  • Evaluate vendor trade-offs

38.4 Advanced AI engineering competency

  • Train large models at scale
  • Fine-tune LLMs efficiently
  • Build distributed training systems
  • Build high-throughput inference systems
  • Build agent platforms
  • Build evaluation infrastructure
  • Build AI security testing pipelines
  • Build governance registries
  • Design multimodal AI systems
  • Design multi-tenant AI infrastructure
  • Build AI developer platforms
  • Research and reproduce new methods

38.5 AI safety and governance competency

  • Threat model LLM applications
  • Test prompt injection
  • Test unsafe tool use
  • Test data leakage
  • Create model cards
  • Create dataset cards
  • Run red team evaluations
  • Map controls to risk frameworks
  • Monitor safety incidents
  • Review high-risk use cases
  • Design human oversight
  • Prepare audit evidence