ML Field Manual

Complete Edition

Machine Learning,
engineered for the plant.

A practitioner's textbook written for one reader: a Planning Engineer at an LPG cylinder filling plant who will use Claude and ChatGPT to write the code — but insists on truly understanding every concept behind it.

DOC

ML-FM-001

REV

1.0

STATUS

16 CHAPTERS · FULL EDITION

CH-01COMMISSIONED

What Machine Learning Actually Is

Before touching a single line of code, you need one mental model so solid that no jargon can shake it. This chapter builds it.

1.1The one-sentence definition

Traditional software works because a human writes explicit rules: if pressure > 8 bar, open the relief valve. Machine Learning flips this. Instead of writing the rules, you show the computer many examples, and it discovers the rules by itself.

◈ Formal Definition

Tom Mitchell's classic textbook definition (1997): a program learns from experience E with respect to a task T and a performance measure P, if its performance at T, measured by P, improves with E. In plain words: the more good examples it sees, the better it gets at the job.

⬡ Plant Analogy

A new operator on the filling carousel doesn't memorize a rulebook for every possible situation. They watch thousands of cylinders pass, and gradually develop an instinct: "that hissing sound plus that gauge flicker means a bad valve seat." Nobody wrote that rule down — it was learned from experience. ML gives software the same ability, at industrial scale.

FIG 1.1 — The fundamental inversion. In ML, rules come OUT of the computer instead of going in. This single diagram is the whole field.

Now decode the six words you will hear more than any others. Each card follows the same 5-step format used across the entire manual.

Modelنموذج

Technical	A mathematical function, learned from data, that maps inputs to outputs.
Simple	The "rules" the computer discovered, saved as a file you can reuse.
Why it matters	The model IS the product. Everything else — data cleaning, training, tuning — exists to produce a good model.
Analogy	The experienced operator's instinct, bottled. You can copy it to every shift.
You'll use it	"Claude, train a model to predict tomorrow's cylinder demand from this CSV."

Trainingالتدريب

Technical	The optimization process where the algorithm adjusts internal parameters to minimize error on example data.
Simple	Showing the computer many examples until its guesses stop being wrong.
Why it matters	Training quality decides everything. Garbage examples → garbage model, no exceptions.
Analogy	Commissioning a new compressor: run it, measure, adjust, repeat until performance meets spec.
You'll use it	You'll split your plant data, train on one part, and keep the rest hidden to test honestly.

Featuresالخصائص / المدخلات

Technical	The input variables the model uses to make a prediction.
Simple	The columns of your spreadsheet that describe each example.
Why it matters	Good features beat fancy algorithms. This is where YOUR plant knowledge becomes a superpower no data scientist has.
Analogy	The readings you'd check before diagnosing a pump: vibration, temperature, hours since service.
You'll use it	Deciding that "days since last maintenance" belongs in the demand model — that decision is feature engineering.

Labelالتصنيف / الإجابة

Technical	The known correct answer attached to each training example (also called the "target").
Simple	The answer column. What you want the model to predict.
Why it matters	No labels = no supervised learning. Most workplace ML projects die because labels don't exist or are unreliable.
Analogy	QC inspection results stamped on each cylinder: PASS / FAIL. That stamp is the label.
You'll use it	Your historical "actual filled quantity" column becomes the label when forecasting production.

Algorithmالخوارزمية

Technical	The learning procedure (e.g., Random Forest, Linear Regression) that produces a model from data.
Simple	The recipe. Data goes in, model comes out.
Why it matters	Beginners obsess over algorithms; practitioners know data quality matters 10× more. Chapter 7 gives you a decision tree so you never guess.
Analogy	Algorithm : Model :: Filling procedure : A filled cylinder. One is the process, the other is the output.
You'll use it	"Claude, compare a Random Forest and Gradient Boosting on this dataset and tell me which wins."

Inference / Predictionالاستدلال / التنبؤ

Technical	Running new, unseen inputs through a trained model to get outputs.
Simple	Using the model. Training is school; inference is the job.
Why it matters	Training happens once (or occasionally); inference runs daily in production. Their requirements differ completely — this drives Chapter 11.
Analogy	Training = certifying the operator. Inference = the operator working today's shift.
You'll use it	Your morning Power BI dashboard showing tomorrow's predicted demand — that number came from inference.

1.2AI vs Machine Learning vs Deep Learning

These three terms are used interchangeably in meetings, and that causes real confusion in procurement documents and project scoping. They are nested circles, not synonyms.

FIG 1.2 — Every deep learning system is ML; every ML system is AI. The reverse is false. Claude and ChatGPT live in the innermost ring.

⚠ Meeting-Room Trap

When a vendor says "our system uses AI", it may be a 1990s-style rulebook. Ask one question: "What data was it trained on?" If there's no training data, it's not machine learning — and it won't improve over time.

1.3A 70-year history in one timeline

You don't need dates memorized — you need to understand why ML exploded now and not in 1980. Three ingredients had to arrive together: data (digitization), compute (GPUs), and algorithms (backpropagation → transformers).

FIG 1.3 — The ideas are old; the enabling conditions are new. Your plant's SCADA and ERP data are exactly the fuel this era runs on.

1.4The three families of Machine Learning

Almost every ML technique belongs to one of three families, defined by what kind of feedback the algorithm learns from.

Family	Learns from	Question it answers	Plant example
Supervised ~80% of business ML	Examples with known answers (labels)	"Given these inputs, what is the output?"	Predict next month's 12kg cylinder demand from 5 years of history
Unsupervised Pattern discovery	Examples with NO answers	"What structure hides in this data?"	Group filling-line stoppages into natural clusters → discover unknown failure modes
Reinforcement Rare in business	Trial, error, and rewards	"What sequence of actions maximizes reward?"	Optimizing a filling carousel's speed schedule (advanced — not a first project)

Supervised learning further splits into two tasks, and telling them apart is a skill you'll use in every project scoping meeting:

Regression → predict a number (tomorrow's demand: 4,120 cylinders)
Classification → predict a category (this cylinder: PASS or FAIL)

⚙ Interactive — Which family is it?

Tap a real plant problem. Check your instinct before revealing.

Supervised · Classification

You have historical labels (failed / didn't fail within 7 days). Predicting a category → classification.

Unsupervised · Anomaly detection

No labels for "unusual" — the algorithm learns what normal looks like and flags deviations.

Supervised · Classification

Past QC decisions are your labels. Two categories → binary classification.

Supervised · Regression (time series)

Predicting a quantity → regression. The Ramadan seasonality makes it a time-series problem (Ch. 7).

Unsupervised · Clustering

No predefined groups exist — you're asking the data to reveal them.

1.5Where ML creates real business value

McKinsey, Google, and Microsoft case studies converge on the same pattern: ML pays off where decisions are frequent, repetitive, data-rich, and currently made by rough rules of thumb. It does NOT pay off on rare, one-off strategic decisions.

Value pattern	Generic example	Your world (GFB-U / GFB-S)
Predict to plan earlier	Retail demand forecasting	Cylinder demand & filling schedule by season
Detect before failure	Aircraft engine monitoring	Compressor / carousel predictive maintenance
Inspect at scale	PCB visual inspection	Cylinder valve & body defect detection (camera)
Optimize allocation	Route optimization	Truck dispatch between Umm Al-Aish & Al-Shuaiba
Find the needle	Fraud detection	Anomalies in OPEX line items or energy consumption
Unlock documents	Contract analysis	RAG over your SharePoint spec corpus — already on your radar

1.6Where ML fails — the honest section

Most ML failures are predictable before a single model is trained. Industry post-mortems (Google's ML guides are blunt about this) repeat the same causes:

Failure condition	Why it kills the project	Pre-flight check
Too little data	Models need hundreds–thousands of examples of the thing you predict. 6 failures in 5 years ≠ a dataset.	Count the rows of the rare class first.
The answer isn't in the data	If demand is driven by a ministry decision, no sensor data will predict it.	Ask: could a smart human predict this from these columns?
The world changes (drift)	A model trained pre-2020 died in 2020. Patterns expire.	Plan for monitoring & retraining from day one (Ch. 11).
Labels are wrong	If QC stamps were inconsistent, the model learns the inconsistency perfectly.	Audit 50 random labels manually before training.
A simple rule already works	If "average of last 4 weeks" forecasts within 3%, ML adds cost, not value.	Always build the dumb baseline first — this is Rule #1 of the whole manual.

▲ Rule #1 of this manual

Never deploy a model that can't beat a simple baseline. Before any ML: compute the naive answer (last value, moving average, "always predict PASS"). Every model must earn its complexity by beating it. You'll see this rule again in Chapters 7, 9, and 14 — it is the single habit that separates practitioners from hobbyists.

1.7Ethics, bias, hallucinations & explainability

Four risk concepts every practitioner must carry — especially in a safety-critical plant environment.

Biasالتحيّز

Technical	Systematic error where a model performs worse for certain groups or conditions, usually inherited from unrepresentative training data.
Simple	The model is unfair or blind to situations it rarely saw.
Why it matters	A defect detector trained only on summer daylight photos will miss defects on night shift. Bias isn't only a social issue — it's an operational one.
Analogy	An inspector who only ever worked day shift judging night-shift conditions.
You'll use it	Checklist question in every project: "Which situations are under-represented in my training data?"

Hallucinationالهلوسة

Technical	A generative model (LLM) producing fluent, confident output that is factually false.
Simple	Claude/ChatGPT sometimes makes things up — smoothly.
Why it matters	You will rely on AI assistants for code. Hallucinated function names, invented library parameters, and fake citations are your #1 daily risk. Chapter 15 is a full defense course.
Analogy	A confident contractor quoting a spec clause that doesn't exist. Verify against the document, not the confidence.
You'll use it	Habit: run AI-generated code on a tiny sample and check one output by hand before trusting it.

Explainabilityقابلية التفسير

Technical	The degree to which a model's predictions can be understood by humans; tools like SHAP and LIME attribute predictions to input features.
Simple	Can the model tell you WHY it decided that?
Why it matters	In a plant, "the AI said shut down line 2" is unacceptable without a reason. Management, HSE, and auditors will demand explanations. Interpretable models often win in industry even when slightly less accurate.
Analogy	A junior engineer recommending a shutdown must show the readings that led there. Same standard for models.
You'll use it	"Claude, add SHAP analysis showing the top 5 drivers of each prediction" — a prompt you'll reuse constantly.

⚠ Ethics in one paragraph

Three commitments cover 90% of workplace AI ethics: (1) never let a model make a safety-critical decision without a human in the loop, (2) never feed personal or confidential data into external AI tools without clearance — your on-prem LLM plan exists precisely for this, and (3) always disclose when a number in a report is a model's prediction, not a measurement.

CH-01 · CLOSE-OUT

01Key Takeaways

ML inverts programming: data + answers → rules, instead of rules + data → answers.
AI ⊃ ML ⊃ Deep Learning ⊃ Generative AI. They are nested, not synonyms.
~80% of business ML is supervised learning: regression (numbers) or classification (categories).
ML thrives on frequent, repetitive, data-rich decisions — not rare strategic ones.
Rule #1: no model deploys without beating a simple baseline.
Your unfair advantage is domain knowledge: choosing features is where a Planning Engineer beats a generic data scientist.

02What Beginners Usually Misunderstand

"ML is smart like a human"

It's pattern-matching at scale, with zero understanding. It will happily learn a nonsense pattern if your data contains one. Treat it like a brilliant but extremely literal intern.

"More complex algorithm = better results"

In tabular business data, simple models (linear regression, gradient-boosted trees) routinely match or beat deep learning — while being faster and explainable. Complexity is a cost you must justify.

"The model is done once it's trained"

Training is ~20% of a real project. Data preparation before, and monitoring/retraining after, are the other 80%. Patterns drift; models expire.

"ChatGPT/Claude = Machine Learning"

They are one product OF machine learning (generative AI). Your demand forecast will use classical ML — a completely different toolset that the same assistants can help you write.

"We need big data"

A clean spreadsheet with 2,000 rows can power a genuinely useful forecasting model. You need enough relevant data, not "big" data.

03Practical Workplace Applications

Idea	Family	Data you already have	Difficulty
Monthly cylinder demand forecast	Regression / time series	Filling records, seasonality, holidays	Starter
Energy-consumption anomaly alerts	Unsupervised	Utility bills, production volumes	Starter
Downtime-cause classification from logs	Classification / NLP	Maintenance & stoppage logs	Medium
Valve defect detection by camera	Deep learning / vision	Needs new labeled images	Advanced

Chapter 13 ranks ~30 of these by ROI, data readiness, and difficulty.

05Exercises

Decision inventory (30 min). List 10 decisions made weekly at GFB-U/GFB-S. For each, mark: frequent? data exists? currently rule-of-thumb? Any row with 3 ✓ is an ML candidate.
Family sorting. Take your 10 decisions and assign each to supervised-regression, supervised-classification, unsupervised, or "not ML" — using the table in §1.4.
Baseline drill. Pick one metric you report monthly. Compute what a naive forecast (last month's value) would have predicted for the past 6 months. Note the average error — this is the number any future model must beat.
Label audit. Find one dataset at work with a "result" column. Sample 20 rows: would two colleagues assign the same label? Estimate the % of disagreement.

06Reflection Questions

Which decision at your plant is currently made by pure experience, where a wrong call costs the most money?
Where does your domain knowledge exceed anything written in the data? How could that knowledge become a feature?
If a model contradicted your 10-year intuition, what evidence would make you trust it?

07Common Mistakes

Starting with an algorithm ("let's use deep learning!") instead of a decision that needs improving.
Calling a project "AI" in a proposal without specifying the task, the data, and the metric.
Trusting a confident LLM answer about your own plant data — it has never seen your data.
Skipping the baseline and celebrating a model that a moving average would beat.
Assuming labels are correct because they're in the system.

08Cheat Sheet — Chapter 1

Core inversion

Data + Answers → Computer → Rules (model)

Nesting

AI ⊃ ML ⊃ DL ⊃ GenAI

Predict a number

Regression (demand, energy, cost)

Predict a category

Classification (pass/fail, will-fail/won't)

No labels

Unsupervised (clustering, anomaly detection)

Good ML problem

Frequent + repetitive + data-rich + rule-of-thumb today

Rule #1

Beat the naive baseline or don't ship

4 risks

Bias · Hallucination · Drift · No explainability

09AI Prompts You Can Use Today

PROMPT 1 · PROBLEM SCOPING

I am a Planning Engineer at an LPG cylinder filling plant. Here is a decision we make regularly: [describe decision]. The data we have: [list columns/sources]. 
1) Is this a machine learning problem? 
2) If yes, is it regression, classification, clustering, or anomaly detection — and why?
3) What simple non-ML baseline should I compute first?
4) What data problems should I expect?
Explain in simple English. Do not write code yet.

PROMPT 2 · JARGON DECODER

Explain the term "[TERM]" to me using this exact format:
1) Technical definition, 2) Simple English, 3) Why it matters in practice, 4) An analogy from an industrial plant, 5) How I will actually use it as a beginner working with plant data. Keep each part under 3 sentences.

PROMPT 3 · VENDOR/PROPOSAL STRESS-TEST

Here is a description of an "AI solution" proposed for our plant: [paste]. Act as a skeptical ML engineer. Tell me: (a) is this actually machine learning or a rule-based system, (b) what training data it would require, (c) what questions I should ask the vendor, (d) what a cheap in-house alternative might look like.

PROMPT 4 · BASELINE FIRST

I will paste a small CSV of monthly [metric] data. Before any ML: compute two naive baselines (last value, and 3-month moving average), report their average absolute error, and tell me the error number any future model must beat. Show results as a small table and explain like I'm new to this.

REFChapter 1 Bibliography

Mitchell, T. (1997). Machine Learning. McGraw-Hill — source of the E/T/P definition (§1.1).
Samuel, A. L. (1959). "Some Studies in Machine Learning Using the Game of Checkers." IBM Journal of Research and Development — origin of the term (§1.3).
Turing, A. M. (1950). "Computing Machinery and Intelligence." Mind (§1.3).
Rumelhart, Hinton & Williams (1986). "Learning representations by back-propagating errors." Nature (§1.3).
Krizhevsky, Sutskever & Hinton (2012). "ImageNet Classification with Deep Convolutional Neural Networks." NeurIPS — AlexNet (§1.3).
Vaswani et al. (2017). "Attention Is All You Need." NeurIPS — the Transformer (§1.3).
Google Developers — Machine Learning Crash Course & Rules of ML (developers.google.com/machine-learning) — basis for §1.6 failure conditions and the baseline rule.
Ng, A. — Machine Learning Specialization, DeepLearning.AI / Stanford — supervised/unsupervised framing (§1.4).
Lundberg & Lee (2017). "A Unified Approach to Interpreting Model Predictions" (SHAP), NeurIPS; Ribeiro et al. (2016). "Why Should I Trust You?" (LIME), KDD (§1.7).
scikit-learn documentation (scikit-learn.org) — terminology alignment for estimator/model/fit.

Full consolidated bibliography will appear as an appendix when the final section is commissioned.

CH-02COMMISSIONED

Python for Machine Learning

You won't write most of your code — Claude will. But you must read it, verify it, and fix it. This chapter teaches exactly that reading fluency, nothing more.

2.1Why Python, and how much of it you need

Python won ML for one reason: it reads almost like English, and every major ML library is built for it. You need roughly 15% of the Python language to be a fully effective ML practitioner. This chapter is that 15%.

⬡ Plant Analogy

You don't need to machine your own valves to run a filling plant — you need to read P&IDs fluently and know when something looks wrong. Python fluency for you means reading code like you read a P&ID.

2.2Variables and the four containers

# A variable is a labeled box holding a value
plant = "GFB-U"          # text (string)
daily_target = 4200      # whole number (int)
fill_rate = 0.94         # decimal (float)
line_active = True       # yes/no (boolean)

Container	Syntax	Rules	You'll use it for
List	`[10, 12, 50]`	Ordered, changeable, duplicates OK	90% of cases: rows of readings, filenames, results
Tuple	`(29.3, 47.9)`	Ordered, frozen after creation	Fixed pairs: coordinates, (width, height). Rarely written by you
Set	`{"GFB-U","GFB-S"}`	No duplicates, no order	De-duplicating; "is X in this group?" checks
Dictionary	`{"size": 12, "qty": 480}`	Key → value pairs	Everywhere: configs, one record, JSON from APIs

◈ Reading shortcut

[ ] = list · ( ) = tuple · { } with : = dictionary · { } without : = set. This one line lets you identify 95% of data structures in AI-generated code instantly.

2.3Loops, conditions, functions

# Loop: do something for each item
for cyl in cylinders:
    if cyl["weight"] < 14.5:      # condition
        flag(cyl)

# Function: a reusable named block. def = define
def fill_efficiency(filled, target):
    return filled / target * 100

fill_efficiency(3990, 4200)   # → 95.0

Indentation is grammar in Python. The spaces at the start of a line define what belongs inside the loop/function. When AI-generated code fails with IndentationError, a line is mis-aligned — the most common copy-paste bug you'll meet.

2.4Classes — the 5-minute version

A class is a blueprint bundling data + functions. You will rarely write classes, but you use them constantly: every scikit-learn model is one.

model = RandomForestRegressor()   # create an object from the class blueprint
model.fit(X, y)                   # .fit() = a method (function attached to the object)
model.predict(X_new)              # the dot means "belonging to model"

That's the entire mental model you need: object dot method. The pattern .fit() then .predict() repeats across all of scikit-learn — learn it once, use it for every algorithm in Chapter 7.

2.5Imports, pip, and virtual environments

Term	Simple English	Analogy
`import pandas as pd`	Load a toolbox into your session; nickname it `pd`	Bringing the right toolkit to the job site
`pip install pandas`	Download a library from the internet (once)	Procuring the toolkit before it can be brought on-site
Virtual environment (`venv`)	An isolated folder of libraries per project, so projects don't break each other	Each project gets its own dedicated tool crib — no shared, mixed-up tools

⚠ #1 beginner error

ModuleNotFoundError: No module named 'X' simply means: run pip install X. Not a bug — a missing procurement step.

2.6Where you'll actually work

Tool	What it is	When you use it
Google Colab	Free Jupyter notebooks in the browser — zero installation, free GPU	Start here. All Chapter 14 projects run in Colab
Jupyter Notebook	Documents mixing code cells + results + notes; run cells one at a time	Exploration & EDA — see each step's output immediately
VS Code	Professional code editor	Later: real scripts, apps, and when you outgrow notebooks
Git / GitHub	Version control = "track changes" for code; GitHub = the cloud home for it	Four commands cover you: `clone · add · commit · push`. You already use GitHub for discovery — now use it for backup

2.7Files, CSV, Excel, JSON, APIs

import pandas as pd
df = pd.read_csv("filling_log.csv")     # CSV → table
df = pd.read_excel("opex_2026.xlsx")    # Excel → table
df.to_excel("report.xlsx")              # table → Excel

import requests, json
r = requests.get("https://api.example.com/prices")
data = r.json()                          # API response → dictionary

CSV — plain-text table; the universal currency of ML data.
JSON — nested dictionaries as text; how APIs and configs speak. If you can read a Python dict, you can read JSON.
API — a service you call over the internet that returns data. requests.get(url) is the whole pattern.

01Key Takeaways

You need reading fluency, not authorship: ~15% of Python covers all of ML practice.
Brackets tell you the structure: [] list, {}+: dict, () tuple.
The universal ML pattern is object.method(): model.fit() → model.predict().
pip installs once; import loads each session; venv keeps projects isolated.
Start in Google Colab — zero setup, and Claude's code pastes straight in.

02Beginner Misunderstandings

"I must memorize syntax"

No. You must recognize structures and read error messages. Claude writes; you verify. Memorization comes free with repetition.

"Errors mean I failed"

Errors are the normal workflow. Professionals see dozens daily. The skill is pasting the full error back to Claude with context.

"Notebooks are for beginners only"

Professional data scientists live in notebooks for exploration. Scripts come later, for automation.

03Workplace Applications

Read your monthly OPEX Excel into pandas and automate the summary you currently build by hand.
A Colab notebook that ingests the GFB-U filling log CSV and produces the charts for your report.
A GitHub private repo as the single home for all your plant analysis notebooks.

04Mini Quiz

1. {"line": 2, "status": "run"} is a…

ListDictionarySet

Curly braces with key: value pairs = dictionary.

2. ModuleNotFoundError: No module named 'seaborn' — the fix is:

pip install seabornRestart the computerRewrite the code

Missing library = missing procurement. Install once, then import works.

3. In model.fit(X, y), .fit is a…

VariableLibraryMethod — a function belonging to the model object

The dot means "belonging to". fit = train this model on data X with answers y.

05Exercises

Open colab.research.google.com, create a notebook, run print("GFB-U online").
Ask Claude for a 10-row fake cylinder-filling CSV, upload it to Colab, load it with pd.read_csv, and run df.head().
Deliberately break the code (delete a bracket), read the error, fix it yourself before asking Claude.

06Reflection

Which Excel task do you repeat monthly that a 20-line script could kill forever?
What in AI-generated code would you currently accept blindly because you can't read it?

07Common Mistakes

Installing Python locally and fighting setup for days — use Colab first.
Retyping code from a screenshot instead of copy-pasting (indentation breaks).
Pasting only the last error line to Claude — always paste the full traceback.
Skipping venv, then breaking an old project by upgrading a library for a new one.

08Cheat Sheet

Load table

pd.read_csv("f.csv") / read_excel

Peek

df.head(), df.info(), df.describe()

Install / load

pip install X → import X

Function

def name(inputs): return output

Loop + condition

for x in items: · if cond:

The ML pattern

model.fit(X,y) → model.predict(X)

09AI Prompts

CODE EXPLAINER

Explain this Python code line by line, in simple English, for a non-programmer engineer. For each line: what it does, and what would break if it were removed. Then list the 3 lines I should verify most carefully.
[paste code]

ERROR FIXER

I ran your code in Google Colab and got this full error. Explain the cause in one sentence, give the fixed code, and tell me how to prevent this class of error in future.
[paste FULL traceback]

COLAB SETUP

Give me a Google Colab starter cell for a data analysis project: install and import pandas, numpy, matplotlib, seaborn; set display options for wide tables; include a test that prints library versions. Add a comment above every line.

REFBibliography

Python official documentation & tutorial — docs.python.org (language semantics, §2.2–2.4).
pip & venv official guides — packaging.python.org (§2.5).
Project Jupyter documentation — jupyter.org; Google Colab documentation — research.google.com/colaboratory (§2.6).
pandas official documentation, IO tools — pandas.pydata.org (§2.7).
Pro Git (Chacon & Straub) — git-scm.com/book (§2.6).

CH-03COMMISSIONED

The Data Science Ecosystem

Sixteen libraries look intimidating until you see them as one plant flow diagram: raw material in, refined product out, each unit doing one job.

3.1The ecosystem as a process flow

FIG 3.1 — Storage → Wrangling → Modeling → Delivery, with an ops layer beneath. Every library below slots into exactly one box.

3.2The 16-library register

Library	What / why it exists	Use when	Avoid when
NumPy	Fast math on arrays of numbers; the engine every other library runs on	Rarely directly — it's under the hood	For labeled tables → use pandas on top
pandas	Excel-like tables (DataFrames) in code; the daily workhorse	Every project, every day	Data > RAM (millions of rows crawling) → Polars/DuckDB
Matplotlib	The original plotting library; full control, verbose	Static report figures, fine-grained control	Quick statistical charts → Seaborn is faster
Seaborn	Beautiful statistical charts on top of Matplotlib in one line	EDA: distributions, correlations, boxplots	Interactive dashboards → Plotly
Plotly	Interactive charts (hover, zoom) for HTML/dashboards	Anything a manager will click	Static PDF/print figures
scikit-learn	THE classical-ML library: all Chapter 7 algorithms, one consistent API	Every tabular ML project	Deep learning (→ PyTorch) or huge data
SciPy	Scientific computing: stats tests, optimization, signal processing	A specific statistical test or optimization	General ML — sklearn wraps what you need
statsmodels	Statistics-first modeling: p-values, confidence intervals, ARIMA	When you need to EXPLAIN (inference), classic time series	When you only need to PREDICT accurately
Polars	pandas' younger, multi-core, much faster rival	pandas becomes slow on large files	Starting out — pandas has 100× more tutorials/AI training data
DuckDB	SQL engine on your laptop; queries huge CSV/Parquet instantly, no server	SQL on big local files; pre-aggregating before pandas	Multi-user production DB → real SQL server
SQL	The language of company databases; how you'll pull COGNOS/ERP data	Extracting plant data at the source	— (non-negotiable skill; your text-to-SQL pilot proves it)
PyArrow	Columnar memory format + Parquet files; the plumbing between tools	Reading/writing Parquet; pandas↔DuckDB handoff	You'll rarely call it directly
joblib	Saves trained sklearn models to a file	`joblib.dump(model,"m.pkl")` after every successful training	Cross-platform model exchange → ONNX
MLflow	Logs every experiment: parameters, metrics, model versions	Once you train the same model 10+ times and lose track	Your very first project — a notes cell is enough
DVC	Git for datasets — versions large data files alongside code	Datasets change and results must be reproducible	Small static CSVs — overkill
ONNX	Universal saved-model format, portable across languages/tools	Deploying a model into non-Python systems	Simple Python-to-Python workflows → joblib

▲ Your starter loadout

For the next 3 months you need exactly five: pandas + scikit-learn + Seaborn + Matplotlib + joblib, all preinstalled in Colab. Everything else enters when a specific pain appears — Polars when pandas slows, MLflow when experiments multiply, DuckDB when SQL-on-files beckons. Tools on demand, not tools in advance.

3.3How they connect in one real script

import pandas as pd                          # wrangling
from sklearn.ensemble import RandomForestRegressor
import seaborn as sns, joblib

df = pd.read_csv("filling_log.csv")          # STORAGE → table
df = df.dropna()                              # WRANGLING
sns.lineplot(data=df, x="date", y="qty")      # DELIVERY (EDA chart)
model = RandomForestRegressor().fit(X, y)     # MODELING
joblib.dump(model, "demand_model.pkl")        # DELIVERY (saved model)

Five lines, four ecosystem boxes. Every project in Chapter 14 is an elaboration of this skeleton.

01Key Takeaways

The ecosystem is a pipeline: Storage → Wrangling (pandas) → Modeling (sklearn) → Delivery (charts/saved models).
NumPy is the invisible engine; pandas is your daily interface to data.
scikit-learn's uniform .fit()/.predict() API means learning one algorithm teaches you all of them.
statsmodels explains (p-values); sklearn predicts (accuracy). Different questions, different tools.
Start with 5 libraries; adopt the rest only when a specific pain arrives.

02Beginner Misunderstandings

"I must learn all 16 before starting"

Five carry you for months. The register above is a map for later, not a syllabus for now.

"pandas is a database"

pandas holds data in RAM for analysis. Databases store data permanently for many users. You pull FROM databases INTO pandas.

"Newer = better (Polars over pandas)"

AI assistants were trained on oceans of pandas code — their pandas answers are far more reliable. As an AI-assisted practitioner, ecosystem maturity IS a feature.

03Workplace Applications

DuckDB to run SQL directly on years of exported filling-log CSVs — no IT ticket required.
Plotly charts embedded in HTML reports for management (interactive, like your dashboards).
joblib-saved demand model that a scheduled script reloads every morning for the Power BI feed.

04Mini Quiz

1. Your daily tool for loading and cleaning tabular plant data:

NumPypandasONNX

pandas is the Excel-like DataFrame layer; NumPy powers it underneath.

2. You need p-values and confidence intervals to EXPLAIN a relationship for an audit. Reach for:

Plotlyjoblibstatsmodels

statsmodels is statistics-first (inference); sklearn is prediction-first.

3. Saving a trained sklearn model for tomorrow's use:

joblib.dump(model, "m.pkl")Screenshot the notebookRetrain it daily from scratch

joblib serializes the model object to disk; reload with joblib.load.

05Exercises

In Colab: import pandas, numpy, sklearn, seaborn and print each .__version__.
Take any work CSV, load with pandas, plot one Seaborn chart, and save the figure to PNG.
Ask Claude to rewrite one of your repetitive Excel calculations as pandas — compare results cell by cell.

06Reflection

Which box of FIG 3.1 is weakest in your current workflow — storage, wrangling, modeling, or delivery?
Where does your data actually live today (COGNOS, SharePoint, Excel folders), and which tool bridges it into pandas?

07Common Mistakes

Tool-collecting: installing MLflow+DVC+Polars before finishing one project with pandas+sklearn.
Fighting Matplotlib syntax for an hour when sns.barplot(...) is one line.
Retraining a model every run because nobody saved it with joblib.
Copy-pasting between Excel and Python instead of read_excel — silent data corruption.

08Cheat Sheet

Tables

pandas (default) · Polars (speed) · DuckDB (SQL on files)

Charts

Seaborn (EDA) · Matplotlib (control) · Plotly (interactive)

Models

sklearn (predict) · statsmodels (explain) · SciPy (tests)

Persistence

joblib (save model) · PyArrow/Parquet (save data)

Ops

MLflow (experiments) · DVC (data versions) · ONNX (portability)

Starter five

pandas · sklearn · seaborn · matplotlib · joblib

09AI Prompts

LIBRARY CHOOSER

My task: [describe]. My data: [size, format, where it lives]. From the standard Python data ecosystem, tell me the MINIMUM set of libraries needed, why each one, and explicitly which popular libraries I should NOT use yet and why.

PIPELINE SKELETON

Write a commented Colab notebook skeleton with these sections: 1) load data from [source], 2) quick quality check, 3) one EDA chart with seaborn, 4) placeholder for a sklearn model, 5) save outputs. Add a markdown cell before each section explaining its purpose in simple English.

REFBibliography

Official documentation: numpy.org · pandas.pydata.org · scikit-learn.org · matplotlib.org · seaborn.pydata.org · plotly.com/python · scipy.org · statsmodels.org · pola.rs · duckdb.org · arrow.apache.org · joblib.readthedocs.io · mlflow.org · dvc.org · onnx.ai.
Harris et al. (2020). "Array programming with NumPy." Nature 585.
Pedregosa et al. (2011). "Scikit-learn: Machine Learning in Python." JMLR 12.
McKinney, W. (2010). "Data Structures for Statistical Computing in Python." SciPy Conf. (pandas).

CH-04COMMISSIONED

Data — The 80% of Every Project

Practitioners spend most of their time here, not on models. Master this chapter and you're ahead of most people who "know ML".

4.1Three shapes of data

Shape	Looks like	Plant examples	ML readiness
Structured	Rows × columns	Filling logs, OPEX tables, SCADA exports	Ready — start here
Semi-structured	Nested tags/keys (JSON, XML, emails)	API responses, maintenance emails, SAP exports	Needs flattening into a table
Unstructured	Free text, images, audio	Shift reports, cylinder photos, spec PDFs on SharePoint	Needs deep learning / LLMs (Ch. 8, 12)

4.2Data quality: the six killers

Problem	How it hides	Standard treatment
Missing values	Blanks, zeros-that-mean-blank, "N/A", -999	Drop rows (if few) · fill with median (numeric) or mode (category) · add a "was_missing" flag column — missingness itself is often a signal
Duplicates	Same record entered twice; system re-exports	`df.duplicated().sum()` then `drop_duplicates()` — always check before counting anything
Wrong types	Numbers stored as text ("4,200"), dates as strings	`pd.to_numeric`, `pd.to_datetime` immediately after loading
Inconsistent categories	"GFB-U", "GFBU", "Umm Al-Aish" all meaning one plant	Standardize with a mapping dictionary; lowercase and strip spaces
Outliers	A 500kg "cylinder", a negative fill time	Investigate FIRST — is it an error or a real rare event? Errors get fixed; real extremes often carry the most information (they may be the failures you want to predict)
Unit chaos	kg vs tonnes, KWD vs fils, mixed fiscal/calendar years	One unit per column, documented. Your planning background makes you the natural enforcer

4.3Preparing features: encoding & scaling

Models eat numbers only. Two conversions handle almost everything:

Encoding (categories → numbers). One-hot encoding turns "Plant = GFB-U/GFB-S" into two 0/1 columns. Never encode categories as 1, 2, 3 when no order exists — the model will invent a fake ranking.
Scaling (numbers → comparable ranges). Standardization rescales each column to mean 0, spread 1; normalization squeezes into 0–1. Needed for distance-based models (KNN, SVM, neural nets); NOT needed for tree models (Random Forest, XGBoost). This one fact resolves 90% of "should I scale?" confusion.

⬡ Plant Analogy

Scaling is unit conversion before comparison. Comparing pressure in bar against temperature in °C on raw magnitude is meaningless — you first bring both to a common reference, exactly like standardization does for features.

4.4Feature engineering — your superpower

Feature engineering = creating new input columns from raw data using domain knowledge. It is the highest-ROI activity in classical ML, and it's where a Planning Engineer beats any outside data scientist:

From a date → day_of_week, is_ramadan, is_summer_peak, days_to_holiday
From maintenance logs → hours_since_last_service, failures_last_90d
From production → rolling_7d_average, qty_vs_same_month_last_year

Feature selection is the reverse: removing columns that add noise. Fewer, stronger features → simpler, more robust, more explainable models.

4.5Data leakage — the silent project killer

⚠ The most expensive mistake in applied ML

Leakage = the model accidentally sees information during training that won't exist at prediction time. Example: predicting cylinder QC failure using a column filled in after inspection. Training accuracy looks miraculous (98%!); production accuracy collapses. Symptom: results too good to be true. Diagnosis: for every feature ask, "would I know this value BEFORE the moment of prediction?" If not — remove it.

4.6Splitting data honestly

FIG 4.1 — The test set is a sealed sample kept for final QC. For time series, always split by time — random splits leak the future into training.

Cross-validation improves on a single split: divide training data into K folds (usually 5), train on 4, validate on the 5th, rotate, and average. The result is a stable performance estimate not dependent on one lucky split. In sklearn: cross_val_score(model, X, y, cv=5).

4.7Pipelines, versioning, labeling

Pipeline — chaining all preprocessing + the model into one object (sklearn.pipeline.Pipeline), so the exact same transformations apply in training and production. Prevents an entire class of leakage bugs; from Chapter 14 onward every project uses one.
Data versioning — "which dataset produced this result?" At minimum: dated, immutable snapshot files (filling_log_2026-07-01.parquet); DVC when it grows serious.
Labeling / annotation — manually attaching correct answers (drawing boxes on defect photos, tagging stoppage causes). Budget it honestly: labeling is often the true cost of an ML project, and consistent labeling guidelines matter more than volume.

01Key Takeaways

Data preparation is most of the job; model training is the short final step.
One-hot encode unordered categories; scale features for distance-based models, skip scaling for trees.
Feature engineering from domain knowledge is the highest-leverage skill you own.
Leakage check for every feature: "known before prediction time?" Too-good results = investigate, don't celebrate.
Test set is sealed until the end; time series split by time; cross-validation for stable estimates.

02Beginner Misunderstandings

"Cleaning means deleting anything strange"

Outliers are investigated, not auto-deleted. In predictive maintenance, the outliers ARE the events you're trying to predict.

"98% training accuracy — we're done!"

Training accuracy is nearly meaningless; models can memorize. Only performance on held-out (or future) data counts.

"Filling missing values with 0 is neutral"

Zero is a real value with meaning (zero production ≠ unrecorded production). Use median/mode + a missing-flag column.

03Workplace Applications

A reusable "GFB data quality report" notebook: missing %, duplicates, type issues, category chaos — run on any new export.
An engineered feature library for LPG demand: Ramadan/Eid flags, summer index, days-since-holiday.
Dated Parquet snapshots of monthly planning tables — instant reproducibility for any past analysis.

04Mini Quiz

1. Your QC-failure model scores 99% in training using a column recorded during final inspection. This is:

Excellent — deploy itData leakage — that column won't exist at prediction timeUnderfitting

Classic leakage: the feature is a consequence of the answer. Production performance will collapse.

2. Forecasting 2026 demand — the correct split is:

Random 80/20Test on 2021, train on 2022–2025Train on the past, test on the most recent period

Time flows one way. Random splits let the model peek at the future.

3. Encoding plants GFB-U/GFB-S as 1 and 2 for a linear model is risky because:

It invents a fake order/magnitude — use one-hot insteadNumbers are slower than textTwo categories is too few

1 vs 2 implies GFB-S is "twice" GFB-U. One-hot creates neutral 0/1 columns.

05Exercises

Run a quality audit on one real work table: % missing per column, duplicate count, and every distinct spelling of one category.
Design (on paper) 8 engineered features for monthly LPG demand. Mark each: available before prediction time? ✓/✗
Ask Claude for a leakage review: paste your feature list + prediction moment, ask it to flag suspects.

06Reflection

Which dataset at KNPC do you trust least, and is the cause missing values, inconsistent entry, or duplicates?
What do you know about demand drivers that is written in NO database — and how could it become a column?

07Common Mistakes

Scaling before splitting (statistics computed on test data = subtle leakage; pipelines prevent this).
Dropping every row with any missing value and silently losing 40% of the data.
Treating -999 / "N/A" sentinel codes as real numbers.
Touching the test set repeatedly until results look good — that's just slow training on test.

08Cheat Sheet

Audit trio

df.info() · df.isna().sum() · df.duplicated().sum()

Missing

median/mode fill + was_missing flag

Exploratory Data Analysis

EDA is the structured conversation you have with a dataset before trusting it with predictions. Skip it and every later step inherits your blindness.

5.1The EDA protocol — same 6 steps, every dataset

#	Step	Core question	Tools
1	Shape & types	How many rows/columns? Right types?	`df.shape`, `df.info()`
2	Summary statistics	Do min/max/mean pass the sanity check?	`df.describe()`
3	Each variable alone	What does its distribution look like? Skewed? Two humps?	Histogram, boxplot, `value_counts()`
4	Pairs of variables	What moves with what?	Scatter plots, correlation heatmap
5	Time behavior	Trend? Seasonality? Level shifts?	Line plots, rolling means
6	The weird stuff	What surprised me, and is it error or signal?	Filtering, sorting, asking colleagues

5.2Choosing the right chart

You want to see…	Chart	Seaborn one-liner
Distribution of one number	Histogram / KDE	`sns.histplot(df, x="qty")`
Distribution across groups	Boxplot	`sns.boxplot(df, x="plant", y="fill_time")`
Relationship of two numbers	Scatter	`sns.scatterplot(df, x="temp", y="demand")`
Change over time	Line	`sns.lineplot(df, x="date", y="qty")`
All correlations at once	Heatmap	`sns.heatmap(df.corr(numeric_only=True), annot=True)`
Category counts	Bar	`sns.countplot(df, x="stoppage_cause")`

⚠ Correlation ≠ causation

Ice-cream sales correlate with drowning (both follow summer). Your energy use may correlate with a variable that merely shadows production volume. EDA finds candidates for relationships; domain reasoning — yours — decides which are real.

5.3Thinking like a data scientist

The mindset, condensed to five habits:

Hypothesis first. Before plotting, write what you expect ("demand peaks in July"). Surprises are only visible against expectations.
Distrust aggregates. A healthy average can hide two sick groups (one line over-performing, one failing). Always disaggregate by plant, shift, season.
Chase anomalies to their source. The weird spike in March? That investigation usually produces the project's most valuable insight.
Quantify, don't adjective. Replace "demand is much higher in summer" with "July demand runs 38% above the annual mean."
End with decisions. An EDA that doesn't change what you do next was decoration.

▲ Deliverable habit

Every EDA ends in a one-page summary: 3 findings, 3 charts, 3 decisions. That page is what management reads — and it's the specification for the model you build next.

01Key Takeaways

EDA is a fixed 6-step protocol, not artistic wandering: shape → stats → single → pairs → time → anomalies.
One table maps every question to its chart; six chart types cover 95% of EDA.
Correlation nominates suspects; your domain knowledge convicts.
Averages hide; disaggregate by plant, line, shift, season.
EDA output = findings that change decisions, captured in one page.

02Beginner Misunderstandings

"EDA is optional — the model will figure it out"

Models amplify whatever the data contains, including its errors. EDA is where you catch the leakage column, the duplicate export, the broken sensor.

"More charts = better EDA"

Twenty aimless plots < five plots each answering a written hypothesis.

"A 0.9 correlation proves causation"

It proves co-movement. A third factor (season, production volume) may drive both.

03Workplace Applications

EDA on 3 years of stoppage logs before any downtime model — the cluster of causes usually surprises everyone.
Seasonal decomposition of cylinder demand: quantify the Ramadan and summer effects you know qualitatively.
Correlation heatmap of OPEX line items vs production volume — instant candidates for cost drivers.

04Mini Quiz

1. To compare fill-time distributions between GFB-U and GFB-S, best chart:

Pie chartLine plotBoxplot by plant

Distribution across groups = boxplot. Shows median, spread, and outliers per plant at once.

2. df.describe() shows max fill_time = 9,600 minutes. First move:

Delete itInvestigate the row — data-entry error or a real extreme event?Ignore, the model will handle it

Anomalies are investigated first. Error → fix; real → possibly your most informative data point.

3. Energy cost correlates 0.85 with cylinder output. Correct statement:

They move together; whether output drives cost needs domain reasoningOutput causes 85% of costReducing output cuts cost by 0.85%

Correlation measures co-movement only, and its value is not a percentage of anything.

05Exercises

Write 5 hypotheses about your filling data BEFORE looking. Then run the 6-step protocol and score yourself.
Build the correlation heatmap for one plant dataset; pick the strongest surprising pair and investigate it to the source.
Produce the one-page output: 3 findings, 3 charts, 3 decisions. Show it to a colleague — do they reach the same decisions?

06Reflection

Which "known fact" about plant operations have you never actually verified in data?
When did an average last mislead your team, and what disaggregation would have caught it?

07Common Mistakes

Plotting before cleaning — beautiful charts of broken data.
Pie charts for anything with more than 3 categories.
Correlation on categories encoded as arbitrary numbers.
Never plotting the target variable itself over time — the single most informative chart.

08Cheat Sheet

Protocol

shape → describe → single vars → pairs → time → anomalies

One number

histplot · groups: boxplot

Two numbers

scatterplot · all: heatmap

Over time

lineplot + rolling mean

Golden rule

correlation ≠ causation

Output

3 findings · 3 charts · 3 decisions

09AI Prompts

FULL EDA

Here is df.info() and df.describe() for my dataset about [context]: [paste]. Generate a complete EDA notebook following this protocol: 1) quality checks, 2) distribution of every numeric column, 3) counts of every category, 4) correlation heatmap, 5) target variable over time, 6) an "anomalies to investigate" list. Use seaborn. After the code, write the 3 most important questions I should ask about this data as the domain expert.

CHART CRITIC

I attach a chart from my analysis. Critique it: is this the right chart type for the question "[question]"? What could mislead a manager viewing it? Rewrite the code for a clearer version and suggest one alternative view of the same data.

REFBibliography

Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley — the founding text of the field.
seaborn official tutorial — seaborn.pydata.org/tutorial.
pandas documentation: Descriptive statistics, GroupBy — pandas.pydata.org.
NIST/SEMATECH e-Handbook of Statistical Methods, ch. 1 (EDA) — itl.nist.gov.

CH-06COMMISSIONED

Statistics for Machine Learning

Only the statistics you'll actually use — as intuition, not derivation. Every concept here earns its place by appearing later in evaluation, tuning, or interpretation.

6.1Spread: variance & standard deviation

Variance measures how far values scatter from their mean; standard deviation (σ) is its square root, expressed in the original units — which is why you'll quote σ, not variance.

⬡ Plant Analogy

Two filling lines both average 14.9 kg per cylinder. Line A: σ = 0.05 kg (tight, consistent). Line B: σ = 0.4 kg (all over the place). Same mean, completely different quality story — the mean without σ is half a report.

6.2Distributions — the shapes data takes

FIG 6.1 — The normal distribution and the 68–95–99.7 rule. A reading beyond ±3σ is a 1-in-370 event — exactly the logic of SPC control charts and simple anomaly detectors.

You'll meet three shapes constantly: normal (measurement noise, fill weights), skewed (repair costs, downtime durations — many small, few huge; the mean gets dragged, so report the median), and bimodal (two humps = two hidden populations, like two shifts or two machines mixed in one column — split them).

6.3Probability & Bayes in working form

Probability quantifies uncertainty from 0 to 1. The one theorem worth internalizing is Bayes' rule: update your belief when evidence arrives. Its practical sting is the base-rate trap:

⚠ The base-rate trap (real numbers)

A defect detector is 95% accurate. Defects occur in 1% of cylinders. A cylinder is flagged — probability it's truly defective? Not 95%. Out of 10,000 cylinders: 100 defective → 95 true flags; 9,900 good → 495 false flags. Truly defective among flags = 95/(95+495) ≈ 16%. When the event is rare, even good models produce mostly false alarms. This single calculation explains why Chapter 9 refuses to accept "accuracy" for rare events, and why precision/recall exist.

6.4Hypothesis tests & confidence intervals

Hypothesis test — "is this difference real or luck?" You assume no difference (null hypothesis), then compute how surprising your data would be if that were true (the p-value). Convention: p < 0.05 → the difference is unlikely to be luck. Use it when comparing line A vs line B, before/after a modification.
Confidence interval (CI) — a range, not a point: "mean fill time = 42.3 s, 95% CI [41.8, 42.8]". Rule of thumb: if two CIs don't overlap, the difference is real. Quote CIs in reports; single numbers overstate certainty.
Correlation vs covariance — both measure co-movement; correlation is covariance standardized to [-1, +1] so it's comparable across variable pairs. You'll use correlation; covariance lives inside the math.

6.5The math intuitions (no equations required)

Field	The one intuition you need	Where it surfaces
Linear algebra	Your data table IS a matrix; each row is a point in a space with one dimension per feature. Models draw boundaries or surfaces in that space	Why features must be numeric; embeddings (Ch. 8, 12); PCA
Calculus	The derivative is a slope — it tells the model which direction reduces error	"Gradient" in gradient descent / gradient boosting
Optimization	Training = walking downhill on an error landscape, step by step, until a low point. The step size is the "learning rate"	Why training iterates; why it can stall in a local dip; hyperparameters (Ch. 10)

⬡ Gradient descent, plant edition

You're on dunes at night with a flashlight, seeking the lowest point. You feel the slope under your feet (the gradient) and step downhill (learning rate = stride length). Too-long strides overshoot the valley; too-short takes forever. That is — genuinely — how every neural network trains.

01Key Takeaways

Always report spread (σ) with the mean; for skewed data (costs, durations) prefer the median.
68–95–99.7: the ±σ rule powers control charts and threshold-based anomaly detection.
Bayes' base-rate trap: with rare events, most alarms are false even from good models — the reason precision/recall exist.
p-values test "real or luck"; confidence intervals replace false-precision single numbers.
Training = gradient descent = walking downhill on an error landscape. That intuition unlocks Chapters 8 and 10.

02Beginner Misunderstandings

"p < 0.05 proves my hypothesis"

It says the data would be surprising if nothing were going on. With enough comparisons, 1-in-20 flukes appear by design — beware testing many things and reporting the one that "worked".

"95% accurate means 95% of alarms are true"

Only if the event is common. For 1% events, see §6.3 — it can be 16%.

"I need to master the math first"

Practitioners need the three intuitions in §6.5. Depth can come later, on demand, per topic.

03Workplace Applications

±3σ control limits on daily energy per cylinder — a statistically honest anomaly alarm before any ML.
A proper hypothesis test on "did the carousel upgrade actually reduce fill time?" instead of eyeballing two averages.
Confidence intervals on your demand forecast in the annual plan — ranges managers can trust.

04Mini Quiz

1. Downtime durations: many short, a few multi-day monsters. Report the typical duration using:

MeanMedianMaximum

Skewed data drags the mean toward the monsters; the median resists.

2. A rare defect (0.5%) is flagged by a 95%-accurate model. The flag is:

95% certainly a defectCertain — the model is accurateProbably a false alarm — the base-rate trap

Rare event + imperfect model = false flags outnumber true ones. Compute it Bayes-style.

3. In gradient descent, the "learning rate" is:

The size of each downhill step on the error landscapeHow fast the human learns MLThe GPU clock speed

Stride length in the night-dunes analogy — too big overshoots, too small crawls.

05Exercises

Compute mean, median, σ for one real KPI. Which two numbers tell the honest story?
Redo the base-rate calculation of §6.3 with your own numbers: pick a rare plant event, assume a 90%-accurate detector, compute the true-alarm rate.
Ask Claude to run a t-test comparing a KPI before/after a real change at the plant, and to explain the p-value in one sentence a manager accepts.

06Reflection

Which number in your monthly report is presented with false precision and deserves a confidence interval?
Where has a "difference" between two periods been celebrated that a hypothesis test might expose as noise?

07Common Mistakes

Comparing two averages without asking whether the gap exceeds normal variation.
Using the mean on skewed cost/duration data.
Testing 20 relationships and reporting the one with p < 0.05.
Reading correlation strength as effect size or causation.

08Cheat Sheet

Spread

σ with every mean; median for skew

Sigma rule

68 / 95 / 99.7 at ±1/2/3σ

Bayes trap

rare event → most alarms false

p-value

<0.05 ⇒ unlikely to be luck (not proof)

CI habit

report ranges, not points

Training

= gradient descent downhill; step = learning rate

09AI Prompts

STAT TEST RUNNER

I have two groups of measurements: [context]. Data: [paste or describe columns]. Choose the appropriate statistical test (explain WHY that one), run it in Python with scipy, and interpret the result in two sentences for a non-statistician manager, including a clear statement of what it does NOT prove.

BASE-RATE CHECK

Event base rate: [X]% . Detector accuracy: assume [Y]% sensitivity and [Z]% specificity (suggest defaults if unknown). Build the 10,000-case table showing true/false alarms, compute the probability a flagged case is real, and state the operational implication for alarm handling.

REFBibliography

OpenIntro Statistics (open textbook, openintro.org) — distributions, tests, CIs.
NIST/SEMATECH e-Handbook — control charts & the 3σ convention.
SciPy stats documentation — docs.scipy.org (tests used in exercises).
3Blue1Brown, "Essence of Linear Algebra" & "Essence of Calculus" video series — the visual intuitions of §6.5.
Ng, A., Machine Learning Specialization (DeepLearning.AI) — gradient descent framing.

CH-07COMMISSIONED

Machine Learning Algorithms

The full toolbox, organized so you never guess. One selection flowchart, then a register of every major algorithm with its honest strengths, weaknesses, and plant use cases.

7.1The selection flowchart

FIG 7.1 — The arrows inside each box are an escalation ladder: start simple, escalate only when the simple model underperforms the baseline meaningfully.

7.2The algorithm register — supervised

Algorithm	Idea in one line	Strengths	Weaknesses	Interpretability / Speed	Plant use
Linear Regression	Best straight-line fit through the data	Instant, fully explainable coefficients ("each °C adds 12 cylinders")	Only straight-line relationships	High / Fast	First model for demand vs drivers; the explainability benchmark
Logistic Regression	Linear model squeezed into a 0–1 probability	Calibrated probabilities, explainable, robust	Linear boundaries only	High / Fast	Probability a cylinder fails QC; baseline for any yes/no
Decision Tree	Learned flowchart of if/then splits	Reads like a procedure; handles mixed data; no scaling needed	A single tree overfits easily	High / Fast	Communicating rules to operators; root-cause style views
Random Forest	Hundreds of varied trees voting	Strong default accuracy, hard to break, feature importances	Slower; explains less than one tree	Medium / Medium	The workhorse: downtime prediction, demand, failure classification
Gradient Boosting (XGBoost / LightGBM)	Trees built one at a time, each fixing the last one's errors	Usually the top accuracy on tabular data; wins most Kaggle tables	Sensitive to tuning; easier to overfit	Medium / Medium	Squeezing the last accuracy from forecasts once RF plateaus
SVM	The widest possible margin between classes	Strong in high-dimensional, small-sample settings	Slow on large data; needs scaling; opaque	Low / Slow	Niche today — tree ensembles usually win on plant tabular data
KNN	Copy the answer of the k most similar past cases	Zero training; intuitive ("cases like this one")	Slow predictions; needs scaling; struggles with many features	Medium / Fast-train, slow-predict	"Find the 5 most similar historical stoppages" — great as a lookup tool
Naive Bayes	Bayes' rule assuming features are independent	Extremely fast; shines on text/word counts	The independence assumption is usually false	High / Fast	Quick classifier for maintenance-log text categories

7.3The register — unsupervised & special families

Algorithm / family	Idea	Watch out	Plant use
K-Means (clustering)	Split data into K round-ish groups around centers	You choose K (use the elbow method); needs scaling; assumes blob shapes	Segmenting stoppage events, customer ordering patterns, operating regimes
DBSCAN (clustering)	Groups = dense regions; sparse points = noise	Two sensitive parameters; struggles with mixed densities	Finds odd-shaped clusters AND flags outliers in one pass
PCA (dimensionality reduction)	Compress many correlated columns into a few "summary directions"	Components lose physical meaning; scale first	Compressing 40 correlated sensor channels before modeling or plotting
Isolation Forest (anomaly)	Anomalies are easier to isolate with random splits — few cuts needed = suspicious	Set expected contamination %; validate flags with experts	Multivariate anomaly detection on energy + production + downtime jointly
Time series: ARIMA / SARIMA / Prophet / gradient boosting with lags	Model trend + seasonality + autocorrelation of a value over time	NEVER random-split; watch for regime changes (drift)	The LPG demand forecast — your flagship use case. Prophet handles Ramadan-style moving holidays via custom events
Recommenders	"Users like you also…" (collaborative) or "items similar to this" (content-based)	Cold start: new users/items have no history	Suggesting relevant spec documents to engineers; spare-part co-occurrence
Ranking	Learn the best ORDER of items, not their values	Needs preference/relevance data	Ordering maintenance backlog by urgency; prioritizing CAPEX candidates

▲ The escalation ladder (memorize this)

For any tabular problem: Baseline → Linear/Logistic → Random Forest → Gradient Boosting. Stop at the first rung whose improvement over the previous is too small to matter operationally. Deep learning enters only for images, audio, and free text — Chapter 8.

7.4Decision boundaries — seeing how models think

FIG 7.2 — Same data, three ways of cutting it. Flexibility is bought with overfitting risk — the central trade-off Chapter 10 manages.

Overfitting = memorizing training noise instead of learning the pattern (great in training, poor on new data). Underfitting = model too simple to capture the real pattern (poor everywhere). The escalation ladder walks you from underfit toward the sweet spot, and validation data tells you when to stop.

CH-08COMMISSIONED

Deep Learning

Deep learning is what unlocked images, speech, and language for machines — and it's the engine inside Claude. You need working intuition, not implementation.

8.1The neuron and the network

FIG 8.1 — A neural network: layers of simple units, each computing a weighted sum + a squash. "Deep" just means many hidden layers. Training adjusts the millions of line-weights downhill on the error landscape.

Why layers matter: each layer learns features OF the previous layer's features. For images: edges → shapes → parts → objects. This automatic feature engineering is exactly why deep learning wins on raw data (pixels, audio, text) where nobody can hand-craft columns — and why it's usually unnecessary for your tabular data, where good columns already exist.

8.2The architecture zoo

Architecture	Built for	Core trick	Status 2026	Plant relevance
CNN (convolutional)	Images	Small filters slide across the image detecting local patterns, position-independent	Standard for vision	Cylinder defect detection, gauge reading, PPE compliance cameras
RNN / LSTM	Sequences	A memory state carried step to step; LSTM gates decide what to keep/forget	Largely superseded by Transformers, still in lightweight sensor models	Legacy sequence models on sensor streams
Transformer	Originally language; now everything	Attention: every element looks at every other and weighs relevance ("In 'the valve failed because IT froze', attention links IT→valve")	The dominant architecture; powers Claude, ChatGPT, Qwen	Your on-prem Qwen3 deployment IS a transformer
Vision / Multimodal models	Images+text together	Transformers applied to image patches; joint text-image training	Mainstream (Claude reads your photos)	"Describe the defect in this cylinder photo" without training anything

8.3Embeddings — the bridge concept

Embeddingالتمثيل الشعاعي

Technical	A learned list of numbers (vector) representing an item, where geometric distance ≈ semantic similarity.
Simple	Meaning turned into coordinates. Similar meanings land close together.
Why it matters	Embeddings power search-by-meaning: "pump cavitation" finds documents saying "impeller vapor damage" with zero shared words. This is the engine of RAG (Ch. 12) — your SharePoint corpus project depends on it.
Analogy	Plot equipment on a map by behavior instead of location: pumps that fail similarly become neighbors, regardless of which plant they're in.
You'll use it	An embedding model + vector database over the LPG spec corpus = semantic search for the whole division.

8.4LLMs in three sentences

A Large Language Model is a giant transformer trained on enormous text to predict the next token (word piece), then refined with human feedback to be helpful and follow instructions. That simple objective, at extreme scale, produces translation, reasoning, and code as side effects. Two consequences you must carry: LLMs are probabilistic, not databases — hence hallucination (§1.7) — and their knowledge freezes at training time — hence RAG to feed them your current documents (Ch. 12).

▲ The practitioner's split

Numbers in tables → classical ML (Ch. 7). Images → CNN/vision models, usually fine-tuned, not built from scratch. Text/documents/questions → LLMs via prompting + RAG, almost never trained by you. This one routing rule prevents the most common architecture mistakes in industry.

01Key Takeaways

Deep learning = automatic feature learning through stacked layers; essential for raw data, usually overkill for tables.
CNNs own images; Transformers (attention) own language and increasingly everything else.
Embeddings turn meaning into geometry — the foundation of semantic search and RAG.
LLMs predict next tokens at scale; probabilistic by nature → verify, and feed them fresh context via RAG.
You will USE deep models (fine-tune, prompt, API), not build them — that's the practitioner position.

02Beginner Misunderstandings

"I should build a neural network for my forecast"

Tabular demand data → gradient boosting almost always wins on cost, speed, and explainability. Deep learning earns its complexity only on raw data.

"The LLM understands like a human"

It models statistical patterns of language extraordinarily well. Fluency ≠ comprehension ≠ factual reliability.

"Training a vision model needs millions of images"

Transfer learning: start from a pre-trained model, fine-tune on hundreds of your defect photos. This is what makes cylinder defect detection feasible.

03Workplace Applications

Fine-tuned CNN on cylinder photos: refurbish / scrap / OK — starting from a pre-trained backbone.
Multimodal Claude reading gauge photos from inspection rounds into structured readings.
Embedding-powered semantic search over specs — the underrated first LLM win you already identified.

04Mini Quiz

1. Detecting valve defects from photos calls for:

A CNN / vision model (fine-tuned)Linear regressionK-Means

Raw images = deep learning territory; convolution detects local visual patterns.

2. "Attention" in a transformer means:

The model asks the user to pay attentionTraining with louder alarmsEach element weighs its relevance to every other element in the input

Attention lets "it" find "the valve" across a sentence — context-dependent meaning.

3. Two maintenance reports use completely different words for the same failure. What finds them as similar?

Keyword searchEmbeddings — nearby vectors in meaning-spaceA histogram

Semantic similarity = geometric closeness. Keywords fail; embeddings don't.

05Exercises

Send Claude a photo of any equipment nameplate and ask for structured JSON output — you just used a multimodal transformer.
List 5 documents-or-images problems at the plant; route each: prompt an LLM / fine-tune vision / classical ML.
Ask Claude to compute embeddings for 10 short maintenance phrases and show their similarity matrix — watch synonyms cluster.

06Reflection

Which unstructured data at GFB (photos, reports, PDFs) holds value that tabular systems can't reach?
Where would a wrong-but-confident model output be dangerous enough to require a human gate?

07Common Mistakes

Reaching for deep learning on tabular data as a first move.
Training vision models from scratch instead of fine-tuning pre-trained ones.
Treating LLM output as a database lookup rather than a probabilistic draft.
Ignoring embeddings — the cheapest, highest-ROI deep-learning product for document-heavy teams.

08Cheat Sheet

Routing

tables→GBM · images→CNN · text→LLM+RAG

Deep =

many layers learning features of features

Transformer

attention: everything weighs everything

Embedding

meaning → coordinates; close = similar

LLM

next-token predictor at scale; verify outputs

Vision shortcut

fine-tune pre-trained, hundreds of images suffice

09AI Prompts

VISION FEASIBILITY

I want to detect [defect type] on [object] using a camera at [location/conditions]. Assess feasibility: what image quantity/quality/labeling would I need, whether transfer learning applies, expected pitfalls (lighting, night shift, class imbalance), and a phased plan starting with a manual-photo pilot before any hardware purchase.

EMBEDDING DEMO

Write Colab code using sentence-transformers to embed these 15 maintenance phrases [paste], compute cosine similarity, and visualize as a heatmap. Then explain in simple English why phrase pairs with no shared words still score high.

REFBibliography

LeCun, Bengio & Hinton (2015). "Deep learning." Nature 521.
Vaswani et al. (2017). "Attention Is All You Need." NeurIPS.
Hochreiter & Schmidhuber (1997). "Long Short-Term Memory." Neural Computation.
Dosovitskiy et al. (2021). "An Image is Worth 16×16 Words" (ViT). ICLR.
DeepLearning.AI — Deep Learning Specialization (Andrew Ng); PyTorch & TensorFlow official tutorials; Hugging Face course — huggingface.co/learn.

CH-09COMMISSIONED

Model Evaluation

The chapter that keeps you honest. A model is worth exactly what its evaluation proves — measured on data it never saw, with the right metric for the business cost.

9.1The confusion matrix — read it once, use it forever

FIG 9.1 — One realistic day of cylinder QC. Accuracy looks superb while precision is terrible — this is the base-rate trap from §6.3 wearing its evaluation costume.

⚠ The accuracy trap

With 1% defects, a model that flags NOTHING scores 99% accuracy while catching zero defects. Never accept accuracy alone for imbalanced problems — which describes almost every failure/defect/anomaly problem at a plant.

9.2Choosing the classification metric

Metric	Answers	Optimize it when the costly error is…	Plant framing
Precision	Of flagged, how many real?	False alarms (wasted rechecks, alarm fatigue)	Inspection team drowning in false flags → raise precision
Recall	Of real cases, how many caught?	Misses (defect reaches a customer, failure unpredicted)	Safety-critical detection → recall first, always
F1	Harmonic balance of both	You need one number and costs are comparable	Model comparison shorthand
ROC-AUC	Across ALL thresholds, how well does the model separate classes? (1.0 perfect, 0.5 coin-flip)	Comparing models before choosing an operating threshold	Model A AUC 0.91 vs B 0.84 → A separates better, then set the threshold by business cost
Calibration	When it says "70%", does it happen ~70% of the time?	Probabilities feed decisions (maintenance scheduling by risk)	Risk-ranked maintenance lists need calibrated probabilities, not just rankings

The threshold is the dial: a classifier outputs a probability, and YOU choose the cutoff. Lower it → more recall, more false alarms; raise it → cleaner flags, more misses. Setting this dial is a business decision about error costs — yours, not the algorithm's.

9.3Regression metrics

Metric	Meaning	Character	Use
MAE	Average absolute miss, in real units	Robust, honest, explainable: "off by 210 cylinders on average"	Default for management reporting
RMSE	Root of mean squared miss	Punishes big misses extra — sensitive to catastrophic errors	When one huge miss hurts far more than many small ones
MAPE	Average % miss	Comparable across products/scales; explodes near zero actuals	"Forecast within 6.5%" — planning language; avoid for low-volume items

▲ Reporting pattern

Quote MAE in units + MAPE in % + the naive-baseline comparison: "MAE 210 cylinders (4.8% MAPE) vs 9.1% for the seasonal-naive baseline — error roughly halved." One sentence, complete evaluation, management-ready.

01Key Takeaways

Every classification metric derives from four cells: TP, FP, FN, TN. Master the matrix, own them all.
Accuracy lies on imbalanced data; precision vs recall encodes which error costs more.
AUC compares models; the threshold operationalizes one — and setting it is a business call.
MAE for honesty, RMSE when big misses are catastrophic, MAPE for planning language.
Always evaluate on unseen data, always against the baseline.

02Beginner Misunderstandings

"97% accuracy = excellent model"

Not with 1% positives — the do-nothing model scores 99%. Ask for the confusion matrix before applauding.

"Maximize precision AND recall"

They trade off via the threshold. You choose the balance the business needs; you can't max both.

"A 70% predicted probability is a fact"

Only if calibrated. Uncalibrated models rank well but their probability numbers can be fantasy.

03Workplace Applications

Set the failure-prediction threshold from real costs: false alarm = 2 tech-hours vs miss = 8h line-down. The math gives the threshold.
Monthly forecast scorecard: MAE, MAPE, vs baseline — the honesty layer under the planning dashboard.
Confusion-matrix review meeting per quarter: are FN (misses) drifting up? That's your retraining trigger.

04Mini Quiz

1. Missing a defective cylinder is far costlier than a false alarm. Prioritize:

PrecisionRecallAccuracy

Recall counts the fraction of real defects caught. Safety-critical = recall first.

2. Forecast A: MAE 200. Forecast B: MAE 220 but far fewer huge misses. Metric that would favor B:

RMSEMAPEAccuracy

RMSE squares errors, so it punishes catastrophic misses — B's strength.

3. Lowering the classification threshold generally:

Raises precision, lowers recallImproves bothRaises recall, lowers precision

A looser trigger flags more: catches more real cases AND more false alarms.

05Exercises

Build FIG 9.1 for a real or imagined plant detector with your own numbers; compute precision, recall, F1 by hand once.
Write the cost of a false alarm and of a miss for one plant use case in KWD/hours — then state which metric leads.
Ask Claude for a threshold-tuning plot (precision & recall vs threshold) on any classifier and pick the operating point.

06Reflection

Which report at KNPC currently quotes a single accuracy-style number that hides its error structure?
Who should own threshold decisions — engineering, operations, or HSE — for each of your candidate models?

07Common Mistakes

Reporting accuracy on a 1%-positive problem.
Evaluating on training data (Ch. 4's sealed-test rule exists for this).
Leaving the threshold at the 0.5 default without a cost analysis.
Using MAPE on series with near-zero months.

08Cheat Sheet

Precision

TP/(TP+FP) — trust in flags

Recall

TP/(TP+FN) — coverage of real cases

balance · AUC: separation across thresholds

MAE

avg miss in units · RMSE: punishes big misses

MAPE

% language; beware near-zero actuals

Golden rule

unseen data + baseline comparison, always

09AI Prompts

FULL EVALUATION

Evaluate this trained classifier properly: confusion matrix (labeled with my domain terms: [terms]), precision, recall, F1, ROC-AUC, and a calibration curve. Then a plain-English paragraph: what the model is good at, its failure mode, and whether it beats the baseline of [describe]. Flag anything suspiciously good.

THRESHOLD BY COST

False alarm cost: [X]. Missed case cost: [Y]. Positives are [Z]% of cases. Compute the expected cost per prediction across thresholds 0.05–0.95 for my model, plot it, recommend the operating threshold, and show the resulting daily alarm volume for [N] predictions/day.

REFBibliography

scikit-learn User Guide: Model evaluation — scikit-learn.org/stable/modules/model_evaluation.
Fawcett, T. (2006). "An introduction to ROC analysis." Pattern Recognition Letters.
Saito & Rehmsmeier (2015). "The Precision-Recall Plot Is More Informative than ROC on Imbalanced Data." PLOS ONE.
Hyndman & Athanasopoulos, Forecasting: Principles and Practice — otexts.com/fpp3 (forecast accuracy metrics).
Niculescu-Mizil & Caruana (2005). "Predicting Good Probabilities with Supervised Learning." ICML (calibration).

CH-10COMMISSIONED

Model Improvement

Your first model works. Now the disciplined art of making it better — in the right order, because most improvement effort is spent in the wrong place.

10.1The improvement hierarchy (spend effort top-down)

Rank	Lever	Typical gain	Why
1	Better data — more rows, fixed errors, corrected labels	Large	Garbage-in ceiling: no tuning outruns bad data
2	Better features — domain-driven engineering (Ch. 4)	Large	The model can only combine what you give it
3	Better algorithm — next rung on the ladder (Ch. 7)	Moderate	RF → GBM often helps; beyond that, diminishing
4	Hyperparameter tuning	Small–moderate	Polish, not transformation — despite its glamour
5	Ensembling — combine models	Small	Last percent; adds complexity to maintain

⚠ Beginner gravity

Beginners are pulled toward levers 4–5 (they feel technical) while levers 1–2 sit ignored. When a model disappoints, your first question is never "which hyperparameters?" — it's "what data or feature is missing?"

10.2Hyperparameter tuning

Parameters are learned by training (the weights). Hyperparameters are the settings YOU choose before training: tree depth, number of trees, learning rate, regularization strength. Three search strategies:

Method	How	Verdict
Grid search	Try every combination in a grid	Exhaustive but explodes combinatorially; fine for 2–3 knobs
Random search	Sample random combinations	Surprisingly effective — usually beats grid at equal budget; your default
Bayesian optimization (e.g., Optuna)	Each trial informs where to look next	Most sample-efficient; use when training is slow/expensive

Non-negotiable rule: tune against cross-validation scores, never against the test set — otherwise you've quietly trained on your exam.

10.3Regularization — the overfitting brake

Regularization penalizes model complexity during training, forcing it to prefer simpler explanations. In linear models: L1/Lasso (can zero-out useless features — built-in feature selection) and L2/Ridge (shrinks all coefficients smoothly). In trees: limiting depth and requiring minimum samples per leaf. In neural nets: dropout (randomly silencing neurons during training) and early stopping (halt when validation error turns upward).

⬡ Plant Analogy

Regularization is a design margin against over-optimization. A procedure tuned to perfection for last July's exact conditions fails in August; you deliberately keep it slightly general so it survives conditions it hasn't seen.

10.4Data augmentation & imbalance handling

Augmentation — synthetically expanding training data: for images, rotate/flip/brighten defect photos (one photo becomes twenty); for tabular data, use with caution (SMOTE creates synthetic minority samples — helpful sometimes, dangerous if it invents impossible cases).
Class weights — the cleaner first move for imbalance: tell the model that missing a defect costs 50× a false alarm (class_weight in sklearn). Try weights before synthetic data.

10.5Ensembling

Three classical flavors: bagging (parallel diverse models voting — Random Forest is bagging built-in), boosting (sequential error-correction — XGBoost/LightGBM), and stacking (a meta-model learns to combine base models' outputs). Practical guidance: you already use ensembling every time you pick RF or GBM; explicit stacking is competition territory, rarely worth its maintenance cost at work.

10.6The improvement loop, operationalized

# The disciplined loop — one change at a time, always vs CV score
1. Baseline + current model score (CV)     # where are we?
2. Error analysis: WHERE does it fail?      # worst months? one plant? rare class?
3. Hypothesis: one change likely to fix it  # new feature? more data? deeper trees?
4. Apply ONE change → re-score              # never bundle changes
5. Keep if better, revert if not, log it    # notebook or MLflow

Error analysis (step 2) is the professional's secret: look at the actual worst predictions. Ten minutes reading the 20 biggest forecast misses teaches more than a day of tuning — you'll spot the missing holiday flag, the sensor outage, the unrecorded line shutdown.

01Key Takeaways

Improvement hierarchy: data → features → algorithm → tuning → ensembling. Effort flows top-down.
Random search is your tuning default; Bayesian (Optuna) when training is expensive; always against CV, never test.
Regularization trades a little training fit for generalization — the deliberate design margin.
For imbalance: class weights first, synthetic data (SMOTE) second, with skepticism.
One change at a time, driven by error analysis of actual worst cases.

02Beginner Misunderstandings

"Tuning will rescue my weak model"

Tuning polishes; it doesn't transform. A model failing by 30% has a data or feature problem, not a hyperparameter problem.

"I tuned on the test set and improved!"

You leaked. Every peek at test during development converts it into slow training data. Tune on CV; open test once, at the end.

"More complex always wins"

Complexity must pay rent in validation gains AND be worth its maintenance cost. A 0.5% gain from stacking five models is a bad trade at a plant.

03Workplace Applications

Error analysis ritual on the demand forecast: every quarter, read the 10 worst months — feed findings into the feature library.
Class weights on the QC model reflecting the true KWD cost ratio of miss vs false alarm.
An Optuna tuning run logged with MLflow once the forecast model matures — polish captured, reproducibly.

04Mini Quiz

1. Your model is 25% worse than needed. Highest-leverage first move:

Bayesian hyperparameter searchStack five modelsError analysis → better data/features

Large gaps live at the top of the hierarchy. Tuning buys single digits, not 25%.

2. Tuning must be scored against:

Cross-validation on training dataThe test set, repeatedlyTraining accuracy

The test set stays sealed. Repeated test peeks = leakage into your choices.

3. L1 (Lasso) regularization is special because it:

Makes models deeperCan shrink useless feature weights to exactly zero — automatic selectionOnly works on images

Lasso's zeroing property doubles as built-in feature selection.

05Exercises

Take any Ch. 7 exercise model; run error analysis on its 10 worst predictions and write one hypothesis per finding.
Run RandomizedSearchCV on a Random Forest (ask Claude for the code); record gain vs default settings — calibrate your expectations.
Add class_weight to an imbalanced classifier and watch the precision/recall trade shift.

06Reflection

In your engineering work, where do you already apply the "one change at a time" discipline — and where is it violated?
What plant knowledge would explain your forecast's worst month better than any hyperparameter?

07Common Mistakes

Tuning before establishing a clean baseline comparison.
Changing features AND algorithm AND hyperparameters at once — gains unattributable.
SMOTE-ing physically impossible synthetic samples into safety data.
Not logging experiments; two weeks later nobody can reproduce "the good run".

08Cheat Sheet

Hierarchy

data → features → algo → tune → ensemble

random default · Optuna when slow · vs CV only

Regularize

L1 zeros · L2 shrinks · trees: depth/leaf · NN: dropout, early stop

Imbalance

class_weight first · SMOTE with care

Loop

score → error analysis → 1 change → re-score → log

Secret weapon

read the 20 worst predictions

09AI Prompts

ERROR ANALYST

Here are my model's 20 worst predictions with their feature values: [paste]. Find patterns: do errors cluster by time, category, or range? For each pattern, hypothesize a cause and propose ONE specific fix (feature, data, or model change), ranked by expected impact. Domain context: [describe].

TUNING RUN

Write a RandomizedSearchCV (or Optuna if training is slow) for my [model] with sensible search ranges you justify, 5-fold CV, and a final table: default score vs tuned score vs baseline. End with an honest one-liner: was the tuning worth it?

REFBibliography

Bergstra & Bengio (2012). "Random Search for Hyper-Parameter Optimization." JMLR.
Akiba et al. (2019). "Optuna: A Next-generation Hyperparameter Optimization Framework." KDD.
Tibshirani (1996). "Regression Shrinkage and Selection via the Lasso." JRSS-B.
Srivastava et al. (2014). "Dropout." JMLR; Chawla et al. (2002). "SMOTE." JAIR.
scikit-learn User Guide: Tuning the hyper-parameters, Ensemble methods — scikit-learn.org.
Ng, A. — "Machine Learning Yearning" (error analysis methodology).

CH-11COMMISSIONED

Production ML — Models That Keep Working

A notebook that predicted well once is a demo. Production means the model runs on schedule, survives change, and someone knows when it breaks. This is where most real-world ML fails.

11.1Deployment patterns — pick the simplest that works

Pattern	How it runs	Latency	Fits when	Your likely uses
Batch	Scheduled script: load model → score new data → write results to a table/file	Hours–daily	Decisions aren't second-by-second	90% of plant use cases: nightly demand forecast → Power BI, weekly failure-risk list
Real-time API	Model behind a web endpoint (FastAPI); systems POST features, get predictions back	Milliseconds	A system needs answers on demand	QC station querying defect probability per cylinder
Edge	Model runs on the device itself (camera, PLC-adjacent box)	Instant, offline-capable	No connectivity / privacy / speed at source	Vision inspection camera on the filling carousel

▲ The boring truth

A cron-scheduled Python script writing to a database table is a deployment. Start there. FastAPI + Docker enter only when another system (not a person) needs predictions on demand.

11.2The minimum production stack

Serialization — joblib.dump the entire pipeline (preprocessing + model), never the model alone, or production data won't be transformed identically.
FastAPI — the standard Python web framework for serving models: ~20 lines wraps your pipeline in an HTTP endpoint.
Docker — packages code + libraries + Python version into a container that runs identically anywhere. Solves "works on my machine" permanently; it's also how your Ollama/vLLM stack ships.
CI/CD — automation that tests and deploys on every code change (e.g., GitHub Actions). For you initially: automated tests that run before any model update goes live.

11.3Drift — why every model dies

Model Driftانحراف النموذج

Technical	Data drift: input distributions shift from training-time. Concept drift: the input→output relationship itself changes.
Simple	The world changes; the model's frozen worldview doesn't. Accuracy decays silently.
Why it matters	The #1 killer of deployed models. New subsidy policy, new customer segment, replaced equipment — your 2024-trained forecast quietly loses touch with 2026.
Analogy	Instrument calibration drift. Nobody asks IF the gauge drifts — only how often to recalibrate. Models are gauges pointed at a moving process.
You'll use it	Monthly drift check on the demand model: compare recent MAE vs commissioning MAE; alert at +25%.

11.4Monitoring & retraining — the operating procedure

Monitor	What to track	Trigger
System health	Did the job run? On time? Errors?	Any failure → alert same day
Input data	Volumes, missing %, ranges vs training profile	Distribution shift → investigate before trusting outputs
Prediction quality	Rolling MAE/recall once actuals arrive	Sustained degradation past threshold → retrain
Business impact	Is the decision the model feeds still improving?	No → question the model's existence, not just its tuning

Retraining policy — decide at deployment, not during the crisis: scheduled (e.g., quarterly with latest data) or triggered (when monitoring crosses thresholds). Either way: retrain → evaluate against the incumbent on recent data → promote only if better → keep the old version for rollback. Version everything: model file, training data snapshot, code commit.

11.5MLOps — the name for all of this

MLOps = the discipline of running ML reliably: versioning (Git + DVC + model registry), automation (CI/CD), monitoring, and governance. Your maturity path: Level 0 — manual notebook runs (fine for pilots) → Level 1 — scheduled scripts, saved pipelines, basic monitoring (your target for year one) → Level 2 — automated retraining pipelines, registries, full observability (justified only when several models run in production). For your on-prem LLM stack, this same layer is exactly litellm (routing) + langfuse (observability) + ragas (evaluation) — MLOps with LLM-flavored names.

01Key Takeaways

Batch scheduling covers ~90% of plant ML; APIs/edge only when a system needs on-demand answers.
Serialize the full pipeline; Docker guarantees "runs the same everywhere"; CI/CD tests before deploy.
All models drift. Plan recalibration at commissioning, like any instrument.
Monitor four layers: system ran → inputs sane → predictions accurate → business impact real.
Retrain-evaluate-promote-rollback, all versioned. MLOps is just operational discipline you already respect.

02Beginner Misunderstandings

"Deployed = done"

Deployment is commissioning day. The operating life — monitoring, drift, retraining — is where value is won or silently lost.

"I need Kubernetes"

You need cron + joblib + a database table. Industrial-scale orchestration enters with industrial-scale model counts.

"Accuracy was validated at launch, so it holds"

Validation certifies the past. Only monitoring certifies the present.

03Workplace Applications

Nightly batch job: score demand model → write to the table Power BI reads → your dashboard becomes model-fed.
A one-page "Model Operating Procedure" per deployed model — owner, schedule, monitors, retrain triggers, rollback. Pure KNPC procedure culture, applied to ML.
Drift dashboard: rolling forecast MAE with a control-limit line (Ch. 6's 3σ thinking, recycled).

04Mini Quiz

1. A weekly failure-risk list for maintenance planning should be served as:

Real-time API with KubernetesA scheduled batch job writing to a tableEdge deployment on each pump

Weekly cadence = batch. Simplest pattern that meets the latency requirement wins.

2. Forecast MAE has crept from 210 to 340 over four months. Most likely:

The GPU is agingjoblib corrupted the fileDrift — the demand pattern has shifted since training

Gradual decay is drift's signature. Investigate what changed, then retrain on recent data.

3. Why serialize the whole pipeline rather than just the model?

So production data receives the exact same preprocessing as training dataSmaller file sizeDocker requires it

Mismatched preprocessing is a classic silent production failure — the pipeline object prevents it.

05Exercises

Draft the one-page Model Operating Procedure for your future demand model: owner, schedule, 4 monitors, retrain trigger, rollback step.
Ask Claude to wrap a trained pipeline in a 20-line FastAPI app and explain every line — just to see how small "deployment" is.
Define the drift alert for one KPI model: which metric, computed how often, alarm threshold, and who gets notified.

06Reflection

Your plant has recalibration schedules for instruments. What's the organizational equivalent for models — and who owns it?
Which upcoming change at KNPC (pricing, subsidy, equipment) would silently break a model trained today?

07Common Mistakes

No baseline snapshot of input distributions at deployment — drift becomes undetectable.
Retraining on data that includes the degraded period's corrupted inputs without review.
Deploying a new version with no rollback path.
Monitoring accuracy but not whether the job even ran (silent 3-week outages are real).

08Cheat Sheet

Pattern

batch (default) · API (on-demand) · edge (at source)

Stack

pipeline + joblib → FastAPI → Docker → CI/CD

Drift

data drift (inputs) · concept drift (relationship)

Monitor

ran? inputs? accuracy? business impact?

Retrain

schedule or trigger → beat incumbent → promote → keep rollback

Version

model + data snapshot + code commit, together

09AI Prompts

DEPLOYMENT PLANNER

My model: [describe]. Consumers of predictions: [who/what systems]. Required freshness: [latency]. Infrastructure available: [describe — e.g., one on-prem server, no cloud]. Recommend the simplest deployment pattern, the exact minimal stack, and a monitoring plan with 4 specific checks and thresholds. Explicitly list what I should NOT build yet.

BATCH JOB BUILDER

Write a production-quality Python batch script that: loads pipeline.pkl, reads new rows from [source], validates inputs against expected ranges (fail loudly with a clear log message), writes predictions to [destination] with a timestamp and model version column, and logs a run summary. Include error handling that never writes partial results.

REFBibliography

Sculley et al. (2015). "Hidden Technical Debt in Machine Learning Systems." NeurIPS — the founding MLOps paper.
Google Cloud Architecture Center: "MLOps: Continuous delivery and automation pipelines in ML" (maturity levels).
Gama et al. (2014). "A Survey on Concept Drift Adaptation." ACM Computing Surveys.
FastAPI documentation — fastapi.tiangolo.com; Docker documentation — docs.docker.com.
Huyen, C. (2022). Designing Machine Learning Systems. O'Reilly.

CH-12COMMISSIONED

Generative AI & LLMs in Practice

The chapter closest to your day-to-day: you're already deploying Qwen on-prem and planning RAG over SharePoint. Here's the complete conceptual map behind those decisions.

12.1Prompt engineering — the six patterns that matter

Pattern	What it is	Example fragment
Role	Assign expertise + audience	"You are a senior reliability engineer writing for plant operators…"
Context loading	Give the situation, constraints, data — the model can't read your mind	"Two filling plants, 12kg focus, Kuwaiti summer peak, fiscal year Apr–Mar…"
Output specification	Dictate format, length, structure	"Return a markdown table with columns X, Y, Z; max 10 rows"
Few-shot examples	Show 1–3 examples of input→desired output; the strongest single lever for consistency	"Here are two correctly classified logs: …"
Step-by-step reasoning	Ask it to reason before answering — measurably improves multi-step accuracy	"Think through the calculation first, then state the answer"
Self-critique	Second pass: "review your answer for errors/omissions"	Catches a surprising share of first-draft mistakes

12.2RAG — your most important architecture

RAG — Retrieval-Augmented Generationالتوليد المعزز بالاسترجاع

Technical	Pipeline: documents → chunks → embeddings → vector DB; at query time, retrieve the most relevant chunks and inject them into the LLM's prompt as grounding context.
Simple	Give the model an open-book exam with YOUR books, instead of asking it to answer from memory.
Why it matters	Solves the three LLM killers at once: stale knowledge, no access to private documents, hallucination (answers now cite retrieved text). It's the standard enterprise pattern — and cheaper/safer than fine-tuning for knowledge.
Analogy	A brilliant new engineer who hasn't read KNPC's archives. RAG hands them the right 5 pages from the archive before every question — instead of sending them to a 6-month training course (fine-tuning).
You'll use it	Your SharePoint spec corpus → markitdown/MinerU for conversion → chunking → embeddings → vector DB → Qwen answers with citations. Quality measured with ragas.

FIG 12.1 — Your SharePoint RAG pipeline end to end. The top row runs once (and on document updates); the bottom row runs per question.

What actually determines RAG quality (in order): document conversion fidelity (garbage extraction = garbage answers — why MinerU matters for engineering PDFs with tables), chunking strategy (split by section, not blindly by character count), retrieval quality (embedding model choice; hybrid keyword+vector search for spec numbers), and only then the LLM. Evaluate with ragas-style metrics: faithfulness (answer grounded in retrieved text?), relevance, and retrieval hit rate on a golden set of ~50 real engineer questions.

12.3Fine-tuning vs RAG vs prompting

Need	Right tool	Why
Model lacks YOUR documents/knowledge	RAG	Knowledge stays fresh, citable, updatable without retraining
Model output format/style/behavior wrong	Prompting first, fine-tuning if truly persistent	Few-shot examples fix most format problems free
Specialized vocabulary/task at high volume	Fine-tuning (LoRA)	Teaches behavior patterns; does NOT reliably add facts
General capability shortfall	Bigger/better model	No amount of tuning turns a 3B model into a 30B one

⚠ The classic enterprise mistake

Fine-tuning to inject company knowledge. It's expensive, freezes knowledge at tuning time, and hallucinates confidently about the gaps. Knowledge → RAG. Behavior → fine-tune. Tattoo this distinction.

12.4Agents, tools, and MCP

An agent is an LLM in a loop: it plans, calls tools (run SQL, search documents, call an API), observes results, and iterates until the task completes. MCP (Model Context Protocol) is the open standard for plugging tools into LLMs — you've already used it connecting Power BI to Claude. Your text-to-SQL pilot is exactly a single-tool agent: question → generated SQL → execute → summarize results. The engineering discipline: constrain tools (read-only DB access!), log every step (langfuse), and keep a human gate on anything that writes.

12.5Your on-prem serving stack, mapped

Layer	Your component	Role
Inference engine	Ollama (pilot) → vLLM (production)	Runs Qwen3-30B; vLLM adds high-throughput serving for ~30 users
Gateway / routing	litellm	One API for all models; per-user keys, quotas, fallbacks
Observability	langfuse	Logs every prompt/response/cost/latency — your COGNOS for LLM usage
Evaluation	ragas + golden question set	Regression-tests RAG quality after every change
Document ingestion	markitdown / MinerU	PDF/Office → clean text for the RAG index

01Key Takeaways

Six prompt patterns (role, context, output spec, few-shot, reasoning, self-critique) cover professional prompting.
RAG = open-book exam with your books; quality lives in conversion + chunking + retrieval, not just the LLM.
Knowledge → RAG; behavior → fine-tune; format → prompt. Never fine-tune to inject facts.
Agents = LLM + tools in a loop; MCP standardizes the plugs; constrain, log, human-gate writes.
Your stack is coherent: Ollama/vLLM serve, litellm routes, langfuse watches, ragas grades.

02Beginner Misunderstandings

"RAG eliminates hallucination"

It dramatically reduces it and adds citations, but bad retrieval or over-creative generation still fabricates. That's why ragas faithfulness scoring exists.

"We should fine-tune Qwen on all our documents"

Fine-tuning teaches style, not a reliable document memory. Your corpus belongs in a vector DB, retrieved per-question.

"Better prompts are about magic words"

They're about information: context, examples, and explicit output contracts. The six patterns are engineering, not incantation.

03Workplace Applications

Spec-RAG assistant: "What's the test pressure for 12kg cylinders per our spec?" → answer + clause citation.
Text-to-SQL on planning tables (your live pilot) — with read-only credentials and langfuse audit trail.
Bilingual report drafting: Arabic/English management summaries from structured monthly data.
Golden-question regression suite: 50 real engineer questions, scored monthly with ragas.

04Mini Quiz

1. Engineers need current spec answers with citations. The right architecture:

Fine-tune Qwen on the PDFsRAG over the spec corpusA bigger base model

Private, updatable, citable knowledge = RAG. Fine-tuning won't reliably memorize documents.

2. The single strongest prompt lever for consistent output format:

Few-shot examples of input→desired outputTyping in capital lettersLonger prompts of any kind

Showing beats telling. 1–3 examples anchor structure better than paragraphs of instructions.

3. In your stack, the component that answers "which users asked what, at what cost, with what latency":

vLLMragaslangfuse

langfuse = observability. vLLM serves; ragas evaluates quality.

05Exercises

Rewrite one of your recent Claude prompts applying all six patterns; compare outputs side by side.
Draft the 50-question golden set for your spec-RAG from real questions engineers actually ask (start with 10).
Design the guardrails table for your text-to-SQL agent: allowed schemas, read-only enforcement, row limits, logging, human review triggers.

06Reflection

Which division questions are asked repeatedly whose answers already exist in documents nobody re-reads?
Where must a human stay in the loop permanently, regardless of how good the models get?

07Common Mistakes

Skipping document-conversion quality checks — the #1 silent RAG killer for engineering PDFs.
Chunking by fixed character count through the middle of specification tables.
Granting an SQL agent write permissions "for convenience".
Deploying RAG with no golden-set evaluation — quality regressions go unnoticed for months.

08Cheat Sheet

Prompt six

role · context · output spec · few-shot · reasoning · critique

RAG chain

convert → chunk → embed → store → retrieve → generate

Decision

knowledge→RAG · behavior→fine-tune · format→prompt

Agent rules

constrain tools · log all · gate writes

Your stack

Ollama/vLLM · litellm · langfuse · ragas

RAG quality

conversion > chunking > retrieval > model

09AI Prompts

RAG ARCHITECT

Design a RAG pipeline for [N] documents of type [engineering specs with tables/scanned PDFs/mixed], serving [M] users on-prem with [hardware]. Specify: conversion tool and quality checks, chunking strategy with sizes and WHY, embedding model (multilingual — Arabic+English), vector DB choice, retrieval approach (hybrid?), and an evaluation plan with a golden question set. Flag the top 3 failure risks for my document type.

PROMPT UPGRADER

Here is a prompt I use often: [paste]. Rewrite it applying: role, context loading, explicit output specification, one few-shot example, and a self-critique instruction. Then explain each change's purpose in one line.

AGENT GUARDRAILS

I'm building a text-to-SQL agent over [describe tables] for [users]. Write its system prompt including: allowed schemas only, read-only enforcement, automatic LIMIT clauses, refusal rules for destructive requests, and a response format that always shows the SQL before results. Then list 5 test prompts a malicious or careless user might try.

REFBibliography

Lewis et al. (2020). "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." NeurIPS — the RAG paper.
Brown et al. (2020). "Language Models are Few-Shot Learners." NeurIPS (GPT-3, few-shot).
Wei et al. (2022). "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." NeurIPS.
Hu et al. (2021). "LoRA: Low-Rank Adaptation of Large Language Models." ICLR.
Anthropic prompt engineering documentation — docs.claude.com; Model Context Protocol — modelcontextprotocol.io.
Es et al. (2023). "RAGAS: Automated Evaluation of Retrieval Augmented Generation." arXiv:2309.15217.
Kwon et al. (2023). "Efficient Memory Management for LLM Serving with PagedAttention." SOSP (vLLM).

CH-13COMMISSIONED

ML at the LPG Filling Plant

Everything so far, aimed at GFB-U and GFB-S. A ranked application portfolio, honest difficulty ratings, and the sequencing logic of which to build first.

13.1The application portfolio

Application	Technique (Ch.)	Data readiness	Difficulty	Impact
Cylinder demand forecasting (monthly/weekly, by size)	Time series: Prophet/GBM+lags (7)	High — years of filling logs	Low	High: inventory, shifts, distribution planning
Energy anomaly detection (kWh per cylinder drifting)	3σ → Isolation Forest (6, 7)	High — meter + production data	Low	Medium: OPEX + early fault signal
Spec-RAG assistant (SharePoint corpus)	RAG (12)	High — corpus exists	Medium	High: division-wide time savings
Text-to-SQL on planning data	LLM agent (12)	High — your pilot	Medium	High: self-serve analytics for 30 users
Downtime/stoppage classification & prediction	Classification: RF/GBM (7)	Medium — depends on log discipline	Medium	High: maintenance planning
Maintenance-log mining (cluster failure texts)	Embeddings + clustering (8, 7)	Medium — free-text quality varies	Medium	Medium: recurring-failure visibility
Cylinder visual defect detection	Fine-tuned CNN (8)	Low — needs labeled photos	High	High: QC consistency & safety
Filling-line predictive maintenance (sensor-based)	Anomaly + classification (7)	Low — sensor history & failure labels scarce	High	High: unplanned downtime ↓
OPEX/CAPEX forecast & variance early warning	Regression + control limits (7, 6)	High — your own tables	Low	Medium: planning accuracy
Distribution/truck scheduling optimization	Optimization (+ forecast input)	Medium	High	Medium–high, shared with logistics

13.2Sequencing — the dependency logic

FIG 13.1 — Waves ordered by data readiness × skill building. Wave 3's gate is data collection that must START during Wave 1 — the classic sequencing insight.

▲ The Wave-3 unlock

Vision QC and predictive maintenance aren't blocked by algorithms — they're blocked by data you aren't collecting yet. The highest-leverage action this quarter costs almost nothing: standardized defect photo capture at QC stations, and disciplined failure-cause codes in the maintenance log. Wave 1 delivers value; its quiet job is seeding Wave 3.

13.3Fitting your existing landscape

Existing asset	Role in the ML architecture
COGNOS / planning tables	Source of record → SQL extracts feed pandas & the text-to-SQL agent
Power BI + DAX layer	Delivery surface: model outputs land in tables Power BI already reads — predictions appear inside dashboards management already trusts
SharePoint spec system	RAG corpus + the change-request workflow keeps the index fresh
On-prem server (Ollama→vLLM)	LLM inference for RAG, text-to-SQL, and report drafting — data never leaves KNPC
Your HTML dashboards	Rapid UI layer for pilot tools before formal IT integration

The selling framework for each proposal, in planning language: baseline cost of current practice → pilot scope (one plant, one product, 3 months) → success metric fixed in advance (e.g., "forecast MAPE under 7% vs 11% manual") → decision gate. Pilots framed as reversible experiments pass approval chains that "AI transformation programs" never survive.

01Key Takeaways

Ten credible applications exist today; three are Wave-1 ready with data you already control.
Sequence by data readiness × skill building, not by impressiveness.
Wave 3 is gated by data collection that must start now — photos and failure codes.
Deliver predictions through Power BI surfaces management already trusts.
Sell pilots with baselines, pre-agreed metrics, and decision gates — planning language, not AI hype.

02Beginner Misunderstandings

"Start with the most impressive project"

Vision QC demos well and fails first — no labeled photos. Credibility compounds from boring wins; start where the data is.

"We need new sensors/systems before any ML"

Wave 1 runs entirely on data already in COGNOS, meters, and your Excel exports.

"One big integrated AI platform"

Portfolio of small, independently valuable tools > monolith. Each survives on its own ROI.

03Workplace Applications

This chapter IS the application catalog — take FIG 13.1 into your 5-year plan as the digital initiatives roadmap.
Quarterly portfolio review: re-score data readiness as logging discipline improves.
Pair each wave with a procedure doc (your LPG-ENG-PROC pattern) so tools outlive their builder.

04Mini Quiz

1. The correct first flagship project is demand forecasting because:

Data exists, difficulty is low, impact is high, and it builds the full skill loopIt uses the most advanced algorithmsIt requires no evaluation

Readiness × impact ÷ difficulty. The flagship also exercises every chapter of this manual once.

2. Vision QC's real blocker is:

CNNs aren't accurate enoughGPU costNo labeled defect photo dataset exists yet

Algorithms are commoditized; labeled data isn't. Hence: start collecting now.

3. The smartest delivery surface for model predictions at KNPC:

A new standalone AI portalTables feeding the existing Power BI dashboardsEmail attachments

Meet users where trust already exists. Batch job → table → dashboard.

05Exercises

Score each portfolio row for YOUR reality: adjust data readiness based on what you know about log quality at both plants.
Draft the one-page pilot proposal for the demand forecast: baseline, scope, metric, gate — in your annual-plan format.
Design the defect-photo collection SOP: angle, lighting, naming convention, label categories, storage location. One page, deployable this month.

06Reflection

Which stakeholder loses convenience if these tools succeed, and how do you bring them along early?
If you could only ship ONE application this year, which maximizes both plant value and your own capability growth?

07Common Mistakes

Leading with Wave-3 ambitions in management presentations — then owning the failure.
Building tools without a named operational owner for the post-you era.
Ignoring the data-collection SOPs that unlock future waves.
Measuring pilots by enthusiasm instead of the pre-agreed metric.

08Cheat Sheet

Wave 1

demand forecast · energy anomaly · OPEX variance

Wave 2

spec-RAG · text-to-SQL · downtime classification

Wave 3

vision QC · predictive maintenance · scheduling

Unlock now

photo SOP + failure-cause codes

Delivery

batch → table → Power BI

Selling

baseline → pilot → metric → gate

09AI Prompts

PILOT PROPOSAL

Write a one-page pilot proposal for [application] at an LPG filling plant, in formal planning language: current-practice baseline and its cost, pilot scope (one plant, 3 months), data sources, success metric with target number, resources needed, risks with mitigations, and a go/no-go decision gate. Audience: division management. No AI hype vocabulary.

DATA COLLECTION SOP

Draft a standard operating procedure for collecting [defect photos / failure-cause codes] at a filling plant QC station: step-by-step capture instructions, naming and labeling convention, quality requirements, storage path, roles and responsibilities, and a weekly compliance check. Keep it to one page; operators are the audience.

REFBibliography

Hyndman & Athanasopoulos, Forecasting: Principles and Practice (3rd ed.) — otexts.com/fpp3 (demand forecasting methodology).
Lee, Davari et al. (2018). "Industrial AI and predictive analytics in manufacturing" — Manufacturing Letters (industrial ML application patterns).
Google, "Rules of Machine Learning" (M. Zinkevich) — developers.google.com (start-simple sequencing doctrine).
McKinsey Global Institute reports on AI in operations — application economics framing (directional, not gospel).

CH-14COMMISSIONED

Your First Projects — Three Guided Builds

Theory ends here. Three complete project walkthroughs, each with the exact structure, the Claude prompts to use, and the verification steps that keep you the engineer of record.

14.1The universal project skeleton

Phase	Deliverable	Chapters used	Share of effort
1. Frame	One paragraph: decision served, prediction target, prediction moment, success metric vs baseline	1, 9	10%
2. Data	Clean dataset + quality report + leakage check	4	35%
3. EDA	3 findings, 3 charts, 3 decisions page	5	15%
4. Model	Baseline + escalation ladder comparison table	7, 10	20%
5. Evaluate	Right metric, unseen data, error analysis	9	10%
6. Deliver	Saved pipeline + batch script + one-page summary	11	10%

14.2Project A — Monthly cylinder demand forecast Wave 1 flagship

Frame: Predict next-month 12kg cylinder demand per plant, at month-start, to set filling schedules and inventory. Success: MAPE < the seasonal-naive baseline ("same month last year") by at least 25% relative.

Data: 4–5 years of monthly filled quantities by plant and size + engineered features (Ch. 4): month, Ramadan overlap %, Eid flags, summer index, lags (1, 2, 12 months), rolling means. Leakage check: every feature known on day 1 of the forecast month.
EDA: demand line per plant, seasonal decomposition, Ramadan-shift visualization (it moves ~11 days/year — your feature must track the Hijri calendar, not fixed months).
Model ladder: seasonal-naive baseline → linear with seasonality dummies → GBM with lags → (optional) Prophet with custom Ramadan/Eid events. Time-based split: train ≤2024, validate 2025.
Evaluate: MAE + MAPE per plant; error analysis on worst 5 months (expect: Ramadan boundary months and any subsidy/policy change).
Deliver: pipeline.pkl + monthly batch script writing to the Power BI table + one-page metric card.

PROJECT A MASTER PROMPT

You are my ML pair programmer. Project: monthly LPG cylinder demand forecast for two plants, Kuwait (fiscal year Apr–Mar, Ramadan/Eid effects follow the Hijri calendar). I have monthly data: [describe columns/years]. Build it in phases and STOP after each phase for my review: 1) feature engineering incl. Hijri-aware Ramadan overlap fraction per Gregorian month, lags and rolling stats, with a leakage check table; 2) EDA with seasonal decomposition; 3) seasonal-naive baseline, then linear, then LightGBM — time-split train ≤2024 / test 2025, compare MAE and MAPE in one table; 4) error analysis of the 5 worst months; 5) save the full pipeline with joblib and write a monthly scoring script. Explain every step in simple English comments.

14.3Project B — Stoppage-cause classification Wave 2

Frame: From stoppage log fields (line, duration, shift, preceding readings, free-text note), classify the cause category — building toward "which line stops next week". Success: F1 above the majority-class baseline; recall prioritized for the costliest cause.

The distinctive steps: standardize the cause labels first (the "GFB-U/GFBU" problem of Ch. 4 always lives here); use embeddings on the free-text notes as features (Ch. 8 meets Ch. 7); class weights for rare causes (Ch. 10); confusion matrix review with maintenance staff — their reading of the errors is the real evaluation (Ch. 9).

PROJECT B MASTER PROMPT

Project: classify plant stoppage causes from a log with columns [list]. Free-text notes are mixed Arabic/English. Phase plan with stops for my review: 1) label audit — list all distinct cause spellings, propose a standardized mapping for my approval; 2) features: categorical encoding + multilingual sentence-embeddings of the notes; 3) Logistic → Random Forest with class weights, stratified 5-fold CV, report per-class precision/recall (NOT just accuracy — classes are imbalanced); 4) confusion matrix with my domain labels + the 10 most confident WRONG predictions for my expert review; 5) plain-English summary of which causes are predictable and which need better logging.

14.4Project C — Energy-per-cylinder anomaly monitor Wave 1

Frame: Flag days where energy consumption per filled cylinder deviates abnormally — early warning for equipment degradation or metering issues. Success: operators confirm ≥50% of flags as "worth investigating" (precision proxy), with <3 flags/week (alarm-fatigue budget).

The distinctive steps: start with the 3σ control chart (Ch. 6) — deploy it, it's already valuable; then Isolation Forest on [energy/cylinder, volume, downtime, ambient temp] jointly to catch multivariate oddities the univariate chart misses; tune the contamination parameter to the alarm budget; every flag gets a one-line operator verdict — this feedback becomes labels for a future supervised model (the data flywheel).

PROJECT C MASTER PROMPT

Project: daily energy-per-cylinder anomaly monitor. Data: daily [energy kWh, cylinders filled, downtime minutes, ambient temp] for [period]. Build: 1) 3-sigma control chart on energy/cylinder with clear matplotlib styling for a wall display; 2) Isolation Forest on all four variables, contamination tuned so total flags ≈ 2–3/week; 3) a comparison: which anomalies does each method catch, with dates listed; 4) a daily batch script that appends flags to a CSV with a blank "operator_verdict" column — explain how this column becomes training labels later. Simple English comments throughout.

▲ The verification discipline (applies to all three)

For every AI-generated phase: (1) read the code and explain it back in one paragraph — if you can't, ask Claude to simplify until you can; (2) check one calculation by hand or in Excel; (3) run the leakage question on every feature; (4) confirm the metric beats the baseline on unseen data before telling anyone. You are the engineer of record; Claude is the contractor.

01Key Takeaways

One skeleton fits all projects: frame → data → EDA → model ladder → evaluate → deliver.
Framing fixes the prediction moment and the baseline BEFORE any code — everything downstream depends on it.
Ramadan follows the Hijri calendar; your features must too. Domain detail beats model sophistication.
Operator feedback loops (Project C's verdict column) turn unsupervised pilots into future supervised gold.
Verification protocol: explain back, hand-check, leakage-check, baseline-check.

02Beginner Misunderstandings

"I'll do all phases in one giant prompt"

Phase-gated prompting ("STOP after each phase") keeps you in control and catches errors before they compound.

"The project ends when the model works"

It ends when the batch script runs on schedule and someone uses the output. Phase 6 is the project.

"My first project should impress"

Your first project should FINISH. Completion teaches the full loop; ambition teaches abandonment.

03Workplace Applications

These ARE the workplace applications — Projects A and C are deployable within weeks on data you control.
Each finished project becomes a template: swap the dataset, keep the skeleton.
The one-page metric cards accumulate into your evidence file for the Wave-2 proposals.

04Mini Quiz

1. Project A's Ramadan feature must be computed from:

Fixed Gregorian monthsThe Hijri calendar overlap with each Gregorian monthIt doesn't matter

Ramadan shifts ~11 days/year against the Gregorian calendar. A fixed-month flag decays into noise.

2. Project C limits flags to 2–3/week because:

Alarm fatigue kills adoption — a budget keeps flags credibleThe algorithm can't produce moreWeekly reports look better

An ignored monitor is a dead monitor. The alarm budget is a product decision (Ch. 9's threshold logic).

3. "STOP after each phase for my review" exists so that:

Claude restsPrompts stay shortYou verify each stage before errors compound downstream

Phase gates = your engineering hold points, applied to AI-assisted work.

05Exercises

Execute Project A end to end this month — it's the manual's graduation exam.
Write the framing paragraph (decision, target, moment, metric, baseline) for a 4th project of your choosing.
After Project A, write a half-page retrospective: where did you lose the most time, and which chapter would have saved it?

06Reflection

Which project's output would YOUR manager notice first — and does that change your starting order?
What does "engineer of record" mean to you when the code is 90% AI-written?

07Common Mistakes

Skipping the framing paragraph and discovering mid-project that the prediction moment makes half the features illegal.
Accepting phase outputs without the explain-back test.
Random-splitting Project A (yes, again — it's the most repeated error in applied ML).
Building all three at once; finish A before opening B.

08Cheat Sheet

Skeleton

frame → data → EDA → ladder → evaluate → deliver

Project A

demand · Hijri features · time split · MAPE vs naive

Project B

stoppage causes · label audit first · per-class metrics

Project C

3σ chart → IsoForest · alarm budget · verdict column

Verify

explain back · hand-check · leakage · baseline

Rule

finish > impress

09AI Prompts

PHASE-GATE WRAPPER

For this entire project, operate in phase-gate mode: complete only the current phase, then STOP and give me: (a) what you did in 3 bullets, (b) what I should verify and how, (c) one risk you see. Do not proceed until I write "approved". Current phase: [X].

RETROSPECTIVE

Project finished. Here's what happened: [notes]. Write an honest retrospective: what consumed the most time vs its value, which decisions were wrong in hindsight, what to templatize for next time, and 3 specific skills to strengthen based on where I struggled.

REFBibliography

Hyndman & Athanasopoulos, FPP3 — otexts.com/fpp3 (baseline & accuracy methodology for Project A).
scikit-learn: TimeSeriesSplit, class_weight, IsolationForest docs — scikit-learn.org.
Taylor & Letham (2018), Prophet — custom seasonality/holiday events.
Reimers & Gurevych (2019). "Sentence-BERT." EMNLP (multilingual embeddings for Project B).
NIST/SEMATECH e-Handbook — control chart construction for Project C.

CH-15COMMISSIONED

Working With AI Assistants

Your force multiplier, used professionally. How to direct, verify, and debug with Claude/ChatGPT so that AI-written code carries your engineering signature.

15.1The collaboration model

Treat the assistant as a brilliant, tireless contractor with no site knowledge and occasional confident errors. Contractors get: clear scopes (framing paragraphs), hold points (phase gates), inspections (verification), and documentation requirements (comments in simple English). The quality of AI output is a direct function of the quality of your direction — which is why Chapters 1–14 make you a better AI user: you now know what to ask for and what to distrust.

15.2Hallucination detection in code & analysis

Red flag	What it looks like	Your check
Phantom functions/parameters	Code calls a library function that doesn't exist (plausible name, wrong reality)	Run it — `AttributeError`/`TypeError` exposes it instantly; or check the official docs
Too-smooth results	Every step "works", metrics look great first try	Great first-try metrics = leakage until proven otherwise (Ch. 4)
Invented facts/citations	Confident references, standards, or numbers you can't trace	Ask "give the exact source"; verify independently before it enters a KNPC document
Silent assumption changes	You said fiscal year; the code quietly uses calendar year	Ask for an "assumptions list" with every deliverable; diff it against your framing
Agreement drift	You push back; it capitulates even when it was right	Ask "steelman both options" instead of "am I right?"

15.3The debugging protocol

# When AI code fails — in this order
1. Paste the FULL traceback, not the last line
2. Include: what you ran, what you expected, what happened
3. Ask for the cause in ONE sentence before any fix       # forces diagnosis over guessing
4. If two fixes fail: "step back — question your approach,
   list 3 alternative root causes, rank by likelihood"      # breaks the patch-loop
5. After the fix works: "explain what was wrong so I
   recognize this class of error next time"                 # convert bugs into skill

15.4Testing & refactoring with AI

Tests — ask for pytest tests with every non-trivial function: normal case, edge case (empty data, single row, missing month), and a deliberately wrong input that must raise an error. Run tests before trusting any refactor.
Refactoring — "same behavior, better structure": ask for it only AFTER tests exist, then run tests to prove behavior held. This is how notebooks graduate into the batch scripts of Ch. 11.
Code review — paste working code and ask for a review against a checklist: correctness risks, leakage, hard-coded values, missing error handling, unclear names. A second AI session reviewing the first one's code catches a surprising amount.

15.5Session craft

Context files beat re-explaining — maintain a project brief (your /handoff pattern is exactly this) and paste it at session start.
One session, one mission — mixing the forecast project with dashboard styling degrades both; long mixed sessions drift.
Checkpoint the state — at milestones, ask for a "current state summary + exact next step" and save it (your /save pattern). Sessions die; briefings persist.
You own the domain — never let generated text about LPG operations, Kuwait context, or KNPC procedure pass without your correction. The model writes; you author.

01Key Takeaways

AI = brilliant contractor, no site knowledge: give scopes, hold points, inspections.
Five hallucination red flags; the universal antidotes are "run it", "source it", and "list your assumptions".
Debug protocol: full traceback → one-sentence diagnosis → escalate to root-cause listing after two failed patches.
Tests before refactors; review sessions catch the builder session's blind spots.
Session craft: briefs in, checkpoints out, one mission per session.

02Beginner Misunderstandings

"AI code that runs is code that's right"

Running means syntactically valid. Right means correct logic, no leakage, sane assumptions — that's your inspection.

"Longer conversations = smarter assistant"

Long mixed sessions accumulate stale context and drift. Fresh session + good brief beats a marathon.

"Asking AI to check its own work is pointless"

Self-critique passes and fresh-session reviews measurably catch errors — cheap insurance you should always buy.

03Workplace Applications

A verification SOP for AI-assisted work in your division — the quality procedure nobody has written yet.
Project briefs as standing artifacts (you already run /handoff — formalize it per KNPC project).
Review-session ritual before any AI-built tool touches real planning data.

04Mini Quiz

1. First-try model shows 99.2% accuracy. Professional reflex:

Celebrate and present itAsk for 99.5%Suspect leakage or evaluation error; audit before believing

Too-good-too-fast is the signature of leakage (Ch. 4) or broken evaluation (Ch. 9).

2. Two AI fixes in a row failed. Best next prompt:

"Step back — list 3 alternative root causes ranked by likelihood""Try again""Fix it properly this time"

Breaks the patch-loop by forcing re-diagnosis instead of a third guess.

3. Refactoring is safe when:

The code looks cleanerTests written beforehand still pass afterwardThe AI says it preserved behavior

"Same behavior" is provable only by tests that existed before the change.

05Exercises

Take any AI code you accepted this month; run the review-checklist prompt on it in a fresh session. Count the findings.
Practice the debugging protocol on a deliberately broken script — including step 4's root-cause escalation.
Write your personal AI-verification SOP: 5 checks, one page, pinned next to your monitor.

06Reflection

Where in your workflow do you currently accept AI output on trust — and what's the worst case if it's wrong there?
Which of your engineering habits (hold points, checklists, sign-offs) transfer directly to AI collaboration?

07Common Mistakes

Pasting the last error line instead of the full traceback.
Letting the assistant "fix" by rewriting everything — diff discipline lost.
Accepting apologetic capitulation as confirmation you were right.
No project brief: re-explaining context badly every session, getting inconsistent architecture.

08Cheat Sheet

Model

brilliant contractor, no site knowledge

Red flags

phantom APIs · too-smooth · unsourced facts · silent assumptions

Debug

full traceback → 1-sentence cause → root-cause list after 2 fails

Safety

tests → refactor → tests again

Sessions

brief in · checkpoint out · one mission

Authority

the model writes; you author

09AI Prompts

CODE REVIEW CHECKLIST

Review this code as a skeptical senior engineer. Checklist: 1) correctness risks and edge cases, 2) data leakage anywhere in the pipeline, 3) hard-coded values that should be parameters, 4) missing error handling, 5) misleading names/comments. Rate each finding severity high/medium/low. Do NOT rewrite the code yet — findings only. [paste code]

ASSUMPTIONS AUDIT

List every assumption embedded in the work you just produced: about the data, the time ranges, calendars (fiscal vs calendar, Hijri effects), units, business rules, and defaults you chose silently. Format: assumption | where it appears | risk if wrong.

TEST GENERATOR

Write pytest tests for this function: one normal case with hand-computed expected values, edge cases (empty input, single row, missing values), and one invalid input that must raise a clear error. Explain what each test protects against. [paste function]

REFBibliography

Anthropic — Claude documentation & prompt engineering guides — docs.claude.com.
Chen et al. (2021). "Evaluating Large Language Models Trained on Code." arXiv:2107.03374 (Codex — capabilities & error modes).
Ji et al. (2023). "Survey of Hallucination in Natural Language Generation." ACM Computing Surveys.
pytest documentation — docs.pytest.org.
Fowler, M. Refactoring (2nd ed.) — behavior-preserving change discipline.

CH-16COMMISSIONED

Your Learning Roadmap

The manual ends; the practice begins. A time-phased plan calibrated to a working engineer's calendar — roughly 5 focused hours a week, compounding for two years.

16.1The phased plan

Horizon	Mission	Concrete outputs	Manual chapters in play
Days 1–30	Environment + fluency	Colab running · Ch. 2–5 exercises done on one REAL work dataset · first EDA one-pager delivered to yourself	2, 3, 4, 5
Days 31–60	First model, honestly evaluated	Project A (demand forecast) through phase 4 · baseline beaten and documented · error analysis written	6, 7, 9, 14
Days 61–90	Ship something	Project A deployed as a scheduled batch job feeding Power BI · Model Operating Procedure page signed · Project C (3σ chart) live	10, 11, 14
Months 4–6	The LLM layer	Spec-RAG pilot with golden-question evaluation · text-to-SQL hardened with guardrails · one Kaggle "Playground" competition entered (for calibration against the world)	8, 12, 13, 15
Year 1	Portfolio + credibility	3 deployed tools with metric cards · Wave-3 data collection running 6+ months · one internal talk: "what ML actually did for GFB"	All
Year 2	Depth + leadership	Chosen specialization (forecasting, industrial LLM systems, or vision QC) studied to depth · you review others' AI-assisted work · Wave 3 unblocked by the data you seeded	Beyond the manual

⚠ The honest failure mode

The plan above fails one way: courses without projects. Every horizon is defined by outputs, not by content consumed. If a week produced no artifact — code, page, chart, decision — it didn't count, whatever you watched.

16.2Curated resources (deliberately short)

Type	Resource	When
Course	Machine Learning Specialization — Andrew Ng, DeepLearning.AI/Coursera	Months 1–3, alongside projects (the theory backbone)
Book	Hands-On Machine Learning (3rd ed.) — Aurélien Géron, O'Reilly	The permanent desk reference; read chapters as projects demand
Book (free)	Forecasting: Principles and Practice — Hyndman & Athanasopoulos, otexts.com/fpp3	Before/during Project A — the forecasting bible
Book (free)	An Introduction to Statistical Learning — statlearning.com	Months 4+, when you want the "why" behind Ch. 7
Practice	Kaggle — Learn courses, then Playground competitions	Month 4+; calibrates your skill against reality
Course	DeepLearning.AI short courses (RAG, LLMOps, prompt engineering)	Months 4–6, feeding the Wave-2 builds
Community	r/MachineLearning · Hugging Face forums · Kaggle discussions	Ongoing; lurk, then ask, then answer
Reference	scikit-learn User Guide · Anthropic docs	Always open in a tab

16.3Staying current without drowning

The field moves fast at the frontier and slowly at the foundations. Everything in Chapters 1–11 will still be true in five years; Chapter 12's tool names will churn. Strategy: foundations deep, frontier sampled — one weekly digest (your Scout agent already does this), one experiment per month with something new, and zero guilt about ignoring the rest. Depth on your three specialization candidates beats breadth across everything.

▲ Final word

You started this manual as a Planning Engineer who uses AI. You finish it knowing the full pipeline: what ML is, how data becomes models, how models are judged, shipped, and kept alive, and how to command AI assistants with an engineer's skepticism. The 16 valves on your manifold are open. Now go build Project A — the plant is waiting.

01Key Takeaways

Outputs define progress: every horizon has artifacts, not watch-lists.
90 days: fluency → honest model → shipped batch job. That's a complete practitioner loop.
Eight resources suffice; the constraint is focused hours, not content supply.
Foundations deep, frontier sampled — Ch. 1–11 age slowly; tool names churn.
Year 2 is specialization + reviewing others — the transition from practitioner to leader.

02Beginner Misunderstandings

"I'll start after finishing one more course"

The course queue is infinite by design. Project A requires only Chapters 1–9 of this manual — which you now have.

"Two years is too slow"

Two years of shipped tools at 5 hrs/week beats any bootcamp. You're compounding domain expertise WITH ML — the rare combination.

"I must keep up with every model release"

You must keep your tools working and your foundations sharp. Scout samples the frontier for you.

03Workplace Applications

Fold the 90-day plan into your personal objectives for the next review cycle — make the learning officially visible.
The Year-1 internal talk doubles as the Wave-2 budget pitch.
Reviewing colleagues' AI-assisted work (Year 2) is how the division's quality bar gets set — by you.

04Mini Quiz

1. A week of tutorials, zero artifacts. Per this roadmap, that week:

Counts — learning happenedDidn't count — progress is measured in outputsCounts double if videos were 2×

Course-consumption without artifacts is the plan's one failure mode.

2. "Foundations deep, frontier sampled" means:

Ignore new developments entirelyChase every releaseMaster the stable core; experiment monthly with the new; skip the rest guilt-free

Ch. 1–11 barely age; tool churn is delegated to a weekly digest and one monthly experiment.

3. The Day-90 milestone is:

A scheduled model feeding Power BI, with an operating procedureFinishing three coursesReading all 16 chapters twice

Ship something that runs without you watching it — the complete loop, once.

05Exercises

Block the recurring 5 hours in your calendar now — the plan is a calendar artifact or it's fiction.
Write your Day-30, Day-60, Day-90 outputs as three one-line commitments; pin them where you work.
Enroll in the Ng specialization and open Project A's Colab — today, both under 15 minutes.

06Reflection

Which of the three Year-2 specializations (forecasting, industrial LLM systems, vision QC) pulls you — and which does GFB need most?
Who at KNPC should you teach first — because teaching is the fastest consolidation of everything in this manual?

07Common Mistakes

Course-collecting (the tutorial treadmill) — the plan's named enemy.
Restarting from zero after a busy month instead of resuming mid-plan.
Learning in private: no shipped tools, no talk, no visibility — compounding forfeited.
Chasing frontier tools before Project A runs on schedule.

08Cheat Sheet

30d

Colab + Ch.2–5 on real data + EDA page

60d

Project A modeled, baseline beaten

90d

Project A shipped + 3σ monitor live

6mo

RAG pilot + text-to-SQL hardened + 1 Kaggle

1yr

3 deployed tools + internal talk

2yr

specialization + reviewer role + Wave 3

09AI Prompts

WEEKLY PLANNER

I'm in [phase] of my ML roadmap with 5 hours this week. Last week I produced: [artifacts]. Currently blocked on: [if any]. Plan this week's 5 hours into 2–3 sessions with a concrete artifact per session, and tell me what to explicitly SKIP this week.

MONTHLY REVIEW

Month review. Planned outputs: [list]. Actual outputs: [list]. Honestly assess: am I on the 90-day track? What's the single highest-leverage correction for next month? Be direct — no encouragement padding.

REFBibliography & Master Resource List

Ng, A. — Machine Learning Specialization, DeepLearning.AI / Coursera.
Géron, A. (2022). Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow, 3rd ed. O'Reilly.
James, Witten, Hastie, Tibshirani — An Introduction to Statistical Learning — statlearning.com (free).
Hyndman & Athanasopoulos — Forecasting: Principles and Practice, 3rd ed. — otexts.com/fpp3 (free).
Huyen, C. (2022). Designing Machine Learning Systems. O'Reilly.
kaggle.com/learn · huggingface.co/learn · scikit-learn.org · docs.claude.com.
Plus: every chapter's bibliography above — the full manual cites ~60 primary sources.

Machine Learning,engineered for the plant.

What Machine Learning Actually Is

1.1The one-sentence definition

1.2AI vs Machine Learning vs Deep Learning

1.3A 70-year history in one timeline

1.4The three families of Machine Learning

⚙ Interactive — Which family is it?

1.5Where ML creates real business value

1.6Where ML fails — the honest section

1.7Ethics, bias, hallucinations & explainability

Python for Machine Learning

2.1Why Python, and how much of it you need

2.2Variables and the four containers

2.3Loops, conditions, functions

2.4Classes — the 5-minute version

2.5Imports, pip, and virtual environments

2.6Where you'll actually work

2.7Files, CSV, Excel, JSON, APIs

The Data Science Ecosystem

3.1The ecosystem as a process flow

3.2The 16-library register

3.3How they connect in one real script

Data — The 80% of Every Project

4.1Three shapes of data

4.2Data quality: the six killers

4.3Preparing features: encoding & scaling

4.4Feature engineering — your superpower

4.5Data leakage — the silent project killer

4.6Splitting data honestly

4.7Pipelines, versioning, labeling

Exploratory Data Analysis

5.1The EDA protocol — same 6 steps, every dataset

5.2Choosing the right chart

5.3Thinking like a data scientist

Statistics for Machine Learning

6.1Spread: variance & standard deviation

6.2Distributions — the shapes data takes

6.3Probability & Bayes in working form

6.4Hypothesis tests & confidence intervals

6.5The math intuitions (no equations required)

Machine Learning Algorithms

7.1The selection flowchart

7.2The algorithm register — supervised

7.3The register — unsupervised & special families

7.4Decision boundaries — seeing how models think

Deep Learning

8.1The neuron and the network

8.2The architecture zoo

8.3Embeddings — the bridge concept

8.4LLMs in three sentences

Model Evaluation

9.1The confusion matrix — read it once, use it forever

9.2Choosing the classification metric

9.3Regression metrics

Model Improvement

10.1The improvement hierarchy (spend effort top-down)

10.2Hyperparameter tuning

10.3Regularization — the overfitting brake

10.4Data augmentation & imbalance handling

10.5Ensembling

10.6The improvement loop, operationalized

Production ML — Models That Keep Working

11.1Deployment patterns — pick the simplest that works

11.2The minimum production stack

11.3Drift — why every model dies

11.4Monitoring & retraining — the operating procedure

11.5MLOps — the name for all of this

Generative AI & LLMs in Practice

12.1Prompt engineering — the six patterns that matter

12.2RAG — your most important architecture

12.3Fine-tuning vs RAG vs prompting

12.4Agents, tools, and MCP

12.5Your on-prem serving stack, mapped

ML at the LPG Filling Plant

13.1The application portfolio

13.2Sequencing — the dependency logic

13.3Fitting your existing landscape

Your First Projects — Three Guided Builds

14.1The universal project skeleton

14.2Project A — Monthly cylinder demand forecast Wave 1 flagship

Machine Learning,
engineered for the plant.