FRESHER

AI Engineering Fundamentals

Outcome: junior AI engineer–ready

You don’t need a math degree to start here. “Do I need to learn ML math first?” is the most common thing that stops people from starting — and for this path, the answer is no. That question belongs to Path 3 (ML Engineering on AWS), which is about training and owning models. This path is about applied AI engineering: calling LLMs well, wiring them into real systems, and shipping things that work in production. If you can write Python and call an API, you can do every project below.

The order matters. Each project below builds a specific, real skill — not a toy demo — and the 12 are sequenced so each one assumes the skills from the ones before it. By the end you’ll have built and deployed a full AI assistant, not just followed a tutorial.

← Back to all paths

Start with the smallest real thing: a CLI that loads a PDF or text file and actually answers questions about it, with streaming output and exact source citations for every answer. This is the project that proves you can wire an LLM into a real tool a person could actually use — everything after this assumes you’re comfortable with that loop.

What you’re building

document-chat sends the entire document to the model as context on every question and requires the model to quote its source for every claim — no retrieval, no vector database, no chunking. That’s deliberate: it’s the simplest version of “an LLM that knows about your data,” and it’s also the version that breaks in an instructive way once the document gets too big (more on that below). Project 3 in this path builds the retrieval-based version that fixes that limit — this project is what makes you feel the limit firsthand first.

The request lifecycle

Every question your CLI answers moves through this same path. The two decision points — “fits in context window?” and “cites real source text?” — are exactly the two places a naive implementation fails silently instead of failing loudly.

Core concepts, three levels deep

1. Tokens

Definition: the sub-word units the model actually counts, prices, and limits on — not the same thing as words or characters. A single word can be one token or several.
In this project: every question you ask against a document sends the entire document as tokens, on every single request — there’s no chunking or caching in this version, so you’ll watch the token count add up directly as you test.
Practical consequence: pin down the real per-request cost using your provider’s pricing before you ship. For this project, document size — not code complexity — is the dominant cost driver.

2. Context window

Definition: the fixed per-request token budget covering system prompt + document + question + answer, combined. It is not persistent memory — it resets on every call.
In this project: this is your CLI’s core constraint. A document only “works” if document tokens + prompt scaffolding + expected answer tokens fits inside the model’s context window, full stop.
Practical consequence: count tokens and check them against your chosen model’s limit before calling the API, and fail with a clear message. A silent truncation or a raw provider error is a worse user experience than an honest “this document is too large for this tool” — and it’s exactly the gap Project 3’s retrieval pipeline exists to close.

3. Streaming generation

Definition: the model produces output by repeatedly sampling one token at a time from a probability distribution over its vocabulary. There is no “generate the whole answer, then send it” step happening behind the scenes — streaming just exposes that one-token-at-a-time process to your CLI instead of buffering it.
In this project: streaming is what makes a multi-paragraph answer feel instant instead of a silent multi-second wait. You consume the provider’s stream and print tokens to the terminal as they arrive.
Practical consequence: your error handling has to account for partial output — what does your CLI do if the connection drops halfway through an answer? Decide that behavior and test it; it’s easy to only ever exercise the happy path.

4. Hallucination and citation grounding

Definition: the model can state a fabricated fact with exactly the same fluent confidence as a true one, because next-token prediction has no built-in truth-check step.
In this project: this is why “exact source citations for every answer” is a hard requirement, not a nice-to-have. A citation is a claim the user can verify (“this is from page 4, paragraph 2”) — an answer without one is a claim they just have to trust.
Practical consequence: design the prompt so the model is instructed to quote directly from the supplied document, then validate in code that the quoted text actually appears in the source before you show it to the user. An unverified citation is barely better than no citation — the model can hallucinate a citation exactly as easily as it hallucinates a fact.

Decision rules

If…	Then…
The document comfortably fits well under the model’s context window	Stuff the whole document into context — this project’s approach is the right one; don’t reach for retrieval you don’t need yet
The document exceeds the context window, or users need to search across many documents	You need retrieval (RAG) — that’s Project 3, not this one
The answer needs to be traceable to an exact quote	Always validate the citation against the source text in code — never trust the model’s claim about its own citation
Users will ask multi-turn follow-up questions	Track conversation history as tokens too — it counts against the same context-window budget as the document

Common mistakes

Truncating the document silently instead of surfacing a clear “this document is too large” error before the API call fails confusingly on its own.
Rendering a citation before validating it — you can’t check a quote against the source text until you have the complete sentence, so validate after the stream finishes, not before.
Trusting a model-stated citation (“this is from page 4”) without checking it — the model can hallucinate a citation exactly like it hallucinates a fact; the check has to happen in your code, not in the prompt alone.

Key concepts at a glance

Concept	One-line definition	Why it matters for `document-chat`
Token	The unit the model counts, prices, and limits on	Determines your real per-request cost
Context window	Fixed per-request budget for prompt + document + answer	The hard limit this project’s whole design works within
Streaming	Model output produced one token at a time	Why the CLI can show partial output instantly
Hallucination	Fluent but potentially false output, with no built-in truth-check	Why every citation must be verified in code, not just requested in the prompt

Go deeper: How LLMs Actually Work — the full concept walkthrough (tokens, embeddings, attention, temperature, hallucination) →

The baseline for any AI Engineer / LLM Application Developer role — proves you can take an LLM call from idea to a working CLI tool, not just a notebook experiment.

Clone this project, run it locally, and compare your output against the description above.

Document Chat on GitHub ↗

Self-check only — no score is recorded anywhere. Answer each question yourself, then click to reveal.

1. Your CLI throws a "context length exceeded" error when a user points it at a 300-page manual. What's the most accurate explanation?
- a) The API key doesn't have permission to read PDFs
- b) The combined tokens of the document, prompt scaffolding, and expected answer exceed the model's context window
- c) The tokenizer failed to parse a special character in the file
- d) The temperature parameter is set too high
Reveal answer

B. The context window is a fixed per-request token budget covering everything you send plus everything you expect back. A 300-page manual is a document-size problem, not an auth error or a sampling-parameter problem — this is exactly the wall Project 3 (the RAG pipeline) exists to get around.
2. Which change actually reduces the per-request token cost of every follow-up question a user asks against the same document?
- a) Increasing max_tokens on the response
- b) Raising the temperature
- c) Keeping the document in one ongoing context (or using provider-side prompt caching) instead of resending the full file with every new question
- d) Switching to a larger, more capable model
Reveal answer

C. Every token you resend — including the whole document, again — is billed again. max_tokens caps output length but doesn't reduce input cost, temperature only changes sampling randomness, and a bigger model makes the same resend pattern more expensive, not less.
3. In 1-2 sentences: why isn't "the model cited page 4" proof, by itself, that the citation is accurate?

Reveal answer

Model answer: the model generates a citation the same way it generates any other text — by predicting the next most likely token, with no built-in fact-check step — so it can hallucinate a page number or a quote just as confidently as it hallucinates a fact. A citation is only trustworthy once you've verified the quoted text actually appears in the source document.
4. Why does document-chat stream its answer token-by-token instead of waiting for the full response?
- a) Streaming is a hard requirement of every LLM provider's API
- b) It reduces the total token cost of the request
- c) It makes a multi-second generation feel responsive instead of a blank wait, since the model is producing one token at a time regardless
- d) It lets the model revise earlier tokens if it detects a mistake
Reveal answer

C. The model always generates one token at a time — streaming just means you show each token to the user as it arrives instead of buffering the whole answer first. It has no effect on token cost, and models can't revise tokens they've already emitted.
5. The connection drops halfway through a streamed answer. Your CLI has already printed a partial sentence that starts a quote from the document. What should it do, and why does "just leave what's on screen" carry a specific risk here?

Reveal answer

Model answer: it should clearly mark the answer as incomplete and either retry or tell the user to re-ask, not leave the partial text looking like a finished answer. The specific risk is the citation: a cut-off quote can't be checked against the source document, so an unfinished citation left on screen looks verified when it isn't — worse than showing nothing.
6. A user asks a follow-up in the same session: "what did it say about refunds two questions ago?" What does answering that correctly cost you, token-wise?
- a) Nothing — follow-up questions are free once the document is loaded
- b) The full conversation history so far (document plus every prior question and answer) counts against the same context-window budget on the new request
- c) Only the new question counts; earlier turns are cached for free by the provider
- d) It only matters after 10 turns in the same session
Reveal answer

B. There's no free persistent memory — if the model needs to see prior turns to answer a follow-up, those turns have to be sent again as part of the current request's context, which is the same budget the document itself is competing for.

Do these three tasks before and while you build — they force you to design against real token numbers and a real failure mode instead of discovering both by accident in front of a user. This is separate from the "What to do" tab's clone-and-run pointer: that gets you the reference implementation running; this is what to build and verify yourself.

1. Token & cost budget — before you write code
30 min

Pick a real file you'd actually want to query (a spec, a contract, a long doc from your own work). Count its tokens with a tokenizer — tiktokenizer.vercel.app or platform.openai.com/tokenizer (cl100k_base encoding is close enough for planning purposes). Then calculate: if this document is sent in full on every question, and you expect roughly 50 questions/day against it, what's the daily input-token cost at your chosen model's per-1M-token input price? Write down the token count, the daily cost estimate, and one sentence on whether that number changes how you'd design the tool (for example: keep the document in one ongoing conversation instead of resending it from scratch on every call).
2. Ship the citation-grounded Q&A loop
90-120 min

Build the CLI: load a file, assemble a prompt that requires the model to quote directly from the source document for every claim, stream the response token-by-token, and print the citation alongside the answer. Add the context-window guard from the Overview — check the combined token count against your model's limit before calling the API, and fail with a clear error instead of silently truncating the document. Test with a question whose answer is genuinely not in the document, and confirm your prompt makes the model say so instead of inventing an answer.
3. Stress-test the context limit
30-45 min

Feed the tool a document that's deliberately too large for the context window (concatenate a few chapters together, or find a genuinely long document). Observe: does it fail the way your Task 2 guard predicted, or does something else happen — silent truncation, a raw provider error, a hang? Write two sentences on what would have to change architecturally to handle this document instead of rejecting it. You don't have to build that part — this is exactly what Project 3's RAG pipeline does.

Deliverable

A short write-up (a README section or a notes file in your project repo) with: Task 1's token count and cost estimate, a link to or description of your working CLI from Task 2, and Task 3's observation plus your two-sentence design note.

What's next

If you want to apply what you just built inside a real engineering job right away, Path 2 takes the same project-based approach and points it at workflows you already do every day, like code review and PR summaries. If your goal is owning full ML systems instead, training, serving, and monitoring, Path 3 covers that ground directly. Neither path requires you finish here first. AI-Augmented Engineering →