Ai · Intermediate
AI Application Development with Python
Build a retrieval-based question-answering app in Python: model calls, tools, embeddings, RAG, evaluation and guardrails, all runnable offline.
About this course
A language model on its own answers from memory and will confidently invent facts. A useful AI application does more: it retrieves the right source material, grounds the model's answer in it, cites where the answer came from, refuses when it does not know, is measured by an evaluation suite, and is wrapped in guardrails that keep secrets and bad input out. This course builds exactly that, in Python 3.12 on Ubuntu Server 24.04. You start with the anatomy of an LLM application and a provider-agnostic client, then learn to call a model well (messages, system prompts, parameters, streaming, tokens and cost), get structured output and tool calls, prepare and chunk documents, embed them and search by similarity, assemble a retrieval-augmented-generation pipeline with citations, make prompts reliable, build a deterministic evaluation harness, add safety guardrails, and finally package, observe and deploy a small local service responsibly. Because a hosted lab must stay reproducible, keyless and offline, everything you run is backed by a local deterministic stub — a small module you are given that mimics a chat/completions and an embeddings interface with repeatable, rule-based outputs. The real Anthropic Claude Messages API is shown as the worked example throughout, and the code is structured so you swap in a real provider by changing one adapter and a config value. The outputs shown in the lessons are produced by the stub, not by a real model; a real model generates fluent, novel prose, while the stub returns text it extracts by fixed rules so the patterns you learn are real and the results repeat exactly. The final project is a retrieval Q&A application over a provided synthetic document set, with grounding and citations, an evaluation harness with pass/fail cases, and guardrails — all runnable offline against the stub, with a documented adapter to swap in a real provider. This is a learning pathway toward an AI application developer role. It does not promise employment, seniority, salary or any vendor certification; completing it earns an Ultiblob Certificate of Completion.
- Content time
- 16 h 50 min
- Lessons
- 10
- Certificate
- Yes
- on completion
Lesson 1 is free. Enroll in a career path to access its full courses.
Lesson 1 is a free preview — read it without an account.
Outline
Lessons
Lesson 1: What an AI application isFree preview
The anatomy of an application built on a language model, and the provider-agnostic client plus local stub you will use for the whole course.
1 h 30 minLesson 2: Calling a model well
Messages and system prompts, the parameters that matter, streaming, and how to account for tokens, cost and latency.
1 h 40 minLesson 3: Structured output and tools
Make a model return validated JSON, and let it call functions you define, with the round trip and the validation that keep it safe.
1 h 40 minLesson 4: Preparing data for retrieval
Clean documents, split them into right-sized chunks with overlap, and attach the metadata that lets an answer cite its source.
1 h 40 minLesson 5: Embeddings and vector search
Turn text into vectors, measure similarity with cosine, and retrieve the nearest chunks — in memory and persisted to SQLite.
1 h 50 minLesson 6: Retrieval-augmented generation
Ground a model's answer in retrieved context, make it cite its source, and have it abstain instead of fabricating.
1 h 50 minLesson 7: Prompt engineering and templating
Make prompts reliable: a stable system prompt, clear delimiters, versioned templates, and resistance to text that tries to hijack instructions.
1 h 30 minLesson 8: Evaluation and eval harnesses
Build a deterministic eval suite with pass/fail cases that measures answer quality and catches regressions before your users do.
1 h 40 minLesson 9: Guardrails and safety
Validate input, check output, refuse unsafe requests, and keep secrets and personal data out of prompts and logs.
1 h 40 minLesson 10: Packaging, observability and deploying responsibly
Package the app with console scripts, run it as a local service, log tokens/cost/latency, control cost, and swap in a real provider by configuration.
1 h 50 min
Where it leads