The AI Forge · Advanced · F6
MLOps & LLMOps
CI/CD for AI, observability, drift, evals-in-prod, FinOps, versioning, compliance
- Duration
- 2 days · 9:00–17:00 each day (13h contact, incl. breaks + lunch)
- Participant level
- Advanced: experienced practitioners
- Format
- Instructor-led labs, build-along
- Participants
- 8 included · 12 maximum (flexible group size)
- Prerequisite
- F2 or F3 (or equivalent)
- Ecosystem
- Agnostic: any model provider or stack
- Price
- EUR 7,000 (EUR 250 per extra participant)
Amounts in EUR, excl. VAT.
ai-generatedWhat participants will be able to do
- Design a production ML/LLM deployment pipeline
- Implement model monitoring, drift detection, and alerting
- Build cost and latency optimisation strategies for LLM APIs
- Manage model versioning, rollback, and A/B testing
Tools & resources
What changes after this module
Engineers run AI in production reliably, automated deployment, tracing and evaluation in the live system, drift and cost control, and audit-ready versioning, built on operational patterns that persist as platforms change.
Who should attend
Engineers and data professionals. Advanced: experienced practitioners.
Programme
This agenda is indicative. Content, sequencing, and examples are adapted to your team's context, tools, and objectives.
Before you start
- A laptop with a working development environment and admin rights
- Proficiency in at least one general-purpose language (Python and/or JavaScript/TypeScript)
- A code editor/IDE with an approved AI coding assistant enabled
- An API key for an approved model provider (issued by the company)
- A Git repository and command-line comfort
- Completion of F2 or F3 or equivalent
- An AI service to operationalise + access to a CI/CD runner and a cloud or container environment
| Day 1 | 9:00–10:30 | What changes for AI in production: Why AI ops differs from classic MLOps and app ops: non-determinism, prompt/model versions, eval-in-the-loop Lab: Audit an existing AI service: the one you’ll operationalise across both days: against a production-readiness checklist and rank the gaps. |
| 10:30–10:45 · ☕ Coffee break | ||
| Day 1 | 10:45–12:30 | CI/CD for AI: Automated testing (incl. evals), prompt/config as code, safe rollout, canary and rollback Lab: Build a pipeline that runs evals on every change and blocks a bad prompt from shipping. |
| 12:30–13:30 · 🍽 Lunch break | ||
| Day 1 | 13:30–15:30 | Observability & tracing: Treating traces, tool calls and outcomes as queryable data; the metrics that matter in production Lab: Instrument end-to-end tracing and build a dashboard of quality, latency and cost signals: no inference goes unlogged (the compliance floor, not just the debugging aid). |
| 15:30–15:45 · ☕ Coffee break | ||
| Day 1 | 15:45–17:00 | Lab: observable deployment: Deploy with tracing + evals wired in Lab: Deploy your service with tracing and in-workflow eval probes running. |
| Day 2 | 9:00–10:30 | Evals & drift in production: Online evaluation, sampling, drift detection and closing the feedback loop back into test sets Lab: Add production sampling + drift alerts and feed a flagged case back into the eval set: the drift signal becomes a rollback trigger in your runbook. |
| 10:30–10:45 · ☕ Coffee break | ||
| Day 2 | 10:45–12:30 | FinOps for AI: Cost attribution, budgets and alerts, model routing and caching at the ops layer Lab: Add per-feature cost attribution and a budget alert, then cut cost with routing/caching. |
| 12:30–13:30 · 🍽 Lunch break | ||
| Day 2 | 13:30–15:30 | Versioning & compliance: Versioning prompts, models, data and evals; audit trails; mapping to EU AI Act / NIST AI RMF Lab: Set up versioned prompt/model/eval artefacts and produce an audit-ready change record. |
| 15:30–15:45 · ☕ Coffee break | ||
| Day 2 | 15:45–17:00 | Runbook & wrap: Incident basics, on-call for AI, and the operating rhythm; peer review Lab: Write the service runbook (alerts, rollback, escalation) and present your ops setup. |
Deliverables
- CI/CD pipeline with eval gates
- End-to-end tracing + production dashboard
- Drift detection + feedback loop
- Cost attribution + budget alerts
- Versioned prompt/model/eval artefacts + audit record
- Service runbook
Interested in running this module for your team? Get in touch and we'll tailor the format, dates, and delivery to your context.
Request this trainingYour trainer
Runs LLM systems in production; monitoring, cost control and rollback experience
Discovery Session
This module requires a full day to deliver a meaningful learning experience. We only offer it in its complete format.
Frequent questions
Can modules be taken individually?
Yes. Every module stands alone at a fixed price, and every module counts toward a programme if you continue.
Where does training happen?
At your premises or remote, on your dates, for private cohorts. Open sessions run on a fixed monthly calendar.
Which AI tools do you train on?
Yours. Every module ships in four ecosystem editions and runs its exercises on your real stack.
Who delivers?
Inforca's senior consultants and trainers. Flagged modules and the executive track are delivered at senior-expert level.
Discuss this module in a First Call: fit, dates, and the path around it.
No commitment · our team responds within one business day