Enterprise LLM Evaluation Platform

Evaluate, compare and trace LLM configurations in a secure, regulated environment

→ Business impact: From internal prototype to a reliable, reproducible, production-operable solution.

Business context

The team needed to evaluate, compare and trace LLM configurations reliably before any production use — without a platform, every model choice stayed an opinion.

Constraints

Sensitive data
Enterprise identity mandatory
Auditability
Reproducibility
Security at the release gate
Production operability

Architecture

Anonymised architecture diagram — edge

Architecture diagram — anonymised

Components: Python · FastAPI · Kubernetes · Helm · Argo CD · PostgreSQL · OAuth2/OIDC

Evaluation workflow

The platform runs comparable LLM configurations against evaluation sets, measures quality and cost, and traces every run for review — comparison becomes a reproducible fact, not a debate.

Key architecture decisions

Full delegation of authentication to enterprise identity (OIDC/SSO)
Environment promotion exclusively through GitOps
Run audit and reproducibility as design requirements

My contribution

Architecture and integration of SSO/OIDC authentication
Python / FastAPI backend services
Kubernetes deployment (Helm, Argo CD)
CI/CD pipelines with security controls
Traceability of runs and results

Outcomes

Prototype taken to production
Reproducible GitOps deployments
Enterprise authentication operational
No critical CVE at the release gate

Confidentiality

Context-specific details (volumes, topology, internal names) are not publishable.

Learnings

Reproducibility beats velocity in regulated environments.
Delegating identity simplifies the rest of the security architecture.