promptfoo
Open SourceObservability & Evals
Test and evaluate prompts like code.
What it’s for
promptfoo runs your prompts across a matrix of models and test cases with assertions on the output — then fails your CI when something regresses. The quickest way to make prompt and model changes safe instead of a leap of faith.
How to use it
npm install -g promptfoo, describe providers, prompts and assertions in YAML, and run promptfoo eval. Inspect results in the local web UI or wire it into GitHub Actions as a CI gate.
Pricing
Free and open source (MIT).