Story in Technology

GitHub Copilot Evals enters public preview

Developers can rerun realistic checks after changing skills, agents or MCP tools.

By
Chris Hahn via ChatGPT
Published
Source date

GitHub launched Copilot Evals in public preview on November 17. It uses the local Vally evaluation engine to create checks, run them with approval, inspect failures and compare behavior before and after customization changes. It is available through VS Code, Copilot CLI and the Copilot app.

Suggested experiment: choose a small set of recurring engineering tasks and rerun the same checks after a prompt or tool change. This makes regressions easier to detect than relying on a single successful answer.

Source: GitHub changelog.

Source published: 2026-11-17T18:00:33Z.

Tags

  • AI evaluations
  • Developer Experience
  • GitHub Copilot

Sources

Links supplied by the author. 540 Daily does not fetch or verify them.