testing
Run the same checks across every client
Your server behaves differently in ChatGPT than it does in Claude or Cursor. Test suites exercise every tool across clients and models, with model-judged pass/fail per combination.
Our open source tools are used by developers at top companies
How it runs
From a flaky tool call to a green pipeline
- 1
Describe
Write a suite in plain language.
- 2
Fan out
Run it across clients and models.
- 3
Judge
Model-graded pass/fail per combination.
- 4
Gate
Block the merge on any regression.
Every client, every run
ChatGPT, Claude, Cursor, VS Code and custom clients exercised from one suite definition.
San Francisco, CA
72°F
Partly cloudy · H 74° L 58°
→ tools/call
{
"name": "display_weather",
"arguments": { "city": "San Francisco" }
}← result20041ms
{ "content": [{ "type": "resource" }] }GPT-5.2
18/18
Claude
18/18
Gemini
17/18
Model swap
Hold the suite constant and vary the model to see exactly which combinations drift.
➜ npx create-mcpfy-app
✓ tools/chart-sales.ts
✓ widgets/Chart.tsx
✓ mcpfy.config.ts
➜ mcpfy dev
ready on :4141 · 6 tools registered
tools/chart-sales.ts
export const chartSales = tool({
name: "chart_sales",
input: z.object({ period: z.string() }),
});CI gating
Suites run on every pull request and fail the check when any client regresses.
- mainProduction12s
- feat/chartsPreview9s
- fix/authPreview11s
➜ npx mcpfy deploy
✓ live → my-server.mcpfy.ai/mcp
In the box
Built for how MCP actually breaks
Schema drift
Catch tool signatures that changed shape.
Client quirks
Per-client conformance, not a single mock.
Latency budgets
Fail on slow tools before users feel it.
Deterministic reruns
Replay any failed case exactly.
Stop shipping regressions to one client at a time
Explore the rest of the platform
Everything on mcpfy Cloud shares one pipeline.