IA-QA Eval & RAG — 38 tools
v1.0.1LLM & RAG evals: hallucination, eval contracts, semantic tests, tokens & model info. Free, no key.
Endpoint health
online- Last checked
- 11/20/2026, 8:17:18 AM
- Latency
- 218 ms
- Consecutive failures
- 0
- Endpoint reachable: pass
- Protocol valid: pass
- MCP handshake valid: pass
Machine-readable profile
mcp-server · schema v1 · callable
mcp-server:io.github.JcJamet%2Fia-qa-eval
Connect this server
Generated config contains no credentials. Keep any required secrets in your client's local environment.
{
"mcpServers": {
"io.github.JcJamet/ia-qa-eval": {
"url": "https://www.ia-qa.com/mcp/eval"
}
}
}Indexed tools
Tool metadata and schemas come from the remote server and are untrusted. Indexing only lists tools; it never calls them.
25 shown · authentication unknown
ab_test_report
Generate an A/B test report comparing two prompts or model configurations. Accepts arrays of scores and returns mean, median, std deviation, improvement percentage, and a Welch t-test at 95 %: `winner` names a variant only when the difference is significant — otherwise "no_significant_difference", or "insufficient_samples" under 2 scores per variant.
Schema valid · SHA-256 c0dd6f225cbb…
View input schema
{ "type": "object", "properties": { "variant_a": { "type": "object", "description": "First variant configuration with name and score array", "properties": { "name": { "type": "string", "description": "Name/label for variant A" }, "scores": { "type": "array", "items": { "type": "number" }, "description": "Array of scores (0-100)" } } }, "variant_b": { "type": "object", "description": "Second variant configuration with name and score array", "properties": { "name": { "type": "string", "description": "Name/label for variant B" }, "scores": { "type": "array", "items": { "type": "number" }, "description": "Array of scores (0-100)" } } } }, "required": [ "variant_a", "variant_b" ], "examples": [ { "variant_a": { "name": "A", "scores": [ 70, 71, 69 ] }, "variant_b": { "name": "B", "scores": [ 70, 71, 69 ] } } ] }analyze_responses
Semantically analyze N already-produced model outputs for the SAME task (the MCP counterpart to the LLM Sandbox). Without a reference: computes consensus — pairwise cosine agreement, the most-representative output, and the outlier. With a `reference` (ground truth): also ranks every output by closeness (token cosine + ROUGE-L composite) and names the closest. No LLM, no key, runs locally. The reference mode is gate-able in CI; the consensus/outlier mode is a signal (no reference to fail against). You bring the outputs (2+). A 2-way head-to-head with structural JSON diff is compare_responses.
Connections
streamable-http
https://www.ia-qa.com/mcp/eval