Hi OFA-Sys/AIR-Bench maintainers,
EvalScope now supports AIR-Bench-Chat and AIR-Bench-Foundation. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.
This may be useful for users who find this repository while setting up AIR-Bench-Chat and AIR-Bench-Foundation and want a quick path to evaluate a hosted or locally served model.
Example:
evalscope eval \
--model YOUR_MODEL \
--api-url OPENAI_API_COMPAT_URL \
--api-key EMPTY_TOKEN \
--datasets air_bench_chat air_bench_foundation \
--limit 10
Docs:
EvalScope repo: https://github.com/modelscope/evalscope
Some benchmarks may require additional assets, Docker, media files, or judge-model configuration for a full run; the EvalScope guide above includes the benchmark-specific setup notes.
If this is useful for your community, we can follow up with a small README/Docs PR linking to the EvalScope usage guide.
Hi OFA-Sys/AIR-Bench maintainers,
EvalScope now supports AIR-Bench-Chat and AIR-Bench-Foundation. This gives users another way to run the benchmark when they want a standardized evaluation workflow around OpenAI-compatible model endpoints, saved predictions, scoring, and report generation.
This may be useful for users who find this repository while setting up AIR-Bench-Chat and AIR-Bench-Foundation and want a quick path to evaluate a hosted or locally served model.
Example:
evalscope eval \ --model YOUR_MODEL \ --api-url OPENAI_API_COMPAT_URL \ --api-key EMPTY_TOKEN \ --datasets air_bench_chat air_bench_foundation \ --limit 10Docs:
EvalScope repo: https://github.com/modelscope/evalscope
Some benchmarks may require additional assets, Docker, media files, or judge-model configuration for a full run; the EvalScope guide above includes the benchmark-specific setup notes.
If this is useful for your community, we can follow up with a small README/Docs PR linking to the EvalScope usage guide.