mirror of
https://github.com/ThearchyHelios/AgentLab.git
synced 2026-11-15 00:21:58 +00:00
定时评测的测试不依赖同进程里的数据源个数(同 843be9a5 给 test_eval_runner 的做法):这里的题没有限定数据源,评测按进程里全部的数据源拍目录快照,别的测试留下的数据源超过 app/eval/snapshots.MAX_SOURCES 时整次评测按设计失败(「这次评测用到的数据源超过 20 个」),断言的 partial 就成了 failed;后端全量并行时分到同一个进程就复现。这里不测这个上限,测试里把上限放大
This commit is contained in:
@@ -39,7 +39,12 @@ URL = "https://hooks.example.com/agentlab/eval-7c2a"
|
|||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
@pytest.fixture(autouse=True)
|
||||||
async def engine_up():
|
async def engine_up(monkeypatch):
|
||||||
|
# 这里的题(ASK)没有限定数据源,评测按进程里全部的数据源拍目录快照:同一个测试进程里别的测试留下的数据源超过上限
|
||||||
|
# (app/eval/snapshots.MAX_SOURCES)时,评测按设计报「数据源超过 20 个」整次失败。这里不测这个上限(同 test_eval_runner)
|
||||||
|
from app.eval import snapshots
|
||||||
|
|
||||||
|
monkeypatch.setattr(snapshots, "MAX_SOURCES", 10_000)
|
||||||
await run_manager.setup()
|
await run_manager.setup()
|
||||||
async with SessionLocal() as session:
|
async with SessionLocal() as session:
|
||||||
await session.execute(delete(Schedule))
|
await session.execute(delete(Schedule))
|
||||||
|
|||||||
Reference in New Issue
Block a user