mirror of
https://github.com/ThearchyHelios/AgentLab.git
synced 2026-11-20 22:22:00 +00:00
收尾超时一律记时长上限:循环里先查满了次数、再在交结果时超时的,limit 也是 timeout_s
上一个提交在模型自己停了查询时补上了 timeout_s,但循环里先撞上别的上限(查询次数、轮数)、交结果那次又超时的,仍记 先撞上的那个:质询会显示「未能回答(已达查询次数上限):回答超过时长上限…」,自相矛盾;双盲复算会说盲算查满了次数、 反方审稿说「每句最多查询 N 次」——查满了照样能交结果,没交成是因为超时。现在交结果那次超时一律记 timeout_s,和报错 原因一致(运行时长上限将至、时长被收紧过的,双盲复算、反方审稿照旧按 timeout_s 说成运行时长上限将至)。金额被拒的 收尾不变,仍报循环里先撞上的那个(test_verify_truncation 钉着它,原因见那里)。
This commit is contained in:
@@ -1028,9 +1028,10 @@ async def _verify(state: _State, *, spec: ModelSpec, system: str, task: str, sou
|
||||
why = "收尾时超过时长上限,没能交结果" if timed_out else \
|
||||
f"核查模型没有按结构交结果({explain_model_error(failure, model_id)})" if failure is not None else \
|
||||
"核查模型没有按结构交结果"
|
||||
# 收尾超时也是触了时长上限:模型自己停了查询(stopped 为 None)时照实记 timeout_s,和原因对得上(质询的报错说
|
||||
# 时长上限,界面按 limit 标「已达时长上限」)。循环里先撞上的上限照旧优先,同上面金额被拒的收尾
|
||||
return outcome(limit=stopped or ("timeout_s" if timed_out else None), answered=answered, error=why)
|
||||
# 收尾超时:没交成结果是因为时长上限,limit 照实记 timeout_s,和原因对得上——模型自己停了查询(stopped 为 None)
|
||||
# 时以前记成没触顶,质询的报错说时长上限、界面却没有「已达时长上限」;循环里先查满了次数的,记成查询次数上限
|
||||
# 会让双盲复算、反方审稿说错原因(查满了照样能交结果)。上面金额被拒的收尾仍报循环里先撞上的那个
|
||||
return outcome(limit="timeout_s" if timed_out else stopped, answered=answered, error=why)
|
||||
declared = list((resolved.get("properties") or {}) if _is_object(resolved) else {})
|
||||
if declared and not any(k in parsed for k in declared):
|
||||
# 一个要求的字段都没交:不是「查不到」,是没按结构交。不这么分,字段名对不上的回复会被当成什么都没查到
|
||||
|
||||
@@ -111,16 +111,19 @@ async def test_a_final_call_that_runs_out_of_time_records_the_time_limit(monkeyp
|
||||
assert out.limit == "timeout_s"
|
||||
|
||||
|
||||
async def test_a_limit_hit_in_the_loop_is_still_the_one_reported(monkeypatch, shop):
|
||||
"""循环先撞上查询次数上限、收尾又超时:触顶原因照旧报先撞上的那个(同金额被拒的收尾,见 test_verify_truncation)。"""
|
||||
async def test_a_final_call_that_runs_out_of_time_after_the_query_limit_records_the_time_limit(monkeypatch, shop):
|
||||
"""循环先撞上查询次数上限、收尾又超时:没交成结果是因为超时,limit 记 timeout_s,和原因一致——记成查询次数上限的话,
|
||||
质询会显示「未能回答(已达查询次数上限):回答超过时长上限…」,双盲复算会说盲算查满了次数(查满了照样能交结果)。
|
||||
金额被拒的收尾照旧报循环里先撞上的那个(见 test_verify_truncation)。"""
|
||||
name, _ = shop
|
||||
monkeypatch.setattr(verify_agent, "SETTLE_GRACE_S", 0.3)
|
||||
Script(monkeypatch, [q(name, VALID_SUM, "c1"), q(name, "SELECT COUNT(*) AS n FROM orders", "c2")],
|
||||
{**FOUND, "rationale": "x"})
|
||||
slow_final(monkeypatch)
|
||||
out = await run(name, timeout_s=0.5, max_queries=1)
|
||||
assert [qd["alias"] for qd in out.queries] == ["Q1"], "查满了一次"
|
||||
assert out.result is None and out.error == "收尾时超过时长上限,没能交结果"
|
||||
assert out.limit == "max_queries"
|
||||
assert out.limit == "timeout_s"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user