合入期 4 期 3 收尾:核查缺口分「核查未通过」「核查未完成」两组,时间线认得核查类日志代码;双盲复算写目录的预告和真写逐字一致(被目录挡下的那一边也说清原因,几个指标落到同一项时按组合预告);反方审稿说清「根本没反查」和「到底降不降档」;字段叫法挪到 labels.py;记录页的核查工件有名字和说明

This commit is contained in:
ThearchyHelios
2026-10-06 11:31:11 +08:00
32 changed files with 3213 additions and 660 deletions
+131 -36
View File
@@ -36,10 +36,11 @@ is_test = 0、is_valid = 1、deleted_at IS NULL);只涉及一张表。拿不
也不写。
**预告。** 审批卡上要让审核人看得出选了之后会往目录里写什么(preview):按发起审批时的目录,选原流程、选盲算各会
写什么、不写的为什么。和真写是同一套判断(derive 认事实、_plan 对目录、_revives 认驳回),所以只在两处可能和最后写
的不一样:审批停下期间目录被人改了;同一次裁决里几个指标落到同一项上(上一段)——写进去的是合起来的说法,预告说会
写的也可能因为说法互相矛盾、和已有条件的关系拿不准、合起来又成了驳回过的说法而不写。反过来不会:预告说不写的,
不会因为和别的指标并成一项被写进去。
写什么、不写的为什么。认出了事实、只是被目录挡下(已确认、驳回过、和已有条件合不起来)的那一边也带上写到哪张表的
哪一项、本来要写什么,卡上才认得出它和别的指标是不是落到同一项上。同一次裁决里几个指标落到同一项上时,写进去的是
合起来的说法(上一段),单个指标的预告说不准:preview_joint 把这几个指标各自选哪一边的每种组合都按真写的办法算一遍
(_merged 归拢、_screen 拿掉驳回的、_plan 对目录),卡上按各指标当前的选择取对应的那一种。和真写是同一套判断、同一句
原因,只有审批停下期间目录被人改了时才会和最后写的不一样。组合太多的(JOINT_LIMIT)那一项不预告,卡上有条件地说。
catalog.save_patch 不收备注,这里用它的同一套原料(read_entry、apply_patch、write_entry 的乐观锁)写,只多把备注放进项里。
"""
@@ -73,6 +74,10 @@ ASPECT_LABEL = {
_VERB = {"count": "计数", "sum": "求和", "avg": "求平均"}
#: 写入时版本对不上(别人刚改过这张表)最多重试几次
_RETRIES = 2
#: 同一次裁决里几个指标落到同一项上时,这一项最多预告多少种组合(preview_joint,每一项各算各的,不共用额度)。
#: 每个指标两边都落在这一项上时近于 3 的指标数次方:四个指标 72 种还在里面,再多的那一项不预告,审批卡回到按其余指标
#: 当前的选择有条件地说。组合数先算出来再比(_combos),不把组合列出来比:二十个指标就是几十亿种
JOINT_LIMIT = 128
#: 无效记录的取值:被排除的取值(或码值表里它的意思)里有这些词,才算在排除无效记录
_INVALID_WORDS = frozenset({
@@ -94,7 +99,7 @@ class Fact:
"""要写进目录的一项:哪个数据源的哪张表、哪一项、写什么。metric、choice 只用来写备注和说明;sql、dialect 是被采纳
那一边的写法和方言,有效记录条件和目录里已有的条件合并时要看它还按哪些条件筛了。同一次裁决里几个指标落到同一项上
时并成一项:members 是并进来的每个指标各自的那一项(各带自己的说法,审批卡上各自的预告就是按它算的),value 是
合起来之后的说法;单个指标的 members 为空。"""
合起来之后的说法;单个指标的 members 为空。metric_id 是指标 ID:几个指标并成一项时,预告按它说清每个指标的结果。"""
source: str
table: str
@@ -105,6 +110,7 @@ class Fact:
sql: str = ""
dialect: str | None = None
members: tuple[Fact, ...] = ()
metric_id: str = ""
def _members(fact: Fact) -> tuple[Fact, ...]:
@@ -160,7 +166,8 @@ def derive(candidate: Mapping[str, Any], schema_cache: Mapping[str, Any] | None,
dialect = candidate.get("dialect")
return Fact(source=str(candidate.get("source") or ""), table=resolved, item=key, value=value, metric=name,
choice=str(side), sql=str(sql.get(side) or ""),
dialect=dialect if isinstance(dialect, str) else None), None
dialect=dialect if isinstance(dialect, str) else None,
metric_id=str(candidate.get("metric_id") or "")), None
def _business_date(item: Mapping[str, Any], side: str) -> tuple[tuple[str, str, Any] | None, str | None]:
@@ -631,33 +638,35 @@ def _rejected_text(fact: Fact, rejected: Any, members: Sequence[Fact]) -> str:
"的选择相反,未写入。如需更改,请到数据目录中修改")
def _screen(notes: Mapping[str, Any], fact: Fact) -> tuple[Fact | None, list[str]]:
"""几个指标并成的一项,先拿掉自己采纳的说法曾被人工驳回的指标:(剩下的重新并成的一项, 拿掉的为什么不写),一个
都不剩时前者为 None。单个指标的、这一项没被驳回过的原样返回,由 _plan 照常判断。
def _screen(notes: Mapping[str, Any], fact: Fact) -> tuple[Fact | None, list[Fact], list[str]]:
"""几个指标并成的一项,先拿掉自己采纳的说法曾被人工驳回的指标:(剩下的重新并成的一项, 拿掉的指标, 没写的原因),
一个都不剩、或者剩下的又合不起来时前者为 None;原因先是拿掉的那几个的(一条),剩下的合不起来时再跟一条。单个指标
的、这一项没被驳回过的原样返回,由 _plan 照常判断。
审批卡上每个指标的预告是按它自己的说法单独算的(preview → _plan),在那里被驳回检查拦下、卡上写着「不写」的,
并进来以后也得拦下。只拿合起来的整句去比,「status <> 'refunded'」和别的条件 AND 在一起就不再等于驳回的那句,
会被写成人工确认,和卡上说的正相反。拿掉的指标照实说原因(和它的预告同一句话),其余的照写,也和各自的预告一致。"""
rejected, said = _rejected(notes, fact.item)
if not rejected or not fact.members:
return fact, []
return fact, [], []
out: list[Fact] = []
kept: list[Fact] = []
for member in fact.members:
(out if _revives(member, said) else kept).append(member)
if not out:
return fact, []
return fact, [], []
reasons = [_rejected_text(fact, said, out)]
if not kept:
return None, reasons
return None, out, reasons
narrowed, why = _merged(kept)
return narrowed, reasons if why is None else [*reasons, why]
return narrowed, out, reasons if why is None else [*reasons, why]
async def record_choices(candidates: list[Mapping[str, Any]], *, run_id: str, node_title: str,
actor: str | None) -> dict[str, list[dict[str, Any]]]:
"""把审核人的选择写进数据目录:{written: [{source, table, item}], skipped: [{reason}]}。永远不抛:数据源没了、
目录写不进去,都落在 skipped 里照实说。
"""把审核人的选择写进数据目录:{written: [{source, table, item, value}], skipped: [{reason}]}。value 是真写进去的
说法(和已有条件合起来的就是合起来的整句),审批卡上的预告说「会记为」的就是它。永远不抛:数据源没了、目录写不
进去,都落在 skipped 里照实说。
先把每个候选认成目录项,再按(数据源、表、项)归拢成一项(_merged:说法一样的、有效记录条件合起来写,说法
不同的整组不写),逐项写(_write:按读到的目录先拿掉各自带回驳回说法的指标,其余的照写)。"""
@@ -690,24 +699,25 @@ async def record_choices(candidates: list[Mapping[str, Any]], *, run_id: str, no
if fact is None:
skipped.append({"reason": why})
continue
done, reasons = await _record(fact, run_id=run_id, node_title=node_title, actor=actor)
if done:
written.append({"source": fact.source, "table": fact.table, "item": fact.item})
value, reasons = await _record(fact, run_id=run_id, node_title=node_title, actor=actor)
if value is not None:
written.append({"source": fact.source, "table": fact.table, "item": fact.item, "value": value})
skipped += [{"reason": r} for r in reasons]
return {"written": written, "skipped": skipped}
async def _record(fact: Fact, *, run_id: str, node_title: str, actor: str | None) -> tuple[bool, list[str]]:
"""写一项:(写了没有, 没写的原因)。几个指标并成的一项可能写了、同时有几个指标被拿出来没写,原因照样交回。"""
async def _record(fact: Fact, *, run_id: str, node_title: str, actor: str | None) -> tuple[Any, list[str]]:
"""写一项:(写进去的说法,没写为 None, 没写的原因)。几个指标并成的一项可能写了、同时有几个指标被拿出来没写,原因
照样交回。"""
try:
async with SessionLocal() as session:
row = (await session.execute(select(DataSource).where(DataSource.name == fact.source))).scalar_one_or_none()
if row is None:
return False, [f"数据源「{fact.source}」已不存在,指标{_names(_members(fact))}的选择未写入数据目录"]
return None, [f"数据源「{fact.source}」已不存在,指标{_names(_members(fact))}的选择未写入数据目录"]
return await _write(session, row, fact, run_id=run_id, node_title=node_title, actor=actor)
except Exception: # noqa: BLE001 - 目录写不进去不能毁掉审批的结果:缺口照样按选择记
logger.exception("双盲复算的选择写入数据目录失败(数据源 %s)", fact.source)
return False, [f"指标{_names(_members(fact))}的选择写入数据目录时出错,未写入"]
return None, [f"指标{_names(_members(fact))}的选择写入数据目录时出错,未写入"]
def _plan(notes: Mapping[str, Any], fact: Fact, tables: Mapping[str, Any] | None) -> tuple[Any, str | None]:
@@ -767,7 +777,9 @@ def preview(candidate: Mapping[str, Any], schema_cache: Mapping[str, Any] | None
candidate 同 derive(choice 不用给);notes 是这个数据源的目录({表名: notes})。返回 {a: …, b: …},每边是
{write: true, source, table, item, value}(value 是真要写进去的说法:和目录里已有的有效记录条件合起来的,是合起来
之后的整句)或 {write: false, reason}。和恢复后真写是同一套判断,只在模块说明里说的两处可能不一样。永远不抛。"""
之后的整句)或 {write: false, reason};认出了事实、只是被目录挡下的不写,照样带上 source、table、item 和本来要写
的 value(这个指标自己的说法)——卡上据此认出几个指标落到同一项上。这个指标单独落在这一项上时,reason 和真写时
跳过的原因一字不差;和别的指标落到同一项上时看 preview_joint。永远不抛。"""
notes = notes or {}
tables = schema_cache.get("tables") if isinstance(schema_cache, Mapping) else None
out: dict[str, dict[str, Any]] = {}
@@ -780,15 +792,98 @@ def preview(candidate: Mapping[str, Any], schema_cache: Mapping[str, Any] | None
except Exception: # noqa: BLE001 - 预告算不出来不能拦住审批:照实说,以写入后的结果为准
logger.exception("双盲复算预告数据目录的写入失败(指标 %s)", candidate.get("metric"))
fact, why = None, f"指标「{candidate.get('metric')}」的选择核对数据目录时出错,无法预告是否写入"
out[side] = {"write": True, "source": fact.source, "table": fact.table, "item": fact.item, "value": value} \
if fact is not None and why is None else {"write": False, "reason": why}
if fact is None:
out[side] = {"write": False, "reason": why}
continue
where = {"source": fact.source, "table": fact.table, "item": fact.item}
out[side] = {"write": True, **where, "value": value} if why is None else \
{"write": False, **where, "value": fact.value, "reason": why}
return out
def _joint_outcome(notes: Mapping[str, Any], members: list[Fact], tables: Mapping[str, Any] | None
) -> dict[str, dict[str, Any]]:
"""几个指标落到同一项上、各按自己的选择并成一项时,每个指标的结果:{指标 ID: {write: true, value} 或 {write: false,
reason}}。和 record_choices 逐步一样:_merged 归拢(说法不同的整组不写),_screen 拿掉带回驳回说法的(它们说驳回
那一句,剩下的合不起来时说合不起来的那一句),剩下的按 _plan 对目录(写的是同一个值,不写的是同一句原因)。"""
fact, why = _merged(members)
if fact is None:
return {m.metric_id: {"write": False, "reason": why} for m in members}
kept, out, reasons = _screen(notes, fact)
result = {m.metric_id: {"write": False, "reason": reasons[0]} for m in out}
rest = [m for m in members if m.metric_id not in result]
if kept is None:
result.update({m.metric_id: {"write": False, "reason": reasons[-1]} for m in rest})
return result
value, why = _plan(notes, kept, tables)
result.update({m.metric_id: {"write": False, "reason": why} if why is not None else {"write": True, "value": value}
for m in rest})
return result
def _combos(sides: Sequence[int], limit: int) -> int:
"""几个指标落到同一项上时有几种组合(至少两个指标落上来的):每个指标要么没落上来、要么落上来的那几边之一,
∏(1 + k) − 1 − Σk(k 是这个指标落到这一项上的边数,减去一个都没有、只有一个的)。连乘到超过 limit 能确定的地方
就停,返回 limit + 1:只用来和上限比,不需要真算出几十亿。"""
total, cap = 1, limit + 1 + sum(sides)
for k in sides:
total *= 1 + k
if total > cap:
return limit + 1
return total - 1 - sum(sides)
def preview_joint(candidates: Sequence[Mapping[str, Any]],
catalogs: Mapping[str, tuple[Mapping[str, Any] | None, Mapping[str, Mapping[str, Any]]]]
) -> list[dict[str, Any]]:
"""同一次裁决里几个指标落到同一项上时的预告:[{source, table, item, members: [{metric_id, choice}], outcomes:
{指标 ID: {write: true, value} 或 {write: false, reason}}}],每条是一种组合(members 是按各自的选择落到这一项上的
指标,照候选的先后,至少两个)。
candidates 照节点交给真写的先后(不一致的指标的先后),带 metric_id,不带 choice;catalogs 是 {数据源名: (表结构,
{表名: notes})},同 preview 的两个参数。每个指标的两边先各按 derive 认出落到哪一项(同 preview),落到同一项上的
几个指标,每种「谁选了哪一边」的组合按 _joint_outcome 算一遍:真写时这几个指标就是这样被归拢、写或不写的。
上限按每一项算:一项的组合数(_combos,先算个数,不先列出来)超过 JOINT_LIMIT 的,这一项整项不预告,别的项照常
预告,几项之间不共用额度。一项不超过 JOINT_LIMIT 种、每个指标最多落到两项上,总的计算量和指标数成正比。
永远不抛:算不出来的那一项不预告。"""
from itertools import product
landed: dict[tuple[str, str, str], dict[int, list[tuple[str, Fact]]]] = {}
for index, candidate in enumerate(candidates):
schema, notes = catalogs.get(str(candidate.get("source") or ""), (None, {}))
for side in SIDE_LABEL:
try:
fact, _ = derive({**candidate, "choice": side}, schema, notes)
except Exception: # noqa: BLE001 - 认不出就当这一边落不到任何一项上,同 preview 照实不说
fact = None
if fact is not None:
landed.setdefault((fact.source, fact.table, fact.item), {}).setdefault(index, []).append((side, fact))
out: list[dict[str, Any]] = []
for (source, table, item), by_metric in landed.items():
if len(by_metric) < 2 or _combos([len(sides) for sides in by_metric.values()], JOINT_LIMIT) > JOINT_LIMIT:
continue
options = [[None, *sides] for _, sides in sorted(by_metric.items())]
combos = [[c for c in combo if c is not None] for combo in product(*options)]
combos = [c for c in combos if len(c) >= 2]
schema, notes = catalogs.get(source, (None, {}))
tables = schema.get("tables") if isinstance(schema, Mapping) else None
try:
entries = [{"source": source, "table": table, "item": item,
"members": [{"metric_id": fact.metric_id, "choice": side} for side, fact in combo],
"outcomes": _joint_outcome((notes or {}).get(table) or {}, [fact for _, fact in combo], tables)}
for combo in combos]
except Exception: # noqa: BLE001 - 预告算不出来不能拦住审批:这一项不预告,卡上有条件地说
logger.exception("双盲复算预告几个指标落到同一项上的写入失败(%s 的 %s)", table, item)
continue
out.extend(entries)
return out
async def _write(session: Any, row: DataSource, fact: Fact, *, run_id: str, node_title: str,
actor: str | None) -> tuple[bool, list[str]]:
"""写一项:(写了没有, 没写的原因)。几个指标并成的一项先按读到的目录拿掉各自带回驳回说法的指标(_screen),写什么、
写不写再由 _plan 定;备注只列真写进去的指标。版本对不上(别人刚改过)重读再来。"""
actor: str | None) -> tuple[Any, list[str]]:
"""写一项:(写进去的说法,没写为 None, 没写的原因)。几个指标并成的一项先按读到的目录拿掉各自带回驳回说法的指标
(_screen),写什么、写不写再由 _plan 定;备注只列真写进去的指标。版本对不上(别人刚改过)重读再来。"""
label = catalog.TABLE_FIELD_LABEL.get(fact.item, fact.item)
tables = (row.schema_cache or {}).get("tables") if isinstance(row.schema_cache, dict) else None
reasons: list[str] = []
@@ -796,19 +891,19 @@ async def _write(session: Any, row: DataSource, fact: Fact, *, run_id: str, node
entry = await catalog.read_entry(session, row.id, fact.table)
notes = entry.notes if entry else {}
version = entry.version if entry else 0
kept, reasons = _screen(notes, fact)
kept, _, reasons = _screen(notes, fact)
if kept is None:
return False, reasons
return None, reasons
value, why = _plan(notes, kept, tables)
if why is not None:
return False, [*reasons, why]
return None, [*reasons, why]
note = note_of(kept, run_id=run_id, node_title=node_title)
at = catalog.now_iso()
try:
merged = catalog.apply_patch(notes, [{"path": kept.item, "value": value, "reason": note}],
table=kept.table, tables=tables, at=at)
except catalog.CatalogInvalid as e:
return False, [*reasons, f"「{kept.table}」的{label}未写入数据目录:{e}"]
return None, [*reasons, f"「{kept.table}」的{label}未写入数据目录:{e}"]
# 不论原来是谁提的:这一项现在出自人的选择,记为人工填写、已确认,备注写明出处
merged[kept.item] = catalog.make_item(value, "human", "confirmed", note=note, at=at)
try:
@@ -817,9 +912,9 @@ async def _write(session: Any, row: DataSource, fact: Fact, *, run_id: str, node
await session.rollback()
continue
except catalog.CatalogInvalid as e:
return False, [*reasons, f"「{kept.table}」的{label}未写入数据目录:{e}"]
return True, reasons
return False, [*reasons, f"「{fact.table}」的数据目录正被修改,{label}未能写入,请稍后在数据目录中确认"]
return None, [*reasons, f"「{kept.table}」的{label}未写入数据目录:{e}"]
return value, reasons
return None, [*reasons, f"「{fact.table}」的数据目录正被修改,{label}未能写入,请稍后在数据目录中确认"]
__all__ = ["FACT_ITEMS", "Fact", "derive", "note_of", "preview", "record_choices"]
__all__ = ["FACT_ITEMS", "JOINT_LIMIT", "Fact", "derive", "note_of", "preview", "preview_joint", "record_choices"]
+22
View File
@@ -66,6 +66,24 @@ FIELD_LABEL: dict[str, str] = {
"defaults": "工作流默认设置",
}
#: 双盲复算节点各配置项的叫法(lib/terms.ts 的 DUAL_CHECK_TEXT.field,check-verify 逐项比对)。模型两项照结论句裁判
#: 「裁判模型 · 接入」「裁判模型」的写法
DUAL_CHECK_FIELD_LABEL: dict[str, str] = {
"card": "口径卡", "metrics": "复算的指标", "definitions": "口径说明", "provider": "盲算模型 · 接入",
"model": "盲算模型", "tolerance": "允许的相对误差", "on_reject": "未采用原流程时", "enabled": "启用双盲复算",
}
#: 「未采用原流程时」两个取值的短说法:检查器下拉的选项以它开头(DUAL_CHECK_TEXT.onReject),后面可以带一句用途
DUAL_CHECK_ON_REJECT_LABEL: dict[str, str] = {"degrade": "计入缺口、出具降档", "fail": "判为失败"}
#: 反方审稿节点各配置项的叫法(lib/terms.ts 的 CHALLENGE_TEXT.field,check-challenge 逐项比对)
CHALLENGE_LABEL: dict[str, str] = {
"report": "要审的报告", "max_claims": "最多审几句", "max_queries": "每句最多查询次数",
"timeout_s": "每句时长上限(秒)", "provider": "反方模型 · 接入", "model": "反方模型",
"on_refuted": "结论被推翻时", "enabled": "启用",
}
#: 「结论被推翻时」三个取值的叫法:和检查器下拉(CHALLENGE_TEXT.onRefuted)一致,只去掉「(默认)」。报错让人去改
#: 某一项时,人在下拉里找得到这个名字
CHALLENGE_ON_REFUTED_LABEL: dict[str, str] = {"degrade": "出具降档", "flag": "只标注", "fail": "报错并停止运行"}
_FIELD_BY_TYPE: dict[str, dict[str, str]] = {
"input": {"fields": "输入字段"},
"agent": {"prompt": "任务"},
@@ -81,6 +99,8 @@ _FIELD_BY_TYPE: dict[str, dict[str, str]] = {
"stop_on_reject": "驳回即终止运行"},
"validate": {"source": "待校验内容", "fail_fast": "校验失败即中断"},
"merge": {"inputs": "输入", "sql": "合并 SQL"},
"dual_check": DUAL_CHECK_FIELD_LABEL,
"challenge": CHALLENGE_LABEL,
}
#: 出具契约里的键(canvas/issues.ts 的 CONTRACT_KEYS、SUB_KEYS)
@@ -114,6 +134,8 @@ _OPTION_LABEL: dict[str, dict[str, str]] = {
"incomparable": "标注与历史不可比"},
"isolation": {"strict": "严格:microVM", "fast": "快速:系统沙箱"},
"thinking": {"summarized": "开启并显示摘要", "adaptive": "开启但不显示", "off": "关闭"},
"on_reject": DUAL_CHECK_ON_REJECT_LABEL,
"on_refuted": CHALLENGE_ON_REFUTED_LABEL,
}
#: 让人在几个取值之间选时的完整选项文字(发布前修复的候选),和检查器下拉一致,只去掉「(默认)」
CHOICE_LABEL: dict[str, dict[str, str]] = {
+43 -14
View File
@@ -22,8 +22,11 @@
没审全(这两个代码还不在期 3 约定的日志代码表里,界面认不得时照原话显示)
5. 处置:被推翻的按 on_refuted 处置。degrade(默认)在产出里写 issuance_gaps,成果节点的出具契约照单收进缺口、
出具降档(nodes/io.py 的 _upstream_issuance_gaps,所以反方审稿要接在成果节点上游,画布校验会提醒);fail 判节点
失败;flag 只记录。产出和审批卡里的 on_refuted 是实际生效的那一种(只有 degrade / flag,见 _finish)。审批卡
(human.py 的 _reviewed_reports)附上审稿概况
失败;flag 只记录。产出和审批卡里的 on_refuted 是实际生效的那一种(只有 degrade / flag,见 _finish);degrades
说被推翻的到底会不会让出具降档(按降档处置、又接在带出具契约的成果节点上游),unreviewed 是未能反查的句数(核查
模型不可用、触顶没给出结论、数据源不在了、报告没有引用数据库查询……这几句记为无法判断,但反查没有完成,触顶的
可能已经查过几次)。每一句都记为无法判断,所以 unreviewed 不会大于 counts 里的 inconclusive。审批卡(human.py 的
_reviewed_reports)附上审稿概况,据这两项说「出具会降档」「N 句未能反查」
上限:句数、每句查询次数、每句时长、全局每日金额(核查和裁判各记各的账,engine/spend.py)。都按发起时快照的核查
设置(run.options["verify"]),节点上写了的盖过它:审批停下期间改了设置,恢复、接着跑仍按发起时的那份。没有快照
@@ -55,8 +58,8 @@ from app.engine.evidence import ENTITY_KINDS, iter_units, ledger_enabled, next_e
from app.engine.judge import _usd
from app.engine.merge_query import MERGE_SOURCE
from app.engine.replay import _in_graph
from app.engine.schema import CHALLENGE_LABEL, CHALLENGE_ON_REFUTED_LABEL, GraphSpec, NodeType, challenge_problems, \
challenge_report
from app.engine.labels import CHALLENGE_LABEL, CHALLENGE_ON_REFUTED_LABEL
from app.engine.schema import GraphSpec, NodeType, _ancestors, challenge_problems, challenge_report
from app.engine.state import GraphState
from app.engine.verify_agent import RATIONALE, VerifyOutcome, run_verify_agent
from app.tools.datasource import run_versions
@@ -466,9 +469,15 @@ def _named(claims: list[dict[str, Any]]) -> str:
return names + ("等" if len(claims) > _NAMED else "")
def _refuted_gap(claims: list[dict[str, Any]]) -> str:
"""出具缺口里的那句:「推翻了 N 句结论:「…」」。不带节点名也不带「反方审稿」:出具契约统一加「反方审稿「节点名」:」
(没改名的节点只加「反方审稿:」),正文再说一遍就成了「反方审稿:反方审稿推翻了……」。"""
return f"推翻了 {len(claims)} 句结论:{_named(claims)}"
def _refuted_words(claims: list[dict[str, Any]]) -> str:
"""出具缺口、时间线、报错里的那句:「反方审稿推翻了 N 句结论:「…」」。不带节点名:出具契约统一加前缀。"""
return f"反方审稿推翻了 {len(claims)} 句结论:{_named(claims)}"
"""时间线、报错里的那句:「反方审稿推翻了 N 句结论:「…」」。它们不经出具契约加前缀,主语得自己带上。"""
return f"反方审稿{_refuted_gap(claims)}"
def pick_items(claims: list[dict[str, Any]]) -> list[dict[str, Any]]:
@@ -616,6 +625,9 @@ class _Review:
unreviewed.append(why)
counts = {v: sum(1 for c in claims if c["verdict"] == v) for v in VERDICTS}
# 没能反查的句数:没派出去、派出去却没查成的(unreviewed 里各有原因),加上触顶时没能给出结论的。都记为无法判断,
# 可「无法判断 2 句」既可能是查了、数据不够,也可能是根本没查:审批卡要分开说
missed = len(unreviewed) + sum(skipped for _, skipped in hit.values())
artifact = await self._store({"report": self.report_id, "doc_artifact": self.doc_artifact, "claims": claims,
"model": model})
for limit, (touched, skipped) in hit.items():
@@ -645,7 +657,7 @@ class _Review:
ctx.emit(EventType.LOG, level="warn", code="challenge_refuted",
items=[{"unit": c["unit"], "text": c["text"]} for c in refuted[:ITEMS_MAX]],
message=_refuted_words(refuted))
return {"artifact": artifact, "counts": counts, "usage": usage,
return {"artifact": artifact, "counts": counts, "usage": usage, "unreviewed": missed,
"claims": [{k: c[k] for k in ("unit", "text", "verdict")} for c in claims]}
async def _store(self, record: dict[str, Any]) -> str | None:
@@ -674,9 +686,13 @@ def _finish(ctx: NodeContext, state: GraphState, outcome: dict[str, Any], *, on_
f"「{CHALLENGE_LABEL['on_refuted']}」改为「{CHALLENGE_ON_REFUTED_LABEL['flag']}」")
on_refuted = "degrade" if refuted else "flag"
result: dict[str, Any] = {"challenge_artifact": outcome.get("artifact"), "counts": outcome.get("counts") or {},
"items": pick_items(claims), "on_refuted": on_refuted}
"items": pick_items(claims), "on_refuted": on_refuted,
"degrades": on_refuted == "degrade" and feeds_issuance(ctx.run.spec, ctx.node.id)}
# checkpoint 里升级前审的结果没有这一项:不知道就不写,审批卡照旧不说
if isinstance(outcome.get("unreviewed"), int):
result["unreviewed"] = outcome["unreviewed"]
if refuted and on_refuted == "degrade":
result["issuance_gaps"] = [_refuted_words(refuted)]
result["issuance_gaps"] = [_refuted_gap(refuted)]
updates: dict[str, Any] = {"nodes": {ctx.node.id: result}}
spent = outcome.get("usage") or {}
if spent.get("calls"):
@@ -694,10 +710,19 @@ def _finish(ctx: NodeContext, state: GraphState, outcome: dict[str, Any], *, on_
# --------------------------------------------------------------------------
def feeds_issuance(spec: GraphSpec, node_id: str) -> bool:
"""这个节点在不在某个带出具契约的成果节点上游。成果节点的出具契约只收上游报上来的缺口(nodes/io.py 的
_upstream_issuance_gaps):接在侧枝上、或者在成果之后,被推翻的结论记了缺口也进不了出具。认法同画布校验
(schema._check_challenge 的「不在成果节点上游」警告)。"""
return any(node_id in _ancestors(spec, n.id) for n in spec.nodes
if n.type == NodeType.OUTPUT and n.config.get("contract"))
def challenge_summary(spec: GraphSpec, outputs: dict[str, Any], report_id: str) -> dict[str, Any] | None:
"""审批卡上这份报告的反方审稿概况 {counts, items, artifact, on_refuted}:审它的、已经审完的反方审稿节点给的。
没有(没加反方审稿、还没审到、直通了)返回 None,审批卡不加这个键。几个反方审稿节点审同一份报告时取画布上
靠后的那个。只看图和状态:重放时算出来的一样。"""
"""审批卡上这份报告的反方审稿概况 {counts, items, artifact, on_refuted, unreviewed, degrades}:审它的、已经审完的
反方审稿节点给的。没有(没加反方审稿、还没审到、直通了)返回 None,审批卡不加这个键。几个反方审稿节点审同一份报告
时取画布上靠后的那个。只看图和状态:重放时算出来的一样。unreviewed、degrades 是期 4 加的:升级前的产出没有,
概况也不带(审批卡按旧规则说)。"""
found: dict[str, Any] | None = None
for node in spec.nodes:
result = outputs.get(node.id)
@@ -707,8 +732,12 @@ def challenge_summary(spec: GraphSpec, outputs: dict[str, Any], report_id: str)
continue
found = {"counts": dict(result["counts"]), "items": list(result.get("items") or [])[:ITEMS_MAX],
"artifact": result.get("challenge_artifact"), "on_refuted": result.get("on_refuted") or "degrade"}
if isinstance(result.get("unreviewed"), int) and not isinstance(result["unreviewed"], bool):
found["unreviewed"] = result["unreviewed"]
if isinstance(result.get("degrades"), bool):
found["degrades"] = result["degrades"]
return found
__all__ = ["CHALLENGE_SYSTEM", "ITEMS_MAX", "VERDICTS", "challenge_summary", "claim_sources", "mark_original",
"pick_claims", "pick_items", "run_challenge", "with_note"]
__all__ = ["CHALLENGE_SYSTEM", "ITEMS_MAX", "VERDICTS", "challenge_summary", "claim_sources", "feeds_issuance",
"mark_original", "pick_claims", "pick_items", "run_challenge", "with_note"]
+22 -9
View File
@@ -21,7 +21,8 @@ SQL,就跟着差异说明进了审批卡、对照工件和目录日志。完
缺口;b、neither、没选的写进产出的 issuance_gaps,成果节点的出具契约据此降档(on_reject 为 fail 时节点直接失败)。
两边只差一个方面、又对得上数据目录事实的,选择写进数据目录(engine/dual_check_facts.py,指标自己的筛选范围不写)。
选了会怎样要在卡上看得出:payload 在约定的几项之外带上实际生效的 on_reject,摘要里每个指标带 catalog——选 a、选 b
按发起审批时的目录各会往目录里写什么、不写的为什么(dual_check_facts.preview,和真写同一套判断)。
按发起审批时的目录各会往目录里写什么、不写的为什么(dual_check_facts.preview,和真写同一套判断);几个指标落到目录
同一项上时每种选择组合合起来会怎样写(preview_joint),放在对照工件的 catalog_joint 里。
盲算触顶、出错、交不出数,或者输入本身没法单独复算的指标按「无法复算」记缺口,发 dual_check_unchecked。
**产出的状态。** 全部一致是 agreed;没有不一致、但有无法复算的:一个都没核成是 unchecked,核成了一部分是 partial;
@@ -57,12 +58,13 @@ from app.db.models import DataSource
from app.engine import recompute
from app.engine.approval import read_decision
from app.engine.context import NodeContext, NodeError
from app.engine.dual_check_facts import preview, record_choices
from app.engine.dual_check_facts import preview, preview_joint, record_choices
from app.engine.evidence import build_catalog, is_number, next_exec
from app.engine.expressions import ExpressionError, _cell_call, _chain_root, cell_value, column_kind, numeric_text, \
parse_expression
from app.engine.replay import _in_graph, ask, once
from app.engine.schema import DUAL_CHECK_FIELD_LABEL, DUAL_CHECK_ON_REJECT, NodeType, dual_check_card
from app.engine.labels import DUAL_CHECK_FIELD_LABEL, DUAL_CHECK_ON_REJECT_LABEL
from app.engine.schema import DUAL_CHECK_ON_REJECT, NodeType, dual_check_card
from app.engine.state import GraphState
#: 节点产出的状态:跳过、全部一致、核成了一部分(其余无法复算)、一个都没核成、人已裁决
@@ -109,10 +111,11 @@ SAME_SQL_ASPECT = "data"
DIFF_TIMEOUT = "两条 SQL 比较超时,未能归纳成具体差异,请对照原文"
DIFF_FAILED = "两条 SQL 比较时出错,未能归纳成具体差异,请对照原文"
NO_SQL = "有一边没有可比较的 SQL,请对照原文"
#: 交给成果节点的缺口(不带节点名:出具契约统一加「双盲复算「节点名」:」)
GAP_BLIND = "指标「{name}」双盲复算不一致,审核人选择了盲算结果"
GAP_NEITHER = "指标「{name}」双盲复算不一致,审核人判断两边都不对"
GAP_UNANSWERED = "指标「{name}」双盲复算不一致,审核人没有作出选择"
#: 交给成果节点的缺口。不带节点名,也不再说「双盲复算」:出具契约统一加「双盲复算「节点名」:」(没改名的节点只加
#: 「双盲复算:」),正文再说一遍就成了「双盲复算:指标「X」双盲复算不一致……」
GAP_BLIND = "指标「{name}」两路结果不一致,审核人选择了盲算结果"
GAP_NEITHER = "指标「{name}」两路结果不一致,审核人判断两边都不对"
GAP_UNANSWERED = "指标「{name}」两路结果不一致,审核人没有作出选择"
GAP_UNCHECKED = "指标「{name}」无法复算:{reason}"
BLIND_SYSTEM = (
@@ -208,7 +211,7 @@ async def run_dual_check(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
# 目录照样写了:审核人的选择是对数据的判断,和这次运行停不停无关
names = "、".join(f"「{r['name']}」" for r in rejected[:5])
raise NodeError(ctx.node.id, f"双盲复算不一致,审核人没有采用原流程的结果(指标{names})。"
f"「{DUAL_CHECK_FIELD_LABEL['on_reject']}」设为「判为失败」,运行已停止;"
f"「{DUAL_CHECK_FIELD_LABEL['on_reject']}」设为「{DUAL_CHECK_ON_REJECT_LABEL['fail']}」,运行已停止;"
"请按审核意见修正取数或口径后重新运行")
return _finish(state, ctx, card_id, card, checked, status=RESOLVED, gaps=gaps, choices=choices, note=note,
artifact=review["card_artifact"], catalog=catalog)
@@ -805,6 +808,8 @@ def _slim(outcome: Any) -> dict[str, Any]:
"row": cell.get("row"), "column": cell.get("column"), "eid": cell.get("eid"),
"sql": query.get("sql"), "source": query.get("source"),
"rationale": (outcome.result or {}).get("rationale") or "",
# 理由组装成的文档(evidence.compose_doc):数字有出处的挂着盲算自己查询的格子,裸数字已标出。审批卡照它画理由
"rationale_doc": outcome.rationale if isinstance(outcome.rationale, dict) else None,
"queries": [{k: q.get(k) for k in ("alias", "artifact", "sql", "row_count", "truncated", "source")}
for q in outcome.queries],
"notes": list(outcome.notes),
@@ -964,7 +969,8 @@ async def _review(state: GraphState, ctx: NodeContext, checked: dict[str, Any],
{"path": i["path"], "value": (results.get(i["key"]) or {}).get("value"),
"sql": (results.get(i["key"]) or {}).get("sql"), "source": (results.get(i["key"]) or {}).get("source"),
"alias": (results.get(i["key"]) or {}).get("alias"),
"rationale": (results.get(i["key"]) or {}).get("rationale") or ""}
"rationale": (results.get(i["key"]) or {}).get("rationale") or "",
"rationale_doc": (results.get(i["key"]) or {}).get("rationale_doc")}
for i in row["inputs"]]},
"diff": {"equivalent": merged["equivalent"],
"items": [{"aspect": i.get("aspect"), "text": i.get("text")} for i in merged["items"]],
@@ -989,6 +995,13 @@ async def _review(state: GraphState, ctx: NodeContext, checked: dict[str, Any],
"catalog": preview(facts[row["metric_id"]], info.get("schema"), info.get("notes")),
})
content = {"metrics": metrics, "model": checked.get("model"), "same_model": bool(checked.get("same_model"))}
# 同一次裁决里几个指标落到数据目录的同一项上时,按各自的选择合起来写(dual_check_facts._merged):每个指标单独的
# 预告说不准,把每种组合的结果都算好放进对照工件(payload 里放不下,审批卡打开时就取工件),卡上按当前的选择取
catalogs = {name: (info.get("schema"), info.get("notes") or {}) for name, info in infos.items()}
joint = await asyncio.to_thread(preview_joint, [facts[r["metric_id"]] for r in rows if r["metric_id"] in facts],
catalogs)
if joint:
content["catalog_joint"] = joint
try:
artifact = await artifact_store.put_json(content, kind="dual_check", run_id=ctx.run.run_id,
node_id=ctx.node.id, meta={"metrics": len(metrics)})
+4 -2
View File
@@ -322,7 +322,8 @@ def _upstream_issuance_gaps(state: GraphState, ctx: NodeContext) -> list[str]:
这是给「把关类」节点留的通用口子:节点觉得这次的结论不该完整出具(双盲复算两路对不上、
反方审稿的反驳没有回应),在自己的产出里写 issuance_gaps,成果节点的出具契约照单收进缺口,
出具随之降档。写的时候不用带节点名,这里统一加上「双盲复算「复核销售额」:」这样的前缀,
和别的缺口一样读得出是谁报的。
和别的缺口一样读得出是谁报的。节点名就是类型的叫法(拖进画布没改名)时只写一次:「反方审稿:」,
不写成「反方审稿「反方审稿」:」。缺口正文也不必再提节点类型,前缀已经说了。
只认这个成果节点的上游:另一条支路上的节点管不到这份成果。不是字符串、空白的条目跳过——
这个键谁都能写(数据整形的 JSON 模板也行),写坏了只当没写,不能让出具崩掉。
@@ -340,7 +341,8 @@ def _upstream_issuance_gaps(state: GraphState, ctx: NodeContext) -> list[str]:
reported = payload.get("issuance_gaps")
if not isinstance(reported, list):
continue
prefix = f"{type_label(node.type)}「{node.title}」:"
kind = type_label(node.type)
prefix = f"{kind}:" if node.title == kind else f"{kind}「{node.title}」:"
out.extend(prefix + gap.strip() for gap in reported if isinstance(gap, str) and gap.strip())
return out
+17
View File
@@ -111,6 +111,23 @@ WARN_CODES: dict[str, tuple[str, str]] = {
# 估不出裁判的金额、裁判和写作是同一个模型:配置上的提醒,答案本身没受影响
"judge_unpriced": ("judge_unpriced", BENIGN),
"judge_same_model": ("judge_same_model", BENIGN),
# 双盲复算、反方审稿的核查模型和被核查的模型相同:同 judge_same_model,只关乎独立性,结论本身没受影响
"dual_check_same_model": ("dual_check_same_model", BENIGN),
"challenge_same_model": ("challenge_same_model", BENIGN),
# 核查 agent(engine/verify_agent.py,双盲复算、反方审稿共用)估不出金额:同 judge_unpriced,配置上的提醒
"verify_unpriced": ("verify_unpriced", BENIGN),
# 核查没做完、没通过:盲算触顶、两路不一致、指标无法复算;反方审稿触顶、推翻了结论、有句子没能反查、只查了
# 部分数据源。答案照常交付(出具契约另按缺口降档),是缺口不是断裂,按各自的 kind 记,界面说得清是哪一种
"dual_check_limit": ("dual_check_limit", DEGRADED),
"dual_check_disagreed": ("dual_check_disagreed", DEGRADED),
"dual_check_unchecked": ("dual_check_unchecked", DEGRADED),
"challenge_limit": ("challenge_limit", DEGRADED),
"challenge_refuted": ("challenge_refuted", DEGRADED),
"challenge_unreviewed": ("challenge_unreviewed", DEGRADED),
"challenge_sources_lost": ("challenge_sources_lost", DEGRADED),
# 核查 agent 自己触顶、按已有结果收尾:多半和节点自己的 *_limit 说的是同一次触顶,照样是缺口,记成它自己的 kind,
# 不落成笼统的 warn——步数上限(max_steps)只有它说
"verify_limit": ("verify_limit", DEGRADED),
}
# 这些 kind 重跑一次大概率能好:原因说得清,且不是「问题本身没问明白」。
+11 -18
View File
@@ -9,6 +9,13 @@ from typing import Any, Iterable, Literal
from pydantic import BaseModel, Field, field_validator, model_validator
from app.engine.labels import TYPE_LABEL, field_label, judge_label, option_label, q, type_label # noqa: F401
# 双盲复算、反方审稿的叫法在 labels.py;校验、执行器和老的调用方照旧从这里取
from app.engine.labels import ( # noqa: F401
CHALLENGE_LABEL,
CHALLENGE_ON_REFUTED_LABEL,
DUAL_CHECK_FIELD_LABEL,
DUAL_CHECK_ON_REJECT_LABEL,
)
class NodeType(StrEnum):
@@ -425,15 +432,9 @@ def _check_report_sources(node: GraphNode, spec: GraphSpec, result: ValidationRe
node_id=node.id, field="metrics_from", code=code)
#: 双盲复算节点各配置项在检查器上的叫法(前端 nodeDefs 的 dual_check 字段)。按 labels.py 的规矩应放在那里,
#: 那个文件不归双盲复算的工作包,先放在校验旁边,节点执行时报错也从这里取
DUAL_CHECK_FIELD_LABEL = {
"card": "口径卡", "metrics": "复算的指标", "definitions": "口径说明", "provider": "盲算模型 · 接入",
"model": "盲算模型", "tolerance": "允许的相对误差", "on_reject": "未采用原流程时", "enabled": "启用双盲复算",
}
#: 审核人没有采用原流程的结果(选了盲算,或者判断两边都不对)时:degrade 计入缺口、出具降档(默认)/ fail 判为失败
#: 审核人没有采用原流程的结果(选了盲算,或者判断两边都不对)时:degrade 计入缺口、出具降档(默认)/ fail 判为失败。
#: 各配置项和这两个取值的叫法(DUAL_CHECK_FIELD_LABEL、DUAL_CHECK_ON_REJECT_LABEL)在 labels.py
DUAL_CHECK_ON_REJECT = ("degrade", "fail")
DUAL_CHECK_ON_REJECT_LABEL = {"degrade": "计入缺口、出具降档", "fail": "判为失败"}
def dual_check_card(spec: GraphSpec, node: GraphNode) -> tuple[str | None, str | None]:
@@ -721,17 +722,9 @@ def _check_report_from(node: GraphNode, report_from: Any, spec: GraphSpec,
# 反方审稿节点(engine/nodes/challenge.py):审哪份报告、各项配置
# --------------------------------------------------------------------------
#: 反方审稿节点各项配置在界面上的叫法:报错写叫法,不写键名
CHALLENGE_LABEL = {
"report": "要审的报告", "max_claims": "最多审几句", "max_queries": "每句最多查询次数",
"timeout_s": "每句时长上限(秒)", "provider": "反方模型 · 接入", "model": "反方模型",
"on_refuted": "结论被推翻时", "enabled": "启用",
}
#: 结论被推翻时怎么办:degrade 计入出具缺口、出具降档(默认)/ flag 只标注 / fail 判节点失败、运行停止
#: 结论被推翻时怎么办:degrade 计入出具缺口、出具降档(默认)/ flag 只标注 / fail 判节点失败、运行停止。
#: 各项配置和这三个取值在界面上的叫法(CHALLENGE_LABEL、CHALLENGE_ON_REFUTED_LABEL)在 labels.py:报错写叫法,不写键名
CHALLENGE_ON_REFUTED = ("degrade", "flag", "fail")
#: 选项的叫法和检查器下拉一致(lib/terms.ts 的 CHALLENGE_TEXT.onRefuted),只去掉「(默认)」:报错让人去改某一项时,
#: 人在下拉里找得到这个名字
CHALLENGE_ON_REFUTED_LABEL = {"degrade": "出具降档", "flag": "只标注", "fail": "报错并停止运行"}
def _challenge_field(key: str) -> str:
+4 -2
View File
@@ -87,10 +87,11 @@ class Blind:
没对上指标名的盲算交 null。不是盲算的调用(报告撰写)返回 writer 的正文并记下收到的消息。
calls 记下每次盲算调用(step / final),tasks 记下盲算收到的任务原文(几个盲算同时进行时也是每个任务一条)。
on_query:盲算第一次查库之前调用一次(比如往库里插一行,造出「两次取数之间数据变了」)。
rationale:盲算交的理由,{ref} 换成交的那一格的引用(缺省「按口径说明统计,结果是 {ref}。」)。
"""
def __init__(self, monkeypatch, source: str, plans: dict, *, writer: str = "本周销售额见口径卡。", row: int = 0,
on_query=None):
on_query=None, rationale: str = "按口径说明统计,结果是 {ref}。"):
from app.providers import mock_model
self.source = source
@@ -98,6 +99,7 @@ class Blind:
self.writer = writer
self.row = row
self.on_query = on_query
self.rationale = rationale
self.calls: list[str] = []
self.tasks: list[str] = []
self.writer_prompts: list[str] = []
@@ -114,7 +116,7 @@ class Blind:
_, column = plan
alias = f"Q{len(blind.sqls(plan))}"
body = {"value": {"alias": alias, "row": blind.row, "column": column},
"rationale": f"按口径说明统计,结果是 [[v:{alias}.r{blind.row}.{column}]]。"}
"rationale": blind.rationale.format(ref=f"[[v:{alias}.r{blind.row}.{column}]]")}
return AIMessage(content=json.dumps(body, ensure_ascii=False))
if model.tools:
blind.calls.append("step")
+68 -7
View File
@@ -265,7 +265,7 @@ async def test_a_refuted_conclusion_degrades_the_issuance(monkeypatch, shop):
issuance = row.output["_issuance"]
assert issuance["tier"] == "degraded"
assert issuance["gaps"] == [f"反方审稿「审稿」:反方审稿推翻了 1 句结论:「{SOUTH}」"], "缺口不带节点名,前缀由出具契约加"
assert issuance["gaps"] == [f"反方审稿「审稿」:推翻了 1 句结论:「{SOUTH}」"], "缺口不带节点名,前缀由出具契约加"
doc = await report_doc(row.id)
east, south = unit_of(doc, EAST), unit_of(doc, SOUTH)
@@ -275,7 +275,9 @@ async def test_a_refuted_conclusion_degrades_the_issuance(monkeypatch, shop):
assert out["items"] == [{"unit": east, "text": EAST, "verdict": "weakened"},
{"unit": south, "text": SOUTH, "verdict": "refuted"}]
assert out["on_refuted"] == "degrade"
assert out["issuance_gaps"] == [f"反方审稿推翻了 1 句结论:「{SOUTH}」"]
assert out["issuance_gaps"] == [f"推翻了 1 句结论:「{SOUTH}」"]
# 两句都真查过了;接在带出具契约的成果节点上游、按降档处置:被推翻的真的会让出具降档
assert out["unreviewed"] == 0 and out["degrades"] is True
record = artifact_store.load(out["challenge_artifact"])
assert set(record) == {"report", "doc_artifact", "claims", "model"}
@@ -317,6 +319,19 @@ async def test_a_refuted_conclusion_degrades_the_issuance(monkeypatch, shop):
assert finished.seq <= row.manifest_seq
async def test_a_challenge_named_after_its_type_is_named_once_in_the_gap(monkeypatch, shop):
"""拖进画布没改名的反方审稿:缺口读成「反方审稿:推翻了 1 句结论:…」,前缀和正文都不再重复「反方审稿」"""
Models(monkeypatch, shop)
spec = graph(shop)
next(n for n in spec["nodes"] if n["id"] == "chal")["data"]["label"] = "反方审稿"
row = await finish(spec)
assert row.status == "succeeded", row.error
assert row.output["_issuance"]["gaps"] == [f"反方审稿:推翻了 1 句结论:「{SOUTH}」"]
# 时间线、报错里那一句照旧带主语:它们不经出具契约加前缀
[refuted] = await logs(row.id, "challenge_refuted")
assert refuted["message"] == f"反方审稿推翻了 1 句结论:「{SOUTH}」"
async def test_flag_only_records_and_fail_stops_the_run(monkeypatch, shop):
Models(monkeypatch, shop)
row = await finish(graph(shop, on_refuted="flag"))
@@ -324,6 +339,7 @@ async def test_flag_only_records_and_fail_stops_the_run(monkeypatch, shop):
assert row.output["_issuance"]["tier"] == "formal" and row.output["_issuance"]["gaps"] == []
out = await output_of(row.id, "chal")
assert out["on_refuted"] == "flag" and "issuance_gaps" not in out and out["counts"]["refuted"] == 1
assert out["degrades"] is False, "只标注:被推翻的不会让出具降档"
assert len(await logs(row.id, "challenge_refuted")) == 1, "flag 也照实记一条"
Models(monkeypatch, shop)
@@ -395,6 +411,8 @@ async def test_a_verdict_without_any_query_is_not_taken(monkeypatch, shop):
assert east["verdict"] == "inconclusive" and "「经受住」" in east["rationale"]
[warn] = await logs(row.id, "challenge_unreviewed")
assert warn["claims_skipped"] == 2 and "没有执行成功任何查询" in warn["reason"]
assert out["unreviewed"] == 2, "一条查询都没执行成功:两句都是未能反查,不是查过了判断不了"
assert out["unreviewed"] <= out["counts"]["inconclusive"], "未能反查的每一句都记为无法判断,审批卡上两个数对得上"
async def test_a_refutation_that_cites_none_of_its_queries_is_not_taken(monkeypatch, shop):
@@ -411,7 +429,8 @@ async def test_a_refutation_that_cites_none_of_its_queries_is_not_taken(monkeypa
assert south["verdict"] == "inconclusive" and len(south["queries"]) == 1 and south["rationale_doc"] is None
assert "没有引用" in south["rationale"] and "扣除退款后华南仍低于华东。" in south["rationale"]
assert east["verdict"] == "stands", "「经受住」不要求引用:它不改变出具"
assert await logs(row.id, "challenge_unreviewed") == [], "查过了,不算没能反查"
assert await logs(row.id, "challenge_unreviewed") == [], "查过了,不算未能反查"
assert out["unreviewed"] == 0
@pytest.mark.parametrize("queries, said, quoted", [
@@ -457,6 +476,7 @@ async def test_claims_whose_sources_are_gone_are_not_sent_out(monkeypatch, shop)
assert all(f"「{shop}」" in c["rationale"] and "不存在或已停用" in c["rationale"] for c in claims)
[warn] = await logs(row.id, "challenge_unreviewed")
assert warn["claims_skipped"] == 2 and f"「{shop}」" in warn["reason"]
assert out["unreviewed"] == 2
async def test_a_claim_with_part_of_its_sources_gone_says_so(monkeypatch, tmp_path, shop):
@@ -518,7 +538,7 @@ async def test_a_claim_with_part_of_its_sources_gone_says_so(monkeypatch, tmp_pa
[lost] = await logs(row.id, "challenge_sources_lost")
assert lost["level"] == "warn" and lost["claims"] == 2 and lost["sources"] == [other]
assert f"「{other}」" in lost["message"] and "2 句" in lost["message"]
assert await logs(row.id, "challenge_unreviewed") == [], "两句都查过了,不算没能反查"
assert await logs(row.id, "challenge_unreviewed") == [], "两句都查过了,不算未能反查"
finally:
await engines.invalidate(other_id)
@@ -652,6 +672,9 @@ async def test_the_time_limit_applies_to_each_claim_and_still_settles(monkeypatc
assert south["verdict"] == "inconclusive" and 1 <= len(south["queries"]) < 20
limits = await logs(row.id, "challenge_limit")
assert [(lm["limit"], lm["claims_skipped"]) for lm in limits] == [("timeout_s", 0)]
# 触顶但按查到的结果收了尾的那句,判断是反方自己给的,不算未能反查;另一句一条查询都没执行,算
out = await output_of(row.id, "chal")
assert out["unreviewed"] == 1 and out["counts"]["inconclusive"] == 2
async def test_the_daily_cap_stops_the_remaining_claims(monkeypatch, shop):
@@ -671,6 +694,8 @@ async def test_the_daily_cap_stops_the_remaining_claims(monkeypatch, shop):
"第一句派出去就撞上上限、一次模型都没调用,和后面不再派出的句子是同一回事,同一种说法"
[limit] = await logs(row.id, "challenge_limit")
assert limit["limit"] == "daily_max_usd" and limit["claims_skipped"] == 2
assert out["unreviewed"] == 2, "撞上每日金额上限没反查的,也是未能反查"
assert out["unreviewed"] <= out["counts"]["inconclusive"]
assert len(await logs(row.id, "verify_limit")) == 1, "第一句撞上每日上限之后,后面的句子不再派 agent 去试"
assert row.output["_issuance"]["tier"] == "formal"
@@ -712,13 +737,45 @@ async def test_the_approval_card_carries_the_challenge_summary(monkeypatch, shop
out = await output_of(row.id, "chal")
assert report["node_id"] == "write"
assert report["challenge"] == {"counts": out["counts"], "items": out["items"],
"artifact": out["challenge_artifact"], "on_refuted": "degrade"}
"artifact": out["challenge_artifact"], "on_refuted": "degrade",
"unreviewed": 0, "degrades": False}
assert report["challenge"]["counts"]["refuted"] == 1
# human.requested 事件里也带着:回放时界面一样读得到
[requested] = await events(row.id, "human.requested", "gate")
assert requested.data["reports"] == approval.payload["reports"]
async def test_the_approval_card_says_whether_refutations_degrade(monkeypatch, shop):
"""审批卡上「出具会降档」只在真的会降档时写:degrades 按「按降档处置、又接在带出具契约的成果节点上游」算。
审批之后的成果节点带了出具契约:会降档;没带(gated 的缺省):不会,见上一个用例"""
Models(monkeypatch, shop)
spec = gated(shop)
next(n for n in spec["nodes"] if n["id"] == "out")["data"]["config"]["contract"] = {"report_from": "write"}
_, approval = await paused(spec)
challenge = approval.payload["reports"][0]["challenge"]
assert challenge["degrades"] is True and challenge["on_refuted"] == "degrade"
async def test_a_challenge_off_the_issuance_path_does_not_degrade(monkeypatch, shop):
"""反方审稿接在侧枝上(成果节点的出具契约收不到它的缺口):按降档处置也降不了档,产出照实写 degrades 为假"""
Models(monkeypatch, shop)
spec = chain(
node("start", "input"),
node("fetch", "tool", tool=f"db_query__{shop}", args={"sql": QUERY}),
node("write", "report", "写周报", instructions="写周报"),
node("out", "output", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}],
contract={"report_from": "write"}),
)
spec["nodes"].append(node("chal", "challenge", "审稿", report="write"))
spec["edges"].append({"source": "write", "target": "chal"})
row = await finish(spec)
assert row.status == "succeeded", row.error
out = await output_of(row.id, "chal")
assert out["on_refuted"] == "degrade" and out["counts"]["refuted"] == 1 and out["issuance_gaps"]
assert out["degrades"] is False
assert row.output["_issuance"]["gaps"] == [], "缺口确实没进出具:和 degrades 说的一致"
async def test_the_approval_card_shows_what_flag_means(monkeypatch, shop):
Models(monkeypatch, shop)
_, approval = await paused(gated(shop, on_refuted="flag"))
@@ -769,8 +826,11 @@ def test_the_card_only_carries_the_review_of_that_report():
items = [{"unit": "u0", "text": "周报甲的一句结论。", "verdict": "stands"}]
outputs = {"chal": {"challenge_artifact": "a1", "counts": counts, "items": items, "on_refuted": "degrade"}}
assert challenge_mod.challenge_summary(spec, outputs, "a") == {
"counts": counts, "items": items, "artifact": "a1", "on_refuted": "degrade"}
"counts": counts, "items": items, "artifact": "a1", "on_refuted": "degrade"}, "老产出没有这两个键:概况也不带"
assert challenge_mod.challenge_summary(spec, outputs, "b") is None
outputs["chal"].update(unreviewed=1, degrades=True)
assert challenge_mod.challenge_summary(spec, outputs, "a") == {
"counts": counts, "items": items, "artifact": "a1", "on_refuted": "degrade", "unreviewed": 1, "degrades": True}
# 直接连着的报告,和直接连着的反方审稿审的那份,都算这个审批节点审的
assert reviewed_report_ids(spec, spec.node_map()["gate"]) == ["a", "b"]
@@ -843,7 +903,7 @@ async def test_a_fail_that_would_be_swallowed_degrades_instead(monkeypatch, shop
row = await finish(graph(shop, on_refuted="fail", on_error="continue"))
assert row.status == "succeeded", row.error
out = await output_of(row.id, "chal")
assert out["on_refuted"] == "degrade" and out["issuance_gaps"] == [f"反方审稿推翻了 1 句结论:「{SOUTH}」"]
assert out["on_refuted"] == "degrade" and out["issuance_gaps"] == [f"推翻了 1 句结论:「{SOUTH}」"]
assert row.output["_issuance"]["tier"] == "degraded"
@@ -957,6 +1017,7 @@ async def test_an_unavailable_model_leaves_the_claims_inconclusive(monkeypatch,
# 时间线上看得出根本没审成:不是只有一条「审了 2 句」
[warn] = await logs(row.id, "challenge_unreviewed")
assert warn["level"] == "warn" and warn["claims_skipped"] == 2 and "核查模型不可用" in warn["reason"]
assert out["unreviewed"] == 2 and out["unreviewed"] <= out["counts"]["inconclusive"]
[done] = await logs(row.id, "challenge_done")
assert "2 句未能反查" in done["message"]
+173 -10
View File
@@ -252,7 +252,8 @@ async def test_a_choice_is_written_as_a_confirmed_human_item_with_its_origin(sho
name, source_id = shop
out = await record_choices([candidate(BY_ORDER_DATE, RIGHT, "b", source=name)], run_id="run-42",
node_title="复核销售额", actor="张三")
assert out == {"written": [{"source": name, "table": "orders", "item": "business_date"}], "skipped": []}
assert out == {"written": [{"source": name, "table": "orders", "item": "business_date",
"value": {"column": "paid_date"}}], "skipped": []}
entry = await entry_of(source_id)
item = entry.notes["business_date"]
assert item["value"] == {"column": "paid_date"} and (item["source"], item["status"]) == ("human", "confirmed")
@@ -290,7 +291,8 @@ async def test_an_inferred_condition_both_sides_use_is_extended(shop):
base = f"SELECT SUM(amount) AS gmv FROM orders WHERE is_test = 0 AND paid_date {WEEK}"
out = await record_choices([candidate(base, extra("status <> 'refunded'", base), "b", source=name)],
run_id="run-1", node_title="复核", actor=None)
assert out["written"] == [{"source": name, "table": "orders", "item": "valid_filter"}], out
assert out["written"] == [{"source": name, "table": "orders", "item": "valid_filter",
"value": "is_test = 0 AND status <> 'refunded'"}], out
item = (await entry_of(source_id)).notes["valid_filter"]
assert item["value"] == "is_test = 0 AND status <> 'refunded'"
assert (item["source"], item["status"]) == ("human", "confirmed")
@@ -327,7 +329,8 @@ async def test_an_inferred_same_condition_is_confirmed(shop):
await seed(source_id, {"valid_filter": catalog.make_item("status <> 'refunded'", "llm")})
out = await record_choices([candidate(WITH_REFUNDS, RIGHT, "b", source=name)], run_id="run-1",
node_title="复核", actor=None)
assert out["written"] == [{"source": name, "table": "orders", "item": "valid_filter"}]
assert out["written"] == [{"source": name, "table": "orders", "item": "valid_filter",
"value": "status <> 'refunded'"}]
item = (await entry_of(source_id)).notes["valid_filter"]
assert item["value"] == "status <> 'refunded'" and (item["source"], item["status"]) == ("human", "confirmed")
@@ -348,7 +351,8 @@ async def test_the_same_column_inferred_is_confirmed_and_keeps_its_rule(shop):
{"column": "paid_date", "rule": "按付款成功时间", "timezone": "Asia/Shanghai"}, "profile")})
out = await record_choices([candidate(BY_ORDER_DATE, RIGHT, "b", source=name)], run_id="run-1",
node_title="复核", actor=None)
assert out["written"] == [{"source": name, "table": "orders", "item": "business_date"}]
assert out["written"] == [{"source": name, "table": "orders", "item": "business_date",
"value": {"column": "paid_date", "rule": "按付款成功时间", "timezone": "Asia/Shanghai"}}]
item = (await entry_of(source_id)).notes["business_date"]
assert item["value"] == {"column": "paid_date", "rule": "按付款成功时间", "timezone": "Asia/Shanghai"}
assert (item["source"], item["status"]) == ("human", "confirmed")
@@ -365,7 +369,8 @@ async def test_a_rejected_item_is_not_brought_back_but_a_rejected_other_value_do
out = await record_choices([candidate(BY_ORDER_DATE, RIGHT, "a", source=name)], run_id="run-2",
node_title="复核", actor=None)
assert out["written"] == [{"source": name, "table": "orders", "item": "business_date"}]
assert out["written"] == [{"source": name, "table": "orders", "item": "business_date",
"value": {"column": "order_date"}}]
assert (await entry_of(source_id)).notes["business_date"]["value"] == {"column": "order_date"}
@@ -394,7 +399,8 @@ async def test_complementary_valid_filters_in_one_decision_are_combined(shop):
# 第三个指标采纳的是和第一个一样的条件:合并时不重复
candidate(COUNT_WEEK, extra("status <> 'refunded'", COUNT_WEEK), "b", metric="订单数", source=name),
], run_id="run-1", node_title="复核", actor=None)
assert out == {"written": [{"source": name, "table": "orders", "item": "valid_filter"}], "skipped": []}
assert out == {"written": [{"source": name, "table": "orders", "item": "valid_filter",
"value": "status <> 'refunded' AND is_test = 0"}], "skipped": []}
entry = await entry_of(source_id)
item = entry.notes["valid_filter"]
assert item["value"] == "status <> 'refunded' AND is_test = 0" and entry.version == 1
@@ -410,7 +416,8 @@ async def test_combined_filters_extend_an_inferred_condition_every_adopted_side_
both = [candidate(base, extra("status <> 'refunded'", base), "b", metric="销售额", source=name),
candidate(base, extra("is_test = 0", base), "b", metric="订单金额", source=name)]
out = await record_choices(both, run_id="run-1", node_title="复核", actor=None)
assert out["written"] == [{"source": name, "table": "orders", "item": "valid_filter"}], out
assert out["written"] == [{"source": name, "table": "orders", "item": "valid_filter",
"value": "channel <> 'test' AND status <> 'refunded' AND is_test = 0"}], out
assert (await entry_of(source_id)).notes["valid_filter"]["value"] == \
"channel <> 'test' AND status <> 'refunded' AND is_test = 0"
@@ -437,7 +444,8 @@ async def test_a_rejected_condition_is_not_written_back_through_a_combined_decis
assert preview(refunds, SCHEMA, notes)["b"]["write"] is False
assert preview(tests, SCHEMA, notes)["b"]["value"] == "is_test = 0"
out = await record_choices([refunds, tests, counted], run_id="run-1", node_title="复核", actor=None)
assert out["written"] == [{"source": name, "table": "orders", "item": "valid_filter"}], out
assert out["written"] == [{"source": name, "table": "orders", "item": "valid_filter",
"value": "is_test = 0"}], out
[skipped] = out["skipped"]
assert "「status <> 'refunded'」曾被人工驳回" in skipped["reason"] and "「销售额」、「订单数」" in skipped["reason"]
assert "订单金额" not in skipped["reason"]
@@ -498,7 +506,8 @@ async def test_combined_filters_do_not_repeat_a_condition_already_in_the_catalog
candidate(base, extra("is_test = 0", base), "b", metric="订单金额", source=name)]
said = preview(picked[1], SCHEMA, {"orders": (await entry_of(source_id)).notes})
out = await record_choices(picked, run_id="run-1", node_title="复核", actor=None)
assert out == {"written": [{"source": name, "table": "orders", "item": "valid_filter"}], "skipped": []}
assert out == {"written": [{"source": name, "table": "orders", "item": "valid_filter",
"value": "status <> 'refunded' AND is_test = 0"}], "skipped": []}
item = (await entry_of(source_id)).notes["valid_filter"]
assert item["value"] == said["b"]["value"] == "status <> 'refunded' AND is_test = 0"
@@ -532,7 +541,8 @@ async def test_the_same_choice_from_two_metrics_is_written_once(shop):
out = await record_choices([candidate(BY_ORDER_DATE, RIGHT, "b", metric="销售额", source=name),
candidate(BY_ORDER_DATE, RIGHT, "b", metric="订单金额", source=name)],
run_id="run-1", node_title="复核", actor=None)
assert out == {"written": [{"source": name, "table": "orders", "item": "business_date"}], "skipped": []}
assert out == {"written": [{"source": name, "table": "orders", "item": "business_date",
"value": {"column": "paid_date"}}], "skipped": []}
entry = await entry_of(source_id)
assert entry.version == 1 and "「销售额」" in entry.notes["business_date"]["note"]
assert "「订单金额」" in entry.notes["business_date"]["note"]
@@ -579,3 +589,156 @@ async def test_a_missing_source_or_an_unrecordable_choice_is_reported(shop):
assert out["written"] == []
assert "数据源「no_such_source」已不存在" in out["skipped"][0]["reason"]
assert "不止差一个方面" in out["skipped"][1]["reason"]
# --------------------------------------------------------------------------
# 预告和真写逐字一致:挡下的一边也说写到哪、为什么;几个指标落到同一项上按合并后的结果预告
# --------------------------------------------------------------------------
def test_a_side_the_catalog_blocks_still_says_where_and_why():
"""认出了事实、只是被目录挡下(已确认、驳回过、和已有条件合不起来)的一边:照样带上写到哪张表的哪一项、本来要写
什么,原因和真写时跳过的原因一字不差。认不出事实的一边只有原因"""
confirmed = {"orders": {"business_date": catalog.make_item({"column": "order_date"}, "human", "confirmed")}}
said = preview(candidate(BY_ORDER_DATE, RIGHT, "a"), SCHEMA, confirmed)
target = {"source": "shop", "table": "orders", "item": "business_date"}
assert said["a"] == {"write": False, **target, "value": {"column": "order_date"},
"reason": "「orders」的业务日期已是人工确认过的「order_date」,无需重复写入"}
assert said["b"]["write"] is False and {k: said["b"][k] for k in target} == target
assert said["b"]["value"] == {"column": "paid_date"} and "已有人工确认过的说法「order_date」" in said["b"]["reason"]
# 认不出事实(采纳的一边没有这个条件):没有写到哪,只有原因
said = preview(candidate(WITH_REFUNDS, RIGHT, "a"), SCHEMA, {})
assert set(said["a"]) == {"write", "reason"} and said["a"]["write"] is False
def test_the_joint_preview_covers_every_way_two_metrics_land_on_one_item():
"""两个指标都只差时间列:各自选原流程是 order_date、选盲算是 paid_date,落到同一张表的业务日期上。四种组合各预告
一次:说法一样的写,说法不同的两个都不写,原因是真写时那一句"""
from app.engine.dual_check_facts import preview_joint
picked = [{**candidate(BY_ORDER_DATE, RIGHT, "a", metric="销售额"), "metric_id": "gmv"},
{**candidate(BY_ORDER_DATE, RIGHT, "a", metric="订单金额"), "metric_id": "total"}]
joint = preview_joint(picked, {"shop": (SCHEMA, {})})
by = {tuple((m["metric_id"], m["choice"]) for m in e["members"]): e for e in joint}
assert set(by) == {(("gmv", a), ("total", b)) for a in "ab" for b in "ab"}
for entry in joint:
assert {k: entry[k] for k in ("source", "table", "item")} == \
{"source": "shop", "table": "orders", "item": "business_date"}
assert by[(("gmv", "b"), ("total", "b"))]["outcomes"] == {
"gmv": {"write": True, "value": {"column": "paid_date"}},
"total": {"write": True, "value": {"column": "paid_date"}}}
split = by[(("gmv", "a"), ("total", "b"))]["outcomes"]
assert split["gmv"] == split["total"] and split["gmv"]["write"] is False
assert "给出了不同的说法" in split["gmv"]["reason"] and "「销售额」" in split["gmv"]["reason"]
def test_the_joint_preview_screens_out_a_rejected_condition_like_the_write():
"""驳回过「status <> 'refunded'」:两个指标并成一项时,带回驳回说法的那个拿出来不写(原因同真写),另一个照写自己的"""
from app.engine.dual_check_facts import preview_joint
notes = {"orders": {"valid_filter": catalog.make_item("status <> 'refunded'", "llm", "rejected")}}
picked = [{**candidate(SUM_WEEK, extra("status <> 'refunded'"), "b", metric="销售额"), "metric_id": "gmv"},
{**candidate(SUM_WEEK, extra("is_test = 0"), "b", metric="订单金额"), "metric_id": "total"}]
[entry] = preview_joint(picked, {"shop": (SCHEMA, notes)})
assert [(m["metric_id"], m["choice"]) for m in entry["members"]] == [("gmv", "b"), ("total", "b")]
assert entry["outcomes"]["total"] == {"write": True, "value": "is_test = 0"}
assert entry["outcomes"]["gmv"]["write"] is False and "曾被人工驳回" in entry["outcomes"]["gmv"]["reason"]
def test_too_many_combinations_are_left_to_the_timeline():
"""几个指标落到同一项上的组合太多(每个两边都落在这一项上,3 的指标数次方):这一项不预告,审批卡回到有条件地说"""
from app.engine import dual_check_facts as facts
picked = [{**candidate(BY_ORDER_DATE, RIGHT, "a", metric=f"指标{i}"), "metric_id": f"m{i}"} for i in range(6)]
assert facts.preview_joint(picked, {"shop": (SCHEMA, {})}) == []
assert len(facts.preview_joint(picked[:3], {"shop": (SCHEMA, {})})) == 3 ** 3 - 1 - 3 * 2
def test_combinations_are_counted_before_they_are_listed(monkeypatch):
"""二十个指标都落到同一项上:组合数是 3 的二十次方量级,得先算出个数、超了就跳过,不能先列出来再比上限——
列一遍就要几十 GB 内存,在服务进程里算(asyncio.to_thread)会把整个服务拖死。预告时从 itertools.product 取了
多少个组合记下来:这一项不预告,取的不该超过上限"""
import itertools
from app.engine import dual_check_facts as facts
drawn = 0
real = itertools.product
def guarded(*iterables, **kw):
nonlocal drawn
for combo in real(*iterables, **kw):
drawn += 1
# 红的时候不至于真把内存吃光:取得比上限多出一截就停
assert drawn <= facts.JOINT_LIMIT + 64, "列出的组合远多于上限:没有先算个数"
yield combo
stem = candidate(BY_ORDER_DATE, RIGHT, "a")
picked = [{**stem, "metric": f"指标{i}", "metric_id": f"m{i}"} for i in range(20)]
monkeypatch.setattr(itertools, "product", guarded)
assert facts.preview_joint(picked, {"shop": (SCHEMA, {})}) == []
assert drawn <= facts.JOINT_LIMIT
def test_each_item_has_its_own_limit():
"""上限按每一项算,不是几项共用一份额度:两个数据源的业务日期各有四个指标落上来(各 72 种组合),两项都预告。
以前共用 JOINT_LIMIT 的额度,前一项用掉 72 种,后一项明明没超上限也被悄悄跳过"""
from app.engine import dual_check_facts as facts
picked = [{**candidate(BY_ORDER_DATE, RIGHT, "a", metric=f"指标{i}", source=source), "metric_id": f"{source}-{i}"}
for source in ("shop", "outlet") for i in range(4)]
joint = facts.preview_joint(picked, {"shop": (SCHEMA, {}), "outlet": (SCHEMA, {})})
per = 3 ** 4 - 1 - 4 * 2
assert per <= facts.JOINT_LIMIT < 2 * per
assert sorted({e["source"] for e in joint}) == ["outlet", "shop"]
assert len(joint) == 2 * per
@pytest.mark.parametrize("choices", [("a", "b"), ("b", "b"), ("b", "a"), ("a", None)])
async def test_the_joint_preview_is_what_gets_written(shop, choices):
"""预告和真写逐字一致:按这组选择,各指标的预告(单独落一项的看 preview,几个落到同一项的看 preview_joint)说写的,
目录里就是那个值;说不写的,原因就在真写跳过的原因里"""
from app.engine.dual_check_facts import preview_joint
name, source_id = shop
await seed(source_id, {"valid_filter": catalog.make_item("is_test = 0", "llm")})
base = f"SELECT SUM(amount) AS gmv FROM orders WHERE is_test = 0 AND paid_date {WEEK}"
picked = [{**candidate(base, extra("status <> 'refunded'", base), "a", metric="销售额", source=name),
"metric_id": "gmv"},
{**candidate(base, extra("channel <> 'test'", base), "a", metric="订单金额", source=name),
"metric_id": "total"}]
notes = {"orders": (await entry_of(source_id)).notes}
singles = {c["metric_id"]: preview(c, SCHEMA, notes) for c in picked}
joint = preview_joint(picked, {name: (SCHEMA, notes)})
chosen = {c["metric_id"]: choice for c, choice in zip(picked, choices) if choice}
expected = said_for(chosen, singles, joint)
out = await record_choices([{**c, "choice": chosen[c["metric_id"]]} for c in picked if c["metric_id"] in chosen],
run_id="run-1", node_title="复核", actor=None)
entry = await entry_of(source_id)
for metric_id, said in expected.items():
if said["write"]:
assert {"source": name, "table": "orders", "item": said["item"], "value": said["value"]} in out["written"]
assert entry.notes[said["item"]]["value"] == said["value"]
else:
assert said["reason"] in [s["reason"] for s in out["skipped"]], (metric_id, said, out)
def said_for(chosen: dict, singles: dict, joint: list) -> dict:
"""审批卡的认法:一个指标选的那一边认出了写到哪,就看按当前的选择还有谁落到同一项上——只有它自己时照单个指标的
预告,几个时照 preview_joint 里这几个成员的那一条"""
out = {}
for metric_id, choice in chosen.items():
side = singles[metric_id][choice]
if "table" not in side:
out[metric_id] = side
continue
where = (side["source"], side["table"], side["item"])
peers = sorted((m, c) for m, c in chosen.items()
if "table" in singles[m][c] and (singles[m][c]["source"], singles[m][c]["table"],
singles[m][c]["item"]) == where)
if len(peers) == 1:
out[metric_id] = side
continue
[entry] = [e for e in joint if sorted((m["metric_id"], m["choice"]) for m in e["members"]) == peers]
out[metric_id] = {**entry["outcomes"][metric_id], "item": entry["item"]}
return out
+72 -18
View File
@@ -16,11 +16,17 @@ variants 和 peers 各自的 catalog)也是服务端的真实输出:按下
服务端合入之前没有这个模块,逐项比对那几项跳过,形状和门槛照样查。
**同一次裁决里几个指标落到同一项上。** 预告是每个指标单独算的,服务端写目录前却要把同一次裁决里落到同一张表同一项上的
说法归拢(业务日期、去重规则说法不同的一项都不写,有效记录条件合成一句写),卡上按其余指标当前的选择补上这一层
(2026-10 第 3 轮复核:两个指标分开选,卡上各自说「记为……」,服务端一项都不写)。peers 是换掉「退款率」对不上的那个
输入的几对 SQL,和「销售额」(或换上的变体)落到数据目录的同一项上;merges 是检查脚本看的几种组合,written 是服务端
按这几种选择真写一遍(record_choices,空目录、临时数据源)的结果。它要建数据源,regenerate 不生成:测试报不一致时,
照报出的实际结果改 written,再看审批卡的说法要不要跟着改。
说法归拢(业务日期、去重规则说法不同的一项都不写,有效记录条件合成一句写,驳回过的拿出来不写)。服务端把每种组合按
真写的办法算好放进对照工件(dual_check_facts.preview_joint → catalog_joint),卡上按各指标当前的选择取;工件里没有时
卡上按其余指标当前的选择有条件地推。peers 是换掉「退款率」对不上的那个输入的几对 SQL,和「销售额」(或换上的变体)
落到数据目录的同一项上;merges 是检查脚本看的几种组合:notes 是这一组合发起审批时的目录(不写就是空目录),catalog
是两个指标按它各自的预告,joint 是 preview_joint 的真实输出(这两项 regenerate 重新生成),written 是服务端按这几种
选择真写一遍(record_choices,临时数据源)的结果——它要建数据源,regenerate 不生成:测试报不一致时,照报出的实际结果
改 written,再看审批卡的说法要不要跟着改。joint 里按这一组合的选择取出的那一条,和 written 一字不差(会写的值、不写的
原因)。
**挡下的一边。** catalog_blocked 是「销售额」那一对 SQL 在目录里已有人工确认的业务日期时的预告(notes 是那时的目录):
两边都认出了事实、都被目录挡下,照样带着写到哪和原因,审批卡逐条说「不会记入:原因」。
重新生成夹具里的差异说明和预告(只改 diff、摘要的 diff 和 fallback、预告,其余原样;没有预告的模块时预告不动):
cd backend && PYTHONPATH=. python tests/test_dual_check_fixture.py
@@ -136,11 +142,9 @@ def _merge_metrics(fx: dict[str, Any], case: dict[str, Any]) -> list[tuple[str,
("refund_rate", PEER_METRIC, tuple(peer["sql"]), peer["catalog"])]
def _shown(side: dict[str, Any]) -> dict[str, Any]:
"""一边的预告里审批卡用得到的:写不写,写的话写到哪、写成什么。不写的原因只在时间线上给人看,措辞不比。"""
if side.get("write") is True:
return {k: side.get(k) for k in ("write", "source", "table", "item", "value")}
return {"write": side.get("write")}
def _merge_candidates(fx: dict[str, Any], case: dict[str, Any]) -> list[dict[str, Any]]:
"""merges 里一种组合交给服务端的候选(不带 choice):两个指标各自的那一对 SQL,带上指标 id(preview_joint 按它说结果)。"""
return [{**_candidate(metric, *sql), "metric_id": metric_id} for metric_id, metric, sql, _ in _merge_metrics(fx, case)]
def _server(name: str) -> Any:
@@ -209,14 +213,41 @@ def test_catalog_previews_have_the_server_shape():
def test_catalog_previews_follow_the_server():
"""预告里审批卡用得到的部分(写不写、写到哪、写成什么)和服务端(dual_check_facts.preview)一致:服务端改了写目录
的规则,这里先变红——审批卡上照预告说的、检查脚本按预告核对的有条件说法都要跟着看一遍。"""
"""预告和服务端(dual_check_facts.preview)一字不差:写不写、写到哪、写成什么,不写的原因(审批卡逐条照原话说「不会
记入:原因」)。服务端改了写目录的规则或说法,这里先变红——审批卡上照预告说的、检查脚本按预告核对的说法都要跟着看一遍。"""
preview = _server("preview")
if preview is None:
pytest.skip("双盲复算的服务端(dual_check_facts.preview)还没合入")
for name, metric, original, blind, _, expected in _preview_cases(_load()):
got = preview(_candidate(metric, original, blind), SCHEMA, {})
assert {s: _shown(got[s]) for s in ("a", "b")} == {s: _shown(expected[s]) for s in ("a", "b")}, name
assert preview(_candidate(metric, original, blind), SCHEMA, {}) == expected, name
def test_a_blocked_side_still_says_where_it_would_write():
"""目录里已有人工确认的业务日期:两边都认出了事实、都被挡下——照样带着写到哪、本来要写什么和原因,和服务端一字不差"""
preview = _server("preview")
if preview is None:
pytest.skip("双盲复算的服务端(dual_check_facts.preview)还没合入")
fx = _load()
blocked = fx["catalog_blocked"]
gmv = fx["card"]["metrics"][0]
(a, b), = _differing(gmv)
assert preview(_candidate(gmv["name"], a["sql"], b["sql"]), SCHEMA, blocked["notes"]) == blocked["catalog"]
for side in ("a", "b"):
said = blocked["catalog"][side]
assert said["write"] is False and said["reason"] and said["table"] == "orders" and said["item"] == "business_date"
def test_merges_carry_the_server_joint_preview():
"""merges 里每一种组合:两个指标各自的预告(catalog)和合起来的预告(joint)都是服务端按这一组合的目录算出来的"""
preview, preview_joint = _server("preview"), _server("preview_joint")
if preview is None or preview_joint is None:
pytest.skip("双盲复算的服务端(dual_check_facts.preview_joint)还没合入")
fx = _load()
for name, case in fx["merges"].items():
notes = case.get("notes") or {}
candidates = _merge_candidates(fx, case)
assert case["catalog"] == {c["metric_id"]: preview(c, SCHEMA, notes) for c in candidates}, name
assert case["joint"] == preview_joint(candidates, {SOURCE: (SCHEMA, notes)}), name
def test_merges_put_two_metrics_on_the_same_item():
@@ -226,8 +257,9 @@ def test_merges_put_two_metrics_on_the_same_item():
fx = _load()
assert fx["merges"]
for name, case in fx["merges"].items():
sides = [said[case["choices"][metric_id]] for metric_id, _, _, said in _merge_metrics(fx, case)]
assert all(s["write"] is True for s in sides), name
sides = [case["catalog"][metric_id][case["choices"][metric_id]] for metric_id, _, _, _ in _merge_metrics(fx, case)]
# 落到同一项上的:会写的、被目录挡下的都算(都带着写到哪),服务端照样把它们归拢成一项
assert all("table" in s for s in sides), name
targets = {(s["source"], s["table"], s["item"]) for s in sides}
assert len(targets) == 1, name
assert all((SOURCE, w["table"], w["item"]) in targets for w in case["written"]), name
@@ -263,15 +295,27 @@ async def test_merged_choices_follow_the_server(make_source):
fx = _load()
for name, case in fx["merges"].items():
source, source_id = await make_source()
candidates = [{**_candidate(metric, *sql), "source": source, "choice": case["choices"][metric_id]}
for metric_id, metric, sql, _ in _merge_metrics(fx, case)]
for table, notes in (case.get("notes") or {}).items():
async with SessionLocal() as session:
await catalog.write_entry(session, source_id, table, notes, if_version=0, actor="数据组")
candidates = [{**c, "source": source, "choice": case["choices"][c["metric_id"]]}
for c in _merge_candidates(fx, case)]
out = await record(candidates, run_id="run-dc-0001", node_title="复核周报口径", actor=None)
written = []
async with SessionLocal() as session:
for w in out["written"]:
entry = await catalog.read_entry(session, source_id, w["table"])
assert w["value"] == entry.notes[w["item"]]["value"], name
written.append({"table": w["table"], "item": w["item"], "value": entry.notes[w["item"]]["value"]})
assert written == case["written"], (name, out["skipped"])
# 审批卡按这一组合的选择从 joint 里取的那一条,就是真写的结果:会写的值、不写的原因一字不差
chosen = {c["metric_id"]: case["choices"][c["metric_id"]] for c in candidates}
[entry] = [e for e in case["joint"] if {m["metric_id"]: m["choice"] for m in e["members"]} == chosen]
for metric_id, said in entry["outcomes"].items():
if said["write"]:
assert {"table": entry["table"], "item": entry["item"], "value": said["value"]} in written, (name, metric_id)
else:
assert said["reason"] in [s["reason"] for s in out["skipped"]], (name, metric_id, out["skipped"])
def regenerate() -> None:
@@ -294,6 +338,16 @@ def regenerate() -> None:
for group, metric_name in SWAPS.items():
for swap in fx[group].values():
swap["catalog"] = preview(_candidate(metric_name, *swap["sql"]), SCHEMA, {})
gmv = fx["card"]["metrics"][0]
(a, b), = _differing(gmv)
blocked = fx["catalog_blocked"]
blocked["catalog"] = preview(_candidate(gmv["name"], a["sql"], b["sql"]), SCHEMA, blocked["notes"])
if (preview_joint := _server("preview_joint")) is not None:
for case in fx["merges"].values():
notes = case.get("notes") or {}
candidates = _merge_candidates(fx, case)
case["catalog"] = {c["metric_id"]: preview(c, SCHEMA, notes) for c in candidates}
case["joint"] = preview_joint(candidates, {SOURCE: (SCHEMA, notes)})
FIXTURE.write_text(json.dumps(fx, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
+121 -11
View File
@@ -401,9 +401,14 @@ async def test_disagreement_pauses_with_the_agreed_payload_and_card(monkeypatch,
assert set(orig) == {"path", "value", "sql", "source", "alias"}
assert (orig["sql"], orig["value"], orig["source"], orig["alias"]) == (BY_ORDER_DATE, 310.0, name, "Q1")
[mine] = metric["blind"]["inputs"]
assert set(mine) == {"path", "value", "sql", "source", "alias", "rationale"}
assert set(mine) == {"path", "value", "sql", "source", "alias", "rationale", "rationale_doc"}
assert (mine["sql"], mine["value"], mine["source"], mine["alias"]) == (RIGHT, 330.0, name, "Q1")
assert mine["path"] == orig["path"] == "cell(nodes.fetch, 0, 'gmv')" and "330" in mine["rationale"]
# 理由组装成的文档:正文同 rationale,数字挂着盲算自己那次查询的格子
doc = mine["rationale_doc"]
assert doc["markdown"] == mine["rationale"] and doc["violations"] == []
[cited] = [seg for block in doc["blocks"] for unit in block["units"] for seg in unit["segments"] if seg.get("cite")]
assert cited["text"] == "330" and cited["cite"]["alias"] == "Q1"
assert metric["original"]["value"] == 310.0 and metric["blind"]["value"] == 330.0
assert set(metric["diff"]) == {"equivalent", "items", "fallback", "text_diff"}
assert metric["diff"]["equivalent"] is False and metric["diff"]["fallback"] is False
@@ -416,6 +421,19 @@ async def test_disagreement_pauses_with_the_agreed_payload_and_card(monkeypatch,
assert len([e for e in evs if e.type == "human.requested"]) == 1
async def test_a_bare_number_in_the_blind_rationale_is_marked(monkeypatch, shop):
"""盲算的理由里写了没有出处的数(「比上周多 20」):对照工件里的理由文档把它标成裸数字,审批卡据此醒目标出"""
name, _ = shop
Blind(monkeypatch, name, {"销售额": (RIGHT, "gmv")}, rationale="按口径说明统计,结果是 {ref},比上周多 20。")
run = await start(graph(name, BY_ORDER_DATE))
card = artifact_store.load((await pending(run.id)).payload["card_artifact"])
[mine] = card["metrics"][0]["blind"]["inputs"]
doc = mine["rationale_doc"]
bare = [seg["text"] for block in doc["blocks"] for unit in block["units"] for seg in unit["segments"]
if seg.get("issue") == "uncited_number"]
assert bare == ["20"] and mine["rationale"] == doc["markdown"]
async def test_choosing_the_original_leaves_no_gap(monkeypatch, shop, client):
name, source_id = shop
Blind(monkeypatch, name, {"销售额": (RIGHT, "gmv")})
@@ -436,7 +454,8 @@ async def test_choosing_the_original_leaves_no_gap(monkeypatch, shop, client):
assert item["value"] == {"column": "order_date"} and (item["source"], item["status"]) == ("human", "confirmed")
assert run.id in item["note"] and "复核销售额" in item["note"] and "销售额" in item["note"]
[wrote] = logs(evs, "dual_check_catalog")
assert wrote["written"] == [{"source": name, "table": "orders", "item": "business_date"}] and wrote["skipped"] == []
assert wrote["written"] == [{"source": name, "table": "orders", "item": "business_date",
"value": {"column": "order_date"}}] and wrote["skipped"] == []
async def test_choosing_the_blind_result_degrades_and_writes_the_catalog(monkeypatch, shop, client):
@@ -446,13 +465,25 @@ async def test_choosing_the_blind_result_degrades_and_writes_the_catalog(monkeyp
done = await decide(client, run.id, {"gmv": "b"})
assert done.status == "succeeded", done.error
out = dual_output(done, await events(run.id))
assert out["issuance_gaps"] == ["指标「销售额」双盲复算不一致,审核人选择了盲算结果"]
assert out["issuance_gaps"] == ["指标「销售额」两路结果不一致,审核人选择了盲算结果"]
issuance = done.output["_issuance"]
assert issuance["tier"] == "degraded"
assert issuance["gaps"] == ["双盲复算「复核销售额」:指标「销售额」双盲复算不一致,审核人选择了盲算结果"]
assert issuance["gaps"] == ["双盲复算「复核销售额」:指标「销售额」两路结果不一致,审核人选择了盲算结果"]
assert (await notes_of(source_id)).notes["business_date"]["value"] == {"column": "paid_date"}
async def test_a_dual_check_named_after_its_type_is_named_once_in_the_gap(monkeypatch, shop, client):
"""拖进画布没改名的双盲复算:缺口读成「双盲复算:指标「销售额」两路结果不一致……」,不写两遍「双盲复算」"""
name, _ = shop
Blind(monkeypatch, name, {"销售额": (RIGHT, "gmv")})
spec = graph(name, BY_ORDER_DATE)
next(n for n in spec["nodes"] if n["id"] == "dual")["data"]["label"] = "双盲复算"
run = await start(spec)
done = await decide(client, run.id, {"gmv": "neither"})
assert done.status == "succeeded", done.error
assert done.output["_issuance"]["gaps"] == ["双盲复算:指标「销售额」两路结果不一致,审核人判断两边都不对"]
async def test_neither_degrades_and_writes_nothing(monkeypatch, shop, client):
name, source_id = shop
Blind(monkeypatch, name, {"销售额": (RIGHT, "gmv")})
@@ -461,7 +492,7 @@ async def test_neither_degrades_and_writes_nothing(monkeypatch, shop, client):
assert done.status == "succeeded", done.error
evs = await events(run.id)
out = dual_output(done, evs)
assert out["issuance_gaps"] == ["指标「销售额」双盲复算不一致,审核人判断两边都不对"]
assert out["issuance_gaps"] == ["指标「销售额」两路结果不一致,审核人判断两边都不对"]
assert done.output["_issuance"]["tier"] == "degraded"
assert not logs(evs, "dual_check_catalog") and await notes_of(source_id) is None
@@ -476,7 +507,7 @@ async def test_a_missing_choice_is_recorded_as_no_choice(monkeypatch, shop, clie
evs = await events(run.id)
out = dual_output(done, evs)
assert out["choices"] == {} and "choice" not in out["metrics"][0]
assert out["issuance_gaps"] == ["指标「销售额」双盲复算不一致,审核人没有作出选择"]
assert out["issuance_gaps"] == ["指标「销售额」两路结果不一致,审核人没有作出选择"]
[resolved] = logs(evs, "dual_check_resolved")
assert resolved["choices"] == {} and resolved["gaps"] == out["issuance_gaps"]
assert "没有作出选择" in resolved["message"] and "两边都不对" not in resolved["message"]
@@ -505,7 +536,7 @@ async def test_on_reject_fail_stops_the_run(monkeypatch, shop, client):
assert done.status == "failed" and "审核人没有采用原流程的结果" in (done.error or "") and "销售额" in done.error
evs = await events(run.id)
assert [d["gaps"] for d in logs(evs, "dual_check_resolved")] == [
["指标「销售额」双盲复算不一致,审核人选择了盲算结果"]]
["指标「销售额」两路结果不一致,审核人选择了盲算结果"]]
# 审核人的选择照样写进了目录:那是对数据的判断,和运行停不停无关
entry = await notes_of(source_id)
assert entry.notes["business_date"]["value"] == {"column": "paid_date"} and entry.version == 1
@@ -581,9 +612,12 @@ async def test_a_confirmed_catalog_entry_is_not_overwritten(monkeypatch, shop, c
preview = (await pending(run.id)).payload["summary"][0]["catalog"]
assert preview["a"]["write"] is False and "无需重复写入" in preview["a"]["reason"]
assert preview["b"]["write"] is False and "已有人工确认过的说法「order_date」" in preview["b"]["reason"]
# 认出了事实、只是被目录挡下:照样说写到哪张表的哪一项、本来要写什么
assert {k: preview["b"][k] for k in ("source", "table", "item", "value")} == \
{"source": name, "table": "orders", "item": "business_date", "value": {"column": "paid_date"}}
done = await decide(client, run.id, {"gmv": "b"})
[wrote] = logs(await events(done.id), "dual_check_catalog")
assert wrote["written"] == []
assert wrote["written"] == [] and [s["reason"] for s in wrote["skipped"]] == [preview["b"]["reason"]]
assert "已有人工确认过的说法「order_date」" in wrote["skipped"][0]["reason"] and "paid_date" in wrote["skipped"][0]["reason"]
entry = await notes_of(source_id)
assert entry.version == 1 and entry.notes["business_date"]["value"] == {"column": "order_date"}
@@ -607,19 +641,95 @@ async def test_a_rejected_filter_is_not_written_back_along_with_another_metric(m
Blind(monkeypatch, name, {"销售额": (RIGHT, "gmv"), "订单金额": (no_test, "total")})
run = await start(graph(name, original, metrics=[GMV, total], dual={"definitions": {}}))
assert run.status == "interrupted", run.error
said = {s["metric_id"]: s["catalog"]["b"] for s in (await pending(run.id)).payload["summary"]}
payload = (await pending(run.id)).payload
said = {s["metric_id"]: s["catalog"]["b"] for s in payload["summary"]}
assert said["gmv"]["write"] is False and "曾被人工驳回" in said["gmv"]["reason"]
assert said["total"]["write"] is True and said["total"]["value"] == "is_test = 0"
done = await decide(client, run.id, {"gmv": "b", "total": "b"})
# 两个指标都落到有效记录条件上:合起来的结果另有预告,和真写一致
choices = {"gmv": "b", "total": "b"}
expected = card_said(payload, choices)
assert expected["gmv"]["write"] is False and expected["gmv"]["reason"] == said["gmv"]["reason"]
assert expected["total"] == {"write": True, "value": "is_test = 0", "item": "valid_filter"}
done = await decide(client, run.id, choices)
assert done.status == "succeeded", done.error
[wrote] = logs(await events(run.id), "dual_check_catalog")
assert wrote["written"] == [{"source": name, "table": "orders", "item": "valid_filter"}]
assert wrote["written"] == [{"source": name, "table": "orders", "item": "valid_filter", "value": "is_test = 0"}]
assert [s["reason"] for s in wrote["skipped"]] == [said["gmv"]["reason"]]
await assert_written_as_said(expected, wrote, source_id)
item = (await notes_of(source_id)).notes["valid_filter"]
assert item["value"] == said["total"]["value"] and (item["source"], item["status"]) == ("human", "confirmed")
assert "「订单金额」" in item["note"] and "「销售额」" not in item["note"]
def card_said(payload: dict, choices: dict) -> dict:
"""审批卡按这组选择会怎么预告(DualCheckApproval 的认法):选的那一边认出了写到哪,就看按当前的选择还有哪些指标
落到同一项上——只有它自己时照摘要里它的预告,几个时照对照工件 catalog_joint 里这几个成员的那一条。"""
summary = {s["metric_id"]: s["catalog"] for s in payload["summary"]}
joint = artifact_store.load(payload["card_artifact"]).get("catalog_joint") or []
def where(metric_id: str, choice: str):
side = summary[metric_id][choice]
return (side["source"], side["table"], side["item"]) if "table" in side else None
out = {}
for metric_id, choice in choices.items():
if choice not in ("a", "b"):
continue
target = where(metric_id, choice)
peers = sorted((m, c) for m, c in choices.items() if c in ("a", "b") and target and where(m, c) == target)
if len(peers) < 2:
out[metric_id] = summary[metric_id][choice]
continue
[entry] = [e for e in joint if sorted((m["metric_id"], m["choice"]) for m in e["members"]) == peers]
out[metric_id] = {**entry["outcomes"][metric_id], "item": entry["item"]}
return out
async def assert_written_as_said(said: dict, wrote: dict, source_id: str) -> None:
"""预告和真写逐字一致:说会记为什么的,目录里就是那个值、日志的 written 里有它;说不记的,原因就在日志的跳过原因里"""
entry = await notes_of(source_id)
for metric_id, side in said.items():
if side["write"]:
assert any(w["item"] == side["item"] and w["value"] == side["value"] for w in wrote["written"]), \
(metric_id, side, wrote)
assert entry.notes[side["item"]]["value"] == side["value"]
else:
assert side["reason"] in [s["reason"] for s in wrote["skipped"]], (metric_id, side, wrote)
#: 两个指标取自同一条查询的两列,都是按下单日筛的:盲算按支付日算,两个指标都只差时间列,落到 orders 的业务日期上
TWO_BY_ORDER_DATE = (f"SELECT SUM(amount) AS gmv, SUM(amount) AS total FROM orders "
f"WHERE status <> 'refunded' AND order_date {WEEK}")
TOTAL = {**GMV, "id": "total", "name": "订单金额", "expression": "cell(nodes.fetch, 0, 'total')"}
@pytest.mark.parametrize("choices", [
{"gmv": "a", "total": "b"}, # 说法不同:两个都不写
{"gmv": "b", "total": "b"}, # 说法一样:写一次
{"gmv": "a", "total": "a"},
{"gmv": "b", "total": "neither"}, # 只有一个落到这一项上:照它自己的预告
])
async def test_the_card_preview_is_what_the_decision_writes(monkeypatch, shop, client, choices):
"""图级:两个指标都只差时间列、落到数据目录的同一项上。审批卡按当前的选择预告的(摘要里各自的预告、对照工件里
合起来的预告),和裁决后真写进目录的逐字一致:写的值一样,不写的原因一句不差"""
name, source_id = shop
right_total = f"SELECT SUM(amount) AS total FROM orders WHERE status <> 'refunded' AND paid_date {WEEK}"
Blind(monkeypatch, name, {"销售额": (RIGHT, "gmv"), "订单金额": (right_total, "total")})
run = await start(graph(name, TWO_BY_ORDER_DATE, metrics=[GMV, TOTAL], dual={"definitions": {}}))
assert run.status == "interrupted", run.error
payload = (await pending(run.id)).payload
card = artifact_store.load(payload["card_artifact"])
assert len(card["catalog_joint"]) == 4, "两个指标两边都落在业务日期上:四种组合各预告一次"
said = card_said(payload, choices)
done = await decide(client, run.id, choices)
assert done.status == "succeeded", done.error
[wrote] = logs(await events(run.id), "dual_check_catalog")
await assert_written_as_said(said, wrote, source_id)
if choices == {"gmv": "a", "total": "b"}:
assert not said["gmv"]["write"] and said["gmv"] == {**said["total"], "item": "business_date"}
assert wrote["written"] == [] and "给出了不同的说法" in said["gmv"]["reason"]
async def test_a_metric_scope_filter_is_not_written_as_a_valid_filter(monkeypatch, shop, client):
"""「华东销售额」原流程按指标名只看华东,盲算漏了地区:审核人选原流程,但 region = '华东' 是这个指标自己的筛选
范围,写成表的有效记录条件,以后所有查订单表的工作流(包括下一次盲算)都会只看华东。"""
@@ -93,3 +93,9 @@ async def test_without_issuance_gaps_nothing_changes():
issuance = row.output["_issuance"]
assert issuance["tier"] == "formal"
assert issuance["gaps"] == []
async def test_a_node_named_after_its_type_is_named_once():
"""节点名就是类型的叫法(拖进画布没改名的):前缀只写一次「数据整形:」,不写成「数据整形「数据整形」:」"""
row = await finish(graph(gaps_node("recheck", "数据整形", ["两路结果不一致"])))
assert row.output["_issuance"]["gaps"] == ["数据整形:两路结果不一致"]
+42
View File
@@ -0,0 +1,42 @@
"""跑完之后的复核层认得双盲复算、反方审稿的 warn 日志代码(engine/review.py 的 WARN_CODES)。
没登记的代码一律按 degraded、kind 记成笼统的 warn:时间线复核和问数据页说不清是哪一种缺口,模型相同这种只关乎独立性、
不影响结论的提醒也会被当成缺口去叫一次复核模型。这里守住两类:
- 模型相同(dual_check_same_model、challenge_same_model)同 judge_same_model,核查模型不在价格目录中(verify_unpriced)
同 judge_unpriced,是配置上的提醒,不进信号;
- 触顶、不一致、无法复算、被推翻、未能反查、只查了部分数据源是缺口,按各自的 kind 记成 degraded。核查 agent 自己触顶
(verify_limit,双盲复算、反方审稿都会发)也是:步数上限只有它说。
"""
from __future__ import annotations
import pytest
from app.core.events import EventType
from app.engine import review as rv
OK_OUTPUT = {"结论": "本周订单 120 单。"}
def warn(message: str, code: str) -> dict:
return {"type": str(EventType.LOG), "node_id": "gate", "data": {"level": "warn", "message": message, "code": code}}
@pytest.mark.parametrize("code", ["dual_check_same_model", "challenge_same_model", "verify_unpriced"])
def test_configuration_reminders_do_not_trigger_review(code):
"""和 judge_same_model、judge_unpriced 一样:配置上的提醒,答案本身没受影响,不值得为它叫一次复核模型、
在答案上多一句降级说明"""
assert rv.WARN_CODES[code] == (code, rv.BENIGN)
assert rv.scan([warn("核查模型与原流程相同(「mx-1」)", code)], OK_OUTPUT) == []
@pytest.mark.parametrize("code", [
"dual_check_limit", "dual_check_disagreed", "dual_check_unchecked",
"challenge_limit", "challenge_refuted", "challenge_unreviewed", "challenge_sources_lost", "verify_limit",
])
def test_verify_gaps_are_degraded_signals_of_their_own_kind(code):
"""核查没做完、没通过:答案能用,但有缺口要说清是哪一种(kind 是代码本身,不是笼统的 warn)"""
assert rv.WARN_CODES[code] == (code, rv.DEGRADED)
[signal] = rv.scan([warn("核查的一条提醒", code)], OK_OUTPUT)
assert (signal.kind, signal.severity, signal.detail) == (code, rv.DEGRADED, "核查的一条提醒")
# 缺口不是断裂:不触发「重跑一次大概率能好」
assert not rv.should_retry([signal])
+17 -1
View File
@@ -91,8 +91,24 @@ V1 = {"明细": [["区域", "数量"], ["分区甲", 1], ["分区乙", 2]]}
V2 = {"明细": [["区域", "数量"], ["分区甲", 60], ["分区乙", 40]], "新增": [["时段", "客流"], ["上午", 7]]}
#: 十六进制里的数字换成字母:名字里不带数字
_NO_DIGITS = str.maketrans("0123456789", "ghijklmnop")
def uniq(prefix: str = "dv") -> str:
return f"{prefix}_{uuid.uuid4().hex[:10]}"
"""一个不会重复的源名,只有字母。
报错原文里有源名(「数据源「dv_…」」),不少断言靠「第二版的 100 不在原文里」证明没有改用当前版本。十六进制名字
每 500 个左右就有一个带着「100」(dv_3a100bc2ef),那几条断言就会无缘无故地失败——这正是
test_snapshot_file_deleted_after_tools_were_built 偶尔失败的原因,和并行、时序无关。名字里不带数字,数字的断言
才只看得见数据。"""
return f"{prefix}_{uuid.uuid4().hex[:12].translate(_NO_DIGITS)}"
def test_source_names_carry_no_digits():
"""守住 uniq 的约定:名字里一个数字都没有,报错原文里出现的数字只能来自数据(见 uniq 的说明)"""
names = {uniq() for _ in range(2000)}
assert len(names) == 2000 and not any(ch.isdigit() for name in names for ch in name.removeprefix("dv_"))
async def publish(name: str, sheets: dict[str, list[list]]) -> tuple[str, str]:
+45
View File
@@ -0,0 +1,45 @@
"""双盲复算、反方审稿各项配置的叫法放在 engine/labels.py(和别的节点一样),校验、执行报错和通用的 field_label /
option_label 取的是同一份。
以前双盲复算的字段叫法写在 schema.py 的 DUAL_CHECK_FIELD_LABEL 里(那时 labels.py 不归双盲复算的工作包),反方审稿的
写在 schema.py 的 CHALLENGE_LABEL 里:发布门禁、升级预览这些走 field_label 的地方拿不到它们,说成通用的「模型」「启用」,
和检查器上的「盲算模型」「启用双盲复算」对不上。和检查器(lib/terms.ts 的 DUAL_CHECK_TEXT.field、CHALLENGE_TEXT.field
与下拉选项)逐项比对在 check-verify、check-challenge 里做。
"""
from __future__ import annotations
from app.engine import labels, schema
def test_the_field_labels_live_in_labels_and_schema_reuses_them():
assert schema.DUAL_CHECK_FIELD_LABEL is labels.DUAL_CHECK_FIELD_LABEL
assert schema.CHALLENGE_LABEL is labels.CHALLENGE_LABEL
assert schema.DUAL_CHECK_ON_REJECT_LABEL is labels.DUAL_CHECK_ON_REJECT_LABEL
assert schema.CHALLENGE_ON_REFUTED_LABEL is labels.CHALLENGE_ON_REFUTED_LABEL
# 每个可选的取值都有叫法,叫法里没有多出来的取值
assert set(labels.DUAL_CHECK_ON_REJECT_LABEL) == set(schema.DUAL_CHECK_ON_REJECT)
assert set(labels.CHALLENGE_ON_REFUTED_LABEL) == set(schema.CHALLENGE_ON_REFUTED)
def test_field_label_names_the_fields_of_the_two_nodes_like_the_inspector():
assert labels.field_label("model", "dual_check") == "盲算模型"
assert labels.field_label("provider", "dual_check") == "盲算模型 · 接入"
assert labels.field_label("enabled", "dual_check") == "启用双盲复算"
assert labels.field_label("on_reject", "dual_check") == "未采用原流程时"
assert labels.field_label("model", "challenge") == "反方模型"
assert labels.field_label("on_refuted", "challenge") == "结论被推翻时"
assert labels.field_label("max_claims", "challenge") == "最多审几句"
# 别的节点上的同名字段照旧
assert labels.field_label("model", "agent") == "模型"
for key, words in labels.DUAL_CHECK_FIELD_LABEL.items():
assert labels.field_label(key, "dual_check") == words, key
for key, words in labels.CHALLENGE_LABEL.items():
assert labels.field_label(key, "challenge") == words, key
def test_option_label_uses_the_short_words_of_the_dropdowns():
assert labels.option_label("on_refuted", "degrade") == "出具降档"
assert labels.option_label("on_refuted", "flag") == "只标注"
assert labels.option_label("on_refuted", "fail") == "报错并停止运行"
assert labels.option_label("on_reject", "degrade") == "计入缺口、出具降档"
assert labels.option_label("on_reject", "fail") == "判为失败"