Merge pull request #6 from ThearchyHelios/feat/evidence-phase3

可点击证据第三期:表名字段名和引文也点得开、可疑实体与结论句、受管门禁 G1–G5、证据审计导出
This commit is contained in:
ThearchyHelios
2026-09-29 11:55:01 +08:00
committed by GitHub
65 changed files with 15316 additions and 378 deletions
+36 -10
View File
@@ -190,19 +190,33 @@ NODE_REFERENCE = """\
写 2 只剩「2%」,0.004 这样的小比率还会显示不出来、报告引用不了。on_missing 默认 fail(缺输入整张卡失败);写 null 则缺输入的指标记为空值,交给出具契约判档
取 tool 节点查库的结果用 cell(nodes.取数节点id, 行, '列名')(行号从 0 数,列按列名):cell() 直接认查询工具
交回的结果,出处精确到快照里的那一格,不用再接 transform 解析。取 agent 的字段写 vars.<agent 的 assign_to>.字段
- report:报告撰写(带引用)。config: {instructions, system, metrics_from:[口径卡id], on_violation, max_repairs, assign_to}
- report:报告撰写(带引用)。config: {instructions, system, metrics_from:[口径卡id], numbers, on_violation, max_repairs,
claims, entities, assign_to}
**凡是给人看的、带数字的报告、分析、结论,都用 report,不要用 llm**。它自动收集上游的口径卡指标、查询结果
(tool 节点和 agent 查过的库,按查询先后编成 Q1、Q2…)和运行输入,模型只能写引用标记,数值由系统从证据里
取出来渲染;模型自己写的数字会被打回重写:
(tool 节点和 agent 查过的库,按查询先后编成 Q1、Q2…)、查库当时的表结构、知识库检索(按先后编成 K1、K2…)
和运行输入,模型只能写引用标记,数值由系统从证据里取出来渲染;模型自己写的数字会被打回重写:
- [[m:指标id]]:口径卡指标。比率、增幅、占比、差值这类派生计算只能先在口径卡里登记再引用
- [[v:Q1.r0.列名]]:第 1 次查询结果里第 0 行那一列的原值(行号从 0 数)
- [[table:Q1 cols=列a,列b rows=0-4]]:把查询结果的几行几列原样生成表格,每一格都点得开出处
- [[i:输入字段]]:运行输入
metrics_from 不写就取所有上游口径卡;on_violation 不写时探索运行是 flag(标出来、照常产出),正式运行是 fail
- [[t:表名]]、[[c:表名.字段名]]:表和字段,点得开是哪次查询用到的、表结构里是什么类型。写在反引号里的名字
(`orders`)也会被核对:只写本次运行查过的数据源里真实存在的名字,表结构、查询、结果列里都没有的会被标成
「可能是编造的名字」(受管模板的正式出具因此降档)
- [[q:K1|逐字引文]]:知识库检索的原话,必须和命中片段里的字逐字一致(空白不计,至少 4 个字),改一个字就算
没有出处;只是拿检索当依据时写 [[see:K1]]。要引原话,在 report 前面接 retrieve 节点
- 句末的 [[see:m:指标id,Q1,K1]]:这句结论的依据,不显示
metrics_from 不写就取所有上游口径卡;on_violation 不写时探索运行是 flag(标出来、照常产出),正式运行是 fail;
numbers 默认 strict(裸数字算违规)。claims 是结论句策略:off(默认,不管)/ require_citation(每句结论都要
挂引用,没挂的计入缺口、出具降档);judge(模型裁判)在后续版本支持,现在写了整张图跑不起来。entities 默认 link
(核对表名、字段名),报告里根本不提表名时才写 off
- output 的出具契约:config.contract = {report_from: 报告节点id, metrics_from:[口径卡id],
required:[必需指标id], expected:[期望指标id], strict: true}。成果字段写 {{ nodes.报告节点id.text }},
前后不要拼别的字——拼了就没法逐段对应证据,出具会降档。受管模板的报告要直接引用查询单元格
([[v:]] / [[table:]])的,契约里写 cells: true,否则这些引用按解析不了算
要按受管级别发布的模板(发布前检查不过就发不出去):报告节点写明 numbers: "strict"、on_violation: "fail"、
claims: "require_citation";契约用 report_from(受管级别不收 narrative),成果字段只取报告节点的正文;给口径卡
供数的 agent 配 output_schema 和 cite_fields: true,code 节点喂口径卡的必须是真取数(evidence_role: "source"),
否则把计算写进口径卡;llm、agent、协作团队写的文字不能绕过报告节点直接进出口,llm 也不能喂口径卡
搭图规则(有数字结论时必须遵守):
1. 取数(tool 节点查库;或者 agent 查库并配 output_schema + cite_fields)→ 口径卡 → report → output
@@ -304,6 +318,10 @@ def _user_message(payload: GenerateIn, *, patch: bool) -> str:
"直接引用 agent 查过的单元格,不必经过口径卡;要比率、增幅再在 agent 和 report 之间加口径卡,agent 配 "
"output_schema + cite_fields,口径卡读 vars.<assign_to>.字段。出口字段写 {{ nodes.report的id.text }}\n"
"- 只问表结构、清单(有哪些表、有哪些字段、列出名单)的,不加 report,agent → output 就行\n"
# 三期起报告会核对写出来的表名、字段名:instructions 里让它「列出用到的表」时,编出来的名字会被标出来
"- report 提到表名、字段名时写 [[t:表名]] / [[c:表名.字段名]] 或放进反引号,只写 agent 真查过的名字,"
"编造的会被标出来;要引用知识库原话就在 report 前接 retrieve,用 [[q:K1|原话]] 逐字引用。问数是探索运行,"
"report 的 claims 不用写\n"
"- 用户要的是周报、要三档出具时,按搭图规则配口径卡和 report_from 契约\n"
"- 不要用 transform 解析 agent / llm 的文字:要把 agent 查到的数交给下游,就给它配 output_schema + "
"cite_fields\n"
@@ -968,7 +986,7 @@ def _sse(obj: dict[str, Any]) -> str:
def _blocking_issues(
nodes: dict[str, dict[str, Any]], edges: list[dict[str, Any]], scope: set[str] | None = None,
*, sources: list[Any] | None = None, baseline: list[dict[str, Any]] | None = None,
level: str | None = None,
level: str | None = None, defaults: dict[str, Any] | None = None,
) -> list[dict[str, Any]] | None:
"""按运行时同一套校验,挑出会挡住运行的问题。图本身不成形时返回 None。
@@ -983,9 +1001,13 @@ def _blocking_issues(
每条是 ValidationIssue 的字典形状(node_id、field 都在),和 final.issues 一样:
界面靠 node_id 定位卡片、靠 field 落到具体的输入框,不用从一行字里抠节点 id。
defaults 是全图默认(graph.defaults):报告撰写节点的 numbers / claims、审批策略都跟随它,不带上的话
自查看到的问题和真正的门禁对不上。
"""
try:
spec = GraphSpec.model_validate({"nodes": list(nodes.values()), "edges": edges})
spec = GraphSpec.model_validate({"nodes": list(nodes.values()), "edges": edges,
**({"defaults": defaults} if isinstance(defaults, dict) else {})})
except Exception: # noqa: BLE001 - 结构不合法留给收尾那一步报
return None
out = ([i.model_dump() for i in validate_graph(spec).issues if i.level == "error"]
@@ -1609,9 +1631,12 @@ _ASSIST_RULES = """\
- 不许降低要求:不删节点、不删连线,add_node 不许用图里已有的节点 id(那等于把旧节点删了重建),也不换节点类型;
不删掉或清空出具契约,不删 required 里的指标,不把 strict 改成 false,不往 allow_numbers 里加数,不替人打开 cells;
审批策略只能往严里改(每次调用都审批 > 仅危险工具需要审批 > 全部自动放行),不放宽、不删掉,更不改成 never;
不取消子工作流钉住的版本;发布级别不是你能改的。结构要动(比如换掉协作团队),输出 question 说明怎么拆,不要自己动手
不取消子工作流钉住的版本;发布级别不是你能改的。结构要动(比如换掉协作团队),输出 question 说明怎么拆,不要自己动手;
报告撰写节点的 numbers、on_violation、claims 只能往严里改(numbers: strict、on_violation: fail、
claims: require_citation),不放宽、不删掉;契约里的 claims 也一样;不关 agent 的 cite_fields,不删它的 output_schema
- 要人拿主意的,不替人选、不编:required 该包括哪些指标、几张口径卡该用哪张、钉哪个版本、协作团队拆成哪几个
固定步骤……每一处输出一行 {"op":"question","node_id":"节点 id","text":"要问的话,把候选列出来"},那一处不改
固定步骤、沙箱代码节点是在取数还是在计算(不许自己把 evidence_role 设成 source)……每一处输出一行
{"op":"question","node_id":"节点 id","text":"要问的话,把候选列出来"},那一处不改
- 只修下面列出的问题,别的不要动
- 可用操作只有 update_node(config 只写要改的字段,没写的原样保留)、add_node、add_edge 和 question;
不要输出 remove_node、remove_edge、reply
@@ -1648,7 +1673,8 @@ async def assist_publish_fix(
before_nodes = {str(n["id"]): n for n in copy.deepcopy(base.get("nodes") or []) if isinstance(n, dict)}
before_edges = [e for e in copy.deepcopy(base.get("edges") or []) if isinstance(e, dict)]
baseline = list(copy.deepcopy(before_nodes).values())
errors = _blocking_issues(before_nodes, before_edges, sources=sources, baseline=baseline, level=level) or []
errors = _blocking_issues(before_nodes, before_edges, sources=sources, baseline=baseline, level=level,
defaults=base.get("defaults")) or []
out: dict[str, Any] = {"accepted": False, "reason": None, "summary": "", "questions": [], "graph": graph,
"ops": []}
if not errors:
@@ -1705,7 +1731,7 @@ async def assist_publish_fix(
proposed = _proposed_graph(base, nodes, edges)
reasons = forbidden_changes(base, proposed, tried)
after = _blocking_issues(_nodes_by_id(proposed), proposed.get("edges") or [], sources=sources,
baseline=baseline, level=level)
baseline=baseline, level=level, defaults=proposed.get("defaults"))
if reasons:
out["reason"] = "Copilot 的修改降低了要求,已作废:" + ";".join(reasons)
elif after is None:
File diff suppressed because it is too large Load Diff
+3 -2
View File
@@ -444,11 +444,12 @@ async def autofix(
bad = _unreadable(payload.level, e)
return {"graph": graph, "changes": [], "ops": [], "applied": [], "rejected": [
{"fix_id": fid, "reason": "工作流的结构读不懂,没法应用修复"} for fid in payload.apply],
"remaining": bad["issues"], "fixes": [], "assist": None, "ok": False}
"remaining": bad["issues"], "fixes": [], "handoff": [], "assist": None, "ok": False}
versions, cards = await _gate_context(session, spec)
out = apply_fixes(graph, payload.apply, payload.choices, level=payload.level, versions=versions, cards=cards)
out["assist"] = None
if payload.assist:
# 人在选项里选了「交给 Copilot」(handoff),和点「交给 Copilot」是一回事
if payload.assist or out["handoff"]:
if out["ok"]:
out["assist"] = {"ok": True, "summary": "修复之后已经没有挡住发布的错误,没有再交给 Copilot",
"questions": []}
+3 -2
View File
@@ -499,13 +499,14 @@ async def run_query(
时限没指定时按数据源自己的配置(query_timeout),由数据库执行(_server_deadline)。
"""
limits = limits or QueryLimits(timeout_seconds=query_timeout(source))
statement = check(sql, readonly=bool(source.readonly), source_name=source.name)
kind = (source.kind or "").lower()
# 按数据源的方言扫:MySQL 的反斜杠转义、PG 的 $$ 字符串,守卫要和数据库读得一样
statement = check(sql, readonly=bool(source.readonly), source_name=source.name, dialect=kind)
engine = await engines.get(source)
started = time.perf_counter()
if not source.readonly and is_write(statement):
if not source.readonly and is_write(statement, dialect=kind):
return await _run_write(engine, kind, statement, limits, started)
async def _read(conn: AsyncConnection) -> tuple[list[str], list[list[Any]], bool, dict[str, str]]:
+292 -78
View File
@@ -16,6 +16,12 @@
刻意不引入 sqlparse 之类的重型依赖:完整解析 SQL 方言是个无底洞,而这里需要的
判断很窄——"第一个关键字是什么"和"有没有第二条语句"。窄问题用窄解法,
把复杂度留在能看懂的范围内。
所有判断都建立在同一次扫描上(_tokens):一遍从左到右,同时认注释、字符串、标识符引号和
分号。以前先用正则去注释、再认字符串,字符串里的 `--`、`/*` 被当成注释删掉,执行的 SQL
和写的不一样。扫描要和数据库的读法一致,而读法跟方言、跟服务器设置走(MySQL 的
NO_BACKSLASH_ESCAPES / ANSI_QUOTES,PG 的 standard_conforming_strings)。守卫不知道
服务器开了哪个,就按每种可能的读法各扫一遍,读出来不一致的直接拒掉。
"""
from __future__ import annotations
@@ -35,9 +41,6 @@ _ALWAYS_DENIED = frozenset({
"attach", "detach",
})
_LINE_COMMENT = re.compile(r"--[^\n]*")
_BLOCK_COMMENT = re.compile(r"/\*.*?\*/", re.S)
# 首关键字是查询、正文却能写的语句。只看首关键字的话它们全都能混过去:
# WITH k AS (SELECT 1) DELETE FROM t SQLite / MySQL / PG 都认
# WITH x AS (DELETE … RETURNING *) SELECT … PG 的数据修改 CTE
@@ -87,89 +90,265 @@ class QueryLimits:
timeout_seconds: float = 30
def strip_comments(sql: str) -> str:
"""去掉注释再判断。否则 `/*x*/ DROP TABLE t` 的首关键字会被看成注释内容。"""
return _BLOCK_COMMENT.sub(" ", _LINE_COMMENT.sub(" ", sql))
@dataclass(frozen=True)
class _Reading:
"""数据库怎么切分一段 SQL:哪些字符开始注释、字符串、带引号的标识符。"""
name: str
#: '…' 里的反斜杠转义下一个字符(MySQL 默认;PG 关了 standard_conforming_strings 时)
backslash: bool = False
#: "…" 是字符串而不是标识符(MySQL 没开 ANSI_QUOTES 时),跟 '…' 一样认反斜杠
dq_string: bool = False
#: `…` 是标识符(MySQL、SQLite)
backtick: bool = False
#: […] 是标识符(SQLite)
brackets: bool = False
#: $$…$$、$tag$…$tag$ 是字符串(PostgreSQL)
dollar: bool = False
#: E'…' 里认反斜杠转义(PostgreSQL)
e_strings: bool = False
#: /* /* */ */ 可以嵌套(PostgreSQL)
nested: bool = False
#: # 开始行注释(MySQL)
hash_comment: bool = False
#: q'[…]' 这类自选定界符的字符串(Oracle)
q_quote: bool = False
def split_statements(sql: str) -> list[str]:
"""按分号切语句,但要认得字符串字面量里的分号。
_GENERIC = _Reading("generic", backtick=True)
_SQLITE = _Reading("sqlite", backtick=True, brackets=True)
_PG = _Reading("postgresql", dollar=True, e_strings=True, nested=True)
_PG_OLD = _Reading("postgresql-backslash", backslash=True, dollar=True, e_strings=True, nested=True)
_MYSQL = _Reading("mysql", backslash=True, dq_string=True, backtick=True, hash_comment=True)
_MYSQL_NBE = _Reading("mysql-no-backslash", dq_string=True, backtick=True, hash_comment=True)
_MYSQL_ANSI = _Reading("mysql-ansi", backslash=True, backtick=True, hash_comment=True)
_MYSQL_BOTH = _Reading("mysql-ansi-no-backslash", backtick=True, hash_comment=True)
_ORACLE = _Reading("oracle", q_quote=True)
`SELECT ';'` 只是一条语句。简单 split(";") 会把它切成两半,然后把后半截
当成"第二条语句"拒掉——那是误杀,比漏杀更让人无从下手。
"""
out: list[str] = []
buf: list[str] = []
quote: str | None = None
# 每种方言的读法:第一种是服务器默认设置下的读法,其余是服务器可能开着的其他设置。
# 守卫看不到服务器的设置,所以每种都要扫,读出来不一样就拒
_READINGS: dict[str, tuple[_Reading, ...]] = {
"sqlite": (_SQLITE,),
"postgresql": (_PG, _PG_OLD),
"postgres": (_PG, _PG_OLD),
"mysql": (_MYSQL, _MYSQL_NBE, _MYSQL_ANSI, _MYSQL_BOTH),
"mariadb": (_MYSQL, _MYSQL_NBE, _MYSQL_ANSI, _MYSQL_BOTH),
"oracle": (_ORACLE,),
}
# 不知道方言时每种读法都算上
_ALL_READINGS: tuple[_Reading, ...] = (_GENERIC, *dict.fromkeys(
r for readings in _READINGS.values() for r in readings))
_AMBIGUOUS = (
"这条 SQL 里的引号、反斜杠或注释,在数据库的不同设置下会读成不同的语句"
"(比如 MySQL 的 NO_BACKSLASH_ESCAPES、ANSI_QUOTES)。"
"字符串里的单引号写成两个单引号 '',不要用反斜杠转义;注释删掉再试"
)
_DOLLAR_TAG = re.compile(r"\$(?:[^\W\d]\w*)?\$")
_Q_CLOSE = {"[": "]", "(": ")", "{": "}", "<": ">"}
def _readings(dialect: str | None) -> tuple[_Reading, ...]:
return _READINGS.get((dialect or "").lower(), _ALL_READINGS)
def _reading(dialect: str | None) -> _Reading:
"""服务器默认设置下的读法。不知道方言时用通用读法(标准 SQL 加反引号)。"""
return _READINGS.get((dialect or "").lower(), (_GENERIC,))[0]
def _word_char(ch: str) -> bool:
return ch.isalnum() or ch in "_$"
def _prefixed(sql: str, i: int, letters: str) -> bool:
"""i 处的引号前面紧挨着一个前缀字母(E'…'),而且这个字母不是某个标识符的尾巴。"""
return i >= 1 and sql[i - 1] in letters and not (i >= 2 and _word_char(sql[i - 2]))
def _q_quote_at(sql: str, i: int) -> bool:
"""Oracle 的 q'…' / nq'…':引号后面紧跟的字符是定界符,不能是空白。"""
if i + 1 >= len(sql) or sql[i + 1].isspace() or not (i >= 1 and sql[i - 1] in "qQ"):
return False
j = i - 2
if j >= 0 and sql[j] in "nN":
j -= 1
return not (j >= 0 and _word_char(sql[j]))
@dataclass(frozen=True)
class _Token:
#: code / comment / string / ident / sep
kind: str
text: str
#: 开、闭定界符的长度,抹内容时保留它们
open: int = 0
close: int = 0
def _quoted_end(sql: str, i: int, quote: str, backslash: bool) -> tuple[int, int]:
"""从 i 处的开引号往后找闭引号。返回 (结束位置, 闭引号长度);没闭合的一直到末尾。"""
j = i + 1
while j < len(sql):
ch = sql[j]
if backslash and ch == "\\":
j += 2
continue
if ch == quote:
if j + 1 < len(sql) and sql[j + 1] == quote: # 两个引号是转义,不是结束
j += 2
continue
return j + 1, 1
j += 1
return len(sql), 0
def _tokens(sql: str, reading: _Reading) -> list[_Token]:
"""一遍扫完:注释、字符串、带引号的标识符、分号各自成段,其余是代码。"""
out: list[_Token] = []
n = len(sql)
code = 0 # 当前这段代码的起点
i = 0
text = strip_comments(sql)
while i < len(text):
ch = text[i]
if quote:
buf.append(ch)
if ch == quote:
# SQL 里连续两个引号是转义('' 表示一个单引号)
if i + 1 < len(text) and text[i + 1] == quote:
buf.append(text[i + 1])
i += 2
continue
quote = None
elif ch in ("'", '"', "`"):
quote = ch
buf.append(ch)
def flush(upto: int) -> None:
if upto > code:
out.append(_Token("code", sql[code:upto]))
while i < n:
ch = sql[i]
nxt = sql[i + 1] if i + 1 < n else ""
prev = sql[i - 1] if i else ""
token: _Token | None = None
end = i
if (ch == "-" and nxt == "-") or (ch == "#" and reading.hash_comment):
end = sql.find("\n", i)
end = n if end < 0 else end
token = _Token("comment", sql[i:end])
elif ch == "/" and nxt == "*":
if reading.nested:
depth, end = 0, i
while end < n:
if sql.startswith("/*", end):
depth, end = depth + 1, end + 2
elif sql.startswith("*/", end):
depth, end = depth - 1, end + 2
if depth == 0:
break
else:
end += 1
end = min(end, n)
else:
close = sql.find("*/", i + 2)
end = n if close < 0 else close + 2
token = _Token("comment", sql[i:end])
elif ch == ";":
stmt = "".join(buf).strip()
end = i + 1
token = _Token("sep", ";")
elif ch == "'":
if reading.q_quote and _q_quote_at(sql, i):
closer = _Q_CLOSE.get(nxt, nxt) + "'"
found = sql.find(closer, i + 2)
end, tail = (n, 0) if found < 0 else (found + 2, 2)
token = _Token("string", sql[i:end], 2, tail)
else:
e_string = reading.e_strings and _prefixed(sql, i, "eE")
end, tail = _quoted_end(sql, i, "'", reading.backslash or e_string)
token = _Token("string", sql[i:end], 1, tail)
elif ch == '"':
end, tail = _quoted_end(sql, i, '"', reading.backslash and reading.dq_string)
token = _Token("string" if reading.dq_string else "ident", sql[i:end], 1, tail)
elif ch == "`" and reading.backtick:
end, tail = _quoted_end(sql, i, "`", False)
token = _Token("ident", sql[i:end], 1, tail)
elif ch == "[" and reading.brackets:
close = sql.find("]", i + 1)
end, tail = (n, 0) if close < 0 else (close + 1, 1)
token = _Token("ident", sql[i:end], 1, tail)
elif ch == "$" and reading.dollar and not (prev and _word_char(prev)):
tag = _DOLLAR_TAG.match(sql, i)
if tag:
found = sql.find(tag.group(0), tag.end())
width = len(tag.group(0))
end, tail = (n, 0) if found < 0 else (found + width, width)
token = _Token("string", sql[i:end], width, tail)
if token is None:
i += 1
continue
flush(i)
out.append(token)
i = code = end
flush(n)
return out
def _joined(tokens: list[_Token]) -> str:
"""注释换成一个空格,其余原样拼回去。空格不能省:'a'/**/'b' 删成 'a''b' 就成了一个字符串。"""
return "".join(" " if t.kind == "comment" else t.text for t in tokens)
def _statements(sql: str, reading: _Reading) -> list[str]:
out: list[str] = []
buf: list[_Token] = []
for token in [*_tokens(sql, reading), _Token("sep", ";")]:
if token.kind == "sep":
stmt = _joined(buf).strip()
if stmt:
out.append(stmt)
buf = []
else:
buf.append(ch)
i += 1
tail = "".join(buf).strip()
if tail:
out.append(tail)
buf.append(token)
return out
def first_verb(sql: str) -> str:
def strip_comments(sql: str, *, dialect: str | None = None) -> str:
"""去掉注释再判断。否则 `/*x*/ DROP TABLE t` 的首关键字会被看成注释内容。
字符串、带引号的标识符里的 `--`、`/*` 不是注释,原样保留。
"""
return _joined(_tokens(sql, _reading(dialect)))
def split_statements(sql: str, *, dialect: str | None = None) -> list[str]:
"""按分号切语句(去掉注释),但要认得字符串字面量里的分号。
`SELECT ';'` 只是一条语句。简单 split(";") 会把它切成两半,然后把后半截
当成"第二条语句"拒掉——那是误杀,比漏杀更让人无从下手。
"""
return _statements(sql, _reading(dialect))
def first_verb(sql: str, *, dialect: str | None = None) -> str:
"""取语句的首关键字(小写)。括号开头的 `(SELECT ...)` 也要认出来。"""
text = strip_comments(sql).lstrip().lstrip("(").lstrip()
text = strip_comments(sql, dialect=dialect).lstrip().lstrip("(").lstrip()
match = re.match(r"[A-Za-z_]+", text)
return match.group(0).lower() if match else ""
def _blank_quoted(sql: str, *, identifiers: bool = True) -> str:
def _blanked(tokens: list[_Token], *, identifiers: bool) -> str:
out: list[str] = []
for t in tokens:
if t.kind == "comment":
out.append(" ")
elif t.kind == "string" or (t.kind == "ident" and identifiers):
inner = len(t.text) - t.open - t.close
out.append(t.text[:t.open] + " " * inner + t.text[len(t.text) - t.close:])
else:
out.append(t.text)
return "".join(out)
def _blank_quoted(sql: str, *, identifiers: bool = True, dialect: str | None = None) -> str:
"""把引号里的内容换成空格:字符串 'please delete' 和标识符 "update" 都不是关键字。
identifiers=False 时只抹字符串、保留 "…" 和 `…`——找函数调用时要看得见
被引号括起来的函数名。
"""
quotes = ("'", '"', "`") if identifiers else ("'",)
out: list[str] = []
quote: str | None = None
i = 0
while i < len(sql):
ch = sql[i]
if quote:
if ch == quote:
if i + 1 < len(sql) and sql[i + 1] == quote: # '' 是转义,不是结束
out.append(" ")
i += 2
continue
quote = None
out.append(ch)
else:
out.append(" ")
else:
if ch in quotes:
quote = ch
out.append(ch)
i += 1
return "".join(out)
return _blanked(_tokens(sql, _reading(dialect)), identifiers=identifiers)
def _pragma_write_reason(stmt: str) -> str | None:
match = _PRAGMA.match(strip_comments(stmt).strip())
def _pragma_write_reason(stmt: str, reading: _Reading = _GENERIC) -> str | None:
match = _PRAGMA.match(_joined(_tokens(stmt, reading)).strip())
if not match:
return "看不出这条 PRAGMA 读的是什么"
name, rest = match.group(1).lower(), match.group(2).strip()
@@ -183,40 +362,63 @@ def _pragma_write_reason(stmt: str) -> str | None:
)
def _write_reason(stmt: str) -> str | None:
def _write_reason(stmt: str, reading: _Reading = _GENERIC) -> str | None:
"""这条语句会不会写。会写就返回一句说给模型听的原因,不会写返回 None。
首关键字只是第一道:以查询开头的语句正文照样能写(见 _HIDDEN_WRITE),
所以 SELECT / WITH / EXPLAIN 还要把正文扫一遍。SHOW / DESCRIBE / VALUES
结构上写不了,不扫——`SHOW CREATE TABLE` 里的 CREATE 不是在建表。
"""
verb = first_verb(stmt)
tokens = _tokens(stmt, reading)
text = _joined(tokens).lstrip().lstrip("(").lstrip()
match = re.match(r"[A-Za-z_]+", text)
verb = match.group(0).lower() if match else ""
if verb not in _READONLY_VERBS:
return f"{verb.upper()} 会修改数据"
if verb == "pragma":
return _pragma_write_reason(stmt)
return _pragma_write_reason(stmt, reading)
if verb in ("select", "with", "explain"):
text = strip_comments(stmt)
hidden = _HIDDEN_WRITE.search(_blank_quoted(text))
hidden = _HIDDEN_WRITE.search(_blanked(tokens, identifiers=True))
if hidden:
word = " ".join(hidden.group(1).split()).upper()
return f"语句里有 {word}——以查询开头的语句也能写数据"
call = _SIDE_EFFECT_CALL.search(_blank_quoted(text, identifiers=False))
call = _SIDE_EFFECT_CALL.search(_blanked(tokens, identifiers=False))
if call:
return f"{call.group(1)}() 有副作用"
return None
def check(sql: str, *, readonly: bool, source_name: str = "") -> str:
def _write_reason_any(stmt: str, readings: tuple[_Reading, ...]) -> str | None:
"""按每种读法都判一遍:哪种读法下会写,就当它会写。"""
reasons = [_write_reason(stmt, r) for r in readings]
if reasons[0]:
return reasons[0]
other = next((r for r in reasons[1:] if r), None)
return f"数据库换一种设置来读的话,{other}。{_AMBIGUOUS}" if other else None
def _one_view(sql: str, readings: tuple[_Reading, ...]) -> list[str] | None:
"""每种读法切出来的语句都一样时返回它,不一样返回 None。"""
views = [_statements(sql, r) for r in readings]
return views[0] if all(v == views[0] for v in views[1:]) else None
def check(sql: str, *, readonly: bool, source_name: str = "", dialect: str | None = None) -> str:
"""校验并返回规范化后的单条 SQL。不通过就抛 SqlRejected。
返回值是去掉尾分号的那一条语句——调用方应该执行这个返回值,
而不是原始输入,否则守卫等于没做。
返回值是去掉注释和尾分号的那一条语句——调用方应该执行这个返回值,
而不是原始输入,否则守卫等于没做。字符串、带引号的标识符原样保留。
dialect 是数据源的 kind(sqlite / postgresql / mysql / oracle …)。
不知道的话按所有方言、所有设置的读法各判一遍,任何一种读法不过都拒。
"""
if not sql or not sql.strip():
raise SqlRejected("SQL 是空的")
statements = split_statements(sql)
readings = _readings(dialect)
statements = _one_view(sql, readings)
if statements is None:
raise SqlRejected(_AMBIGUOUS)
if not statements:
raise SqlRejected("SQL 里没有可执行的语句")
if len(statements) > 1:
@@ -226,7 +428,11 @@ def check(sql: str, *, readonly: bool, source_name: str = "") -> str:
)
stmt = statements[0]
verb = first_verb(stmt)
# 交给数据库的就是 stmt:它在每种读法下都得还是完整的一条,没有注释、没有分号
if any(_tokens_have(stmt, r, ("comment", "sep")) for r in readings):
raise SqlRejected(_AMBIGUOUS)
verb = first_verb(stmt, dialect=dialect)
if not verb:
raise SqlRejected("看不出这条 SQL 要做什么(没有识别到关键字)")
@@ -237,7 +443,7 @@ def check(sql: str, *, readonly: bool, source_name: str = "") -> str:
)
if readonly:
reason = _write_reason(stmt)
reason = _write_reason_any(stmt, readings)
if reason:
where = f"「{source_name}」" if source_name else "这个数据源"
raise SqlRejected(
@@ -249,11 +455,19 @@ def check(sql: str, *, readonly: bool, source_name: str = "") -> str:
return stmt
def is_write(sql: str) -> bool:
def _tokens_have(sql: str, reading: _Reading, kinds: tuple[str, ...]) -> bool:
return any(t.kind in kinds for t in _tokens(sql, reading))
def is_write(sql: str, *, dialect: str | None = None) -> bool:
"""是不是写操作。用来决定要不要走人工审批。
和只读判定用同一份 _write_reason:审批这道门要是只看首关键字,
`WITH … DELETE` 在可写源上就能不经审批直接写。
`WITH … DELETE` 在可写源上就能不经审批直接写。几种读法切出来不一样的,
一律当写:看不清的交给人批。
"""
statements = split_statements(sql)
return not statements or any(_write_reason(s) is not None for s in statements)
readings = _readings(dialect)
statements = _one_view(sql, readings)
if not statements:
return True
return any(_write_reason(s, r) is not None for s in statements for r in readings)
+254 -27
View File
@@ -18,20 +18,26 @@
from __future__ import annotations
import copy
import re
from typing import Any, Callable
from pydantic import BaseModel
from app.engine.governance import publish_issues
from app.engine.schema import GraphNode, GraphSpec, NodeType, _ancestors
from app.engine.governance import REPORT_POLICY, exit_text_fields, publish_issues
from app.engine.schema import REPORT_CLAIMS, GraphNode, GraphSpec, NodeType, _ancestors, claims_problem
#: 被选中的修复按这个顺序应用:先补契约骨架,再补契约里的各项,最后是节点上的开关。
#: 前面的修复可能改变后面的候选(metrics_from 决定 required 从哪几张卡里挑)
#: 前面的修复可能改变后面的候选(metrics_from 决定 required 从哪几张卡里挑,report_from 决定
#: 成果字段默认接哪个报告撰写节点)
_ORDER = (
"governed.no_contract", "contract.metrics_from_missing", "contract.metrics_from_invalid",
"contract.report_from_missing", "contract.report_from_invalid", "report.metrics_from_invalid",
"contract.required_missing", "contract.strict_off", "governed.agent_approval_never",
"contract.report_from_missing", "contract.report_from_invalid", "governed.report_from_required",
"governed.exit_text_source", "report.metrics_from_invalid",
"contract.required_missing", "contract.strict_off", "contract.claims_ignored", "report.claims_invalid",
"governed.report_policy",
"governed.caliber_agent_cite_fields", "governed.caliber_compute_input", "governed.agent_approval_never",
"governed.default_approval_never", "governed.subgraph_unpinned", "governed.supervisor", "contract.not_object",
"governed.text_bypass", "governed.caliber_agent_schema", "governed.caliber_model_input",
)
#: 修复落在全图上、不落在任何节点上的问题:问题各挂在跟随全图默认的那几个节点上(点一条能定位),
@@ -40,14 +46,17 @@ _GRAPH_FIXES = frozenset({"governed.default_approval_never"})
#: 除了 error,这几条警告也给修复(受管档位下)。已发布档位的门禁只给警告,不借「一键修复」
#: 替人把审批、strict 这些往受管的要求上拧
_WARNING_FIXES = {"governed": frozenset({"contract.strict_off"})}
_WARNING_FIXES = {"governed": frozenset({"contract.strict_off", "contract.claims_ignored"})}
#: choice 类修复里「交给 Copilot」那个候选的值。候选上另有 handoff: true 标记:只认标记,不认值——
#: 别的 choice 拿节点 id 当候选值,节点 id 恰好叫 copilot 的报告撰写节点照样是个普通候选
HANDOFF = "copilot"
ASSIST_ONLY = "这一处是结构性的问题,要交给 Copilot:请求里带上 assist: true"
STALE = "这处问题现在已经没有了(修复 id 过期),重新检查一次"
SOLVED = "前面的修复已经一并解决了这一处"
NEED_CHOICE = "这一处要你来选:{label}"
_TEXT_NODES = (NodeType.LLM, NodeType.AGENT)
#: 修复和 Copilot 兜底只许改节点配置、补节点和连线。其余的图操作(删节点、删连线)和
#: 任何不认识的操作(发布、改级别…)一律当降低要求处理
_GRAPH_OPS = frozenset({"add_node", "update_node", "add_edge"})
@@ -59,11 +68,13 @@ _GRAPH_OPS = frozenset({"add_node", "update_node", "add_edge"})
class _Ctx:
def __init__(self, spec: GraphSpec, versions: dict[str, Any] | None, cards: dict[str, Any] | None) -> None:
def __init__(self, spec: GraphSpec, versions: dict[str, Any] | None, cards: dict[str, Any] | None,
level: str = "governed") -> None:
self.spec = spec
self.nodes = spec.node_map()
self.versions = versions or {}
self.cards = cards or {}
self.level = level
def upstream(self, node_id: str, *types: NodeType) -> list[GraphNode]:
"""这个节点上游的某几类节点,按画布上的节点顺序。"""
@@ -148,7 +159,10 @@ def _plan_metrics_from_invalid(ctx: _Ctx, issue: dict[str, Any], node: GraphNode
return _cards_choice(ctx, node, before, "contract.metrics_from")
def _reports_choice(ctx: _Ctx, node: GraphNode, before: Any, *, narrative: bool) -> dict[str, Any]:
def _reports_choice(ctx: _Ctx, node: GraphNode, before: Any) -> dict[str, Any]:
"""report_from 指向哪个报告撰写节点。以前没有报告撰写节点时退回让人选 narrative(旧的叙述模式),
可受管级别的叙述模式过不了 G1(governed.report_from_required),已发布级别这几条又只是警告、不给
修复——那个候选修了也发不出去,所以没有报告撰写节点就交给 Copilot 加一个。"""
field = "contract.report_from"
reports = ctx.upstream(node.id, NodeType.REPORT)
if len(reports) == 1:
@@ -156,27 +170,120 @@ def _reports_choice(ctx: _Ctx, node: GraphNode, before: Any, *, narrative: bool)
before, reports[0].id)
if reports:
return _choice("选出具契约核对哪个「报告撰写」节点的文档(report_from)", field, _node_options(reports))
writers = ctx.upstream(node.id, *_TEXT_NODES) if narrative else []
if writers:
# 没有报告撰写节点时退回旧的叙述模式:数字按叙述回指口径卡。只有一个候选也要人确认——
# 它是不是那段该核对的叙述,只有作者知道
options = [{"value": f"{{{{ nodes.{n.id}.text }}}}", "label": f"「{n.title}」写的文字",
"hint": f"narrative = {{{{ nodes.{n.id}.text }}}}"} for n in writers]
return _choice("选出具契约要核对的叙述来自哪个节点(narrative)", "contract.narrative", options,
default=options[0]["value"] if len(options) == 1 else None)
return _assist("出口上游没有「报告撰写」节点,也没有写叙述的模型节点", field)
return _assist("出口上游没有「报告撰写」节点:要在出口前面加一个,出具契约才有文档可核对", field)
def _plan_report_from_missing(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
if node is None:
return None
return _reports_choice(ctx, node, _contract(node).get("report_from"), narrative=True)
return _reports_choice(ctx, node, _contract(node).get("report_from"))
def _plan_report_from_required(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
if node is None:
return None
return _reports_choice(ctx, node, _contract(node).get("report_from"))
def _report_text(report_id: str) -> str:
return f"{{{{ nodes.{report_id}.text }}}}"
def _plan_exit_fields(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
"""G1:成果字段改成取报告撰写节点的正文。就一个候选也给选项——字段里放什么是作者的事,
改了就是换掉原来的内容;报告撰写节点一个都没有,交给 Copilot 加。"""
if node is None:
return None
bad = exit_text_fields(ctx.spec, node)
if not bad:
return None
named = [f"「{name}」" for _, name, _ in bad if name is not None]
action = f"把成果字段{'、'.join(named)}改成取" if named else "给出口配一个成果字段 result,取"
reports = ctx.upstream(node.id, NodeType.REPORT)
if not reports:
return _assist(f"「{node.title}」上游没有报告撰写节点:要在出口前面加一个,{action}它的正文", "fields")
options = [{**o, "hint": f"{o['hint']} · {_report_text(o['value'])}"} for o in _node_options(reports)]
pinned = _contract(node).get("report_from")
default = pinned if pinned in {r.id for r in reports} else reports[0].id if len(reports) == 1 else None
return _choice(f"{action}哪个报告撰写节点的正文", "fields", options, default=default,
_fields=[field for field, _, _ in bad])
def _plan_report_policy(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
"""G3:报告撰写节点写明最严的三项。答案唯一,只往严里改;写了不支持的 claims 由 claims_invalid 管。"""
if node is None:
return None
writes: dict[str, Any] = {}
before: dict[str, Any] = {}
for key, want, _ in REPORT_POLICY:
value = node.config.get(key)
effective = ctx.spec.defaults.get(key) if value in (None, "") else value
if effective != want and not (key == "claims" and claims_problem(effective)):
writes[key], before[key] = want, copy.deepcopy(value)
if not writes:
return None
keys = list(writes)
preview = ({"field": keys[0], "before": before[keys[0]], "after": writes[keys[0]]} if len(keys) == 1
else {"field": "、".join(keys), "before": before, "after": writes})
return {"kind": "auto", "label": f"给「{node.title}」写明 " + "、".join(f"{k}: {v}" for k, v in writes.items()),
"field": keys[0], "preview": preview, "_set": writes}
def _plan_claims(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
"""claims 写了不支持的值。受管级别只认 require_citation,答案唯一;已发布级别 off 和 require_citation
都行,要人选——写的是 judge(还不支持的模型裁判)时建议 require_citation:它离裁判最近。"""
if node is None:
return None
before = node.config.get("claims")
if ctx.level == "governed":
return _auto(f"把「{node.title}」的 claims 改成 require_citation:受管出具要求每句结论挂引用"
+ ("(结论句裁判在后续版本支持)" if before == "judge" else ""), "claims", before, "require_citation")
hints = {"off": "结论句不参与判档,和以前一样", "require_citation": "没挂引用的结论句计入缺口,按出具档位降档"}
options = [{"value": v, "label": v, "hint": hints[v]} for v in REPORT_CLAIMS]
return _choice(f"选「{node.title}」的结论句策略(claims)", "claims", options,
default="require_citation" if before == "judge" else None)
def _plan_compute_input(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
"""G4:沙箱代码到底是在取数还是在算,只有作者知道。取数的标成 source;在算的交给 Copilot 把计算
挪进口径卡——两样都不替人选,也不给建议。"""
if node is None:
return None
options = [
{"value": "source", "label": "它在取数:标成 source(取数)",
"hint": "evidence_role = source,口径卡读它时记得下出处"},
{"value": HANDOFF, "label": "它在做计算:交给 Copilot 把计算挪进口径卡",
"hint": "Copilot 改的同样只是预览,要人拿主意的会原样问你", "handoff": True},
]
return _choice(f"「{node.title}」的产出喂给了口径卡:它是在取数,还是在做计算?", "evidence_role", options)
def _plan_cite_fields(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
if node is None:
return None
return _auto(f"打开「{node.title}」的 cite_fields:每个字段都核对到查询结果里的那一格", "cite_fields",
node.config.get("cite_fields"), True)
def _plan_agent_schema(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
return _assist(f"给「{_title(node)}」配 output_schema:交哪些字段、什么类型要你或 Copilot 定,自动修复不替你编;"
"配好之后再开 cite_fields", "output_schema")
def _plan_model_input(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
return _assist(f"「{_title(node)}」写的数进了口径卡:要换成 Agent(配 output_schema、开 cite_fields)或「调用工具」"
"节点取数。修复不删节点,Copilot 会拟一个改法")
def _plan_text_bypass(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
return _assist(f"「{_title(node)}」写的文字要改由报告撰写节点来写:在它和出口之间加一个报告撰写节点,出口改成取"
f"报告的正文。修复不删节点,「{_title(node)}」会留在图上")
def _plan_report_from_invalid(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
if node is None:
return None
return _reports_choice(ctx, node, _contract(node).get("report_from"), narrative=False)
return _reports_choice(ctx, node, _contract(node).get("report_from"))
def _plan_report_metrics_from(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
@@ -226,6 +333,18 @@ def _plan_strict(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> di
_contract(node).get("strict"), True)
def _plan_claims_ignored(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
"""受管契约的 claims 写了 on_uncited: ignore:受管正式运行本来就按 degrade 办,改成 degrade 答案唯一、
不降低要求,只是让写法和实际一致。"""
if node is None:
return None
claims = _contract(node).get("claims")
if not isinstance(claims, dict):
return None
return _auto("把出具契约 claims 的 on_uncited 改成 degrade:受管级别的正式运行本来就这么办,写法和实际一致",
"contract.claims.on_uncited", claims.get("on_uncited"), "degrade")
def _plan_approval(ctx: _Ctx, issue: dict[str, Any], node: GraphNode | None) -> dict[str, Any] | None:
if node is None:
return None
@@ -292,7 +411,18 @@ _PLANNERS: dict[str, Callable[[_Ctx, dict[str, Any], GraphNode | None], dict[str
"contract.report_from_invalid": _plan_report_from_invalid,
"contract.required_missing": _plan_required,
"contract.strict_off": _plan_strict,
"contract.claims_ignored": _plan_claims_ignored,
"report.metrics_from_invalid": _plan_report_metrics_from,
# 证据门禁 G1–G5
"governed.report_from_required": _plan_report_from_required,
"governed.exit_text_source": _plan_exit_fields,
"governed.text_bypass": _plan_text_bypass,
"governed.report_policy": _plan_report_policy,
"report.claims_invalid": _plan_claims,
"governed.caliber_compute_input": _plan_compute_input,
"governed.caliber_agent_cite_fields": _plan_cite_fields,
"governed.caliber_agent_schema": _plan_agent_schema,
"governed.caliber_model_input": _plan_model_input,
}
@@ -320,7 +450,7 @@ def plan_fixes(spec: GraphSpec, issues: list[Any], *, level: str, versions: dict
cards:钉在别处(caliber_from)的口径卡节点 id → 那一版里的指标定义,用来列 required 的候选。
返回的 Fix 里以 _ 开头的键是应用时要用的内部信息,交给前端前用 public() 去掉。
"""
ctx = _Ctx(spec, versions, cards)
ctx = _Ctx(spec, versions, cards, level)
warned = _WARNING_FIXES.get(level, frozenset())
out: dict[str, dict[str, Any]] = {}
for issue in map(_as_dict, issues):
@@ -426,11 +556,41 @@ def _edit_defaults(graph: dict[str, Any], fix: dict[str, Any], value: Any) -> li
for path, after in (fix.get("_set") or {}).items() if path.startswith("defaults.")]
#: 每类问题怎么改图。大多是同一种「按节点字段写值」,改全图默认的另走一格;留着这张表是为了哪天
def _edit_exit_fields(graph: dict[str, Any], fix: dict[str, Any], value: Any) -> list[tuple[str, str, Any, Any]]:
"""G1:文字不是来自报告撰写节点的成果字段,改成取选中那个报告的正文;只动门禁点了名的那几个字段。
出口没配字段的补一个 result——没配字段时成果的键就叫 result,下游照旧取得到。"""
node = _node(graph, fix["node_id"])
if node is None:
return []
config = _config(node)
text = _report_text(str(value))
targets = list(fix.get("_fields") or [])
if targets == ["fields"]:
before = copy.deepcopy(config.get("fields"))
config["fields"] = [{"name": "result", "value": text}]
return [(str(node["id"]), "fields", before, copy.deepcopy(config["fields"]))]
fields = config.get("fields") if isinstance(config.get("fields"), list) else []
out = []
for path in targets:
m = re.fullmatch(r"fields\[(\d+)\]\.value", path)
field = fields[int(m.group(1))] if m and int(m.group(1)) < len(fields) else None
if isinstance(field, dict):
out.append((str(node["id"]), path, copy.deepcopy(field.get("value")), text))
field["value"] = text
return out
#: 每类问题怎么改图。大多是同一种「按节点字段写值」,改全图默认、改成果字段的各走一格;留着这张表是为了
#: 某类修复要特殊处理时只换它自己的那一格(测试里也借它模拟一条出错的修复规则)
_EDITORS: dict[str, Callable[[dict[str, Any], dict[str, Any], Any], list[tuple[str | None, str, Any, Any]]]] = {
code: (_edit_defaults if code in _GRAPH_FIXES else _edit) for code in _PLANNERS
}
_EDITORS["governed.exit_text_source"] = _edit_exit_fields
def _handoff(fix: dict[str, Any], value: Any) -> bool:
"""选中的是不是「交给 Copilot」那个候选:只认候选上的 handoff 标记。"""
return any(o.get("handoff") is True and o.get("value") == value for o in fix.get("options") or [])
def _chosen(fix: dict[str, Any], choices: dict[str, Any]) -> tuple[Any, str]:
@@ -460,7 +620,7 @@ def _op(graph: dict[str, Any], node_id: str | None, edits: list[tuple[str | None
keys = dict.fromkeys(path.split(".", 1)[1] for _, path, _, _ in edits if path.startswith("defaults."))
return {"op": "update_defaults", "defaults": {k: copy.deepcopy(defaults.get(k)) for k in keys}}
config = _config(_node(graph, node_id) or {})
tops = dict.fromkeys(path.split(".")[0] for _, path, _, _ in edits)
tops = dict.fromkeys(re.split(r"[.\[]", path, maxsplit=1)[0] for _, path, _, _ in edits)
return {"op": "update_node", "id": node_id, "config": {k: copy.deepcopy(config.get(k)) for k in tops}}
@@ -469,7 +629,8 @@ def apply_fixes(graph: dict[str, Any] | GraphSpec, fix_ids: list[str], choices:
cards: dict[str, list[dict[str, Any]]] | None = None) -> dict[str, Any]:
"""在副本上逐个应用修复,每一个都复核:降低要求、错误没变少或者冒出新错误的,丢弃并写明原因。
返回 {graph, changes, ops, applied, rejected, remaining, fixes, ok}。graph 只是预览,不落库。
返回 {graph, changes, ops, applied, rejected, handoff, remaining, fixes, ok}。graph 只是预览,不落库。
handoff 是人在 choice 里选了「交给 Copilot」的修复 id:图不动,调用方据此把剩下的 error 交给 Copilot。
"""
raw = graph.model_dump(mode="json") if isinstance(graph, GraphSpec) else graph
current = copy.deepcopy(raw)
@@ -481,6 +642,7 @@ def apply_fixes(graph: dict[str, Any] | GraphSpec, fix_ids: list[str], choices:
wanted = sorted(dict.fromkeys(fix_ids), key=lambda fid: rank.get(fid.split(":")[0], len(rank)))
applied: list[str] = []
rejected: list[dict[str, str]] = []
handoff: list[str] = []
changes: list[dict[str, Any]] = []
ops: list[dict[str, Any]] = []
for fid in wanted:
@@ -495,6 +657,9 @@ def apply_fixes(graph: dict[str, Any] | GraphSpec, fix_ids: list[str], choices:
if why:
rejected.append({"fix_id": fid, "reason": why})
continue
if fix["kind"] == "choice" and _handoff(fix, value):
handoff.append(fid)
continue
trial = copy.deepcopy(current)
edits = _EDITORS.get(fix["code"], _edit)(trial, fix, value)
try:
@@ -503,7 +668,9 @@ def apply_fixes(graph: dict[str, Any] | GraphSpec, fix_ids: list[str], choices:
rejected.append({"fix_id": fid, "reason": "改完的图结构读不懂,这条修复已丢弃"})
continue
trial_issues = publish_issues(trial_spec, level=level)
why = ";".join(forbidden_changes(current, trial)) or judge(
# choice 里人亲手选的值不算「替人决定」(比如把沙箱代码标成取数);别的降低要求照样拦
chosen = {(str(nid), path) for nid, path, _, _ in edits} if fix["kind"] == "choice" else set()
why = ";".join(forbidden_changes(current, trial, chosen=chosen)) or judge(
issues, trial_issues, target=(fix["code"], fix["node_id"]))
if why:
rejected.append({"fix_id": fid, "reason": why})
@@ -517,7 +684,7 @@ def apply_fixes(graph: dict[str, Any] | GraphSpec, fix_ids: list[str], choices:
ops.append(_op(current, fix["node_id"], edits))
plan = plan_fixes(spec, issues, level=level, versions=versions, cards=cards)
return {"graph": current, "changes": changes, "ops": ops, "applied": applied, "rejected": rejected,
"remaining": annotate(issues, plan), "fixes": [public(f) for f in plan],
"handoff": handoff, "remaining": annotate(issues, plan), "fixes": [public(f) for f in plan],
"ok": not any(i.level == "error" for i in issues)}
@@ -643,11 +810,31 @@ def _contract_loosened(name: str, prior: Any, contract: Any) -> list[str]:
out.append(f"往「{name}」出具契约的 allow_numbers 里加了 {'、'.join(extra)}(不带出处也能放行的数)")
if contract.get("cells") is True and prior.get("cells") is not True:
out.append(f"替你打开了「{name}」出具契约的 cells(允许报告直接引用查询单元格),这要你自己决定")
was, now = _claims_rank(prior.get("claims")), _claims_rank(contract.get("claims"))
if was is not None and now is not None and now < was:
out.append(f"把「{name}」出具契约的 claims 往宽里改了(没挂依据的结论句{_CLAIMS_WORDS[was]} → "
f"{_CLAIMS_WORDS[now]})")
return out
#: 契约 claims 的严格程度:不查 < 查了不算缺口(ignore)< 计入缺口降档(degrade)< 不予出具(withhold)
_CLAIMS_WORDS = ("不检查", "只标出来", "计入缺口", "不予出具")
def _claims_rank(value: Any) -> int | None:
"""契约里 claims 的严格程度,越大越严。judge 这类还不支持的写法不比较(validate 另报)。"""
name = value.get("policy") if isinstance(value, dict) else value
if name in (None, "", "off"):
return 0
if name != "require_citation":
return None
on_uncited = value.get("on_uncited") if isinstance(value, dict) else None
return {"ignore": 1, "withhold": 3}.get(on_uncited, 2)
def forbidden_changes(before: dict[str, Any], after: dict[str, Any],
ops: list[dict[str, Any]] | None = None) -> list[str]:
ops: list[dict[str, Any]] | None = None, *,
chosen: set[tuple[str, str]] | frozenset[tuple[str, str]] = frozenset()) -> list[str]:
"""降低要求的改动,一条一句。空列表表示没有。确定性修复和 Copilot 兜底过的是同一道检查。
禁止:
@@ -657,6 +844,9 @@ def forbidden_changes(before: dict[str, Any], after: dict[str, Any],
- 审批策略往宽里改(每次调用都审批 > 仅危险工具需要审批 > 全部自动放行)、删掉写明的审批策略,
把审批写成(或新加一个)「全部自动放行」;全图默认同理
- 取消子工作流钉住的版本
- 报告撰写节点的 numbers、on_violation、claims 往宽里改;关掉 agent 的 cite_fields
- 替人把沙箱代码标成取数(evidence_role: source):它在取数还是在算只有作者知道。chosen 里的
(节点 id, 字段) 是人在 choice 里亲手选的,不算
- 改发布级别——级别不在图里,修复接口也从不写工作流的 status,所以只可能以一个不认识的
操作出现,按不认识的操作拒掉
"""
@@ -701,8 +891,21 @@ def forbidden_changes(before: dict[str, Any], after: dict[str, Any],
reasons.append(why)
contract, prior = cfg.get("contract"), prev.get("contract")
reasons += _contract_loosened(name, prior, contract)
if node.get("type") == "code" and cfg.get("evidence_role") == "source" \
and prev.get("evidence_role") != "source" and (nid, "evidence_role") not in chosen:
reasons.append((f"替你把「{name}」标成了取数(evidence_role: source)" if was
else f"新加的「{name}」标成了取数(evidence_role: source)")
+ ",它是在取数还是在算要你自己决定")
if was is None:
continue
if was.get("type") == node.get("type") == "report":
reasons += _report_loosened(name, prev, cfg, before.get("defaults"), after.get("defaults"))
if was.get("type") == node.get("type") == "agent" and prev.get("cite_fields") is True \
and cfg.get("cite_fields") is not True:
reasons.append(f"关掉了「{name}」的 cite_fields(字段不再逐个核对到查询结果)")
elif was.get("type") == node.get("type") == "agent" and prev.get("cite_fields") is True \
and prev.get("output_schema") and not cfg.get("output_schema"):
reasons.append(f"删掉了「{name}」的 output_schema(cite_fields 随之失效,字段不再逐个核对到查询结果)")
if prev.get("workflow_version") and not cfg.get("workflow_version"):
reasons.append(f"取消了「{name}」钉住的版本")
if not isinstance(prior, dict) or not prior:
@@ -722,6 +925,30 @@ def forbidden_changes(before: dict[str, Any], after: dict[str, Any],
return reasons
#: 报告撰写节点三项设置从严到宽的次序:没写时运行时的缺省值(on_violation 没写时探索运行按 flag)
_REPORT_STRICT = {"numbers": ("strict", "strict"), "on_violation": ("fail", "flag"),
"claims": ("require_citation", "off")}
def _report_loosened(name: str, prev: dict[str, Any], cfg: dict[str, Any], old_defaults: Any,
new_defaults: Any) -> list[str]:
"""报告撰写节点的三项设置有没有往宽里改:原来(按节点、再按全图默认)是最严的值,改完不是了。
claims 改成 judge 不算放宽——那是还不支持的更严的写法,validate 会报。"""
def effective(config: dict[str, Any], defaults: Any, key: str, fallback: str) -> Any:
value = config.get(key)
if value in (None, ""):
value = (defaults if isinstance(defaults, dict) else {}).get(key)
return fallback if value in (None, "") else value
out = []
for key, (strict, fallback) in _REPORT_STRICT.items():
was, now = effective(prev, old_defaults, key, fallback), effective(cfg, new_defaults, key, fallback)
if was == strict and now != strict and not (key == "claims" and now == "judge"):
out.append(f"把「{name}」的 {key} 从 {strict} 放宽成了 {now}"
+ ("(去掉了写明的值,运行时退回缺省)" if cfg.get(key) in (None, "") else ""))
return out
def _type_word(node_type: Any, type_label: Callable[[Any], str]) -> str:
try:
return f"「{type_label(NodeType(node_type))}」"
File diff suppressed because it is too large Load Diff
+374 -4
View File
@@ -1,11 +1,22 @@
from __future__ import annotations
from typing import Any
import re
from collections import defaultdict
from typing import Any, Callable
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.engine.schema import GraphSpec, NodeType, ValidationResult, type_label, validate_graph
from app.engine.schema import (
GraphNode,
GraphSpec,
NodeType,
ValidationResult,
_caliber_inputs,
claims_problem,
type_label,
validate_graph,
)
# 口径升版的三种合法处置。没有默认值是刻意的:升版最容易静默翻车的地方
# 就是"上周和这周的数其实不可比,但报表看起来一切正常"。
@@ -26,7 +37,7 @@ def lint_for_publish(spec: GraphSpec, *, level: str) -> ValidationResult:
错误会挡住发布。
每条问题带机器编号(code):发布前自动修复(engine/autofix.py)按它认问题、出修复,
文案可以改,编号不能随便改。
文案可以改,编号不能随便改。证据门禁 G1–G5 见 _lint_evidence。
"""
result = ValidationResult()
strict = level == "governed"
@@ -88,6 +99,8 @@ def lint_for_publish(spec: GraphSpec, *, level: str) -> ValidationResult:
flag(f"{who}打开了「允许联网」:受管模板建议把取数收敛到「调用工具」节点"
"(那里有取数快照)", code="governed.code_network", node_id=node.id, field="network")
_lint_evidence(spec, flag, strict=strict)
if strict and not has_contract_output:
result.add(
"受管模板至少要有一个「成果 / 出具」节点声明出具契约,否则三档出具无从谈起",
@@ -101,7 +114,28 @@ def publish_issues(spec: GraphSpec, *, level: str) -> list[Any]:
发布、发布前检查、自动修复复核都走这一个口径——检查说能发的,真发布时不会被拦。
"""
return [*validate_graph(spec).issues, *lint_for_publish(spec, level=level).issues]
gate = lint_for_publish(spec, level=level).issues
return [*_not_repeated(validate_graph(spec).issues, gate, spec), *gate]
#: validate 里和门禁 G4、G5 说同一件事的两条提示(schema.evidence_issues)
_REPEATS = frozenset({"evidence.caliber_compute_input", "evidence.cite_fields_off"})
def _not_repeated(issues: list[Any], gate: list[Any], spec: GraphSpec) -> list[Any]:
"""门禁已经点了名的,validate 那条同义的提示不再列:门禁那条带编号、带修复,一处问题只说一遍。
G5 落在 agent 上,validate 那条也落在 agent 上;G4 落在喂数的沙箱代码上,validate 那条落在口径卡读它的
那条指标上,按口径卡的输入对回去。没被门禁覆盖的(比如不给口径卡供数的 agent)照旧提示。
"""
cited = {i.node_id for i in gate if i.code == "governed.caliber_agent_cite_fields"}
compute = {i.node_id for i in gate if i.code == "governed.caliber_compute_input"}
covered = {("evidence.cite_fields_off", nid, "output_schema") for nid in cited}
for card in spec.nodes:
if card.type == NodeType.METRICS and compute:
covered |= {("evidence.caliber_compute_input", card.id, f"metrics[{i}].expression")
for i, producer in _caliber_inputs(card, spec) if producer.id in compute}
return [i for i in issues if i.code not in _REPEATS or (i.code, i.node_id, i.field) not in covered]
def _lint_contract(
@@ -153,6 +187,342 @@ def _lint_contract(
"受管模板建议把出具契约设为 strict:非 strict 下未回指的数字只降档不拦截",
code="contract.strict_off", node_id=node_id, field="contract.strict",
)
claims = contract.get("claims")
if strict and isinstance(claims, dict) and claims.get("on_uncited") == "ignore":
# 受管级别的正式运行把 ignore 当 degrade(io._claims),要求没有降低;只是写法看上去像放宽了
flag(
"出具契约的 claims 写了 on_uncited: ignore,但受管级别的正式运行里没挂依据的结论句照样计入缺口"
"(按 degrade 处理),这个写法不会生效。改成 degrade,或者更严的 withhold",
code="contract.claims_ignored", node_id=node_id, field="contract.claims",
)
# --------------------------------------------------------------------------
# 证据门禁 G1–G5:受管出具要在结构上保证「报告里的每个数都点得开出处」
#
# G1 带契约的出口核对报告撰写节点的文档,成果字段的文字只能来自报告撰写节点
# G2 模型写的文字不绕过报告撰写节点直接流进出口(从图上追)
# G3 报告撰写节点写明 numbers: strict、on_violation: fail、claims: require_citation
# G4 口径卡的输入不来自计算角色的沙箱代码
# G5 给口径卡供数的 agent 配 output_schema 并开 cite_fields(模型调用节点不能给口径卡供数)
#
# 只看图和配置,不看运行结果。只在发布和发布前检查时起作用:已经发布的版本照常运行,
# 下次发布才按这些规则查。受管级别挡住发布,已发布级别只给警告。
# --------------------------------------------------------------------------
#: 报告撰写节点在受管模板里要写明的三项:(配置键, 要求的值, 为什么)
REPORT_POLICY = (
("numbers", "strict", "没有出处的裸数字判为违规"),
("on_violation", "fail", "重写之后还违规就让节点失败,不带着问题出具"),
("claims", "require_citation", "没挂引用的结论句计入缺口,按出具档位降档"),
)
#: 产出本身就是证据、或者由系统确定地算出来的节点。成果字段取它们的值,不算「别的节点写的字」
_TRACEABLE = (NodeType.REPORT, NodeType.METRICS, NodeType.INPUT, NodeType.TOOL, NodeType.RETRIEVE)
_WRITERS = (NodeType.LLM, NodeType.AGENT, NodeType.SUPERVISOR)
#: 往对话里写消息的节点(emit_message 没关时),「最后一条消息」就是它们里最后跑的那个写的
_EMITTERS = (*_WRITERS, NodeType.REPORT)
#: 把上游的值原样或加工后往下传的节点:追文字的来路要穿过它们
_RELAYS = (NodeType.TRANSFORM, NodeType.CODE, NodeType.VALIDATE, NodeType.HUMAN, NodeType.LOOP)
#: 往回追口径卡的输入时穿过的整形节点。沙箱代码不穿过:它自己就是 G4 要判的
_RESHAPERS = (NodeType.TRANSFORM, NodeType.VALIDATE)
#: 只决定走哪条路、不把值往下带的配置
_CONTROL_KEYS = frozenset({"skip_if", "condition", "cases"})
_TEMPLATE = re.compile(r"\{\{(.*?)\}\}", re.S)
# 模板和表达式里对值的引用。nodes.<id>…、vars.<名字>… 落得到具体节点;last_message、messages
# 是最后写消息的那个节点写的;output、loops(和整个 nodes / vars)说不准来自谁
_REF = re.compile(
r"(?<![\w.])(?P<root>nodes|vars|last_message|messages|output|loops)(?![\w-])"
r"(?P<path>(?:\s*(?:\.\s*[\w-]+|\[\s*(?:'[^']*'|\"[^\"]*\"|-?\d+)\s*\]))*)")
_SEGMENT = re.compile(r"\.\s*([\w-]+)|\[\s*(?:'([^']*)'|\"([^\"]*)\"|(-?\d+))\s*\]")
#: (根, 名字, 往里取的路径)。last_message 这类没有名字
Ref = tuple[str, str, tuple[str, ...]]
_LAST: Ref = ("last_message", "", ())
def _refs(text: str, *, template: bool) -> list[Ref]:
out: list[Ref] = []
for piece in [m.group(1) for m in _TEMPLATE.finditer(text)] if template else [text]:
for m in _REF.finditer(piece):
path = tuple(next(g for g in s.groups() if g is not None) for s in _SEGMENT.finditer(m.group("path")))
root = m.group("root")
if root in ("nodes", "vars") and path:
out.append((root, path[0], path[1:]))
else:
out.append((root, "", path))
return out
def _strings(value: Any) -> list[str]:
"""配置里所有带 {{ }} 的字符串。"""
if isinstance(value, str):
return [value] if "{{" in value else []
if isinstance(value, dict):
return [s for v in value.values() for s in _strings(v)]
if isinstance(value, list):
return [s for v in value for s in _strings(v)]
return []
def _template_refs(value: Any) -> list[Ref]:
return [r for text in _strings(value) for r in _refs(text, template=True)]
def _flow_refs(node: GraphNode) -> list[Ref]:
"""这个节点的产出读了哪些值。判断走哪条路的条件不算;校验节点没写 source 时校验的是最后一条消息。"""
out: list[Ref] = []
for key, value in node.config.items():
if key in _CONTROL_KEYS:
continue
out += _refs(value, template=False) if key == "expression" and isinstance(value, str) \
else _template_refs(value)
if node.type == NodeType.VALIDATE and node.config.get("source") in (None, ""):
out.append(_LAST)
return out
def _effective(spec: GraphSpec, node: GraphNode, key: str, default: Any = None) -> Any:
"""和运行时 NodeContext.cfg 同一个规则:节点上没写就取图级 defaults。"""
value = node.config.get(key)
return spec.defaults.get(key, default) if value in (None, "") else value
class _Graph:
"""一张图上反复要用的查找:谁产出了某个值、最后一条消息是谁写的。"""
def __init__(self, spec: GraphSpec) -> None:
self.spec = spec
self.nodes = spec.node_map()
self.parents: dict[str, list[str]] = defaultdict(list)
for e in spec.edges:
self.parents[e.target].append(e.source)
def producers(self, root: str, name: str) -> list[GraphNode]:
"""nodes.<name> 就是那个节点;vars.<name> 是用 assign_to 写它的节点、同名的入口字段、
每轮注入它的循环。"""
if not name or root not in ("nodes", "vars"):
return []
if root == "nodes":
return [self.nodes[name]] if name in self.nodes else []
out = []
for n in self.spec.nodes:
cfg = n.config
if n.type == NodeType.INPUT:
writes = {str(f.get("name") or "").strip() for f in cfg.get("fields") or [] if isinstance(f, dict)}
elif n.type == NodeType.LOOP and cfg.get("mode", "foreach") == "foreach":
item = str(cfg.get("item_var") or "item").strip()
writes = {item, f"{item}_index"}
else:
writes = set()
if name in writes or str(cfg.get("assign_to") or "").strip() == name:
out.append(n)
return out
def last_writers(self, node_id: str) -> list[GraphNode]:
"""到这个节点时,最后一条消息可能是谁写的:沿连线往回走,每条路上遇到的第一个写消息的节点。"""
found: set[str] = set()
seen = {node_id}
stack = list(self.parents[node_id])
while stack:
cur = stack.pop()
if cur in seen or cur not in self.nodes:
continue
seen.add(cur)
n = self.nodes[cur]
if n.type in _EMITTERS and _effective(self.spec, n, "emit_message", True):
found.add(cur)
continue
stack.extend(self.parents[cur])
return [n for n in self.spec.nodes if n.id in found]
def sources(self, consumer: str, ref: Ref) -> list[GraphNode]:
root, name, _ = ref
return self.last_writers(consumer) if root in ("last_message", "messages") else self.producers(root, name)
def _who(node: GraphNode) -> str:
return f"「{node.title}」({type_label(node.type)})"
def _traceable(node: GraphNode) -> bool:
return node.type in _TRACEABLE or (node.type == NodeType.CODE and node.config.get("evidence_role") == "source")
def _model_text(node: GraphNode, ref: Ref) -> bool:
"""顺着这个引用拿到的是不是模型写的话。开了 cite_fields 的 agent,assign_to(和 nodes.<id>.data)
拿到的是逐格核对过的字段,那是数据;它的原话、最后一条消息照样是模型写的。"""
if node.type in (NodeType.LLM, NodeType.SUPERVISOR):
return True
if node.type != NodeType.AGENT:
return False
root, _, path = ref
cited = bool(node.config.get("output_schema")) and node.config.get("cite_fields") is True
return not (cited and (root == "vars" or path[:1] == ("data",)))
def _exit_refs(node: GraphNode) -> list[tuple[str, str | None, list[Ref]]]:
"""出口交出去的每个字段:(字段路径, 字段名, 引用)。没配字段时交的是最后一条消息(io.run_output)。"""
fields = node.config.get("fields")
if not fields:
return [("fields", None, [_LAST])]
return [(f"fields[{i}].value", str(f["name"]), _template_refs(f.get("value", "")))
for i, f in enumerate(fields if isinstance(fields, list) else [])
if isinstance(f, dict) and f.get("name")]
def _untraceable_sources(graph: _Graph, exit_id: str, refs: list[Ref]) -> list[str]:
"""一个成果字段里核对不了出处的来源,一个来源一句(「叙述」(模型调用)、最后一条消息…)。"""
out: list[str] = []
for ref in refs:
root, name, _ = ref
if root in ("output", "loops") or (root in ("nodes", "vars") and not name):
out.append(f"{root} 里的值(说不准来自哪个节点)")
elif root in ("last_message", "messages"):
out += [f"最后一条消息({_who(w)}写的)" for w in graph.last_writers(exit_id) if w.type != NodeType.REPORT]
else:
# 开了 cite_fields 的 agent 交的字段逐格核对过,是数据不是模型写的话——和 G2(_model_text)同一口径
out += [_who(p) for p in graph.producers(root, name)
if not _traceable(p) and (p.type != NodeType.AGENT or _model_text(p, ref))]
return list(dict.fromkeys(out))
def exit_text_fields(spec: GraphSpec, exit_node: GraphNode, graph: _Graph | None = None,
) -> list[tuple[str, str | None, list[str]]]:
"""G1:出口里文字不是来自报告撰写节点的字段:(字段路径, 字段名, 来源)。字段名为 None 是没配字段。
发布前自动修复(autofix)按同一份结果改字段,门禁和修复认的是同一批字段。
"""
graph = graph or _Graph(spec)
out = []
for field, name, refs in _exit_refs(exit_node):
if bad := _untraceable_sources(graph, exit_node.id, refs):
out.append((field, name, bad))
return out
def _text_leaks(graph: _Graph, exit_node: GraphNode) -> dict[str, tuple[GraphNode, set[tuple[str, ...]]]]:
"""流进这个出口、没经过报告撰写节点的模型原话:写字的节点 id → (节点, 途经的节点标题)。
途经为空的是出口字段直接取的。沿着模板和表达式里的引用往回追,穿过整形、沙箱代码、校验、人工审批、
循环;碰到报告撰写节点就停——模型写的话进了报告,就只是写作者的素材。
"""
found: dict[str, tuple[GraphNode, set[tuple[str, ...]]]] = {}
seen: set[str] = set()
todo: list[tuple[str, Ref, tuple[str, ...]]] = [
(exit_node.id, ref, ()) for _, _, refs in _exit_refs(exit_node) for ref in refs]
while todo:
consumer, ref, via = todo.pop(0)
for source in graph.sources(consumer, ref):
if source.type == NodeType.REPORT or source.id == exit_node.id:
continue
if source.type in _WRITERS:
if _model_text(source, ref):
found.setdefault(source.id, (source, set()))[1].add(via)
elif source.type in _RELAYS and source.id not in seen:
seen.add(source.id)
todo += [(source.id, r, (*via, source.title)) for r in _flow_refs(source)]
return found
def _card_feeders(graph: _Graph, card: GraphNode) -> list[GraphNode]:
"""口径卡的输入是谁产出的:穿过整形、校验节点往回追到真正的来源。"""
out: dict[str, GraphNode] = {}
seen: set[str] = set()
todo = [producer for _, producer in _caliber_inputs(card, graph.spec)]
while todo:
node = todo.pop(0)
if node.id in seen or node.id == card.id:
continue
seen.add(node.id)
if node.type in _RESHAPERS:
todo += [s for ref in _flow_refs(node) for s in graph.sources(node.id, ref)]
else:
out.setdefault(node.id, node)
return list(out.values())
def _lint_evidence(spec: GraphSpec, flag: Callable[..., None], *, strict: bool) -> None:
graph = _Graph(spec)
outputs = [n for n in spec.nodes if n.type == NodeType.OUTPUT]
contracted = {n.id for n in outputs if isinstance(n.config.get("contract"), dict) and n.config["contract"]}
# G1
for node in outputs:
if node.id not in contracted:
continue
contract = node.config["contract"]
# 两样都没写的由 contract.report_from_missing 报,这里只管「写了叙述、没写 report_from」
if contract.get("report_from") in (None, "") and str(contract.get("narrative") or "").strip():
flag(f"{_who(node)}的出具契约用的是旧的叙述模式(narrative),没有声明 report_from:数字只能按数值"
"回头匹配,出处可能不唯一。受管出具核对「报告撰写」节点产出的文档,每个数都有唯一的出处——把 "
"report_from 指向出口上游的报告撰写节点",
code="governed.report_from_required", node_id=node.id, hard=True, field="contract.report_from")
for field, name, bad in exit_text_fields(spec, node, graph):
head = (f"{_who(node)}的成果字段「{name}」取的是{'、'.join(bad)}的产出" if name is not None
else f"{_who(node)}没有配成果字段,交出去的是{'、'.join(bad)}")
flag(f"{head}:受管出具里给人看的文字只能来自「报告撰写」节点,别的节点写的字核对不了出处。"
+ ("把它改成取报告撰写节点的正文" if name is not None else "给出口配一个字段,取报告撰写节点的正文"),
code="governed.exit_text_source", node_id=node.id, hard=True, field=field)
# G2:每个出口都算。受管级别是 error,已发布级别是警告——和别的 G 规则一样,级别只管轻重,不管范围
flagged: set[str] = set()
for node in outputs:
for writer, vias in _text_leaks(graph, node).values():
indirect = sorted(v for v in vias if v)
# 带契约的出口里字段直接取的,G1 已经点名了那个字段
if writer.id in flagged or (node.id in contracted and not indirect):
continue
flagged.add(writer.id)
via = f"(经过{'、'.join(f'「{t}」' for t in indirect[0])})" if indirect else ""
flag(f"{_who(writer)}写的文字没经过「报告撰写」节点,就流进了出口「{node.title}」{via}:模型写的话不算"
"证据,里面的数字、表名都核对不了出处。改由报告撰写节点来写(它只写引用标记,数字由系统渲染),"
"出口取报告的正文",
code="governed.text_bypass", node_id=writer.id, hard=True)
# G3
for node in spec.nodes:
if node.type != NodeType.REPORT:
continue
loose = []
for key, want, why in REPORT_POLICY:
value = _effective(spec, node, key)
if key == "claims" and claims_problem(value):
continue # 写了不支持的值(比如 judge),validate 的 report.claims_invalid 报
if value != want:
now = "现在没写" if value in (None, "") else f"现在是 {value}"
loose.append((key, f"{key}: {want}({why};{now})"))
if loose:
flag(f"{_who(node)}要写明 {';'.join(text for _, text in loose)}。受管模板的报告撰写节点按最严的规则自查",
code="governed.report_policy", node_id=node.id, hard=True, field=loose[0][0])
# G4、G5
feeders: dict[str, tuple[GraphNode, list[str]]] = {}
for card in spec.nodes:
if card.type == NodeType.METRICS:
for feeder in _card_feeders(graph, card):
feeders.setdefault(feeder.id, (feeder, []))[1].append(f"「{card.title}」")
for feeder, cards in feeders.values():
to = f"口径卡{'、'.join(dict.fromkeys(cards))}"
cfg = feeder.config
if feeder.type == NodeType.CODE and cfg.get("evidence_role") != "source":
role = "是 compute" if cfg.get("evidence_role") else "没写,按计算算"
flag(f"{_who(feeder)}的产出喂给了{to},而它的角色是计算(evidence_role {role}):沙箱里算出来的数核对"
"不了出处。它其实是在取数的话,把 evidence_role 标成 source;是在做计算的话,把计算挪进口径卡的表达式",
code="governed.caliber_compute_input", node_id=feeder.id, hard=True, field="evidence_role")
elif feeder.type == NodeType.AGENT and not cfg.get("output_schema"):
flag(f"{_who(feeder)}给{to}供数,却没有配 output_schema:它交出来的是一段自由文字,口径卡读到的数核对不了"
"出处。给它配 output_schema 并开 cite_fields,每个字段都核对到查询结果里的那一格",
code="governed.caliber_agent_schema", node_id=feeder.id, hard=True, field="output_schema")
elif feeder.type == NodeType.AGENT and cfg.get("cite_fields") is not True:
flag(f"{_who(feeder)}给{to}供数,配了 output_schema 却没开 cite_fields:字段没有逐个核对到查询结果,"
"口径卡读到的仍是模型报的数。打开 cite_fields",
code="governed.caliber_agent_cite_fields", node_id=feeder.id, hard=True, field="cite_fields")
elif feeder.type == NodeType.LLM:
flag(f"{_who(feeder)}给{to}供数:模型调用节点不调工具,写出来的数没有出处。改由 Agent(配 output_schema "
"并开 cite_fields)或「调用工具」节点取数",
code="governed.caliber_model_input", node_id=feeder.id, hard=True)
async def unresolved_caliber_upgrades(
+93 -15
View File
@@ -57,12 +57,13 @@ async def run_output(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
reports = _report_fields(result, mapping, state, ctx)
contract = ctx.cfg("contract")
if contract:
cells = True
cells, governed = True, False
if isinstance(contract, dict) and contract.get("report_from") not in (None, ""):
from app.engine.governance import cells_allowed, governed_formal
cells = cells_allowed(contract, governed=await governed_formal(ctx.run.run_id))
result["_issuance"] = _apply_contract(contract, state, ctx, reports=reports, cells=cells)
governed = await governed_formal(ctx.run.run_id)
cells = cells_allowed(contract, governed=governed)
result["_issuance"] = _apply_contract(contract, state, ctx, reports=reports, cells=cells, governed=governed)
evidence = _evidence_of(reports, contract)
if evidence:
result["_evidence"] = evidence
@@ -142,7 +143,7 @@ def _evidence_of(reports: list[dict[str, Any]], contract: Any) -> dict[str, Any]
def _apply_contract(
contract: dict[str, Any], state: GraphState, ctx: NodeContext,
*, reports: list[dict[str, Any]] | None = None, cells: bool = True,
*, reports: list[dict[str, Any]] | None = None, cells: bool = True, governed: bool = False,
) -> dict[str, Any]:
from datetime import datetime, timezone
@@ -195,7 +196,8 @@ def _apply_contract(
trace = None
cited: dict[str, Any] = {}
if citations:
cited = _check_citations(str(report_from), contract, state, ctx, reports or [], cells=cells)
cited = _check_citations(str(report_from), contract, state, ctx, reports or [], cells=cells,
governed=governed)
gaps.extend(cited["gaps"])
unmatched = cited["unmatched"]
else:
@@ -235,6 +237,8 @@ def _apply_contract(
strict=bool(contract.get("strict")),
gaps=gaps,
unresolved=cited.get("unresolved"),
uncited_claims=cited.get("uncited"),
claims_policy=cited.get("claims_policy"),
)
if citations:
@@ -285,26 +289,31 @@ def _apply_contract(
#: 文档本身对不上(不是某个数没出处):这次复核没法完成,记 gap
_INTEGRITY = {"bad_schema", "segment_mismatch", "structural_text", "render_mismatch", "eid_mismatch",
"state_mismatch"}
#: 出具声明里列多少个可疑名字、多少句没挂依据的结论(计数照实,列表只是给人看的样本)
_LISTED = 20
def _check_citations(
report_from: str, contract: dict[str, Any], state: GraphState, ctx: NodeContext,
reports: list[dict[str, Any]], *, cells: bool = True,
reports: list[dict[str, Any]], *, cells: bool = True, governed: bool = False,
) -> dict[str, Any]:
"""独立复核报告文档:{matched, unmatched, unresolved, gaps, doc_artifact, stats}。
"""独立复核报告文档:{matched, unmatched, unresolved, gaps, doc_artifact, stats, claims_policy, uncited,
claims, entities}。
不信报告节点自己的统计:文档按 id 从工件库取回(取回时复验哈希),目录按报告节点
同一套参数从状态里重建(不用文档里存的那份),每个带引用的片段重新解析、重新渲染、
逐字比对。一个配错的报告节点绕不过这里。
cells:受管级别的正式运行、契约没声明 cells 时为 False,单元格引用一律判解析不了。
governed:受管级别的正式运行。可疑实体(名字哪里都找不到)只在这时按缺口降档,别处只标注。
"""
from app.core.artifact_store import load
from app.engine.evidence import iter_units, resolve_ref, verify_doc
from app.engine.nodes.report import report_catalog
from app.engine.evidence import iter_units, ledger_enabled, resolve_ref, verify_doc
from app.engine.nodes.report import report_catalog, report_entities
out: dict[str, Any] = {"matched": [], "unmatched": [], "unresolved": [], "gaps": [],
"doc_artifact": None, "stats": None}
"doc_artifact": None, "stats": None, "claims_policy": None, "uncited": None,
"claims": None, "entities": None}
gaps = out["gaps"]
spec = ctx.run.spec
node = spec.node_map().get(report_from)
@@ -332,10 +341,12 @@ def _check_citations(
return out
catalog = report_catalog(state, spec, node)
checked = verify_doc(doc, catalog, allow_numbers=contract.get("allow_numbers"), cells_allowed=cells)
checked = verify_doc(doc, catalog, allow_numbers=contract.get("allow_numbers"), cells_allowed=cells,
entities=report_entities(spec, node) == "link")
out["stats"] = checked["stats"]
broken: list[str] = []
flagged: set[str] = set()
suspicious: dict[str, list[dict[str, Any]]] = {"unknown_entity": [], "unverified_entity": []}
for v in checked["violations"]:
where = {k: v[k] for k in ("segment", "unit") if v.get(k)}
if v.get("span"):
@@ -346,11 +357,27 @@ def _check_citations(
out["unmatched"].append({"token": v.get("text", ""), "context": v.get("context", ""), **where})
elif v["code"] == "unresolved_ref":
out["unresolved"].append({"ref": v.get("ref", ""), "message": v["message"], **where})
elif v["code"] in suspicious:
# 写 [[see:t:x]] 时违规上没有正文里的字,名字取自引用
name = v.get("text") or str(v.get("ref") or "").partition(":")[2]
suspicious[v["code"]].append({"name": name, **({"ref": v["ref"]} if v.get("ref") else {}), **where})
elif v["code"] in _INTEGRITY:
broken.append(v["message"])
if broken:
more = f"等 {len(broken)} 处" if len(broken) > 1 else ""
gaps.append(f"报告「{node.title}」的文档没通过复核:{broken[0]}{more}")
unknown, unverified = suspicious["unknown_entity"], suspicious["unverified_entity"]
if unknown and governed:
# 用户拍板:可疑实体在受管级别的正式出具里按「有缺口」降档,不拦截——反引号里的一个业务词
# 也可能被当成名字,为它不予出具太重;探索运行、已发布级别只标注
names = "".join(f"「{n}」" for n in dict.fromkeys(u["name"] for u in unknown[:5]))
more = "等" if len(unknown) > 5 else ""
gaps.append(f"报告「{node.title}」里有 {len(unknown)} 处可疑实体({names}{more}):本次运行的表结构、查询、"
"结果列里都没有这些名字,可能是编造的")
if unknown or unverified:
# 表结构快照不全时核对不了的名字(unverified)在哪个级别都只标注
out["entities"] = {"unknown": unknown[:_LISTED], "unverified": unverified[:_LISTED],
"counted": governed and bool(unknown)}
# 老前端要用的逐个数字出处,从复核通过的数字片段拼出来
for _, unit in iter_units(doc):
@@ -375,10 +402,8 @@ def _check_citations(
out["matched"].append({**hit, "segment": seg["id"], "unit": unit["id"], "start": start, "end": end,
"span": [start, end], "eid": cite["eid"]})
claims = contract.get("claims")
policy = claims.get("policy") if isinstance(claims, dict) else claims
if policy not in (None, "", "off"):
gaps.append("契约声明了结论句检查(claims),这一版还不支持,结论句没有核对")
_claims(out, contract, payload, evidence_on=ledger_enabled(ctx.run), governed=governed,
uncited=checked.get("uncited") or [])
mine = next((r for r in reports if r["node"].id == report_from), None)
edited = mine["edited"] if mine else []
@@ -389,6 +414,48 @@ def _check_citations(
return out
#: 没挂依据的结论句怎么处置,从松到严。认不出的写法按 degrade 算:写错一个词不能让缺口悄悄消失
_ON_UNCITED = ("ignore", "degrade", "withhold")
def _claims(out: dict[str, Any], contract: dict[str, Any], payload: dict[str, Any], *, evidence_on: bool,
governed: bool = False, uncited: list[dict[str, Any]]) -> None:
"""结论句策略:没挂依据的结论句按 claims_policy 交给 decide_tier 判档。
两处可以要求挂依据:报告节点写作时记下的 claims(节点产出里的,写作提示按它要求过),和契约自己写的
require_citation(对象形式可以另写 on_uncited)。两处都写了取更严的那个——契约只能收紧、不能放松:
报告节点要求了,契约写 ignore 或 off 也照旧计入缺口,写 withhold 就收紧成不予出具。受管级别的正式运行
里 ignore 一律当 degrade:没有哪道门禁查契约里的 claims,不能让它把用户要的底线拉低。
数的是这里重新核对出来的 uncited,不信文档里记的 cites。升级前写的报告(节点产出里没有 claims 这个键,
包括跨着升级还没跑完的运行)一律不管,契约写了也照旧记「这一版还不支持」。judge(裁判模型)是后续版本
的事,契约写了照实记缺口。
"""
written = payload.get("claims") if isinstance(payload, dict) else None
# 报告节点升级后才在产出里记 claims(off 也记):没有这个键就是升级前写的报告
upgraded = evidence_on and isinstance(payload, dict) and "claims" in payload
declared = contract.get("claims")
name = declared.get("policy") if isinstance(declared, dict) else declared
asks: list[str] = []
if upgraded and written == "require_citation":
asks.append("degrade")
if upgraded and name == "require_citation":
wanted = declared.get("on_uncited") if isinstance(declared, dict) else None
asks.append(wanted if wanted in _ON_UNCITED else "degrade")
elif name not in (None, "", "off"):
out["gaps"].append("契约声明了结论句检查(claims),这一版还不支持,结论句没有核对")
if not asks:
return
on_uncited = max(asks, key=_ON_UNCITED.index)
if governed and on_uncited == "ignore":
on_uncited = "degrade"
policy: Any = "require_citation" if on_uncited == "degrade" else \
{"policy": "require_citation", "on_uncited": on_uncited}
out["claims_policy"], out["uncited"] = policy, uncited
out["claims"] = {"policy": "require_citation", "uncited_claims": len(uncited), "uncited": uncited[:_LISTED],
**({"on_uncited": on_uncited} if on_uncited != "degrade" else {})}
def _declare_citations(
tier: str, cited: dict[str, Any], *, calibers: list[dict[str, str]], metrics: list[dict[str, Any]],
gaps: list[str], missing_required: list[str], missing_expected: list[str], report_from: str,
@@ -411,7 +478,11 @@ def _declare_citations(
"stats": cited["stats"],
"gaps": gaps,
"declared_at": declared_at.isoformat(),
# 结论句策略和可疑实体:只在用上时才有这两个键,升级前的运行和没用上的契约形状不变
**({"claims": cited["claims"]} if cited.get("claims") else {}),
**({"entities": cited["entities"]} if cited.get("entities") else {}),
}
entities = cited.get("entities")
ctx.emit(
EventType.ISSUANCE,
mode="citations",
@@ -426,6 +497,13 @@ def _declare_citations(
metrics_checked=len(metrics),
matched_numbers=len(cited["matched"]),
matched=cited["matched"][:50],
# on_uncited 只在不是默认的 degrade 时才有(收紧成 withhold、探索运行的契约写了 ignore)
**({"claims": {k: cited["claims"][k] for k in ("policy", "uncited_claims", "on_uncited")
if k in cited["claims"]}} if cited.get("claims") else {}),
# 计数取复核的统计(声明里的名单最多列 _LISTED 个)
**({"entities": {"unknown": (cited["stats"] or {}).get("unknown_entities", len(entities["unknown"])),
"unverified": (cited["stats"] or {}).get("unverified_entities", len(entities["unverified"])),
"counted": entities["counted"]}} if entities else {}),
)
return issuance
+18 -2
View File
@@ -18,7 +18,14 @@ from app.db.base import SessionLocal
from app.db.models import Skill
from app.engine.approval import read_decision
from app.engine.context import NodeContext, NodeError
from app.engine.evidence import cell_eid, ledger_enabled, make_eid, next_exec, query_entry_fields
from app.engine.evidence import (
cell_eid,
ledger_enabled,
make_eid,
next_exec,
query_entry_fields,
schema_ledger_entry,
)
from app.engine.expressions import CellError, cell_value, column_kind, locate_cell, numeric_text, same_value
from app.engine.guards import FALLBACK_CONTEXT, Guard, legacy_hint, node_limits
from app.engine.replay import ask, once
@@ -503,6 +510,8 @@ async def run_agent(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
preview=content[:2000],
artifact=snapshot_id,
**({"query_artifact": query["artifact"]} if query else {}),
# 查询当时数据源的表结构快照:和查询快照一样落在封存范围内,证据接口从这里认它
**({"schema_artifact": query["schema_artifact"]} if query and query.get("schema_artifact") else {}),
**extra,
)
outcome: dict[str, Any] = {"content": content, "ok": ok, "duration_ms": elapsed}
@@ -821,10 +830,17 @@ def _record_call(node_id: str, exec_no: int, call_id: str, name: str, args: dict
return content
alias = f"Q{len(queries) + 1}"
record.update(alias=alias, artifact=artifact)
# 表结构快照:同一件在本节点这次执行里只记一条 schema 条目。快照是内容寻址的,重放时读到的是同一份
schema = fields.get("schema_artifact")
seen = schema and any(e.get("kind") == "schema" and e.get("artifact") == schema for e in entries)
frozen = None if not schema or seen else schema_ledger_entry(schema, node_id=node_id, exec_no=exec_no)
shaped = {"schema_artifact": schema, "tables": fields["tables"]} if seen or frozen else {}
entries.append({"kind": "query", "node_id": node_id, "exec": exec_no, "artifact": artifact,
**({"via": snapshot} if snapshot else {}), "call_id": call_id, "tool": name,
"source": fields["source"], "columns": fields["columns"], "rows": fields["rows"],
"truncated": fields["truncated"]})
"truncated": fields["truncated"], **shaped})
if frozen:
entries.append(frozen)
queries.append({"alias": alias, "call_id": call_id, "tool": name, "sql": str(args.get("sql") or ""),
"artifact": artifact, "via": snapshot, "columns": fields["columns"],
"rows": data.get("rows") or [], "truncated": fields["truncated"]})
+69 -12
View File
@@ -11,6 +11,12 @@
5. 收尾:文档落成 report_doc 工件,report.checked 事件进封存;仍然违规时按
on_violation 失败(fail)或者照常产出、把违规片段标出来(flag)
三期起目录里还有表和字段(这次运行冻结的表结构)、知识库检索的原话。可疑实体(反引号里
写了个哪里都找不到的名字)只标注:不重写、不失败,受管级别的正式出具按缺口降档;表结构快照
不全时核对不了的名字(unverified_entity)只标注。entities: off 整层不管表名、字段名。
结论句策略 claims:off(默认)不管;require_citation 在写作提示里要求结论句挂依据(自动链接的
名字不算依据),没挂的计入缺口、出口降档;judge(模型裁判)是后续版本的事,现在写了直接报错。
出口契约复核(io.py)用 report_catalog 按同一套参数重建目录、独立重算,不信这里的自查。
"""
from __future__ import annotations
@@ -26,6 +32,8 @@ from app.db.base import SessionLocal
from app.engine.context import NodeContext, NodeError
from app.engine.evidence import (
CELLS_REASON,
CLAIMS_RULE,
ENTITY_KINDS,
MARKER_RULES,
StreamRenderer,
build_catalog,
@@ -35,13 +43,20 @@ from app.engine.evidence import (
ledger_enabled,
)
from app.engine.nodes.llm import _invoke_streaming, _model_spec, _report_call
from app.engine.schema import GraphNode, GraphSpec, NodeType, _ancestors
from app.engine.schema import REPORT_CLAIMS, GraphNode, GraphSpec, NodeType, _ancestors, claims_problem
from app.engine.state import GraphState, message_text, thinking_text
from app.engine.toolcalls import leaked_markup, markup_warning
from app.providers.factory import ProviderNotConfigured, get_chat_model
ON_VIOLATION = ("fail", "flag")
NUMBERS = ("strict", "off")
#: 实体层:link(默认)核对表名、字段名并自动链接;off 整层不管(目录里不收表和字段)
ENTITIES = ("link", "off")
#: 结论句策略,和画布校验同一份(schema.REPORT_CLAIMS);judge 是后续版本的事
CLAIMS = REPORT_CLAIMS
#: 只标注、不让写作者重写也不判失败的违规:可疑实体(受管的正式出具按缺口降档)、表结构快照不全时
#: 核对不了的名字(出口只标注)
SOFT = frozenset({"unknown_entity", "unverified_entity"})
#: 重写一次就是多一次整篇的模型调用,写不对的模型多给几次也大多写不对
MAX_REPAIRS = 3
@@ -79,16 +94,38 @@ def report_catalog(state: GraphState, spec: GraphSpec, node: GraphNode) -> dict[
if isinstance(sources, str) and sources != "ancestors":
sources = [sources]
if isinstance(sources, list):
# evidence_from 只管查询、检索这类证据从哪几个节点来;指标从哪来由 metrics_from 管
# evidence_from 只管查询、检索这类证据(连同查询当时的表结构)从哪几个节点来;指标从哪来由 metrics_from 管
keep = {str(s) for s in sources}
ledger = [e for e in ledger if e.get("kind") not in ("query", "retrieval") or e.get("node_id") in keep]
return build_catalog(
ledger = [e for e in ledger
if e.get("kind") not in ("query", "retrieval", "schema") or e.get("node_id") in keep]
catalog = build_catalog(
nodes=state.get("nodes") or {},
ledger=ledger,
inputs=state.get("input") or {},
metrics_from=_setting(spec, node, "metrics_from") or None,
allowed=_ancestors(spec, node.id),
)
if report_entities(spec, node) == "off":
# 目录里没有表和字段,写作目录也就不列;compose / verify 另传 entities=False,解析不了的原因照实说
catalog = {alias: e for alias, e in catalog.items() if e.get("kind") not in ENTITY_KINDS}
return catalog
def report_entities(spec: GraphSpec, node: GraphNode) -> str:
"""报告节点的实体层:link(默认)/ off。出口复核调 verify_doc 时传 entities=(这个值 == "link")。
写错的值执行时节点会报错,这里只把 off 当 off,别的一律按默认。
"""
return "off" if _setting(spec, node, "entities", "link") == "off" else "link"
def report_claims(spec: GraphSpec, node: GraphNode) -> str:
"""报告节点的结论句策略:off / require_citation。出口判档按它(和节点执行时同一个取值规则)。
写错的值(包括 judge)画布校验就拦了,执行时节点也会报错,这里只把没写当成 off。
"""
value = _setting(spec, node, "claims", "off")
return value if isinstance(value, str) and value in CLAIMS else "off"
def report_allowance(spec: GraphSpec, node: GraphNode) -> list[Any]:
@@ -185,6 +222,12 @@ def _options(ctx: NodeContext) -> tuple[str, int]:
numbers = str(ctx.cfg("numbers", "strict") or "strict")
if numbers not in NUMBERS:
raise NodeError(ctx.node.id, f"numbers 只能是 {' / '.join(NUMBERS)},写的是 {numbers!r}")
# 画布校验也拦这两项;执行时再守一道,报的是同一句话,免得写错的值被悄悄当成默认
if problem := claims_problem(ctx.cfg("claims")):
raise NodeError(ctx.node.id, problem)
entities = ctx.cfg("entities", "link") or "link"
if entities not in ENTITIES:
raise NodeError(ctx.node.id, f"entities 只能是 {' / '.join(ENTITIES)},写的是 {entities!r}")
raw = ctx.cfg("max_repairs", 1)
try:
repairs = int(raw)
@@ -196,8 +239,13 @@ def _options(ctx: NodeContext) -> tuple[str, int]:
def _blocking(violations: list[dict[str, Any]], numbers: str) -> list[dict[str, Any]]:
"""要让写作者改的违规。numbers=off 时裸数字只记录不拦(文档和出口复核照样看得到)。"""
return [v for v in violations if not (numbers == "off" and v.get("code") == "uncited_number")]
"""要让写作者改的违规。numbers=off 时裸数字只记录不拦(文档和出口复核照样看得到)。
可疑实体只标注(SOFT):一个反引号里的业务词也可能被当成名字,为它整篇重写、判失败都不值;
它留在文档和出口复核里,受管的正式出具按缺口降档。表结构快照不全时核对不了的名字同样只标注。
"""
return [v for v in violations if v.get("code") not in SOFT
and not (numbers == "off" and v.get("code") == "uncited_number")]
def _named(violations: list[dict[str, Any]], limit: int = 3) -> str:
@@ -216,14 +264,16 @@ def _repair_request(violations: list[dict[str, Any]]) -> str:
)
def _messages(ctx: NodeContext, state: GraphState, catalog: dict[str, Any], cells: bool = True) -> list[BaseMessage]:
def _messages(ctx: NodeContext, state: GraphState, catalog: dict[str, Any], cells: bool = True,
claims: str = "off") -> list[BaseMessage]:
system = ctx.render_str(ctx.cfg("system", ""), state)
instructions = ctx.render_str(ctx.cfg("instructions", ""), state).strip() \
or "根据下面的证据写一份简洁的报告,先总后分。"
rules = f"{MARKER_RULES}\n{CLAIMS_RULE}" if claims == "require_citation" else MARKER_RULES
return [
SystemMessage(content="\n\n".join(p for p in (system, ROLE) if p)),
HumanMessage(content=(
f"{instructions}\n\n{catalog_prompt(catalog, cells_allowed=cells)}\n\n{MARKER_RULES}\n\n"
f"{instructions}\n\n{catalog_prompt(catalog, cells_allowed=cells)}\n\n{rules}\n\n"
"只输出报告正文(Markdown),不要写撰写说明。"
)),
]
@@ -257,7 +307,10 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
catalog = report_catalog(state, spec, ctx.node)
allow = report_allowance(spec, ctx.node)
cells = True
if ledger_enabled(ctx.run):
# 结论句策略只对升级后发起的运行生效:升级前的运行(图里也不会有 claims)事件、产出一个字不加
evidence_on = ledger_enabled(ctx.run)
claims = report_claims(spec, ctx.node) if evidence_on else "off"
if evidence_on:
# 升级前发起的运行没有查询条目,单元格无从谈起,也就不多查这一次
from app.engine.governance import governed_formal
@@ -265,7 +318,7 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
if warning := _no_evidence(catalog, cells):
ctx.emit(EventType.LOG, level="warn", code="report_no_evidence", message=warning)
messages = _messages(ctx, state, catalog, cells)
messages = _messages(ctx, state, catalog, cells, claims)
started = time.perf_counter()
spent: list[dict[str, Any]] = []
nudged = False
@@ -291,9 +344,11 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
message="模型只输出了思考内容,报告正文为空。把 max_tokens 调大,或把 thinking 设为 off。")
return response, text
entities = report_entities(spec, ctx.node) == "link"
def compose(raw: str) -> dict[str, Any]:
return compose_doc(raw, catalog, node_id=ctx.node.id, run_id=ctx.run.run_id, allow_numbers=allow,
cells_allowed=cells)
cells_allowed=cells, entities=entities)
response, raw = await draft(messages)
doc = compose(raw)
@@ -310,7 +365,8 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
doc_artifact = await _store(doc, ctx)
ctx.emit(EventType.REPORT_CHECKED, doc_artifact=doc_artifact, ok=not doc["violations"],
stats=doc["stats"], violations=doc["violations"][:20], repairs=repairs,
on_violation=on_violation, failed=bool(blocking) and on_violation == "fail")
on_violation=on_violation, failed=bool(blocking) and on_violation == "fail",
**({"claims": claims} if evidence_on else {}))
if blocking and on_violation == "fail":
head = ";".join(v["message"] for v in blocking[:5])
more = f";…另有 {len(blocking) - 5} 处" if len(blocking) > 5 else ""
@@ -331,6 +387,7 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
"repairs": repairs,
"model": model_id,
"duration_ms": int((time.perf_counter() - started) * 1000),
**({"claims": claims} if evidence_on else {}),
}
updates: dict[str, Any] = {"nodes": {ctx.node.id: result}, "usage": usage}
if ctx.cfg("emit_message", True):
+10 -3
View File
@@ -12,7 +12,7 @@ from langgraph.func import task
from app.core.artifact_store import canonical_json, content_hash
from app.engine.approval import read_decision
from app.engine.context import NodeContext, NodeError
from app.engine.evidence import ledger_enabled, next_exec, query_entry_fields
from app.engine.evidence import ledger_enabled, next_exec, query_entry_fields, schema_ledger_entry
from app.engine.replay import ask, once
from app.engine.state import GraphState
from app.engine.toolcalls import ToolTimeout, limit_fields, limit_of, run_bounded
@@ -175,9 +175,11 @@ async def run_tool(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
snapshot_id = None
# 只认数据源工具交回的查询:别的工具(MCP、自定义)拼出同样形状的 JSON,里面的工件 id 不是我们落的
query = query_entry_fields(result) if evidence_on and name.startswith(QUERY_PREFIX) else None
schema = query.get("schema_artifact") if query else None
ctx.emit(EventType.TOOL_END, tool=name, duration_ms=elapsed,
preview=json.dumps(result, ensure_ascii=False, default=str)[:2000],
artifact=snapshot_id, **({"query_artifact": query["artifact"]} if query else {}))
artifact=snapshot_id, **({"query_artifact": query["artifact"]} if query else {}),
**({"schema_artifact": schema} if schema else {}))
updates: dict[str, Any] = {"nodes": {ctx.node.id: result}}
if evidence_on:
@@ -188,9 +190,14 @@ async def run_tool(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
entries.append({"kind": "tool", "node_id": ctx.node.id, "exec": exec_no, "artifact": snapshot_id,
"tool": name})
if query:
# 查询当时的表结构快照另记一条 schema 条目;取不回来的话查询条目也不带它(不启用实体核对)
frozen = schema_ledger_entry(schema, node_id=ctx.node.id, exec_no=exec_no) if schema else None
entries.append({"kind": "query", "node_id": ctx.node.id, "exec": exec_no, "artifact": query["artifact"],
**({"via": snapshot_id} if snapshot_id else {}), "tool": name, "source": query["source"],
"columns": query["columns"], "rows": query["rows"], "truncated": query["truncated"]})
"columns": query["columns"], "rows": query["rows"], "truncated": query["truncated"],
**({"schema_artifact": schema, "tables": query["tables"]} if frozen else {})})
if frozen:
entries.append(frozen)
if entries:
updates["evidence"] = entries
var_name = ctx.cfg("assign_to", "")
+35 -4
View File
@@ -422,6 +422,34 @@ def _check_report_sources(node: GraphNode, spec: GraphSpec, result: ValidationRe
node_id=node.id, field="metrics_from", code=code)
#: 报告撰写节点的结论句策略(claims)。off:结论句不参与判档(默认,保持以前的行为);
#: require_citation:没挂引用的结论句计入缺口、按出具档位降档。judge(模型裁判)是后续版本的
REPORT_CLAIMS = ("off", "require_citation")
CLAIMS_LATER = "结论句裁判在后续版本支持"
def claims_problem(value: Any) -> str | None:
"""claims 的取值有什么问题,没问题返回 None。校验和报告撰写节点执行时报的是同一句话。"""
if value in (None, "") or (isinstance(value, str) and value in REPORT_CLAIMS):
return None
if value == "judge":
return f"claims 写的是 judge:{CLAIMS_LATER},现在只能写 off 或 require_citation"
return f"claims 只能是 {' / '.join(REPORT_CLAIMS)},写的是 {value!r}"
def _check_report_claims(node: GraphNode, spec: GraphSpec, result: ValidationResult) -> None:
"""claims 写错了(包括还不支持的 judge)挡住运行:执行时报告节点照样会报这一句,
与其跑完取数、算完口径卡才死在写报告那一步,不如在画图时说破。
和运行时同一个取值规则:节点上没写就取图级 defaults(NodeContext.cfg)。
"""
value = node.config.get("claims")
if value in (None, ""):
value = spec.defaults.get("claims")
if problem := claims_problem(value):
result.add(problem, node_id=node.id, field="claims", code="report.claims_invalid")
def _check_report_from(node: GraphNode, report_from: Any, spec: GraphSpec,
result: ValidationResult) -> None:
"""引用模式的契约:report_from 必须是出口上游的报告撰写节点。"""
@@ -506,6 +534,7 @@ def validate_graph(spec: GraphSpec) -> ValidationResult:
node_id=node.id, field=f"metrics[{i}]")
elif node.type == NodeType.REPORT:
_check_report_sources(node, spec, result)
_check_report_claims(node, spec, result)
elif node.type == NodeType.OUTPUT:
contract = cfg.get("contract")
if contract and not isinstance(contract, dict):
@@ -982,15 +1011,16 @@ def evidence_issues(spec: GraphSpec) -> list[ValidationIssue]:
out: list[ValidationIssue] = []
def add(message: str, node: GraphNode, field: str, level: str = "warning") -> None:
out.append(ValidationIssue(level=level, node_id=node.id, field=field, message=message)) # type: ignore[arg-type]
def add(message: str, node: GraphNode, field: str, level: str = "warning", code: str | None = None) -> None:
out.append(ValidationIssue(level=level, node_id=node.id, field=field, message=message, # type: ignore[arg-type]
code=code))
for node in spec.nodes:
cfg = node.config
if node.type == NodeType.AGENT and cfg.get("output_schema") and cfg.get("cite_fields") is not True:
add("output_schema 要开 cite_fields 才生效,会多一次抽取调用:开了之后循环结束再按 Schema "
"抽一次字段,每个字段都核对到查询结果里的那一格;不开的话 assign_to 拿到的仍是自由文字",
node, "output_schema")
node, "output_schema", code="evidence.cite_fields_off")
elif node.type == NodeType.CODE:
role = cfg.get("evidence_role")
if role not in (None, "") and role not in EVIDENCE_ROLES:
@@ -1007,7 +1037,8 @@ def evidence_issues(spec: GraphSpec) -> list[ValidationIssue]:
warned.add(producer.id)
add(f"口径卡的输入来自沙箱代码「{producer.title}」,它的角色是计算(evidence_role 不是 source):"
"沙箱里算出来的数核对不了出处。负责取数的代码节点把 evidence_role 标成 source;"
"业务计算挪进口径卡的表达式", node, f"metrics[{i}].expression")
"业务计算挪进口径卡的表达式", node, f"metrics[{i}].expression",
code="evidence.caliber_compute_input")
return out
+5 -1
View File
@@ -233,7 +233,9 @@ TEMPLATES: list[dict[str, Any]] = [
"description": "报告里的每个数都点得开出处:tool 节点查库、口径卡登记指标、报告撰写节点只写引用"
"标记,数字由系统从口径卡取出来渲染,出口按引用逐段核对后三档出具。取数节点查的是示例数据源 "
"shop 的 orders 表(week、amount、refunded 三列):先接入你的库(或者上传一张表格、命名为 "
"shop),再把 SQL 换成你的口径。",
"shop),再把 SQL 换成你的口径。报告撰写节点按受管要求写明了 on_violation: fail 和 claims: "
"require_citation:探索运行里修不掉的数字违规也会让报告节点失败,而不只是标出来;正式运行里没挂引用"
"的结论句计入缺口、按档降级。",
"tags": ["出具", "口径卡", "周报", "证据"],
"nodes": [
_n("start", "input", "周期", fields=[
@@ -265,7 +267,9 @@ TEMPLATES: list[dict[str, Any]] = [
{"id": "refund_rate", "name": "退款率", "decimals": 4, "format": "percent_of_ratio",
"expression": "vars.q.rows[0][3] / vars.q.rows[0][2] if vars.q.rows[0][2] else None"},
]),
# 受管出具要求报告撰写节点写明最严的三项(发布门禁 G3),模板照着写,拿来就能发布成受管
_n("write", "report", "报告撰写", metrics_from=["caliber"],
numbers="strict", on_violation="fail", claims="require_citation",
system="你是周报撰写人:结论先行,只写有数据支撑的判断。",
instructions="为 {{ input.week }} 写一份简短的周报:先用一两句话总结本周,"
"再分点说销售额、订单、客单价和退款的变化。"),
+27 -1
View File
@@ -144,6 +144,10 @@ def _make_query_tool(source: DataSource, ctx: ToolContext) -> StructuredTool:
payload = result.to_payload()
payload["source"] = source.name
# 表结构快照:这次查询时数据源的结构冻结下来,报告里写的表名、字段名按它核对。事后数据源
# 重新探查、改了结构,已经跑完的运行核对的还是当时那一份
if schema := await _store_schema(source, ctx):
payload["schema_artifact"] = schema
# 查询快照进工件库:出具体系的数字回指要能下钻到"这个数是哪条 SQL 查出来的"
try:
from app.core.artifact_store import put_json
@@ -172,6 +176,28 @@ def _make_query_tool(source: DataSource, ctx: ToolContext) -> StructuredTool:
)
async def _store_schema(source: DataSource, ctx: ToolContext) -> str | None:
"""把数据源此刻的 schema_cache 存成 schema_snapshot 工件,返回工件 id。
内容寻址:结构没变的话,同一次运行里查多少次都是同一件,文件只有一份。没探查过结构
(或者探查失败、一张表都没有)返回 None:没有东西可以冻结,报告也就不核对表名。
"""
cache = source.schema_cache or {}
tables = cache.get("tables")
if not isinstance(tables, dict) or not tables:
return None
from app.core.artifact_store import put_json
from app.engine.evidence import SCHEMA_SNAPSHOT
content = {"source": source.name, "tables": tables,
**{k: cache[k] for k in ("schema", "synced_at", "truncated", "total") if k in cache}}
try:
return await put_json(content, kind=SCHEMA_SNAPSHOT, run_id=ctx.run_id or "", node_id=ctx.node_id or "",
meta={"source": source.name, "tables": len(tables)})
except Exception: # noqa: BLE001 - 存不下不影响查询本身,只是这次报告不核对表名
return None
def _make_schema_tool(source: DataSource) -> StructuredTool:
async def _run(table: str | list[str] | None = None) -> str:
wanted = [table] if isinstance(table, str) else list(table or [])
@@ -206,4 +232,4 @@ def is_dangerous_call(source: DataSource, sql: str) -> bool:
只读源上的写不需要审批——守卫和连接层都会拒,批了也写不进去。
"""
return not source.readonly and is_write(sql)
return not source.readonly and is_write(sql, dialect=source.kind)
+4 -2
View File
@@ -157,9 +157,11 @@ def _governable(tool: str):
node("q", "tool", tool=tool, args={"sql": "DELETE FROM t WHERE x = 0"}),
node("caliber", "metrics", caliber="删除口径", caliber_version="v1", assign_to="m",
metrics=[{"id": "done", "name": "已执行", "unit": "次", "expression": "1"}]),
# 受管契约要核对报告撰写节点的文档(门禁 G1),叙述模式发不成受管
node("write", "report", instructions="写执行结果",
numbers="strict", on_violation="fail", claims="require_citation"),
node("out", "output", fields=[{"name": "结果", "value": "{{ nodes.q }}"}],
contract={"metrics_from": ["caliber"], "narrative": "执行了 {{ vars.m.done }} 次",
"required": ["done"], "strict": True}),
contract={"metrics_from": ["caliber"], "report_from": "write", "required": ["done"], "strict": True}),
)
+4 -2
View File
@@ -39,7 +39,8 @@ def two_exits() -> dict:
node("fetch", "tool", "取数", tool="db_query__shop", args={"sql": "SELECT SUM(amount) AS gmv FROM orders"}),
node("card", "metrics", "周报口径卡", caliber="周报口径", metrics=[
{"id": "gmv", "name": "销售额", "unit": "元", "expression": "cell(nodes.fetch, 0, 'gmv')"}]),
node("write", "report", "报告撰写", instructions="写周报"),
node("write", "report", "报告撰写", instructions="写周报",
numbers="strict", on_violation="fail", claims="require_citation"),
node("done", "output", "出具", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}]),
node("raw", "output", "原始数", fields=[{"name": "数", "value": "{{ nodes.fetch }}"}]),
]
@@ -227,7 +228,8 @@ def guarded() -> dict:
{"id": "gmv", "name": "销售额", "unit": "元", "expression": "cell(nodes.fetch, 0, 'gmv')"}]),
node("team", "supervisor", "复核团队", goal="复核", agents=[{"name": "checker", "tools": []}]),
node("gate", "human", "人工把关", prompt="确认数字"),
node("write", "report", "报告撰写", instructions="写周报"),
node("write", "report", "报告撰写", instructions="写周报",
numbers="strict", on_violation="fail", claims="require_citation"),
node("done", "output", "出具", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}], contract=CONTRACT),
]
edges = [{"id": f"e{i}", "source": a, "target": b} for i, (a, b) in enumerate(
+495
View File
@@ -0,0 +1,495 @@
"""证据门禁 G1–G5 接进发布前自动修复(engine/autofix.py)。
照发布前自动修复的原则:答案唯一、只提高要求的一键修(auto);要人拿主意的列候选、不替人选(choice);
结构性的交给 Copilot(assist)。修完重跑 validate 和门禁,错误必须变少、不能冒出新的;降低要求的改动
(numbers、on_violation、claims 往宽里改,关掉 cite_fields,替人把沙箱代码标成取数)一律拒绝。
"""
from __future__ import annotations
import copy
import json
import pytest
from httpx import ASGITransport, AsyncClient
from langchain_core.messages import AIMessageChunk
from app.engine.autofix import HANDOFF, apply_fixes, forbidden_changes, plan_fixes
from app.engine.governance import publish_issues
from app.engine.schema import GraphSpec
from app.main import app
STRICT = {"numbers": "strict", "on_violation": "fail", "claims": "require_citation"}
SCHEMA = {"type": "object", "properties": {"gmv": {"type": "number"}}}
def node(nid, ntype, label=None, **config):
return {"id": nid, "type": ntype, "position": {"x": 10, "y": 20}, "width": 240,
"data": {"label": label or nid, "config": config}}
def traceable(*, report=None, **contract_overrides) -> dict:
"""input → 查库 → 口径卡 → 报告撰写 → 出具:受管要求(含 G1–G5)全部满足。"""
contract = {"report_from": "write", "metrics_from": ["card"], "required": ["gmv"], "strict": True}
contract.update(contract_overrides)
nodes = [
node("start", "input", "周期"),
node("fetch", "tool", "取数", tool="db_query__shop",
args={"sql": "SELECT SUM(amount) AS gmv, COUNT(*) AS orders FROM orders"}),
node("card", "metrics", "周报口径卡", caliber="周报口径", metrics=[
{"id": "gmv", "name": "销售额", "unit": "元", "expression": "cell(nodes.fetch, 0, 'gmv')"}]),
node("write", "report", "报告撰写", instructions="写周报", **(STRICT if report is None else report)),
node("done", "output", "出具", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"},
{"name": "指标清单", "value": "{{ nodes.card.text }}"}],
contract={k: v for k, v in contract.items() if v is not None}),
]
edges = [{"id": f"e{i}", "source": a["id"], "target": b["id"]} for i, (a, b) in enumerate(zip(nodes, nodes[1:]))]
return {"nodes": nodes, "edges": edges, "viewport": {"x": 1, "y": 2, "zoom": 1}}
def find(graph: dict, nid: str) -> dict:
return next(n for n in graph["nodes"] if n["id"] == nid)
def config(graph: dict, nid: str) -> dict:
return find(graph, nid)["data"]["config"]
def insert(graph: dict, new: dict, *, before: str, after: str) -> dict:
graph["nodes"].insert(len(graph["nodes"]) - 1, new)
graph["edges"] += [{"source": after, "target": new["id"]}, {"source": new["id"], "target": before}]
return graph
def plan(graph: dict, level: str = "governed") -> dict[str, dict]:
spec = GraphSpec.model_validate(graph)
return {f["id"]: f for f in plan_fixes(spec, publish_issues(spec, level=level), level=level)}
def errors(graph: dict, level: str = "governed") -> list:
return [i for i in publish_issues(GraphSpec.model_validate(graph), level=level) if i.level == "error"]
def codes(issues) -> set[str]:
return {i["code"] if isinstance(i, dict) else i.code for i in issues}
def test_the_baseline_needs_no_fix():
assert errors(traceable()) == [] and plan(traceable()) == {}
# --------------------------------------------------------------------------
# G3:一键写明 numbers / on_violation / claims
# --------------------------------------------------------------------------
def test_report_policy_is_filled_in_with_one_auto_fix():
graph = traceable(report={"numbers": "off"})
fid = "governed.report_policy:write"
fix = plan(graph)[fid]
assert fix["kind"] == "auto" and fix["node_id"] == "write"
assert fix["preview"]["after"] == STRICT and fix["preview"]["before"] == {
"numbers": "off", "on_violation": None, "claims": None}
out = apply_fixes(graph, [fid], level="governed")
assert out["applied"] == [fid] and out["rejected"] == [] and out["ok"] is True
assert {k: config(out["graph"], "write")[k] for k in STRICT} == STRICT
assert config(out["graph"], "write")["instructions"] == "写周报" # 别的配置原样留着
assert [(c["field"], c["before"], c["after"]) for c in out["changes"]] == [
("numbers", "off", "strict"), ("on_violation", None, "fail"), ("claims", None, "require_citation")]
assert out["ops"] == [{"op": "update_node", "id": "write", "config": STRICT}]
# 坐标、视口这些前端的东西原样留着
assert find(out["graph"], "write")["position"] == {"x": 10, "y": 20} and out["graph"]["viewport"]["zoom"] == 1
def test_only_the_missing_settings_are_written():
graph = traceable(report={"numbers": "strict", "claims": "require_citation"})
fix = plan(graph)["governed.report_policy:write"]
assert fix["preview"] == {"field": "on_violation", "before": None, "after": "fail"}
out = apply_fixes(graph, [fix["id"]], level="governed")
assert [c["field"] for c in out["changes"]] == ["on_violation"] and out["ok"] is True
def test_published_level_warnings_get_no_fix():
graph = traceable(report={})
assert plan(graph, "published") == {}
# --------------------------------------------------------------------------
# validate:claims 写了不支持的值
# --------------------------------------------------------------------------
def test_judge_becomes_require_citation_under_governed():
graph = traceable(report={**STRICT, "claims": "judge"})
fid = "report.claims_invalid:write"
fix = plan(graph)[fid]
assert fix["kind"] == "auto" and fix["preview"] == {"field": "claims", "before": "judge",
"after": "require_citation"}
out = apply_fixes(graph, [fid], level="governed")
assert out["ok"] is True and config(out["graph"], "write")["claims"] == "require_citation"
def test_an_invalid_claims_value_is_a_choice_under_published():
graph = traceable(report={"claims": "judge"})
fid = "report.claims_invalid:write"
fix = plan(graph, "published")[fid]
assert fix["kind"] == "choice" and [o["value"] for o in fix["options"]] == ["off", "require_citation"]
assert fix["default"] == "require_citation" # 离裁判最近的是「每句结论挂引用」,仍然要人确认
assert apply_fixes(graph, [fid], level="published")["applied"] == []
out = apply_fixes(graph, [fid], {fid: "off"}, level="published")
assert out["applied"] == [fid] and config(out["graph"], "write")["claims"] == "off" and out["ok"] is True
# 乱写的值没有「最接近的」,不给建议
assert "default" not in plan(traceable(report={"claims": "strict"}), "published")[fid]
# --------------------------------------------------------------------------
# G5:给口径卡供数的 agent
# --------------------------------------------------------------------------
def feeding_agent(**config_) -> dict:
graph = insert(traceable(), node("ask", "agent", "查数员", tools=["db_query__shop"], prompt="查 orders",
assign_to="facts", **config_), before="card", after="start")
config(graph, "card")["metrics"].append({"id": "gmv2", "expression": "vars.facts.gmv"})
return graph
def test_cite_fields_is_switched_on_when_the_schema_exists():
graph = feeding_agent(output_schema=SCHEMA)
fid = "governed.caliber_agent_cite_fields:ask"
fix = plan(graph)[fid]
assert fix["kind"] == "auto" and fix["preview"] == {"field": "cite_fields", "before": None, "after": True}
out = apply_fixes(graph, [fid], level="governed")
assert out["ok"] is True and config(out["graph"], "ask")["cite_fields"] is True
assert config(out["graph"], "ask")["output_schema"] == SCHEMA
def test_a_missing_schema_is_not_invented():
graph = feeding_agent()
fid = "governed.caliber_agent_schema:ask"
fix = plan(graph)[fid]
assert fix["kind"] == "assist" and "output_schema" in fix["label"]
out = apply_fixes(graph, [fid], level="governed")
assert out["applied"] == [] and out["rejected"][0]["fix_id"] == fid
assert "output_schema" not in config(out["graph"], "ask") and out["graph"] == graph
def test_a_model_call_feeding_a_card_goes_to_copilot():
graph = insert(traceable(), node("guess", "llm", "估算", prompt="估一个数", assign_to="g"), before="card",
after="start")
config(graph, "card")["metrics"].append({"id": "g", "expression": "vars.g"})
assert plan(graph)["governed.caliber_model_input:guess"]["kind"] == "assist"
# --------------------------------------------------------------------------
# G4:沙箱代码喂口径卡——它到底是在取数还是在算,要人说
# --------------------------------------------------------------------------
def compute_feeding() -> dict:
graph = insert(traceable(), node("calc", "code", "计算", code="print(1)", assign_to="calc"), before="card",
after="fetch")
config(graph, "card")["metrics"].append({"id": "ratio", "expression": "vars.calc.ratio"})
return graph
def test_compute_code_is_a_choice_not_an_auto_fix():
graph = compute_feeding()
fid = "governed.caliber_compute_input:calc"
fix = plan(graph)[fid]
assert fix["kind"] == "choice" and fix["node_id"] == "calc" and fix["field"] == "evidence_role"
assert [o["value"] for o in fix["options"]] == ["source", HANDOFF]
assert "default" not in fix and "preview" not in fix
# 不选就不动
out = apply_fixes(graph, [fid], level="governed")
assert out["applied"] == [] and "evidence_role" not in config(out["graph"], "calc")
assert out["rejected"][0]["fix_id"] == fid
def test_choosing_source_marks_the_node():
graph = compute_feeding()
fid = "governed.caliber_compute_input:calc"
out = apply_fixes(graph, [fid], {fid: "source"}, level="governed")
assert out["applied"] == [fid] and out["ok"] is True
assert config(out["graph"], "calc")["evidence_role"] == "source"
def test_choosing_copilot_hands_the_node_over_without_touching_it():
graph = compute_feeding()
fid = "governed.caliber_compute_input:calc"
out = apply_fixes(graph, [fid], {fid: HANDOFF}, level="governed")
assert out["applied"] == [] and out["rejected"] == [] and out["handoff"] == [fid]
assert out["graph"] == graph and out["ok"] is False
def test_copilot_may_not_mark_a_code_node_as_source_on_its_own():
before = compute_feeding()
after = copy.deepcopy(before)
config(after, "calc")["evidence_role"] = "source"
[reason] = forbidden_changes(before, after)
assert "「计算」" in reason and "source" in reason
# 新加的沙箱代码直接标成取数也一样
after = copy.deepcopy(before)
after["nodes"].append(node("raw", "code", "取原始数据", code="print(2)", evidence_role="source"))
assert any("「取原始数据」" in r for r in forbidden_changes(before, after))
# --------------------------------------------------------------------------
# G1:契约和成果字段接到报告撰写节点
# --------------------------------------------------------------------------
def narrative() -> dict:
return traceable(report_from=None, narrative="{{ nodes.write.text }}")
def test_report_from_takes_the_only_report_upstream():
graph = narrative()
fid = "governed.report_from_required:done"
fix = plan(graph)[fid]
assert fix["kind"] == "auto" and fix["preview"] == {"field": "contract.report_from", "before": None,
"after": "write"}
out = apply_fixes(graph, [fid], level="governed")
assert out["ok"] is True and config(out["graph"], "done")["contract"]["report_from"] == "write"
def test_several_reports_make_report_from_a_choice():
graph = insert(narrative(), node("write2", "report", "第二份报告", instructions="写月报", **STRICT),
before="done", after="card")
fid = "governed.report_from_required:done"
fix = plan(graph)[fid]
assert fix["kind"] == "choice" and [o["value"] for o in fix["options"]] == ["write", "write2"]
assert apply_fixes(graph, [fid], level="governed")["applied"] == []
out = apply_fixes(graph, [fid], {fid: "write2"}, level="governed")
assert out["applied"] == [fid] and config(out["graph"], "done")["contract"]["report_from"] == "write2"
def test_without_a_report_node_report_from_goes_to_copilot():
graph = narrative()
write = find(graph, "write")
write.update(type="llm", data={"label": "写周报", "config": {"prompt": "写周报"}})
# 受管级别的叙述模式过不了 G1,不再给「选 narrative」这种修了也发不出去的候选
for fid in ("governed.report_from_required:done",):
assert plan(graph)[fid]["kind"] == "assist"
bare = traceable(report_from=None)
find(bare, "write").update(type="llm", data={"label": "写周报", "config": {"prompt": "写周报"}})
assert plan(bare)["contract.report_from_missing:done"]["kind"] == "assist"
def with_story(graph: dict) -> dict:
graph = insert(graph, node("narrate", "llm", "叙述", prompt="写周报", assign_to="story"), before="done",
after="card")
config(graph, "done")["fields"] += [{"name": "摘要", "value": "{{ vars.story }}"},
{"name": "点评", "value": "点评:{{ nodes.narrate.text }}"}]
return graph
def test_exit_fields_are_repointed_at_the_chosen_report():
graph = with_story(traceable())
fid = "governed.exit_text_source:done"
fix = plan(graph)[fid]
assert fix["kind"] == "choice" and fix["field"] == "fields"
assert [o["value"] for o in fix["options"]] == ["write"] and fix["default"] == "write"
assert "「摘要」" in fix["label"] and "「点评」" in fix["label"]
# 就一个候选也不自动改:字段里放什么是作者的事
out = apply_fixes(graph, [fid], level="governed")
assert out["applied"] == [] and out["graph"] == graph
out = apply_fixes(graph, [fid], {fid: "write"}, level="governed")
assert out["applied"] == [fid] and out["ok"] is True
fields = config(out["graph"], "done")["fields"]
assert fields == [{"name": "周报", "value": "{{ nodes.write.text }}"},
{"name": "指标清单", "value": "{{ nodes.card.text }}"},
{"name": "摘要", "value": "{{ nodes.write.text }}"},
{"name": "点评", "value": "{{ nodes.write.text }}"}]
assert [(c["field"], c["before"], c["after"]) for c in out["changes"]] == [
("fields[2].value", "{{ vars.story }}", "{{ nodes.write.text }}"),
("fields[3].value", "点评:{{ nodes.narrate.text }}", "{{ nodes.write.text }}")]
assert out["ops"] == [{"op": "update_node", "id": "done", "config": {"fields": fields}}]
def test_an_exit_without_fields_gets_one_that_takes_the_report():
graph = traceable()
config(graph, "done")["fields"] = []
graph = insert(graph, node("polish", "llm", "润色", prompt="{{ nodes.write.text }}"), before="done",
after="write")
graph["edges"] = [e for e in graph["edges"] if (e["source"], e["target"]) != ("write", "done")]
fid = "governed.exit_text_source:done"
out = apply_fixes(graph, [fid], {fid: "write"}, level="governed")
assert out["applied"] == [fid] and out["ok"] is True
assert config(out["graph"], "done")["fields"] == [{"name": "result", "value": "{{ nodes.write.text }}"}]
def test_exit_fields_without_any_report_go_to_copilot():
graph = with_story(traceable())
find(graph, "write").update(type="llm", data={"label": "写周报", "config": {"prompt": "写周报"}})
config(graph, "done")["contract"].pop("report_from")
assert plan(graph)["governed.exit_text_source:done"]["kind"] == "assist"
# --------------------------------------------------------------------------
# G2:结构性的,交给 Copilot
# --------------------------------------------------------------------------
def test_text_bypass_is_an_assist():
graph = traceable()
graph["nodes"] += [node("chat", "llm", "闲聊", prompt="你好"),
node("side", "output", "附带成果", fields=[{"name": "回答", "value": "{{ nodes.chat.text }}"}])]
graph["edges"] += [{"source": "start", "target": "chat"}, {"source": "chat", "target": "side"}]
fid = "governed.text_bypass:chat"
fix = plan(graph)[fid]
assert fix["kind"] == "assist" and "报告撰写" in fix["label"]
out = apply_fixes(graph, [fid], level="governed")
assert out["applied"] == [] and out["rejected"][0]["fix_id"] == fid
# --------------------------------------------------------------------------
# 修完更好、不降低要求
# --------------------------------------------------------------------------
def test_all_auto_fixes_together_leave_only_the_decisions():
graph = with_story(feeding_agent(output_schema=SCHEMA))
graph = insert(graph, node("calc", "code", "计算", code="print(1)", assign_to="calc"), before="card",
after="fetch")
config(graph, "card")["metrics"].append({"id": "ratio", "expression": "vars.calc.ratio"})
config(graph, "write").clear()
config(graph, "done")["contract"].update(report_from=None, narrative="{{ nodes.write.text }}")
fixes = plan(graph)
autos = [f["id"] for f in fixes.values() if f["kind"] == "auto"]
assert set(autos) == {"governed.report_policy:write", "governed.caliber_agent_cite_fields:ask",
"governed.report_from_required:done"}
before = len(errors(graph))
out = apply_fixes(graph, autos, level="governed")
assert sorted(out["applied"]) == sorted(autos) and out["rejected"] == []
assert len([i for i in out["remaining"] if i["level"] == "error"]) < before
left = {i["code"] for i in out["remaining"] if i["level"] == "error"}
assert left == {"governed.exit_text_source", "governed.caliber_compute_input"}
# 剩下的都要人拿主意,没有一条被替人应用
assert all(fixes[f]["kind"] != "auto" for f in fixes if f.split(":")[0] in left)
@pytest.mark.parametrize("key, before, after", [
("numbers", "strict", "off"), ("on_violation", "fail", "flag"), ("on_violation", "fail", None),
("claims", "require_citation", "off"), ("claims", "require_citation", None),
])
def test_loosening_the_report_policy_is_refused(key, before, after):
old = traceable(report={**STRICT, key: before})
new = copy.deepcopy(old)
if after is None:
config(new, "write").pop(key)
else:
config(new, "write")[key] = after
[reason] = forbidden_changes(old, new)
assert "「报告撰写」" in reason and key in reason
def test_turning_cite_fields_off_is_refused():
old = feeding_agent(output_schema=SCHEMA, cite_fields=True)
new = copy.deepcopy(old)
config(new, "ask")["cite_fields"] = False
[reason] = forbidden_changes(old, new)
assert "「查数员」" in reason and "cite_fields" in reason
def test_tightening_is_not_refused():
old = traceable(report={"numbers": "off", "on_violation": "flag", "claims": "off"})
new = copy.deepcopy(old)
config(new, "write").update(STRICT)
assert forbidden_changes(old, new) == []
# --------------------------------------------------------------------------
# 接口:发布前检查带上 G 规则的修复;选了「交给 Copilot」就真的交给 Copilot
# --------------------------------------------------------------------------
@pytest.fixture
async def client():
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
class Scripted:
"""按剧本吐操作流的 Copilot 模型;记下每次收到的请求。"""
def __init__(self, ops: list[dict]) -> None:
self.lines = [json.dumps(o, ensure_ascii=False) for o in ops]
self.calls: list[list] = []
async def astream(self, messages):
self.calls.append(messages)
for line in self.lines:
yield AIMessageChunk(content=line + "\n")
def script(monkeypatch, ops: list[dict]) -> Scripted:
import app.api.copilot as copilot
model = Scripted([{"op": "plan", "summary": "改图"}, *ops, {"op": "done", "explanation": "改好了"}])
async def _model(*_a, **_k):
return model, "mock-fast"
monkeypatch.setattr(copilot, "get_chat_model", _model)
return model
async def create(client, graph: dict) -> str:
return (await client.post("/api/workflows", json={"name": "证据门禁-修复", "graph": graph})).json()["id"]
async def test_publish_check_lists_the_g_fixes(client):
graph = with_story(compute_feeding())
config(graph, "write").clear()
wf = await create(client, graph)
body = (await client.post(f"/api/workflows/{wf}/publish-check", json={"level": "governed"})).json()
assert body["ok"] is False
kinds = {f["id"]: f["kind"] for f in body["fixes"]}
assert kinds["governed.report_policy:write"] == "auto"
assert kinds["governed.caliber_compute_input:calc"] == "choice"
assert kinds["governed.exit_text_source:done"] == "choice"
assert all(not any(k.startswith("_") for k in f) for f in body["fixes"]) # 内部信息不外露
by_code = {i["code"]: i for i in body["issues"]}
assert by_code["governed.report_policy"]["fix"] == "governed.report_policy:write"
assert by_code["governed.exit_text_source"]["fix"] == "governed.exit_text_source:done"
# 已发布级别:G 规则只是警告,不给修复
body = (await client.post(f"/api/workflows/{wf}/publish-check", json={"level": "published"})).json()
assert {i["level"] for i in body["issues"] if (i["code"] or "").startswith("governed.")} == {"warning"}
assert not [f for f in body["fixes"] if f["code"].startswith("governed.")]
async def test_choosing_copilot_in_the_choice_runs_the_assist(client, monkeypatch):
# Copilot 把口径卡里读沙箱代码的那条指标改成直接读查询结果:计算挪进了口径卡
card = [{"id": "gmv", "name": "销售额", "unit": "元", "expression": "cell(nodes.fetch, 0, 'gmv')"},
{"id": "ratio", "expression": "cell(nodes.fetch, 0, 'gmv') / 100"}]
model = script(monkeypatch, [{"op": "update_node", "id": "card", "config": {"metrics": card}}])
wf = await create(client, compute_feeding())
fid = "governed.caliber_compute_input:calc"
out = (await client.post(f"/api/workflows/{wf}/autofix", json={
"level": "governed", "apply": [fid], "choices": {fid: HANDOFF}})).json()
assert out["handoff"] == [fid] and len(model.calls) == 1
human = next(text for role, text in model.calls[0] if role == "human")
assert "「计算」" in human and "evidence_role" in human
assert out["assist"]["ok"] is True and out["applied"] == ["assist"] and out["ok"] is True, out
assert "evidence_role" not in config(out["graph"], "calc")
# 没选「交给 Copilot」、也没带 assist 的,不调模型
model.calls.clear()
out = (await client.post(f"/api/workflows/{wf}/autofix", json={
"level": "governed", "apply": [fid], "choices": {fid: "source"}})).json()
assert model.calls == [] and out["assist"] is None and out["applied"] == [fid]
@pytest.mark.parametrize("ops, word", [
([{"op": "update_node", "id": "calc", "config": {"evidence_role": "source"}}], "evidence_role"),
([{"op": "update_node", "id": "write", "config": {"numbers": "off"}},
{"op": "update_node", "id": "calc", "config": {"evidence_role": "source"}}], "numbers"),
])
async def test_copilot_cannot_decide_for_the_author_or_loosen_the_report(client, monkeypatch, ops, word):
script(monkeypatch, ops)
wf = await create(client, compute_feeding())
out = (await client.post(f"/api/workflows/{wf}/autofix", json={
"level": "governed", "apply": [], "assist": True})).json()
assert out["assist"]["ok"] is False and out["applied"] == []
[refused] = [r for r in out["rejected"] if r["fix_id"] == "assist"]
assert "降低了要求" in refused["reason"] and word in refused["reason"]
assert out["graph"] == (await client.get(f"/api/workflows/{wf}")).json()["graph"]
+2 -1
View File
@@ -39,7 +39,8 @@ def weekly(default: str | None = "never", **ask_config) -> dict:
node("ask", "agent", "查数员", **ask),
node("card", "metrics", "周报口径卡", caliber="周报口径", metrics=[
{"id": "gmv", "name": "销售额", "unit": "元", "expression": "vars.gmv"}]),
node("write", "report", "报告撰写", instructions="写周报"),
node("write", "report", "报告撰写", instructions="写周报",
numbers="strict", on_violation="fail", claims="require_citation"),
node("done", "output", "出具", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}],
contract={"report_from": "write", "metrics_from": ["card"], "required": ["gmv"], "strict": True}),
]
+9 -7
View File
@@ -32,7 +32,8 @@ def weekly(**contract_overrides) -> dict:
node("card", "metrics", "周报口径卡", caliber="周报口径", metrics=[
{"id": "gmv", "name": "销售额", "unit": "元", "expression": "cell(nodes.fetch, 0, 'gmv')"},
{"id": "orders", "name": "订单数", "unit": "单", "expression": "cell(nodes.fetch, 0, 'orders')"}]),
node("write", "report", "报告撰写", instructions="写周报"),
node("write", "report", "报告撰写", instructions="写周报",
numbers="strict", on_violation="fail", claims="require_citation"),
node("done", "output", "出具", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}],
contract={k: v for k, v in contract.items() if v is not None}),
]
@@ -70,7 +71,8 @@ def add_card(graph: dict, nid: str = "card2") -> dict:
def add_report(graph: dict, nid: str = "write2") -> dict:
graph["nodes"].insert(4, node(nid, "report", "第二份报告", instructions="写月报"))
graph["nodes"].insert(4, node(nid, "report", "第二份报告", instructions="写月报",
numbers="strict", on_violation="fail", claims="require_citation"))
graph["edges"] += [{"source": "card", "target": nid}, {"source": nid, "target": "done"}]
return graph
@@ -174,17 +176,17 @@ def test_a_wrong_report_from_takes_the_only_upstream_report():
assert out["ok"] is True and config(out["graph"], "done")["contract"]["report_from"] == "write"
def test_without_a_report_node_the_narrative_is_a_choice_over_model_nodes():
def test_without_a_report_node_report_from_goes_to_copilot():
# 以前这里让人在模型节点里选一段 narrative。受管级别的叙述模式过不了门禁 G1
# (governed.report_from_required),选了也发不出去:没有报告撰写节点就交给 Copilot 加一个
graph = weekly(report_from=None)
write = find(graph, "write")
write.update(type="llm", data={"label": "写周报", "config": {"prompt": "写周报", "assign_to": "report"}})
fid = "contract.report_from_missing:done"
fix = plan(graph)[fid]
assert fix["kind"] == "choice" and fix["field"] == "contract.narrative"
assert [o["value"] for o in fix["options"]] == ["{{ nodes.write.text }}"]
assert fix["kind"] == "assist" and fix["field"] == "contract.report_from"
out = apply_fixes(graph, [fid], {fid: "{{ nodes.write.text }}"}, level="governed")
assert config(out["graph"], "done")["contract"]["narrative"] == "{{ nodes.write.text }}"
assert "contract.report_from_missing" not in codes(out["remaining"])
assert out["applied"] == [] and "narrative" not in config(out["graph"], "done")["contract"]
def test_strict_off_is_turned_on():
+360
View File
@@ -0,0 +1,360 @@
"""出口契约(引用模式)按报告节点的 claims 和可疑实体判档。
- claims: require_citation:没挂依据的结论句计入缺口,按 decide_tier 的 claims_policy 降档(不是不予出具,
strict 也一样);数的是出口重新核对出来的(verify_doc 的 uncited),不信文档里记的 cites。off(默认)
照旧不管。升级前发起的运行不管图里写没写,一律按 off
- 可疑实体(unknown_entity):只在受管级别的正式运行里按「有缺口」降档,不拦截;探索运行、已发布级别
只在 _issuance.entities 里标注。表结构快照不全时核对不了的名字(unverified_entity)在哪都只标注
- 契约自己写 claims: require_citation 的同样生效;写 judge 的仍是后续版本,照实记缺口
- 契约只能收紧、不能放松:报告节点要求了,契约写 ignore / off 照旧降档,写 withhold 收紧成不予出具;
受管级别的正式运行里 ignore 一律当 degrade(没有哪道门禁查契约里的 claims)
"""
from __future__ import annotations
import asyncio
import sqlite3
import uuid
from types import SimpleNamespace
import pytest
from httpx import ASGITransport, AsyncClient
from langchain_core.messages import AIMessage
from sqlalchemy import select
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import DataSource, Run, RunEvent, Workflow, WorkflowVersion
from app.engine import runner as runner_mod
from app.engine.runner import run_manager
from app.main import app
SOURCE = "warehouse"
TOOL = f"db_query__{SOURCE}"
SQL = "SELECT region, SUM(amount) AS gmv, COUNT(*) AS order_cnt FROM orders GROUP BY region ORDER BY gmv DESC"
CLEAN = "东区销售额 [[m:gmv]],来自 `orders`。[[see:m:gmv]]"
UNCITED = CLEAN + "增长主要来自新客。"
SUSPICIOUS = "东区销售额 [[m:gmv]],来自 `orders`,`refund_log` 另算。[[see:m:gmv]]"
@pytest.fixture(autouse=True)
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
@pytest.fixture
async def client():
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
async def _warehouse(tmp_path, *, truncated: bool = False):
from app.data.introspect import introspect
path = tmp_path / "warehouse.db"
db = sqlite3.connect(path)
db.executescript(
"CREATE TABLE orders (id INTEGER PRIMARY KEY, region TEXT, amount REAL);"
"CREATE TABLE refunds (id INTEGER PRIMARY KEY, order_id INTEGER, amount REAL);")
db.executemany("INSERT INTO orders VALUES (?, ?, ?)", [(1, "east", 100.5), (2, "west", 200.0), (3, "east", 300.0)])
db.commit()
db.close()
probe = SimpleNamespace(id=f"probe-{path.name}", name=SOURCE, kind="sqlite", database=str(path), host=None,
port=None, username=None, password=None, options={}, readonly=True, schema_cache={})
cache = await introspect(probe)
if truncated:
# 库里的表太多、探查只存了一部分:快照不全,找不到的名字只能说核对不了
cache.update(truncated=True, total=500)
await engines.invalidate(probe.id)
async with SessionLocal() as session:
row = (await session.execute(select(DataSource).where(DataSource.name == SOURCE))).scalar_one_or_none()
if row is None:
row = DataSource(name=SOURCE, kind="sqlite", database=str(path), readonly=True, enabled=True)
session.add(row)
row.database, row.kind, row.enabled, row.readonly, row.schema_cache = str(path), "sqlite", True, True, cache
row.options = {}
await session.commit()
source_id = row.id
await engines.invalidate(source_id)
return source_id
@pytest.fixture
async def warehouse(tmp_path):
source_id = await _warehouse(tmp_path)
yield source_id
await engines.invalidate(source_id)
@pytest.fixture
async def partial_warehouse(tmp_path):
source_id = await _warehouse(tmp_path, truncated=True)
yield source_id
await engines.invalidate(source_id)
def node(nid, ntype, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
def weekly(*, report=None, contract=None):
nodes = [
node("start", "input"),
node("fetch", "tool", tool=TOOL, args={"sql": SQL}),
node("card", "metrics", caliber="周报口径", caliber_version="v2", metrics=[
{"id": "gmv", "name": "销售额", "unit": "元", "decimals": 1, "format": "thousands",
"expression": "cell(nodes.fetch, 0, 'gmv')"}]),
node("write", "report", instructions="写周报", on_violation="flag", max_repairs=0, **(report or {})),
node("out", "output", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}],
contract={"report_from": "write", "metrics_from": ["card"], "required": ["gmv"], "strict": True,
**(contract or {})}),
]
return {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
def script(monkeypatch, text):
from app.providers import mock_model
monkeypatch.setattr(mock_model.MockChatModel, "_decide", lambda self, messages: AIMessage(content=text))
async def wait(run_id: str) -> Run:
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run_id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
return row
raise AssertionError(f"run {run_id} 没跑完:{row.status}")
async def explore(graph) -> dict:
run = await run_manager.start(graph=graph, input_payload={})
row = await wait(run.id)
assert row.status == "succeeded", row.error
return row.output["_issuance"]
async def formal(client, graph, *, status: str) -> Run:
"""把这张图发布成 status 级别,再从发布版本发起正式运行。"""
async with SessionLocal() as session:
wf = Workflow(name=f"claims-{status}-{uuid.uuid4().hex[:6]}", graph=graph, status=status, published_version=1)
session.add(wf)
await session.flush()
session.add(WorkflowVersion(workflow_id=wf.id, version=1, graph=graph))
await session.commit()
wf_id = wf.id
res = await client.post("/api/runs", json={"workflow_id": wf_id, "run_class": "formal"})
assert res.status_code == 201, res.text
row = await wait(res.json()["id"])
assert row.status == "succeeded", row.error
return row
async def issuance_event(run_id: str) -> dict:
async with SessionLocal() as session:
[event] = (await session.execute(select(RunEvent).where(
RunEvent.run_id == run_id, RunEvent.type == "issuance"))).scalars()
return event.data
# --------------------------------------------------------------------------
# claims
# --------------------------------------------------------------------------
async def test_require_citation_degrades_uncited_claims(warehouse, monkeypatch):
script(monkeypatch, UNCITED)
issuance = await explore(weekly(report={"claims": "require_citation"}))
assert issuance["tier"] == "degraded", "strict 下也只是降档:结论句没挂依据不是数没有出处"
claims = issuance["claims"]
assert claims["policy"] == "require_citation" and claims["uncited_claims"] == 1
[item] = claims["uncited"]
assert item["text"] == "增长主要来自新客。" and item["unit"] and len(item["span"]) == 2
assert issuance["unmatched_numbers"] == [] and issuance["unresolved"] == []
assert not any("结论句" in g for g in issuance["gaps"]), "判了就不是「没核」"
async def test_require_citation_with_every_claim_cited_is_formal(warehouse, monkeypatch):
script(monkeypatch, CLEAN)
issuance = await explore(weekly(report={"claims": "require_citation"}))
assert issuance["tier"] == "formal"
assert issuance["claims"] == {"policy": "require_citation", "uncited_claims": 0, "uncited": []}
async def test_claims_off_changes_nothing(warehouse, monkeypatch):
script(monkeypatch, UNCITED)
issuance = await explore(weekly())
assert issuance["tier"] == "formal" and "claims" not in issuance
async def test_an_auto_linked_name_does_not_count_as_a_citation(warehouse, monkeypatch):
"""只提到表名的结论句仍算没挂依据:自动链接的名字是系统加的标注,不是写作者给的出处。"""
script(monkeypatch, CLEAN + "增长主要来自 `orders` 里的新客。")
issuance = await explore(weekly(report={"claims": "require_citation"}))
assert issuance["tier"] == "degraded" and issuance["claims"]["uncited_claims"] == 1
async def test_runs_from_before_the_upgrade_ignore_claims(warehouse, monkeypatch):
async def no_snapshot(session):
return None
monkeypatch.setattr(runner_mod, "_agent_limits", no_snapshot)
script(monkeypatch, UNCITED)
issuance = await explore(weekly(report={"claims": "require_citation"}))
assert issuance["tier"] == "formal" and "claims" not in issuance and "entities" not in issuance
async def test_the_contract_can_ask_for_citations_too(warehouse, monkeypatch):
script(monkeypatch, UNCITED)
issuance = await explore(weekly(contract={"claims": {"policy": "require_citation", "on_uncited": "withhold"}}))
assert issuance["tier"] == "withheld" and issuance["claims"]["uncited_claims"] == 1
script(monkeypatch, CLEAN)
issuance = await explore(weekly(contract={"claims": "require_citation"}))
assert issuance["tier"] == "formal" and not any("结论句" in g for g in issuance["gaps"])
LOOSE = {"claims": {"policy": "require_citation", "on_uncited": "ignore"}}
async def test_the_contract_can_tighten_the_report_policy_but_never_loosen_it(warehouse, monkeypatch):
"""报告节点写了 require_citation:契约写 withhold 能收紧成不予出具;写 ignore、写 off 都放不松,照旧降档。"""
script(monkeypatch, UNCITED)
asks = {"claims": "require_citation"}
issuance = await explore(weekly(report=asks, contract=LOOSE))
assert issuance["tier"] == "degraded", "契约的 ignore 不能把报告节点要求的缺口抹掉"
assert issuance["claims"]["uncited_claims"] == 1 and "on_uncited" not in issuance["claims"]
issuance = await explore(weekly(report=asks, contract={"claims": "off"}))
assert issuance["tier"] == "degraded" and issuance["claims"]["uncited_claims"] == 1
issuance = await explore(weekly(report=asks, contract={"claims": {"policy": "require_citation",
"on_uncited": "withhold"}}))
assert issuance["tier"] == "withheld" and issuance["claims"]["on_uncited"] == "withhold"
async def test_ignore_cannot_lower_the_governed_floor(client, warehouse, monkeypatch):
"""受管级别的正式运行:契约写 on_uncited: ignore 不算数(没有哪道门禁查契约里的 claims),没挂依据的结论句照样
计入缺口、降一档。"""
script(monkeypatch, UNCITED)
row = await formal(client, weekly(report={"claims": "require_citation"}, contract=LOOSE), status="governed")
issuance = row.output["_issuance"]
assert issuance["tier"] == "degraded" and issuance["gaps"] == [], issuance
assert issuance["claims"]["uncited_claims"] == 1 and "on_uncited" not in issuance["claims"]
assert (await issuance_event(row.id))["claims"] == {"policy": "require_citation", "uncited_claims": 1}
# 报告节点没写(门禁之前发布的版本)、只有契约写了 ignore:受管级别同样不认 ignore
row = await formal(client, weekly(contract=LOOSE), status="governed")
assert row.output["_issuance"]["tier"] == "degraded"
async def test_outside_governed_a_contract_may_ignore_its_own_policy(client, warehouse, monkeypatch):
"""报告节点没要求时,契约自己写的 ignore 在探索运行、已发布级别照旧生效:不计入缺口,照实记下 ignore。"""
script(monkeypatch, UNCITED)
issuance = await explore(weekly(contract=LOOSE))
assert issuance["tier"] == "formal" and issuance["claims"]["on_uncited"] == "ignore"
row = await formal(client, weekly(contract=LOOSE), status="published")
assert row.output["_issuance"]["tier"] == "formal"
assert (await issuance_event(row.id))["claims"] == {"policy": "require_citation", "uncited_claims": 1,
"on_uncited": "ignore"}
@pytest.mark.parametrize(("written", "declared", "governed", "effect"), [
("require_citation", None, False, "degrade"),
("require_citation", "off", False, "degrade"),
("require_citation", {"policy": "require_citation", "on_uncited": "ignore"}, False, "degrade"),
("require_citation", {"policy": "require_citation", "on_uncited": "degrade"}, False, "degrade"),
("require_citation", {"policy": "require_citation", "on_uncited": "withhold"}, False, "withhold"),
("require_citation", {"policy": "require_citation", "on_uncited": "bogus"}, False, "degrade"),
("off", "require_citation", False, "degrade"),
("off", {"policy": "require_citation", "on_uncited": "withhold"}, True, "withhold"),
("off", {"policy": "require_citation", "on_uncited": "ignore"}, False, "ignore"),
("off", {"policy": "require_citation", "on_uncited": "ignore"}, True, "degrade"),
("off", {"policy": "require_citation", "on_uncited": "bogus"}, False, "degrade"),
("off", None, True, None),
("off", "off", True, None),
])
def test_the_effective_claims_policy_is_the_stricter_one(written, declared, governed, effect):
"""报告节点记下的策略和契约写的取更严的那个;受管级别 ignore 当 degrade;认不出的写法不放松(当 degrade)。"""
from app.engine.issuance import decide_tier
from app.engine.nodes.io import _claims
out = {"gaps": [], "claims_policy": None, "uncited": None, "claims": None}
uncited = [{"unit": "u1", "span": [0, 4], "text": "增长很快"}]
_claims(out, {"claims": declared} if declared is not None else {}, {"text": "x", "claims": written},
evidence_on=True, governed=governed, uncited=uncited)
assert out["gaps"] == []
if effect is None:
assert out["claims_policy"] is None and out["claims"] is None
return
assert out["claims"]["uncited_claims"] == 1 and out["claims"].get("on_uncited", "degrade") == effect
tier = decide_tier(missing_required=[], missing_expected=[], unmatched=[], strict=True, gaps=[],
uncited_claims=out["uncited"], claims_policy=out["claims_policy"])
assert tier == {"degrade": "degraded", "withhold": "withheld", "ignore": "formal"}[effect]
def test_a_run_in_flight_across_the_upgrade_keeps_the_old_gap():
"""升级前写的报告(产出里没有 claims)碰上写了 require_citation 的契约:照旧记「这一版还不支持」,不判档。"""
from app.engine.nodes.io import _claims
out = {"gaps": [], "claims_policy": None, "uncited": None, "claims": None}
_claims(out, {"claims": "require_citation"}, {"text": "x"}, evidence_on=True, governed=False,
uncited=[{"unit": "u1", "span": [0, 4], "text": "增长很快"}])
assert out["claims_policy"] is None and out["claims"] is None
assert len(out["gaps"]) == 1 and "这一版还不支持" in out["gaps"][0]
# --------------------------------------------------------------------------
# 可疑实体
# --------------------------------------------------------------------------
async def test_suspicious_names_are_only_annotated_in_exploratory_runs(warehouse, monkeypatch):
script(monkeypatch, SUSPICIOUS)
issuance = await explore(weekly())
assert issuance["tier"] == "formal" and issuance["gaps"] == []
[fake] = issuance["entities"]["unknown"]
assert fake["name"] == "refund_log" and fake["segment"] and fake["unit"]
assert issuance["entities"]["unverified"] == [] and issuance["entities"]["counted"] is False
async def test_suspicious_names_degrade_governed_formal_runs(client, warehouse, monkeypatch):
script(monkeypatch, SUSPICIOUS)
row = await formal(client, weekly(), status="governed")
issuance = row.output["_issuance"]
assert issuance["tier"] == "degraded", "按缺口降档,不是不予出具(契约是 strict 的)"
assert any("refund_log" in g and "可疑" in g for g in issuance["gaps"]), issuance["gaps"]
assert issuance["entities"]["counted"] is True and issuance["entities"]["unknown"][0]["name"] == "refund_log"
event = await issuance_event(row.id)
assert event["tier"] == "degraded" and event["entities"] == {"unknown": 1, "unverified": 0, "counted": True}
async def test_published_formal_runs_only_annotate(client, warehouse, monkeypatch):
script(monkeypatch, SUSPICIOUS)
issuance = (await formal(client, weekly(), status="published")).output["_issuance"]
assert issuance["tier"] == "formal" and issuance["gaps"] == []
assert issuance["entities"]["unknown"][0]["name"] == "refund_log" and issuance["entities"]["counted"] is False
async def test_names_that_cannot_be_checked_are_only_annotated_even_when_governed(client, partial_warehouse,
monkeypatch):
script(monkeypatch, SUSPICIOUS)
issuance = (await formal(client, weekly(), status="governed")).output["_issuance"]
assert issuance["tier"] == "formal" and issuance["gaps"] == []
assert issuance["entities"]["unknown"] == []
assert [u["name"] for u in issuance["entities"]["unverified"]] == ["refund_log"]
async def test_the_exit_says_the_report_turned_entities_off(warehouse, monkeypatch):
"""entities: off 的报告写了 [[t:]]:出口复核和报告节点一样照实说是这个节点关掉的,不说成「后续版本支持」。"""
from app.engine.evidence import ENTITIES_OFF_REASON, LATER_REASON
script(monkeypatch, "东区销售额 [[m:gmv]],见 [[t:orders]]。[[see:m:gmv]]")
issuance = await explore(weekly(report={"entities": "off"}))
[unresolved] = issuance["unresolved"]
assert unresolved["ref"] == "t:orders" and ENTITIES_OFF_REASON in unresolved["message"]
assert LATER_REASON not in unresolved["message"] and "entities" not in issuance
async def test_clean_reports_carry_no_entity_annotation(warehouse, monkeypatch):
script(monkeypatch, CLEAN)
issuance = await explore(weekly())
assert issuance["tier"] == "formal" and "entities" not in issuance
@@ -0,0 +1,54 @@
"""Copilot 的节点手册要教三期的写法:表名字段名、知识库原话、结论句策略,以及受管模板的报告节点怎么配。
手册里没写的配置,Copilot 就不会用;写错的写法,报告节点会一直被打回。所以这里不只查关键词,
还把手册里举的标记例子交给真正的解析器,确认写法和系统认的是同一套。
"""
from __future__ import annotations
import re
from app.api.copilot import NODE_REFERENCE, GenerateIn, _user_message
from app.engine.evidence import QUOTE_MIN, SUPPORTED_KINDS, parse_markers
from app.engine.schema import REPORT_CLAIMS
def _report_section() -> str:
start = NODE_REFERENCE.index("- report:")
return NODE_REFERENCE[start:NODE_REFERENCE.index("\n- output", start)]
def test_the_report_entry_teaches_entities_quotes_and_claims():
report = _report_section()
for words in (
["[[t:", "[[c:", "反引号", "可能是编造的名字"], # 表名、字段名
["[[q:K1|", "逐字", f"至少 {QUOTE_MIN} 个字", "[[see:K1]]"], # 知识库原话
["claims", "require_citation", "off", "judge", "后续版本"], # 结论句策略
["entities", "link"], # 实体层开关
):
assert all(w in report for w in words), [w for w in words if w not in report]
# 结论句策略的取值和校验认的是同一份
assert all(value in report for value in REPORT_CLAIMS)
def test_the_markers_in_the_guide_parse_as_the_system_reads_them():
kinds = {m["kind"] for m in parse_markers(_report_section())}
assert {"t", "c", "q", "m", "v", "table", "i", "see"} <= kinds
assert kinds <= SUPPORTED_KINDS
[quote] = [m for m in parse_markers(_report_section()) if m["kind"] == "q"]
assert quote["ref"] == "K1" and quote["quote"]
def test_governed_templates_spell_out_the_report_settings():
ref = NODE_REFERENCE
governed = ref[ref.index("受管"):]
for words in (['numbers: "strict"', 'on_violation: "fail"', 'claims: "require_citation"'],
["report_from", "narrative"], ["cite_fields: true"], ["evidence_role", "source"]):
assert all(w in governed for w in words), [w for w in words if w not in governed]
def test_the_answer_branch_mentions_names_and_quotes():
text = _user_message(GenerateIn(instruction="上周哪张表的退款最多", intent="answer"), patch=False)
assert "[[t:" in text and "反引号" in text and "编造" in text
assert "[[q:K1|" in text and "逐字" in text
# 问数是探索运行:不要替人打开结论句策略
assert not re.search(r"claims\s*[:=]\s*['\"]?require_citation", text)
+418
View File
@@ -0,0 +1,418 @@
"""证据图里的表、字段、检索,和记录页审计表用的审计视图(可以导出 JSON / CSV)。
- 证据图的 evidence 列出报告用到的表、字段、知识库检索;它们算不算「已封存」要看背后的
表结构快照、查询快照、检索快照是不是封存范围内的事件交回过的
- 审计视图把报告里每个有状态的片段按状态分组:有出处 / 无证据 / 可疑实体 / 旧运行猜测。
没挂依据的结论句、没有可画线文字的违规(结构片段里的数字、粗体里的可疑名字)也各占一行,
键盘用户在表里能看全
- 附上封存核对的结果;CSV 表头是中文,字段里的逗号、引号、换行按 RFC 4180 转义,
以 = + - @ 开头的文字(不是数)前面加一个 ',免得被电子表格当成公式;每行末尾一列「封存核对」,
文件存下来以后也看得出封存链是好的、断的还是没封存
- 没挂依据的结论句那一行说的「计入缺口」按出具时实际生效的策略(_issuance.claims)说,和出具横幅一致
"""
from __future__ import annotations
import asyncio
import csv
import io
import sqlite3
from types import SimpleNamespace
import pytest
from httpx import ASGITransport, AsyncClient
from langchain_core.messages import AIMessage
from sqlalchemy import select
from app.core import artifact_store
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import DataSource, Run, RunEvent
from app.engine.evidence import iter_segments
from app.engine.runner import run_manager
from app.main import app
SOURCE = "warehouse"
TOOL = f"db_query__{SOURCE}"
SQL = "SELECT region, SUM(amount) AS gmv, COUNT(*) AS order_cnt FROM orders GROUP BY region ORDER BY gmv DESC"
#: 第一句有出处(反引号里的真表名、单元格),ASCII 逗号和引号是给 CSV 转义用的;第二句有编造的名字
#: 和裸数字、没挂依据;第三句是没挂依据的结论
WRITE = ('`orders` 里东区最多, 达 [[v:Q1.r0.gmv]] "含税"。[[see:Q1]]`refund_log` 另算,另有 999 笔待核。'
"增长主要来自新客。")
HEADER = ["分组", "报告节点", "成果字段", "片段", "句子编号", "种类", "原文", "状态", "问题", "引用", "证据",
"证据种类", "证据编号", "工件", "来源节点", "已封存", "句子", "说明", "封存核对"]
@pytest.fixture(autouse=True)
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
@pytest.fixture
async def client():
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
@pytest.fixture
async def warehouse(tmp_path):
from app.data.introspect import introspect
path = tmp_path / "warehouse.db"
db = sqlite3.connect(path)
db.executescript(
"CREATE TABLE orders (id INTEGER PRIMARY KEY, region TEXT, amount REAL, created_at TEXT);"
"CREATE TABLE refunds (id INTEGER PRIMARY KEY, order_id INTEGER, amount REAL);")
db.executemany("INSERT INTO orders VALUES (?, ?, ?, ?)",
[(1, "east", 100.5, "2026-09-01"), (2, "west", 200.0, "2026-09-02"), (3, "east", 300.0, "2026-09-03")])
db.commit()
db.close()
probe = SimpleNamespace(id=f"probe-{path.name}", name=SOURCE, kind="sqlite", database=str(path), host=None,
port=None, username=None, password=None, options={}, readonly=True, schema_cache={})
cache = await introspect(probe)
await engines.invalidate(probe.id)
async with SessionLocal() as session:
row = (await session.execute(select(DataSource).where(DataSource.name == SOURCE))).scalar_one_or_none()
if row is None:
row = DataSource(name=SOURCE, kind="sqlite", database=str(path), readonly=True, enabled=True)
session.add(row)
row.database, row.kind, row.enabled, row.readonly, row.schema_cache = str(path), "sqlite", True, True, cache
row.options = {}
await session.commit()
source_id = row.id
await engines.invalidate(source_id)
yield source_id
await engines.invalidate(source_id)
def node(nid, ntype, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
def chain(*nodes):
return {"nodes": list(nodes), "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
def script(monkeypatch, text):
from app.providers import mock_model
monkeypatch.setattr(mock_model.MockChatModel, "_decide", lambda self, messages: AIMessage(content=text))
async def finish(graph) -> Run:
run = await run_manager.start(graph=graph, input_payload={})
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run.id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
return row
raise AssertionError(f"run {run.id} 没跑完:{row.status}")
async def events(run_id: str, etype: str, node_id: str | None = None) -> list[RunEvent]:
async with SessionLocal() as session:
q = select(RunEvent).where(RunEvent.run_id == run_id, RunEvent.type == etype)
if node_id:
q = q.where(RunEvent.node_id == node_id)
return list((await session.execute(q.order_by(RunEvent.seq))).scalars())
async def report_run(monkeypatch, text=WRITE) -> tuple[Run, dict]:
script(monkeypatch, text)
row = await finish(chain(node("start", "input"), node("fetch", "tool", tool=TOOL, args={"sql": SQL}),
node("write", "report", instructions="写一句", max_repairs=0),
node("out", "output", fields=[{"name": "r", "value": "{{ nodes.write.text }}"}])))
assert row.status == "succeeded", row.error
[checked] = await events(row.id, "report.checked", "write")
return row, artifact_store.load(checked.data["doc_artifact"])
def rows_of(body: dict, group: str) -> list[dict]:
return next(g["rows"] for g in body["groups"] if g["key"] == group)
# --------------------------------------------------------------------------
# 证据图:表、字段、检索
# --------------------------------------------------------------------------
async def test_the_graph_lists_tables_and_columns(client, warehouse, monkeypatch):
row, doc = await report_run(monkeypatch, "`orders` 里东区最多,达 [[v:Q1.r0.gmv]],按 [[c:orders.region]] 汇总。"
"[[see:Q1]]")
[end] = await events(row.id, "tool.end", "fetch")
body = (await client.get(f"/api/runs/{row.id}/evidence")).json()
by_alias = {e["alias"]: e for e in body["evidence"]}
orders = by_alias["t:orders"]
assert orders["kind"] == "table" and orders["name"] == "orders" and orders["sealed"] is True
assert orders["artifact"] == end.data["schema_artifact"] and orders["queries"] == ["Q1"]
assert {s["kind"] for s in orders["sources"]} == {"schema", "sql"}
assert orders["cited_by"] == [next(s["id"] for s in iter_segments(doc) if s.get("ref") == "t:orders")]
region = by_alias["c:orders.region"]
assert region["kind"] == "column" and region["table"] == "orders" and region["sealed"] is True
assert {"from": "t:orders", "to": "Q1", "rel": "appears_in", "report": "write"} in body["edges"]
assert {"from": "c:orders.region", "to": "Q1", "rel": "appears_in", "report": "write"} in body["edges"]
async def test_the_graph_lists_retrievals_with_the_quotes_that_cite_them(client, monkeypatch):
from app.memory import kb
async def fake_search(session, **kw):
return [{"chunk_id": "ch-1", "document_id": "doc-1", "title": "售后月报", "ordinal": 3, "score": 0.9,
"content": "九月退款主要集中在东区,原因是物流延误。"}]
monkeypatch.setattr(kb, "search", fake_search)
script(monkeypatch, "售后说「[[q:K1|退款主要集中在东区]]」。[[see:K1]]")
row = await finish(chain(node("start", "input"), node("docs", "retrieve", query="退款", collection="kb"),
node("write", "report", instructions="引用原话"),
node("out", "output", fields=[{"name": "r", "value": "{{ nodes.write.text }}"}])))
assert row.status == "succeeded", row.error
[checked] = await events(row.id, "report.checked", "write")
doc = artifact_store.load(checked.data["doc_artifact"])
quote = next(s for s in iter_segments(doc) if s["kind"] == "quote")
body = (await client.get(f"/api/runs/{row.id}/evidence")).json()
k1 = next(e for e in body["evidence"] if e["alias"] == "K1")
assert k1["kind"] == "retrieval" and k1["sealed"] is True and k1["source"] == "kb"
assert k1["cited_by"] == [quote["id"]]
audit = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
[cited] = rows_of(audit, "cited")
assert cited["kind"] == "quote" and cited["text"] == "退款主要集中在东区" and cited["evidence_kind"] == "quote"
assert cited["alias"] == "K1" and cited["sealed"] is True and cited["artifact"] == k1["artifact"]
# --------------------------------------------------------------------------
# 审计视图
# --------------------------------------------------------------------------
async def test_the_audit_groups_every_stateful_piece(client, warehouse, monkeypatch):
row, doc = await report_run(monkeypatch)
body = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
assert body["run_id"] == row.id and body["schema"] == "agentlab.evidence.audit/1" and body["mode"] == "cited"
assert [g["key"] for g in body["groups"]] == ["cited", "none", "suspicious", "candidate"]
assert [g["label"] for g in body["groups"]] == ["有出处", "无证据", "可疑实体", "旧运行猜测"]
assert body["counts"] == {g["key"]: len(g["rows"]) for g in body["groups"]}
assert body["total"] == sum(body["counts"].values())
cited = rows_of(body, "cited")
assert [(r["kind"], r["text"]) for r in cited] == [("entity", "`orders`"), ("number", "400.5")]
cell = cited[1]
assert cell["ref"] == "v:Q1.r0.gmv" and cell["evidence"] == "Q1.r0.gmv" and cell["evidence_kind"] == "cell"
assert cell["node_id"] == "fetch" and cell["sealed"] is True and cell["state"] == "deterministic"
assert cell["segment"] and cell["unit"] and doc["markdown"][slice(*cell["span"])] == "400.5"
assert cited[0]["evidence_kind"] == "table" and cited[0]["sealed"] is True
# 组内按在正文里的位置排:没挂依据的句子从句首算
none = rows_of(body, "none")
assert [(r["kind"], r["issue"]) for r in none] == [
("claim", "uncited_claim"), ("number", "uncited_number"), ("claim", "uncited_claim")]
assert none[1]["text"] == "999" and none[1]["state"] == "none"
assert [none[0]["text"], none[2]["text"]] == ["`refund_log` 另算,另有 999 笔待核。", "增长主要来自新客。"]
assert none[0]["segment"] is None and none[0]["unit"] and none[0]["sentence"] == none[0]["text"]
[fake] = rows_of(body, "suspicious")
assert fake["kind"] == "entity" and fake["issue"] == "unknown_entity" and fake["text"] == "`refund_log`"
assert "可能是编造的名字" in fake["note"]
assert rows_of(body, "candidate") == []
seal = body["seal"]
assert seal["sealed"] is True and seal["ok"] is True and seal["manifest_seq"] == row.manifest_seq
assert seal["events"] > 0 and seal["message"] == "事件流与封存时一致"
[report] = body["reports"]
assert report["node_id"] == "write" and report["hash_ok"] is True and report["doc_sealed"] is True
assert report["claims"] == "off" and report["stats"]["unknown_entities"] == 1
async def test_violations_without_a_row_of_their_own_are_listed(client, warehouse, monkeypatch):
"""粗体里的可疑名字、列表序号里的数字没有可以画线的片段,只在违规清单里:审计表照样各给一行。"""
row, _ = await report_run(monkeypatch, "**`refund_log`** 另算。\n\n100. 东区最多,达 [[v:Q1.r0.gmv]]。[[see:Q1]]")
body = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
[fake] = rows_of(body, "suspicious")
assert (fake["kind"], fake["issue"], fake["text"]) == ("violation", "unknown_entity", "refund_log")
assert fake["segment"] and "可能是编造的名字" in fake["note"]
[bare] = rows_of(body, "none")
assert (bare["kind"], bare["issue"], bare["text"]) == ("violation", "uncited_number", "100")
assert [r["text"] for r in rows_of(body, "cited")] == ["400.5"]
async def test_the_audit_can_show_only_some_groups(client, warehouse, monkeypatch):
row, _ = await report_run(monkeypatch)
body = (await client.get(f"/api/runs/{row.id}/evidence/audit", params={"groups": "none,suspicious"})).json()
assert [g["key"] for g in body["groups"]] == ["none", "suspicious"]
assert body["total"] == 4
bad = await client.get(f"/api/runs/{row.id}/evidence/audit", params={"groups": "none,nope"})
assert bad.status_code == 422 and bad.json()["code"] == "evidence_bad_group"
async def test_the_audit_exports_json_and_csv(client, warehouse, monkeypatch):
row, _ = await report_run(monkeypatch)
plain = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
as_json = await client.get(f"/api/runs/{row.id}/evidence/audit", params={"format": "json"})
assert as_json.status_code == 200 and as_json.json() == plain
assert "attachment" in as_json.headers["content-disposition"]
assert as_json.headers["content-disposition"].endswith('.json"')
as_csv = await client.get(f"/api/runs/{row.id}/evidence/audit", params={"format": "csv"})
assert as_csv.status_code == 200 and as_csv.headers["content-type"].startswith("text/csv")
assert as_csv.headers["content-disposition"].endswith('.csv"')
assert as_csv.headers["x-evidence-seal"] == "ok"
raw = as_csv.content.decode("utf-8")
assert raw.startswith(""), "Excel 要 BOM 才认 UTF-8"
table = list(csv.reader(io.StringIO(raw[1:])))
assert table[0] == HEADER
flat = [r for g in plain["groups"] for r in g["rows"]]
assert len(table) - 1 == len(flat) == plain["total"]
cell = next(line for line in table[1:] if line[6] == "400.5")
assert cell[0] == "有出处" and cell[7] == "有出处" and cell[15] == "是" and cell[10] == "Q1.r0.gmv"
# 句子里的 ASCII 逗号和双引号原样读回来
assert cell[16] == '`orders` 里东区最多, 达 400.5 "含税"。'
fake = next(line for line in table[1:] if line[8] == "可能是编造的名字")
assert fake[0] == "可疑实体" and fake[6] == "`refund_log`"
# 存下来的文件自己带着封存核对的结果(响应头离了浏览器就没了)
assert {line[18] for line in table[1:]} == {f"封存核对通过(封存于第 {row.manifest_seq} 条事件)"}
bad = await client.get(f"/api/runs/{row.id}/evidence/audit", params={"format": "xlsx"})
assert bad.status_code == 422 and bad.json()["code"] == "evidence_bad_format"
def test_csv_escapes_commas_quotes_newlines_and_formulas():
from app.api.evidence import audit_csv
rows = [{"group": "none", "text": 'a,b "c"\nd', "sentence": "=HYPERLINK(\"x\")", "note": "-3.2%",
"evidence": "+cmd", "ref": "@x", "sealed": None, "kind": "claim"}]
raw = audit_csv(rows)
assert raw.startswith("") and "\r\n" in raw
[header, line] = list(csv.reader(io.StringIO(raw[1:])))
assert header == HEADER
got = dict(zip(header, line))
assert got["原文"] == 'a,b "c"\nd'
assert got["句子"] == "'=HYPERLINK(\"x\")" and got["证据"] == "'+cmd" and got["引用"] == "'@x"
assert got["说明"] == "-3.2%", "负数不是公式,不加引号"
assert got["已封存"] == "—" and got["分组"] == "无证据" and got["种类"] == "结论句"
assert got["封存核对"] == "", "不给封存结论就空着,不瞎填"
[_, line] = list(csv.reader(io.StringIO(audit_csv(rows, seal="=封存核对没通过")[1:])))
assert line[-1] == "'=封存核对没通过", "封存那一格同样过公式防护"
async def test_a_broken_seal_marks_every_row_unsealed(client, warehouse, monkeypatch):
row, _ = await report_run(monkeypatch)
async with SessionLocal() as session:
started = (await session.execute(select(RunEvent).where(
RunEvent.run_id == row.id, RunEvent.type == "node.started", RunEvent.node_id == "start"))).scalar_one()
started.data = {**(started.data or {}), "note": "事后改过"}
await session.commit()
body = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
assert body["seal"]["ok"] is False and "对不上" in body["seal"]["message"]
assert all(r["sealed"] is False for r in rows_of(body, "cited"))
as_csv = await client.get(f"/api/runs/{row.id}/evidence/audit", params={"format": "csv"})
assert as_csv.headers["x-evidence-seal"] == "broken"
table = list(csv.reader(io.StringIO(as_csv.content.decode("utf-8")[1:])))
assert {line[15] for line in table[1:] if line[0] == "有出处"} == {"否"}
assert {line[18] for line in table[1:]} == {"封存核对没通过:封存之后有事件被改过、删过或插过"}
async def test_a_run_waiting_for_approval_is_audited_as_unsealed(client, warehouse, monkeypatch):
script(monkeypatch, WRITE)
run = await run_manager.start(graph=chain(
node("start", "input"), node("fetch", "tool", tool=TOOL, args={"sql": SQL}),
node("write", "report", instructions="写一句", max_repairs=0),
node("gate", "human", mode="approve", title="确认"),
node("out", "output", fields=[{"name": "r", "value": "{{ nodes.write.text }}"}])), input_payload={})
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run.id)
if row.status in ("interrupted", "failed"):
break
assert row.status == "interrupted", row.error
body = (await client.get(f"/api/runs/{run.id}/evidence/audit")).json()
assert body["mode"] == "cited" and body["seal"]["sealed"] is False and body["seal"]["ok"] is None
assert rows_of(body, "cited") and all(r["sealed"] is False for r in rows_of(body, "cited"))
as_csv = await client.get(f"/api/runs/{run.id}/evidence/audit", params={"format": "csv"})
assert as_csv.headers["x-evidence-seal"] == "unsealed"
table = list(csv.reader(io.StringIO(as_csv.content.decode("utf-8")[1:])))
assert len(table) > 1 and {line[18] for line in table[1:]} == {"未封存:运行还没跑完,或者停在人工审批"}
CLAIMED = "`orders` 里东区最多,达 [[m:gmv]]。[[see:m:gmv]]增长主要来自新客。"
async def claimed_run(monkeypatch, *, report=None, contract=None) -> Run:
"""一句有出处、一句没挂依据的结论;contract 为 None 时出口不带契约。"""
script(monkeypatch, CLAIMED)
card = node("card", "metrics", caliber="周报口径", caliber_version="v1", metrics=[
{"id": "gmv", "name": "销售额", "decimals": 1, "expression": "cell(nodes.fetch, 0, 'gmv')"}])
out = node("out", "output", fields=[{"name": "r", "value": "{{ nodes.write.text }}"}],
**({"contract": {"report_from": "write", "metrics_from": ["card"], "required": ["gmv"], **contract}}
if contract is not None else {}))
row = await finish(chain(node("start", "input"), node("fetch", "tool", tool=TOOL, args={"sql": SQL}), card,
node("write", "report", instructions="写一句", max_repairs=0, **(report or {})), out))
assert row.status == "succeeded", row.error
return row
async def test_uncited_claims_say_what_issuance_did_with_them(client, warehouse, monkeypatch):
"""结论句那一行说的「计入缺口」要和出具横幅一致:按出具时实际生效的策略(_issuance.claims)说,
不只看报告节点自己写的 claims。"""
from app.api.evidence import UNCITED_CLAIM, UNCITED_CLAIM_COUNTED, UNCITED_CLAIM_REQUIRED, UNCITED_CLAIM_WITHHELD
async def claim_notes(row: Run) -> list[str]:
body = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
return [r["note"] for r in rows_of(body, "none") if r["kind"] == "claim"]
# 报告节点没写、契约写了 require_citation:出具因为这句降档,审计表也得说计入缺口
row = await claimed_run(monkeypatch, contract={"claims": "require_citation"})
assert row.output["_issuance"]["tier"] == "degraded"
assert await claim_notes(row) == [UNCITED_CLAIM_COUNTED]
# 契约收紧成 withhold:说的是不予出具
row = await claimed_run(monkeypatch, contract={"claims": {"policy": "require_citation", "on_uncited": "withhold"}})
assert row.output["_issuance"]["tier"] == "withheld"
assert await claim_notes(row) == [UNCITED_CLAIM_WITHHELD]
# 探索运行里契约自己写 ignore:出具没算它,审计表也不说计入缺口
row = await claimed_run(monkeypatch, contract={"claims": {"policy": "require_citation", "on_uncited": "ignore"}})
assert row.output["_issuance"]["tier"] == "formal"
assert await claim_notes(row) == [UNCITED_CLAIM]
# 报告节点要求了、契约的 ignore 放不松:计入缺口
row = await claimed_run(monkeypatch, report={"claims": "require_citation"},
contract={"claims": {"policy": "require_citation", "on_uncited": "ignore"}})
assert row.output["_issuance"]["tier"] == "degraded"
assert await claim_notes(row) == [UNCITED_CLAIM_COUNTED]
# 报告节点要求了、但出口没有契约:没有哪次出具按它判档,不能说「出具时计入缺口」
row = await claimed_run(monkeypatch, report={"claims": "require_citation"})
assert "_issuance" not in (row.output or {})
assert await claim_notes(row) == [UNCITED_CLAIM_REQUIRED]
# 都没要求
row = await claimed_run(monkeypatch, contract={})
assert await claim_notes(row) == [UNCITED_CLAIM]
async def test_the_contract_policy_only_speaks_for_the_report_it_checked(client, warehouse, monkeypatch):
"""两份报告,契约只核对 write:另一份(side)没挂依据的结论句不能跟着说「出具时计入缺口」。"""
from app.api.evidence import UNCITED_CLAIM, UNCITED_CLAIM_COUNTED
script(monkeypatch, CLAIMED)
card = node("card", "metrics", caliber="周报口径", caliber_version="v1", metrics=[
{"id": "gmv", "name": "销售额", "decimals": 1, "expression": "cell(nodes.fetch, 0, 'gmv')"}])
row = await finish(chain(
node("start", "input"), node("fetch", "tool", tool=TOOL, args={"sql": SQL}), card,
node("write", "report", instructions="写一句", max_repairs=0),
node("side", "report", instructions="再写一句", max_repairs=0),
node("out", "output", fields=[{"name": "r", "value": "{{ nodes.write.text }}"},
{"name": "s", "value": "{{ nodes.side.text }}"}],
contract={"report_from": "write", "metrics_from": ["card"], "claims": "require_citation"})))
assert row.status == "succeeded", row.error
body = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
notes = {r["report"]: r["note"] for r in rows_of(body, "none") if r["kind"] == "claim"}
assert notes == {"write": UNCITED_CLAIM_COUNTED, "side": UNCITED_CLAIM}
async def test_runs_without_evidence_say_so(client):
row = await finish(chain(node("start", "input", fields=[{"name": "q", "default": "x"}]),
node("out", "output", fields=[{"name": "r", "value": "{{ input.q }}"}])))
body = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
assert body["mode"] == "none" and body["total"] == 0 and body["note"]
resp = await client.get("/api/runs/no-such-run/evidence/audit")
assert resp.status_code == 404 and resp.json()["code"] == "run_not_found"
+685
View File
@@ -0,0 +1,685 @@
"""实体层:报告里写的表名、字段名能点开出处,写了不存在的名字会被标出来。
以前 `refunds` 这种写在反引号里的名字,报告不管它存不存在:模型编一个听起来很像的表名,
读者无从分辨。三期:
- 数据源工具每次查询都把当时的表结构冻结成 schema_snapshot 工件(内容寻址),台账记一条
schema 条目;查询条目记下 SQL 里 FROM / JOIN 的表(注释和字符串里的不算)
- 目录编出表条目 t:<表> 和列条目 c:<表>.<列>,来源是表结构快照、SQL 用到的表、查询结果的列
- [[t:]] [[c:]] 标记能解析,显示名字本身;正文里的名字按保守规则自动链接
- 反引号里写了像标识符的名字、却哪里都找不到的,标成可疑实体(state none,issue
unknown_entity),记违规但不拦——报告节点不为它失败、不为它重写
- 流式渲染和整篇渲染一致
"""
from __future__ import annotations
import asyncio
import json
import random
import sqlite3
from types import SimpleNamespace
import pytest
from langchain_core.messages import AIMessage, ToolMessage
from sqlalchemy import select
from app.core import artifact_store
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import DataSource, Run, RunEvent
from app.engine import runner as runner_mod
from app.engine.evidence import (
ENTITIES_OFF_REASON,
LATER_REASON,
StreamRenderer,
build_catalog,
catalog_prompt,
compose_doc,
iter_segments,
make_eid,
query_entry_fields,
render_markers,
schema_entry_fields,
sql_tables,
uncited_claims,
verify_doc,
)
from app.engine.runner import run_manager
SOURCE = "warehouse"
TOOL = f"db_query__{SOURCE}"
SQL = "SELECT region, SUM(amount) AS gmv, COUNT(*) AS order_cnt FROM orders GROUP BY region ORDER BY gmv DESC"
SCHEMA_ART = "5" * 64
Q_ART = "a" * 64
TOOL_SNAP = "b" * 64
QUERY = {"kind": "query", "node_id": "fetch", "exec": 1, "artifact": Q_ART, "via": TOOL_SNAP, "tool": TOOL,
"source": SOURCE, "columns": ["region", "gmv", "order_cnt"], "rows": 2, "truncated": False,
"schema_artifact": SCHEMA_ART, "tables": ["orders"]}
SCHEMA = {"kind": "schema", "node_id": "fetch", "exec": 1, "artifact": SCHEMA_ART, "source": SOURCE,
"tables": {"orders": ["id", "region", "amount", "created_at", "name"],
"refunds": ["id", "order_id", "amount"]}}
@pytest.fixture
def catalog():
return build_catalog(nodes={}, ledger=[QUERY, SCHEMA], inputs={"week": "2026-W37"})
def entity_segments(doc):
return [s for s in iter_segments(doc) if s["kind"] == "entity"]
def codes(doc):
return [v["code"] for v in doc["violations"]]
# --------------------------------------------------------------------------
# 从 SQL 里抽表名:注释、字符串里的不算
# --------------------------------------------------------------------------
@pytest.mark.parametrize("sql, tables", [
("-- FROM ghost_a\nSELECT o.region, 'JOIN ghost_b' AS note /* FROM ghost_c */ FROM orders o "
"LEFT JOIN \"refunds\" r ON r.order_id = o.id WHERE o.name <> 'FROM ghost_d'", ["orders", "refunds"]),
("WITH recent AS (SELECT * FROM orders), top AS (SELECT * FROM recent) "
"SELECT * FROM top JOIN ANALYTICS.v_demo_table v ON 1 = 1", ["orders", "ANALYTICS.v_demo_table"]),
("SELECT * FROM orders a, refunds AS b, `sales_daily` WHERE a.id = b.order_id", ["orders", "refunds", "sales_daily"]),
("SELECT * FROM (SELECT id FROM orders) t JOIN refunds USING (id)", ["orders", "refunds"]),
("SELECT * FROM generate_series(1, 3)", []),
("select * from Orders join orders on 1=1", ["Orders"]),
("SELECT 'it''s FROM ghost' AS x FROM orders", ["orders"]),
# 字符串里的 -- 和 /* 不是注释:先认字符串、再认注释,一趟扫完
("SELECT '--x' AS a FROM orders", ["orders"]),
("SELECT '/*' AS a FROM orders WHERE note <> '*/ JOIN ghost_e'", ["orders"]),
("SELECT \"a--b\" FROM orders JOIN refunds ON 1 = 1", ["orders", "refunds"]),
# 引号括起来的名字里的 FROM 不是关键字(MySQL 默认把 "…" 当字符串)
("SELECT \"FROM ghost_f\" AS x FROM orders", ["orders"]),
# 反斜杠转义的引号(MySQL、PostgreSQL 的 E'…'):宁可漏认一张表,不能把字符串里的名字当表
("SELECT 'it\\'s FROM ghost_g' AS x FROM orders", ["orders"]),
("SELECT $$ FROM ghost_h $$ AS a, $tag$ JOIN ghost_i $tag$ FROM orders WHERE id = $1", ["orders"]),
("/* a /* nested */ FROM ghost_j */ SELECT * FROM orders", ["orders"]),
("# FROM ghost_k\nSELECT * FROM orders", ["orders"]),
# SQLite 也认的方括号标识符;数组下标、数组字面量里的字符串不是
("SELECT * FROM [main].[orders] o JOIN [refunds] r ON r.order_id = o.id", ["main.orders", "refunds"]),
("SELECT ARRAY['FROM ghost_l'] AS a, tags[1] FROM orders", ["orders"]),
("", []),
])
def test_sql_tables_ignore_comments_strings_and_cte_names(sql, tables):
assert sql_tables(sql) == tables
def test_query_entry_carries_tables_only_with_a_schema_snapshot():
"""没有表结构快照(数据源没探查过结构)的查询保持二期的形状:只凭 SQL 和结果列核对名字,
反引号里写的真实表名(没进这条 SQL)会被误标成编造的,这种运行不启用实体核对。"""
payload = {"artifact": Q_ART, "columns": ["gmv"], "rows": [[1]], "sql": SQL, "source": SOURCE}
plain = query_entry_fields(payload)
assert "tables" not in plain and "schema_artifact" not in plain
fields = query_entry_fields(json.dumps({**payload, "schema_artifact": SCHEMA_ART}))
assert fields["schema_artifact"] == SCHEMA_ART and fields["tables"] == ["orders"]
# --------------------------------------------------------------------------
# 目录
# --------------------------------------------------------------------------
def test_catalog_lists_tables_and_columns_from_all_three_sources(catalog):
orders = catalog["t:orders"]
assert orders["kind"] == "table" and orders["locator"] == {"table": "orders"}
assert orders["eid"] == make_eid("table", SCHEMA_ART, {"table": "orders"}) and orders["artifact"] == SCHEMA_ART
assert {(s["kind"], s.get("alias")) for s in orders["sources"]} == {("schema", None), ("sql", "Q1")}
assert orders["queries"] == ["Q1"]
assert catalog["t:refunds"]["queries"] == []
amount = catalog["c:orders.amount"]
assert amount["kind"] == "column" and amount["locator"] == {"table": "orders", "column": "amount"}
assert amount["eid"] == make_eid("column", SCHEMA_ART, {"table": "orders", "column": "amount"})
# 结果列落在 SQL 用到的表里:同一个条目多一个来源
region = catalog["c:orders.region"]
assert {(s["kind"], s.get("alias")) for s in region["sources"]} == {("schema", None), ("result", "Q1")}
# 没有表名的结果列(聚合的别名):c:<列>,出处是那份查询快照
cnt = catalog["c:order_cnt"]
assert cnt["locator"] == {"column": "order_cnt"} and cnt["eid"] == make_eid("column", Q_ART, {"column": "order_cnt"})
assert cnt["sources"] == [{"kind": "result", "artifact": Q_ART, "alias": "Q1", "source": SOURCE}]
# 同名字段在好几张表里:c:<列> 列出是哪几张
assert catalog["c:amount"]["tables"] == ["orders", "refunds"]
def test_phase_two_ledgers_build_no_entities():
old = {k: v for k, v in QUERY.items() if k not in ("schema_artifact", "tables")}
catalog = build_catalog(nodes={}, ledger=[old])
assert not [a for a, e in catalog.items() if e["kind"] in ("table", "column")]
doc = compose_doc("看 `ghost_table`。", catalog)
assert doc["violations"] == [] and not entity_segments(doc)
# --------------------------------------------------------------------------
# 标记
# --------------------------------------------------------------------------
def test_entity_markers_render_as_the_name(catalog):
doc = compose_doc("数据来自 [[t:orders]] 的 [[c:orders.amount]] 和 [[c:order_cnt]]。[[see:t:refunds]]", catalog)
assert doc["markdown"] == "数据来自 orders 的 orders.amount 和 order_cnt。"
segs = entity_segments(doc)
assert [(s["text"], s["cite"]["kind"], s["state"]) for s in segs] == [
("orders", "table", "deterministic"), ("orders.amount", "column", "deterministic"),
("order_cnt", "column", "deterministic")]
assert segs[0]["cite"]["eid"] == catalog["t:orders"]["eid"]
assert doc["violations"] == [] and doc["stats"]["entities"] == 3
assert doc["blocks"][0]["units"][0]["cites"] == ["t:orders", "c:orders.amount", "c:order_cnt", "t:refunds"]
def test_unknown_marker_is_suspicious_not_unresolved(catalog):
doc = compose_doc("退款看 [[t:refund_log]]。字段写错 [[t:amount]]。", catalog)
fake, wrong = entity_segments(doc)
assert fake["text"] == "refund_log" and fake["state"] == "none" and fake["issue"] == "unknown_entity"
assert "可能是编造的名字" in fake["cite"]["reason"]
# 字段当表写:名字是真的,写法错了——这是解析不了,要重写
assert wrong["issue"] == "unresolved_ref" and wrong["text"].startswith("⟦?t:")
assert codes(doc) == ["unknown_entity", "unresolved_ref"]
assert doc["stats"]["unknown_entities"] == 1 and doc["stats"]["unresolved"] == 1
def test_entity_markers_without_any_schema_keep_the_phase_one_answer():
"""这次运行没查过库(或者是升级前的运行):没有东西可以核对,t / c 仍按一期判为解析不了。"""
catalog = build_catalog(nodes={}, inputs={"week": "2026-W37"})
doc = compose_doc("看 [[t:orders]],还有 `ghost_table`。", catalog)
[seg] = [s for s in iter_segments(doc) if s.get("ref")]
assert seg["state"] == "none" and LATER_REASON in seg["cite"]["reason"]
assert codes(doc) == ["unresolved_ref"] # 反引号里的名字不核对
# --------------------------------------------------------------------------
# 反引号:一律核对
# --------------------------------------------------------------------------
def test_backticked_names_are_linked_or_flagged(catalog):
doc = compose_doc("看 `orders` 和 `ghost_table`,`refunds.order_id` 与 `SUM(amount)`、`NULL`、`week`。", catalog)
segs = entity_segments(doc)
assert [(s["text"], s["state"], s.get("issue")) for s in segs] == [
("`orders`", "deterministic", None), ("`ghost_table`", "none", "unknown_entity"),
("`refunds.order_id`", "deterministic", None)]
assert segs[0]["ref"] == "t:orders" and segs[0]["code"] is True and segs[0]["auto"] is True
assert segs[2]["cite"]["eid"] == catalog["c:refunds.order_id"]["eid"]
[v] = doc["violations"]
assert v["code"] == "unknown_entity" and v["text"] == "ghost_table" and v["segment"] == segs[1]["id"]
assert doc["markdown"][v["span"][0]:v["span"][1]] == "ghost_table"
assert doc["stats"]["unknown_entities"] == 1
assert verify_doc(doc, catalog)["violations"] == doc["violations"]
def test_verify_finds_suspicious_names_even_if_the_doc_hides_them(catalog):
doc = compose_doc("看 `ghost_table` 的数。", catalog)
for s in iter_segments(doc):
if s.get("issue") == "unknown_entity":
s.update(kind="text", state="neutral")
s.pop("issue")
checked = verify_doc(doc, catalog)
assert [v["code"] for v in checked["violations"]] == ["unknown_entity"]
assert checked["stats"]["unknown_entities"] == 1
def test_names_inside_code_blocks_are_not_checked(catalog):
doc = compose_doc("```sql\nSELECT `ghost_col` FROM ghost_table\n```\n\n正文。", catalog)
assert doc["violations"] == [] and not entity_segments(doc)
# --------------------------------------------------------------------------
# 正文里的裸名字:保守
# --------------------------------------------------------------------------
def test_bare_names_link_conservatively(catalog):
doc = compose_doc("按 region 和 name 统计,orders 表的 order_id、created_at 与 refunds.amount;"
"amount、id、gmv 不链接,Orders 也不链接,order_cnt 链接。", catalog)
assert [s["text"] for s in entity_segments(doc)] == [
"orders", "order_id", "created_at", "refunds.amount", "order_cnt"]
assert doc["violations"] == []
assert all(s["state"] == "deterministic" and s.get("auto") and not s.get("code") for s in entity_segments(doc))
def test_bare_names_inside_links_and_bold_are_left_alone(catalog):
doc = compose_doc("见 [orders 说明](http://example.com/orders) 和 **orders 汇总**。", catalog)
assert not entity_segments(doc)
assert "".join(s["text"] for s in iter_segments(doc)) == doc["markdown"]
# --------------------------------------------------------------------------
# 一致:流式 = 整篇;组装 = 复核
# --------------------------------------------------------------------------
def test_streaming_equals_full_render(catalog):
text = ("## [[t:orders]]\n\n按 [[c:orders.region]] 看,[[c:order_cnt]] 最多;[[t:refund_log]] 没有。"
"`orders` 与 [[t:amount]] 以及 [[i:week]]。[[see:t:orders]]\n\n- [[c:refunds.order_id]]")
whole = render_markers(text, catalog)
assert "[[" not in whole and "orders" in whole
rng = random.Random(7)
for _ in range(300):
renderer = StreamRenderer(catalog)
out, i = [], 0
while i < len(text):
step = rng.randint(1, 9)
out.append(renderer.feed(text[i:i + step]))
i += step
out.append(renderer.flush())
assert "".join(out) == whole
def test_compose_and_verify_agree_and_doc_catalog_stays_small(catalog):
doc = compose_doc("[[t:orders]] 里 `refunds` 和 `ghost_table`,order_cnt。", catalog)
checked = verify_doc(doc, catalog)
assert checked["violations"] == doc["violations"] and checked["stats"] == doc["stats"]
kept = {a for a, e in doc["catalog"].items() if e["kind"] in ("table", "column")}
assert kept == {"t:orders", "t:refunds", "c:order_cnt"} # 只留正文里用到的实体
assert "i:week" in doc["catalog"] and "Q1" in doc["catalog"]
# --------------------------------------------------------------------------
# 表结构快照不全、实体层关掉、老目录:各说各的话
# --------------------------------------------------------------------------
def test_schema_entry_keeps_truncation():
content = {"source": SOURCE, "tables": {"orders": {"columns": [{"name": "id"}]}}, "schema": None}
assert schema_entry_fields(content) == {"source": SOURCE, "schema": None, "synced_at": None,
"tables": {"orders": ["id"]}}
assert schema_entry_fields({**content, "truncated": True, "total": 250}) == {
"source": SOURCE, "schema": None, "synced_at": None, "tables": {"orders": ["id"]},
"truncated": True, "total": 250}
def test_a_truncated_schema_cannot_call_a_name_made_up(catalog):
"""库里的表太多、快照只存了前 200 张:没列出的名字可能是真的,只能说「核对不了」(软问题,出口
不记缺口),不能说「可能是编造的」。已列出的表的字段是全的,表.字段 写错了照样是可疑实体。"""
cut = build_catalog(nodes={}, ledger=[QUERY, {**SCHEMA, "truncated": True, "total": 250}])
text = "看 `ghost_table`、[[t:ghost_two]]、`orders.ghost_col` 和 `orders`。[[see:c:ghost_three]]"
doc = compose_doc(text, cut)
segs = entity_segments(doc)
assert [(s["text"], s["state"], s.get("issue")) for s in segs] == [
("`ghost_table`", "none", "unverified_entity"), ("ghost_two", "none", "unverified_entity"),
("`orders.ghost_col`", "none", "unknown_entity"), ("`orders`", "deterministic", None)]
assert segs[1]["cite"]["unverified"] is True and "unknown" not in segs[1]["cite"]
# 违规按位置排:句末依据 [[see:]] 的位置记的是整句,排在最前
assert codes(doc) == ["unverified_entity", "unverified_entity", "unverified_entity", "unknown_entity"]
assert "核对不了" in doc["violations"][0]["message"] and "编造" not in doc["violations"][0]["message"]
assert doc["stats"]["unverified_entities"] == 3 and doc["stats"]["unknown_entities"] == 1
checked = verify_doc(doc, cut)
assert checked["violations"] == doc["violations"] and checked["stats"] == doc["stats"]
assert "只存了一部分" in catalog_prompt(cut) and "只存了一部分" not in catalog_prompt(catalog)
# 同一篇放到完整的快照下:全都是可疑实体
assert codes(compose_doc(text, catalog)) == ["unknown_entity"] * 4
def test_old_catalogs_keep_the_old_stats_keys(catalog):
"""升级前的运行(目录里没有表、字段、检索):统计的键和以前一模一样,事件一个字不加。"""
old = build_catalog(nodes={}, inputs={"week": "2026-W37"})
doc = compose_doc("本周 [[i:week]],看 [[t:orders]]。", old)
assert list(doc["stats"]) == ["units", "segments", "claims", "connective", "headings", "numbers",
"numbers_cited", "values", "uncited_numbers", "unresolved", "see",
"uncited_claims", "violations"]
assert verify_doc(doc, old)["stats"] == doc["stats"]
for cat in (catalog, build_catalog(nodes={}, ledger=[{"kind": "retrieval", "node_id": "kb", "exec": 1,
"artifact": "c" * 64, "source": "kb", "rows": 1}])):
stats = compose_doc("本周。", cat)["stats"]
assert {"entities", "quotes", "unknown_entities", "unverified_entities"} <= set(stats)
def test_entities_off_says_the_node_turned_it_off(catalog):
"""entities: off 是作者有意关掉的:[[t:]] 解析不了的原因照实说,不能说「后续版本支持」。
目录里就算还有表和字段,关掉了也不链接、不核对反引号。"""
text = "看 [[t:orders]]、`ghost_table` 和 order_cnt。"
doc = compose_doc(text, catalog, entities=False)
[seg] = [s for s in iter_segments(doc) if s.get("ref")]
assert seg["issue"] == "unresolved_ref" and seg["cite"]["reason"] == ENTITIES_OFF_REASON
assert not entity_segments(doc) or all(s.get("ref") for s in entity_segments(doc))
assert codes(doc) == ["unresolved_ref"] and ENTITIES_OFF_REASON in doc["violations"][0]["message"]
assert verify_doc(doc, catalog, entities=False)["violations"] == doc["violations"]
stripped = {a: e for a, e in catalog.items() if e["kind"] not in ("table", "column")}
assert compose_doc(text, stripped, entities=False)["violations"] == doc["violations"]
# --------------------------------------------------------------------------
# 自动链接只是标注,不是写作者声明的依据
# --------------------------------------------------------------------------
def units(doc):
return [u for block in doc["blocks"] for u in block["units"]]
@pytest.mark.parametrize("text", ["增长主要来自新客,见 order_id 字段。", "增长主要来自 `orders` 表的新客。",
"增长主要来自新客,见 orders.region。"])
def test_an_auto_linked_name_is_not_a_citation(catalog, text):
"""结论句的依据只认写作者自己写的 [[…]] 和 [[see:…]]:提一个字段名就算挂了依据的话,
claims: require_citation(受管级别必须写)形同虚设。"""
doc = compose_doc(text, catalog)
[unit] = units(doc)
assert entity_segments(doc) and all(s.get("auto") for s in entity_segments(doc))
assert unit["kind"] == "claim" and unit["cites"] == []
checked = verify_doc(doc, catalog)
assert [u["unit"] for u in checked["uncited"]] == [unit["id"]]
assert checked["uncited"] == uncited_claims(doc) and checked["stats"]["uncited_claims"] == 1
# 链接照样在:点得开,文档目录里也留着它的条目(证据接口要用)
for seg in entity_segments(doc):
assert seg["state"] == "deterministic" and seg["cite"]["alias"] in doc["catalog"]
def test_a_marker_the_writer_wrote_is_a_citation(catalog):
doc = compose_doc("增长主要来自新客,见 [[c:refunds.order_id]]。退款来自 `refunds`。[[see:t:refunds]]", catalog)
first, second = units(doc)
assert first["cites"] == ["c:refunds.order_id"] and second["cites"] == ["t:refunds"]
assert verify_doc(doc, catalog)["uncited"] == [] == uncited_claims(doc)
def test_an_auto_linked_name_alone_does_not_make_a_claim(catalog):
"""自动链接只重新切片:句子算不算结论,和它提没提表名无关——没开实体层时是连接性的话,
开了也还是。"""
text = "下面按 order_id 和 `orders` 展开,`ghost_table` 另说。"
doc = compose_doc(text, catalog)
[unit] = units(doc)
assert unit["kind"] == "connective" and entity_segments(doc)
plain = compose_doc(text, build_catalog(nodes={}, inputs={"week": "2026-W37"}))
assert [u["kind"] for u in units(plain)] == ["connective"]
# 名字里带方向词的(growth_rate 是查询结果的列,会被自动链接),开不开实体层都一样判
rated = build_catalog(nodes={}, ledger=[{**QUERY, "columns": ["growth_rate"]}, SCHEMA])
doc = compose_doc("见 growth_rate。", rated)
assert [s["text"] for s in entity_segments(doc)] == ["growth_rate"]
for cat in (rated, build_catalog(nodes={})):
assert [u["kind"] for u in units(compose_doc("见 growth_rate。", cat))] == ["claim"]
def test_verify_recomputes_uncited_instead_of_trusting_the_doc(catalog):
doc = compose_doc("增长主要来自新客,见 order_id 字段。收入上升。[[see:t:orders]]", catalog)
first, second = units(doc)
checked = verify_doc(doc, catalog)
assert checked["uncited"] == uncited_claims(doc) == [
{"unit": first["id"], "span": first["span"], "text": "增长主要来自新客,见 order_id 字段。"}]
first["cites"] = ["c:refunds.order_id"] # 文档自称挂了依据
assert uncited_claims(doc) == [] # 界面用的这份只照文档说
assert [u["unit"] for u in verify_doc(doc, catalog)["uncited"]] == [first["id"]] # 复核不信
second["see"] = [] # 依据被拿掉了:复核也看得出来
assert [u["unit"] for u in verify_doc(doc, catalog)["uncited"]] == [first["id"], second["id"]]
# --------------------------------------------------------------------------
# 跑一次:表结构快照被冻结,台账记下它
# --------------------------------------------------------------------------
@pytest.fixture
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
@pytest.fixture
async def warehouse(tmp_path):
"""探查过结构的示例库:orders、refunds 两张表。"""
from app.data.introspect import introspect
path = tmp_path / "warehouse.db"
db = sqlite3.connect(path)
db.executescript(
"CREATE TABLE orders (id INTEGER PRIMARY KEY, region TEXT, amount REAL, created_at TEXT);"
"CREATE TABLE refunds (id INTEGER PRIMARY KEY, order_id INTEGER, amount REAL);")
db.executemany("INSERT INTO orders VALUES (?, ?, ?, ?)",
[(1, "east", 100.5, "2026-09-01"), (2, "west", 200.0, "2026-09-02"), (3, "east", 300.0, "2026-09-03")])
db.commit()
db.close()
probe = SimpleNamespace(id=f"probe-{path.name}", name=SOURCE, kind="sqlite", database=str(path), host=None,
port=None, username=None, password=None, options={}, readonly=True, schema_cache={})
cache = await introspect(probe)
await engines.invalidate(probe.id)
async with SessionLocal() as session:
row = (await session.execute(select(DataSource).where(DataSource.name == SOURCE))).scalar_one_or_none()
if row is None:
row = DataSource(name=SOURCE, kind="sqlite", database=str(path), readonly=True, enabled=True)
session.add(row)
row.database, row.kind, row.enabled, row.readonly, row.schema_cache = str(path), "sqlite", True, True, cache
await session.commit()
source_id = row.id
await engines.invalidate(source_id)
yield source_id
await engines.invalidate(source_id)
def node(nid, ntype, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
def chain(*nodes):
return {"nodes": list(nodes), "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
def tool_graph(*after, sql=SQL):
return chain(node("start", "input"), node("fetch", "tool", tool=TOOL, args={"sql": sql}), *after,
node("out", "output", fields=[{"name": "r", "value": "{{ nodes.fetch }}"}]))
async def finish(graph, statuses=("succeeded", "failed")) -> Run:
run = await run_manager.start(graph=graph, input_payload={})
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run.id)
if row.status in statuses and (row.status == "interrupted" or row.manifest_seq is not None):
return row
raise AssertionError(f"run {run.id} 没跑完:{row.status}")
async def events(run_id: str, etype: str, node_id: str | None = None) -> list[RunEvent]:
async with SessionLocal() as session:
q = select(RunEvent).where(RunEvent.run_id == run_id, RunEvent.type == etype)
if node_id:
q = q.where(RunEvent.node_id == node_id)
return list((await session.execute(q.order_by(RunEvent.seq))).scalars())
async def ledger(run_id: str) -> list[dict]:
tup = await run_manager.checkpointer.aget_tuple({"configurable": {"thread_id": run_id}})
return list(tup.checkpoint["channel_values"].get("evidence") or [])
async def test_schema_snapshot_is_frozen_and_recorded(engine_up, warehouse):
row = await finish(tool_graph())
assert row.status == "succeeded", row.error
[end] = await events(row.id, "tool.end", "fetch")
schema_art = end.data["schema_artifact"]
payload = json.loads(row.output["r"])
assert payload["schema_artifact"] == schema_art
snap = artifact_store.load(schema_art)
assert snap["source"] == SOURCE and set(snap["tables"]) == {"orders", "refunds"}
assert artifact_store.load(payload["artifact"])["schema_artifact"] == schema_art
entries = await ledger(row.id)
assert [e["kind"] for e in entries] == ["tool", "query", "schema"]
query, schema = entries[1], entries[2]
assert query["schema_artifact"] == schema_art and query["tables"] == ["orders"]
assert schema == {"kind": "schema", "node_id": "fetch", "exec": 1, "artifact": schema_art, "source": SOURCE,
"schema": snap.get("schema"), "synced_at": snap.get("synced_at"),
"tables": {"orders": ["id", "region", "amount", "created_at"],
"refunds": ["id", "order_id", "amount"]}}
[finished] = await events(row.id, "node.finished", "fetch")
assert finished.data["evidence"] == entries
# 事后改了数据源的结构缓存:已有的快照、台账、目录都不变
async with SessionLocal() as session:
src = await session.get(DataSource, warehouse)
cache = json.loads(json.dumps(src.schema_cache))
cache["tables"]["ghost_new"] = {"qualified": "ghost_new", "columns": [{"name": "x", "type": "INT"}]}
del cache["tables"]["refunds"]
src.schema_cache = cache
await session.commit()
assert artifact_store.load(schema_art) == snap
catalog = build_catalog(nodes={}, ledger=await ledger(row.id))
assert "t:refunds" in catalog and "t:ghost_new" not in catalog
again = await finish(tool_graph())
[end2] = await events(again.id, "tool.end", "fetch")
assert end2.data["schema_artifact"] != schema_art
assert "ghost_new" in artifact_store.load(end2.data["schema_artifact"])["tables"]
async def test_agent_records_one_schema_entry_per_snapshot(engine_up, warehouse, monkeypatch):
from app.providers import mock_model
def decide(self, messages):
done = sum(isinstance(m, ToolMessage) for m in messages)
if done < 2:
sql = SQL if done == 0 else "SELECT COUNT(*) AS n FROM refunds -- FROM ghost_table"
return AIMessage(content="查。", tool_calls=[{"name": TOOL, "args": {"sql": sql}, "id": f"call_{done}"}])
return AIMessage(content="查完了。")
monkeypatch.setattr(mock_model.MockChatModel, "_decide", decide)
row = await finish(chain(node("start", "input"), node("bot", "agent", prompt="查", tools=[TOOL], approval="never"),
node("out", "output", fields=[{"name": "r", "value": "{{ nodes.bot.text }}"}])))
assert row.status == "succeeded", row.error
ends = await events(row.id, "tool.end", "bot")
assert len(ends) == 2 and ends[0].data["schema_artifact"] == ends[1].data["schema_artifact"]
entries = await ledger(row.id)
assert [e["kind"] for e in entries] == ["tool", "query", "schema", "tool", "query"]
assert [e["tables"] for e in entries if e["kind"] == "query"] == [["orders"], ["refunds"]]
catalog = build_catalog(nodes={}, ledger=entries)
assert catalog["t:refunds"]["queries"] == ["Q2"] and "t:ghost_table" not in catalog
async def test_runs_from_before_the_upgrade_record_no_schema(engine_up, warehouse, monkeypatch):
async def no_snapshot(session):
return None
monkeypatch.setattr(runner_mod, "_agent_limits", no_snapshot)
row = await finish(tool_graph())
assert row.status == "succeeded", row.error
[end] = await events(row.id, "tool.end", "fetch")
assert "schema_artifact" not in end.data and "query_artifact" not in end.data
[finished] = await events(row.id, "node.finished", "fetch")
assert "evidence" not in finished.data
async def test_report_flags_a_made_up_name_without_failing(engine_up, warehouse, monkeypatch):
"""可疑实体只标注:on_violation=fail 的报告节点不为它失败,也不为它重写。"""
from app.providers import mock_model
seen: list = []
def decide(self, messages):
seen.append(messages)
return AIMessage(content="`orders` 里东区最多,达 [[v:Q1.r0.gmv]];`refund_log` 另算。[[see:Q1]]")
monkeypatch.setattr(mock_model.MockChatModel, "_decide", decide)
row = await finish(tool_graph(node("write", "report", instructions="写一句", on_violation="fail")))
assert row.status == "succeeded", row.error
assert len(seen) == 1, "可疑实体不该触发重写"
[checked] = await events(row.id, "report.checked", "write")
assert [v["code"] for v in checked.data["violations"]] == ["unknown_entity"]
assert checked.data["stats"]["unknown_entities"] == 1 and checked.data["stats"]["entities"] == 1
prompt = "\n".join(str(m.content) for m in seen[0])
assert "[[t:" in prompt and "可能是编造的名字" in prompt and "orders" in prompt
doc = artifact_store.load(checked.data["doc_artifact"])
[linked] = [s for s in iter_segments(doc) if s.get("ref") == "t:orders"]
assert linked["code"] is True and linked["cite"]["eid"] == doc["catalog"]["t:orders"]["eid"]
schema_art = (await events(row.id, "tool.end", "fetch"))[0].data["schema_artifact"]
assert doc["catalog"]["t:orders"]["artifact"] == schema_art
async def test_report_with_entities_off_leaves_names_alone(engine_up, warehouse, monkeypatch):
"""entities: off:目录里没有表和字段,写作目录不教 [[t:]],反引号不核对;写了 [[t:]] 的,
解析不了的原因照实说是这个节点关掉的。"""
from app.providers import mock_model
seen: list = []
def decide(self, messages):
seen.append(messages)
return AIMessage(content="`orders` 里东区最多,达 [[v:Q1.r0.gmv]];`refund_log` 另算,见 [[t:orders]]。[[see:Q1]]")
monkeypatch.setattr(mock_model.MockChatModel, "_decide", decide)
row = await finish(tool_graph(node("write", "report", instructions="写一句", entities="off", max_repairs=0)))
assert row.status == "succeeded", row.error
[checked] = await events(row.id, "report.checked", "write")
assert [v["code"] for v in checked.data["violations"]] == ["unresolved_ref"]
assert ENTITIES_OFF_REASON in checked.data["violations"][0]["message"]
assert "unknown_entities" not in checked.data["stats"] and "entities" not in checked.data["stats"]
doc = artifact_store.load(checked.data["doc_artifact"])
assert not [a for a, e in doc["catalog"].items() if e["kind"] in ("table", "column")]
assert not [s for s in iter_segments(doc) if s["kind"] == "entity" and not s.get("ref")]
assert "[[t:" not in "\n".join(str(m.content) for m in seen[0])
async def test_report_with_a_bad_entities_value_fails(engine_up, warehouse, monkeypatch):
from app.providers import mock_model
seen: list = []
monkeypatch.setattr(mock_model.MockChatModel, "_decide", lambda self, messages: seen.append(1) or AIMessage("x"))
row = await finish(tool_graph(node("write", "report", instructions="写一句", entities="bogus")))
assert row.status == "failed"
assert "entities 只能是 link / off,写的是 'bogus'" in (row.error or ""), row.error
assert seen == [], "写错的配置不该先花一次模型调用"
def test_soft_violations_never_ask_for_a_rewrite():
from app.engine.nodes.report import SOFT, _blocking
assert SOFT == {"unknown_entity", "unverified_entity"}
soft = [{"code": "unknown_entity"}, {"code": "unverified_entity"}]
assert _blocking(soft, "strict") == []
assert _blocking([*soft, {"code": "unresolved_ref"}], "strict") == [{"code": "unresolved_ref"}]
# --------------------------------------------------------------------------
# 给 B 段的导出:evidence_from 连带表结构、按名字找条目
# --------------------------------------------------------------------------
def test_evidence_from_also_filters_the_schema_entries():
"""evidence_from 只收某几个节点的证据:别的节点查询当时冻结的表结构也不收,否则报告能引用
一张它的证据范围里根本没查过的表。"""
from app.engine.nodes.report import report_catalog, report_entities
from app.engine.schema import GraphSpec
other = {**QUERY, "node_id": "other", "artifact": "c" * 64, "columns": ["n"],
"schema_artifact": "6" * 64, "tables": ["refunds"]}
other_schema = {**SCHEMA, "node_id": "other", "artifact": "6" * 64, "tables": {"refunds": ["id", "order_id"]}}
state = {"nodes": {}, "input": {}, "evidence": [QUERY, SCHEMA, other, other_schema]}
def catalog_for(**config):
spec = GraphSpec.model_validate(chain(node("start", "input"), node("fetch", "tool", tool=TOOL),
node("other", "tool", tool=TOOL), node("write", "report", **config)))
return report_catalog(state, spec, spec.node_map()["write"]), spec
full, _ = catalog_for()
assert {"t:orders", "t:refunds", "c:orders.region"} <= set(full)
only, _ = catalog_for(evidence_from=["other"])
assert [a for a in only if a[0] == "Q"] == ["Q1"] and only["Q1"]["artifact"] == "c" * 64
tables = {a for a, e in only.items() if e["kind"] == "table"}
assert tables == {"t:refunds"} # fetch 的表结构(orders)不收
assert all(o["artifact"] == "6" * 64 for o in only["t:refunds"]["sources"] if o["kind"] == "schema")
off, spec = catalog_for(entities="off")
assert not [a for a, e in off.items() if e["kind"] in ("table", "column")] and "Q1" in off
assert report_entities(spec, spec.node_map()["write"]) == "off"
def test_find_entity_and_closest_entities(catalog):
from app.engine.evidence import closest_entities, find_entity
assert find_entity("ORDERS", catalog) == "t:orders"
assert find_entity(" orders.Amount ", catalog) == "c:orders.amount"
assert find_entity("amount", catalog) == "c:amount" # 好几张表都有的字段
assert find_entity("order_cnt", catalog) == "c:order_cnt" # 没有表名的结果列
assert find_entity("amount", catalog, "table") is None
assert find_entity("orders", catalog, "column") is None
assert find_entity("ghost_table", catalog) is None
qualified = build_catalog(nodes={}, ledger=[{**SCHEMA, "schema": "public"}])
assert find_entity("public.orders", qualified) == "t:orders" == find_entity("PUBLIC.ORDERS", qualified, "table")
assert find_entity("public.orders.region", qualified) == "c:orders.region"
assert closest_entities("ordrs", catalog)[0] == "t:orders"
assert closest_entities("refund", catalog, limit=1) == ["t:refunds"]
assert len(closest_entities("order", catalog, limit=2)) <= 2
assert closest_entities("zzzz_qqq", catalog) == []
assert closest_entities("ordrs", catalog) == closest_entities("ordrs", catalog) # 同样的输入同样的顺序
+368
View File
@@ -0,0 +1,368 @@
"""证据接口的实体步骤和引文步骤:点开报告里的表名、字段名、原话,看得到它们的出处。
- 表名、字段名:出现在哪几次查询里(SQL 用到的表、结果的列),表结构快照是什么时候同步的、
字段是什么类型。表结构快照只认封存范围内的事件交回过的那几件(tool.end.schema_artifact、
node.finished.evidence 的 schema 条目),取回时复验哈希
- 可疑实体(写了一个哪里都找不到的名字):照实说本次运行的表结构、查询、结果列里都没有它,
再附上最接近的几个已知名字当提示——提示只从封存的台账里找
- 引文:原话落在哪次检索的哪条命中、原文里的起止,检索快照同样只认封存范围内的
"""
from __future__ import annotations
import asyncio
import json
import sqlite3
from types import SimpleNamespace
import pytest
from httpx import ASGITransport, AsyncClient
from langchain_core.messages import AIMessage
from sqlalchemy import select
from app.core import artifact_store
from app.core.config import settings
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import DataSource, Run, RunEvent
from app.engine.evidence import (
UNKNOWN_ENTITY_REASON,
UNVERIFIED_ENTITY_REASON,
iter_segments,
make_eid,
normalize_quote,
)
from app.engine.runner import run_manager
from app.main import app
SOURCE = "warehouse"
TOOL = f"db_query__{SOURCE}"
SQL = "SELECT region, SUM(amount) AS gmv, COUNT(*) AS order_cnt FROM orders GROUP BY region ORDER BY gmv DESC"
WRITE = "`orders` 里东区最多,达 [[v:Q1.r0.gmv]],按 [[c:orders.region]] 汇总。`refund_log` 另算。[[see:Q1]]"
HIT = {"chunk_id": "ch-1", "document_id": "doc-1", "title": "售后月报", "ordinal": 3,
"content": "九月退款主要集中在\n东区,原因是物流延误。"}
@pytest.fixture(autouse=True)
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
@pytest.fixture
async def client():
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
async def _warehouse(tmp_path, *, truncated: bool = False) -> str:
"""探查过结构的示例库:orders、refunds 两张表。truncated 装作库里的表太多、快照只存了一部分。"""
from app.data.introspect import introspect
path = tmp_path / "warehouse.db"
db = sqlite3.connect(path)
db.executescript(
"CREATE TABLE orders (id INTEGER PRIMARY KEY, region TEXT, amount REAL, created_at TEXT);"
"CREATE TABLE refunds (id INTEGER PRIMARY KEY, order_id INTEGER, amount REAL);")
db.executemany("INSERT INTO orders VALUES (?, ?, ?, ?)",
[(1, "east", 100.5, "2026-09-01"), (2, "west", 200.0, "2026-09-02"), (3, "east", 300.0, "2026-09-03")])
db.commit()
db.close()
probe = SimpleNamespace(id=f"probe-{path.name}", name=SOURCE, kind="sqlite", database=str(path), host=None,
port=None, username=None, password=None, options={}, readonly=True, schema_cache={})
cache = await introspect(probe)
if truncated:
cache.update(truncated=True, total=500)
await engines.invalidate(probe.id)
async with SessionLocal() as session:
row = (await session.execute(select(DataSource).where(DataSource.name == SOURCE))).scalar_one_or_none()
if row is None:
row = DataSource(name=SOURCE, kind="sqlite", database=str(path), readonly=True, enabled=True)
session.add(row)
row.database, row.kind, row.enabled, row.readonly, row.schema_cache = str(path), "sqlite", True, True, cache
row.options = {}
await session.commit()
source_id = row.id
await engines.invalidate(source_id)
return source_id
@pytest.fixture
async def warehouse(tmp_path):
source_id = await _warehouse(tmp_path)
yield source_id
await engines.invalidate(source_id)
@pytest.fixture
async def partial_warehouse(tmp_path):
source_id = await _warehouse(tmp_path, truncated=True)
yield source_id
await engines.invalidate(source_id)
def node(nid, ntype, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
def chain(*nodes):
return {"nodes": list(nodes), "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
def script(monkeypatch, text):
from app.providers import mock_model
monkeypatch.setattr(mock_model.MockChatModel, "_decide", lambda self, messages: AIMessage(content=text))
async def finish(graph) -> Run:
run = await run_manager.start(graph=graph, input_payload={})
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run.id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
return row
raise AssertionError(f"run {run.id} 没跑完:{row.status}")
async def events(run_id: str, etype: str, node_id: str | None = None) -> list[RunEvent]:
async with SessionLocal() as session:
q = select(RunEvent).where(RunEvent.run_id == run_id, RunEvent.type == etype)
if node_id:
q = q.where(RunEvent.node_id == node_id)
return list((await session.execute(q.order_by(RunEvent.seq))).scalars())
async def reseal(run_id: str) -> None:
"""按改过之后的事件重算封存哈希:模拟「封存链完好、但事件里追不到这件工件」。"""
async with SessionLocal() as session:
run = await session.get(Run, run_id)
rows = [tuple(r) for r in await session.execute(
select(RunEvent.seq, RunEvent.type, RunEvent.node_id, RunEvent.data)
.where(RunEvent.run_id == run_id, RunEvent.seq <= run.manifest_seq).order_by(RunEvent.seq))]
run.manifest_hash = artifact_store.manifest_hash(rows)
await session.commit()
async def rewrite(run_id: str, etype: str, node_id: str, change) -> None:
async with SessionLocal() as session:
for row in (await session.execute(select(RunEvent).where(
RunEvent.run_id == run_id, RunEvent.type == etype, RunEvent.node_id == node_id))).scalars():
row.data = change(dict(row.data))
await session.commit()
async def report_run(monkeypatch, text=WRITE) -> tuple[Run, dict]:
script(monkeypatch, text)
row = await finish(chain(node("start", "input"), node("fetch", "tool", tool=TOOL, args={"sql": SQL}),
node("write", "report", instructions="写一句"),
node("out", "output", fields=[{"name": "r", "value": "{{ nodes.write.text }}"}])))
assert row.status == "succeeded", row.error
[checked] = await events(row.id, "report.checked", "write")
return row, artifact_store.load(checked.data["doc_artifact"])
async def opened(client, run_id: str, seg: dict) -> dict:
resp = await client.get(f"/api/runs/{run_id}/evidence/segments/{seg['id']}")
assert resp.status_code == 200, resp.text
return resp.json()
def seg_where(doc, **match) -> dict:
return next(s for s in iter_segments(doc) if all(s.get(k) == v for k, v in match.items()))
# --------------------------------------------------------------------------
# 表名、字段名
# --------------------------------------------------------------------------
async def test_a_table_opens_its_schema_snapshot_and_queries(client, warehouse, monkeypatch):
row, doc = await report_run(monkeypatch)
[end] = await events(row.id, "tool.end", "fetch")
schema_art, query_art = end.data["schema_artifact"], end.data["query_artifact"]
snap = artifact_store.load(schema_art)
body = await opened(client, row.id, seg_where(doc, kind="entity", ref="t:orders"))
[step] = body["chain"]
assert step["step"] == "entity" and step["status"] == "resolved"
assert step["kind"] == "table" and step["name"] == "orders" and step["alias"] == "t:orders"
assert step["code"] is True and step["auto"] is True
assert step["queries"] == ["Q1"] and step["source"] == SOURCE
assert step["synced_at"] == snap["synced_at"] and step["columns"] == 4 and step["is_view"] is False
kinds = {s["kind"]: s for s in step["sources"]}
assert set(kinds) == {"schema", "sql"}
assert kinds["schema"]["artifact"] == schema_art and kinds["schema"]["sealed"] is True
assert kinds["schema"]["hash_ok"] is True and kinds["schema"]["present"] is True
assert kinds["sql"] == {"kind": "sql", "artifact": query_art, "alias": "Q1", "source": SOURCE,
"node_id": "fetch", "tool": TOOL, "sealed": True}
assert step["eid"] == make_eid("table", schema_art, {"table": "orders"}) and step["eid_ok"] is True
assert step["sealed"] is True and body["seal"]["covered"] is True
assert "表结构" in body["note"]
async def test_a_column_carries_its_type_and_result_source(client, warehouse, monkeypatch):
row, doc = await report_run(monkeypatch)
body = await opened(client, row.id, seg_where(doc, kind="entity", ref="c:orders.region"))
[step] = body["chain"]
assert step["kind"] == "column" and step["table"] == "orders" and step["column"] == "region"
assert step["type"] == "TEXT" and step["type_source"] == "schema" and step["nullable"] is True
assert step["primary_key"] is False and "code" not in step and "auto" not in step
assert {(s["kind"], s.get("alias")) for s in step["sources"]} == {("schema", None), ("result", "Q1")}
assert step["queries"] == ["Q1"] and step["sealed"] is True
async def test_an_aggregate_alias_takes_its_type_from_the_result(client, warehouse, monkeypatch):
"""聚合的别名(gmv)不在表结构里:类型取查询快照记下的列类型,出处是那份查询快照。"""
row, doc = await report_run(monkeypatch, "东区最多,汇总在 `gmv` 这一列,见 [[v:Q1.r0.gmv]]。[[see:Q1]]")
[end] = await events(row.id, "tool.end", "fetch")
body = await opened(client, row.id, seg_where(doc, kind="entity", ref="c:gmv"))
[step] = body["chain"]
assert step["kind"] == "column" and step["column"] == "gmv" and step.get("table") is None
assert step["type"] == "number" and step["type_source"] == "result"
assert [(s["kind"], s["artifact"], s["sealed"]) for s in step["sources"]] == [
("result", end.data["query_artifact"], True)]
assert step["eid"] == make_eid("column", end.data["query_artifact"], {"column": "gmv"})
assert step["sealed"] is True
async def test_a_made_up_name_explains_itself_and_suggests_real_ones(client, warehouse, monkeypatch):
row, doc = await report_run(monkeypatch)
seg = seg_where(doc, kind="entity", issue="unknown_entity")
body = await opened(client, row.id, seg)
[step] = body["chain"]
assert step["step"] == "entity" and step["status"] == "unknown" and step["name"] == "refund_log"
assert step["reason"] == UNKNOWN_ENTITY_REASON
assert 0 < len(step["closest"]) <= 3
assert step["closest"][0] == {"alias": "t:refunds", "kind": "table", "name": "refunds"}
assert step["checked"] == {"schemas": 1, "queries": 1}
assert "可能是编造的名字" in body["note"]
assert [v["code"] for v in body["violations"]] == ["unknown_entity"]
async def test_a_made_up_marker_is_suspicious_too(client, warehouse, monkeypatch):
"""[[t:编造]] 和反引号里编的名字一样是可疑实体:名字取自引用,不是占位。"""
row, doc = await report_run(monkeypatch, "退款看 [[t:refund_logs]],东区最多 [[v:Q1.r0.gmv]]。[[see:Q1]]")
body = await opened(client, row.id, seg_where(doc, kind="entity", issue="unknown_entity"))
[step] = body["chain"]
assert step["status"] == "unknown" and step["name"] == "refund_logs" and step["kind"] == "table"
assert step["closest"][0]["alias"] == "t:refunds"
async def test_a_name_a_partial_snapshot_cannot_check_is_not_called_made_up(client, partial_warehouse, monkeypatch):
row, doc = await report_run(monkeypatch)
body = await opened(client, row.id, seg_where(doc, kind="entity", issue="unverified_entity"))
[step] = body["chain"]
assert step["status"] == "unverified" and step["name"] == "refund_log"
assert step["reason"] == UNVERIFIED_ENTITY_REASON and "编造" not in body["note"] and "核对不了" in body["note"]
table = await opened(client, row.id, seg_where(doc, kind="entity", ref="t:orders"))
assert table["chain"][0]["snapshot_truncated"] is True
assert {s["kind"]: s for s in table["chain"][0]["sources"]}["schema"]["truncated"] is True
async def test_suggestions_come_only_from_the_sealed_ledger(client, warehouse, monkeypatch):
"""台账里的 schema 条目在封存的事件里被去掉了:提示里就不能再有只在那份快照里的表。"""
row, doc = await report_run(monkeypatch)
await rewrite(row.id, "node.finished", "fetch",
lambda d: {**d, "evidence": [e for e in d["evidence"] if e["kind"] != "schema"]})
await reseal(row.id)
body = await opened(client, row.id, seg_where(doc, kind="entity", issue="unknown_entity"))
[step] = body["chain"]
assert all(c["alias"] != "t:refunds" for c in step["closest"]), step["closest"]
assert step["checked"]["schemas"] == 0
async def test_a_schema_snapshot_outside_the_seal_is_not_trusted(client, warehouse, monkeypatch):
row, doc = await report_run(monkeypatch)
[end] = await events(row.id, "tool.end", "fetch")
schema_art = end.data["schema_artifact"]
await rewrite(row.id, "tool.end", "fetch", lambda d: {k: v for k, v in d.items() if k != "schema_artifact"})
await rewrite(row.id, "node.finished", "fetch",
lambda d: {**d, "evidence": [e for e in d["evidence"] if e["kind"] != "schema"]})
await reseal(row.id)
body = await opened(client, row.id, seg_where(doc, kind="entity", ref="c:orders.region"))
[step] = body["chain"]
schema = next(s for s in step["sources"] if s["kind"] == "schema")
assert schema["artifact"] == schema_art and schema["sealed"] is False
assert "hash_ok" not in schema and "present" not in schema
# 类型、同步时间只能从封存的快照取:取不到就不给,不从工件库里「顺手」读
assert "type" not in step and "synced_at" not in step
assert step["sealed"] is False and body["seal"]["covered"] is False
graph = (await client.get(f"/api/runs/{row.id}/evidence")).json()
assert {e["alias"]: e["sealed"] for e in graph["evidence"]}["c:orders.region"] is False
async def test_a_tampered_schema_snapshot_is_reported(client, warehouse, monkeypatch):
row, doc = await report_run(monkeypatch)
[end] = await events(row.id, "tool.end", "fetch")
art = end.data["schema_artifact"]
path = settings.data_dir / "artifacts" / art[:2] / f"{art}.json"
content = json.loads(path.read_text("utf-8"))
content["tables"]["orders"]["columns"][1]["type"] = "INTEGER"
path.write_text(json.dumps(content, ensure_ascii=False), encoding="utf-8")
body = await opened(client, row.id, seg_where(doc, kind="entity", ref="c:orders.region"))
[step] = body["chain"]
schema = next(s for s in step["sources"] if s["kind"] == "schema")
assert schema["sealed"] is True and schema["hash_ok"] is False
assert "type" not in step and step["sealed"] is False
# --------------------------------------------------------------------------
# 引文
# --------------------------------------------------------------------------
async def quote_run(monkeypatch, text="售后说「[[q:K1|退款主要集中在东区]]」。[[see:K1]]") -> tuple[Run, dict]:
from app.memory import kb
async def fake_search(session, **kw):
return [dict(HIT, score=0.9)]
monkeypatch.setattr(kb, "search", fake_search)
script(monkeypatch, text)
row = await finish(chain(node("start", "input"), node("docs", "retrieve", query="退款", collection="kb"),
node("write", "report", instructions="引用售后的原话"),
node("out", "output", fields=[{"name": "r", "value": "{{ nodes.write.text }}"}])))
assert row.status == "succeeded", row.error
[checked] = await events(row.id, "report.checked", "write")
return row, artifact_store.load(checked.data["doc_artifact"])
async def test_a_quote_opens_the_hit_it_was_copied_from(client, monkeypatch):
row, doc = await quote_run(monkeypatch)
[end] = await events(row.id, "retrieve.end", "docs")
body = await opened(client, row.id, seg_where(doc, kind="quote"))
[step] = body["chain"]
assert step["step"] == "quote" and step["alias"] == "K1" and step["text"] == "退款主要集中在东区"
assert step["source"] == {"artifact": end.data["artifact"], "document": "doc-1", "chunk": "ch-1",
"title": "售后月报", "ordinal": 3}
match = step["match"]
assert match["hit"] == 0 and step["content"] == HIT["content"]
assert normalize_quote(step["content"][match["start"]:match["end"]]) == normalize_quote(step["text"])
assert step["collection"] == "kb" and step["query"] == "退款" and step["node_id"] == "docs"
assert step["hash_ok"] is True and step["match_ok"] is True and step["eid_ok"] is True
assert step["sealed"] is True and body["seal"]["covered"] is True
assert "逐字" in body["note"]
async def test_a_retrieval_outside_the_seal_is_not_shown(client, monkeypatch):
row, doc = await quote_run(monkeypatch)
await rewrite(row.id, "retrieve.end", "docs", lambda d: {k: v for k, v in d.items() if k != "artifact"})
await rewrite(row.id, "node.finished", "docs",
lambda d: {**d, "evidence": [e for e in d.get("evidence") or [] if e["kind"] != "retrieval"]})
await reseal(row.id)
body = await opened(client, row.id, seg_where(doc, kind="quote"))
[step] = body["chain"]
assert step["sealed"] is False and step["content"] is None and step["note"]
assert body["seal"]["covered"] is False
async def test_an_unmatched_quote_has_no_chain(client, monkeypatch):
row, doc = await quote_run(monkeypatch, "售后说「[[q:K1|退款主要集中在西区]]」。[[see:K1]]")
seg = seg_where(doc, kind="quote")
assert seg["state"] == "none"
body = await opened(client, row.id, seg)
assert body["chain"] == [] and "找不到这句原话" in body["note"]
+298
View File
@@ -0,0 +1,298 @@
"""旧运行的降级展示:候选只从已封存的工件里找。
- legacy_text:没有契约、没有报告的答案,把数字按数值和封存范围内的查询快照单元格、口径卡指标
比对,每个数字最多 3 个候选,state 记 candidate,写明「猜测的来源,不能当证据」。按
(run_id, manifest_hash) 放进内存 LRU(再带上数据源遮罩的指纹:事后加了遮罩要重算),猜测放到
线程里跑,不堵事件循环
- 二期以前的运行没有 tool.end.query_artifact:查询快照的 id 只嵌在 tool_snapshot 的内容里,
顺着封存事件里的 tool_snapshot(取回时复验哈希)找
- legacy_contract:旧契约 matched 的指标详情同样只从封存的口径卡里取
- 人为插进 artifacts 表的行、封存之后追加的事件指向的工件,一律不当证据
"""
from __future__ import annotations
import asyncio
import sqlite3
import threading
import pytest
from httpx import ASGITransport, AsyncClient
from sqlalchemy import select
from app.api import evidence as evidence_api
from app.core import artifact_store
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import Artifact, DataSource, Run, RunEvent
from app.engine import runner as runner_mod
from app.engine.evidence import GUESS_NOTE, GUESS_SCHEMA
from app.engine.runner import run_manager
from app.main import app
SOURCE = "warehouse"
TOOL = f"db_query__{SOURCE}"
SQL = "SELECT region, SUM(amount) AS gmv, COUNT(*) AS order_cnt FROM orders GROUP BY region ORDER BY gmv DESC"
ANSWER = "东区销售额 400.5,西区 200,一共 7 单。"
@pytest.fixture(autouse=True)
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
@pytest.fixture
async def client():
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
@pytest.fixture
async def warehouse(tmp_path):
path = tmp_path / "warehouse.db"
db = sqlite3.connect(path)
db.executescript("CREATE TABLE orders (id INTEGER PRIMARY KEY, region TEXT, amount REAL);")
db.executemany("INSERT INTO orders VALUES (?, ?, ?)", [(1, "east", 100.5), (2, "west", 200.0), (3, "east", 300.0)])
db.commit()
db.close()
async with SessionLocal() as session:
row = (await session.execute(select(DataSource).where(DataSource.name == SOURCE))).scalar_one_or_none()
if row is None:
row = DataSource(name=SOURCE, kind="sqlite", database=str(path), readonly=True, enabled=True)
session.add(row)
row.database, row.kind, row.enabled, row.readonly, row.schema_cache = str(path), "sqlite", True, True, {}
row.options = {}
await session.commit()
source_id = row.id
await engines.invalidate(source_id)
yield source_id
await engines.invalidate(source_id)
def node(nid, ntype, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
def chain(*nodes):
return {"nodes": list(nodes), "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
def asked(answer=ANSWER):
return chain(node("start", "input"), node("fetch", "tool", tool=TOOL, args={"sql": SQL}),
node("out", "output", fields=[{"name": "answer", "value": answer}]))
async def finish(graph) -> Run:
run = await run_manager.start(graph=graph, input_payload={})
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run.id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
return row
raise AssertionError(f"run {run.id} 没跑完:{row.status}")
async def events(run_id: str, etype: str, node_id: str | None = None) -> list[RunEvent]:
async with SessionLocal() as session:
q = select(RunEvent).where(RunEvent.run_id == run_id, RunEvent.type == etype)
if node_id:
q = q.where(RunEvent.node_id == node_id)
return list((await session.execute(q.order_by(RunEvent.seq))).scalars())
async def reseal(run_id: str) -> None:
async with SessionLocal() as session:
run = await session.get(Run, run_id)
rows = [tuple(r) for r in await session.execute(
select(RunEvent.seq, RunEvent.type, RunEvent.node_id, RunEvent.data)
.where(RunEvent.run_id == run_id, RunEvent.seq <= run.manifest_seq).order_by(RunEvent.seq))]
run.manifest_hash = artifact_store.manifest_hash(rows)
await session.commit()
def numbers(field: dict) -> dict[str, dict]:
return {s["text"]: s for s in field["segments"] if s["kind"] == "number"}
def counting(monkeypatch) -> list[int]:
"""数一数真正猜了几次,并记下在哪个线程里猜的。"""
calls: list[int] = []
real = evidence_api.guess_sources
def spy(*args, **kwargs):
calls.append(threading.get_ident())
return real(*args, **kwargs)
monkeypatch.setattr(evidence_api, "guess_sources", spy)
return calls
# --------------------------------------------------------------------------
# legacy_text
# --------------------------------------------------------------------------
async def test_an_answer_without_a_contract_gets_guessed_candidates(client, warehouse):
row = await finish(asked())
assert row.status == "succeeded", row.error
[end] = await events(row.id, "tool.end", "fetch")
body = (await client.get(f"/api/runs/{row.id}/evidence")).json()
assert body["mode"] == "legacy_text" and body["reports"] == [] and body["evidence"] == []
legacy = body["legacy"]
assert legacy["schema"] == GUESS_SCHEMA and legacy["mode"] == "legacy_text" and legacy["note"] == GUESS_NOTE
assert legacy["note"].startswith("猜测的来源,不能当证据") and legacy["sealed"] is True
[field] = legacy["fields"]
assert field["field"] == "answer" and field["markdown"] == ANSWER
found = numbers(field)
east = found["400.5"]
assert east["state"] == "candidate" and len(east["candidates"]) <= 3
top = east["candidates"][0]
assert top["kind"] == "cell" and top["ref"] == "Q1.r0.gmv" and top["artifact"] == end.data["query_artifact"]
assert top["node_id"] == "fetch" and top["tool"] == TOOL
assert found["200"]["candidates"][0]["ref"] == "Q1.r1.gmv"
assert found["7"]["state"] == "none" and "candidates" not in found["7"]
assert legacy["stats"] == {"numbers": 3, "guessed": 2, "unguessed": 1,
"candidates": sum(len(s.get("candidates") or []) for s in found.values())}
audit = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
assert audit["mode"] == "legacy_text"
guessed = next(g for g in audit["groups"] if g["key"] == "candidate")["rows"]
assert [(r["text"], r["field"], r["state"]) for r in guessed] == [("400.5", "answer", "candidate"),
("200", "answer", "candidate")]
assert guessed[0]["evidence"].startswith("Q1.r0.gmv") and "猜测" in guessed[0]["note"]
assert len(guessed[0]["candidates"]) == len(east["candidates"])
unguessed = next(g for g in audit["groups"] if g["key"] == "none")["rows"]
assert [(r["text"], r["kind"]) for r in unguessed] == [("7", "number")]
async def test_artifacts_outside_the_sealed_events_are_not_candidates(client, warehouse):
"""人为往 artifacts 表插一行(内容里正好有 7),再在封存之后追加一条指向它的 tool.end:都不认。"""
row = await finish(asked())
fake = await artifact_store.put_json(
{"columns": ["n"], "rows": [[7]], "row_count": 1, "truncated": False, "sql": "SELECT 7 AS n",
"source": SOURCE}, kind="query_snapshot", run_id=row.id, node_id="fetch")
async with SessionLocal() as session:
assert await session.get(Artifact, (fake, row.id)) is not None
session.add(RunEvent(run_id=row.id, seq=row.manifest_seq + 3, type="tool.end", node_id="fetch", ts=0.0,
data={"tool": TOOL, "query_artifact": fake}))
await session.commit()
body = (await client.get(f"/api/runs/{row.id}/evidence")).json()
assert body["seal"]["ok"] is True
found = numbers(body["legacy"]["fields"][0])
assert found["7"]["state"] == "none"
assert all(c["artifact"] != fake for s in found.values() for c in s.get("candidates") or [])
async def test_guesses_are_cached_per_seal_and_run_off_the_event_loop(client, warehouse, monkeypatch):
row = await finish(asked())
calls = counting(monkeypatch)
first = (await client.get(f"/api/runs/{row.id}/evidence")).json()
again = (await client.get(f"/api/runs/{row.id}/evidence")).json()
await client.get(f"/api/runs/{row.id}/evidence/audit")
assert len(calls) == 1, "同一次封存只猜一次,证据图和审计表共用"
assert again["legacy"] == first["legacy"]
assert calls[0] != threading.get_ident(), "猜测要放到线程里跑"
async def test_masked_columns_are_never_candidates_even_after_caching(client, warehouse, monkeypatch):
row = await finish(asked())
calls = counting(monkeypatch)
before = numbers((await client.get(f"/api/runs/{row.id}/evidence")).json()["legacy"]["fields"][0])
assert before["400.5"]["state"] == "candidate"
async with SessionLocal() as session:
src = await session.get(DataSource, warehouse)
src.options = {"mask_columns": ["gmv"]}
await session.commit()
after = numbers((await client.get(f"/api/runs/{row.id}/evidence")).json()["legacy"]["fields"][0])
assert len(calls) == 2, "遮罩变了,缓存的猜测作废"
assert after["400.5"]["state"] == "none" and after["200"]["state"] == "none"
assert all(c["locator"]["column"] != "gmv" for s in after.values() for c in s.get("candidates") or [])
async def test_runs_from_before_phase_two_are_followed_through_the_tool_snapshot(client, warehouse, monkeypatch):
async def no_snapshot(session):
return None
monkeypatch.setattr(runner_mod, "_agent_limits", no_snapshot)
row = await finish(asked())
assert row.status == "succeeded", row.error
[end] = await events(row.id, "tool.end", "fetch")
assert "query_artifact" not in end.data and end.data["artifact"]
body = (await client.get(f"/api/runs/{row.id}/evidence")).json()
assert body["mode"] == "legacy_text"
top = numbers(body["legacy"]["fields"][0])["400.5"]["candidates"][0]
assert top["ref"] == "Q1.r0.gmv" and top["kind"] == "cell"
snapshot = artifact_store.load(end.data["artifact"])
assert top["artifact"] in snapshot["result"]
async def test_answers_without_numbers_or_sources_stay_none(client, warehouse):
body = (await client.get(f"/api/runs/{(await finish(asked('东区最多'))).id}/evidence")).json()
assert body["mode"] == "none" and "legacy" not in body
plain = await finish(chain(node("start", "input"), node("out", "output", fields=[{"name": "r", "value": "共 7 单"}])))
body = (await client.get(f"/api/runs/{plain.id}/evidence")).json()
assert body["mode"] == "none"
# --------------------------------------------------------------------------
# legacy_contract:指标详情只从封存的口径卡里取
# --------------------------------------------------------------------------
KPI = "{'gmv': 45678.5, 'orders': 1234}"
STANDARD = [{"id": "gmv", "name": "销售额", "unit": "元", "decimals": 1, "expression": "vars.kpi.gmv"},
{"id": "orders", "name": "订单数", "unit": "单", "expression": "vars.kpi.orders"}]
STORY = "本周销售额 45678.5 元,订单 1234 单,另有 999 笔待核。"
def narrated():
return chain(node("start", "input"), node("fetch", "transform", mode="expression", expression=KPI, assign_to="kpi"),
node("caliber", "metrics", caliber="周报口径", caliber_version="v2", metrics=STANDARD),
node("narr", "transform", mode="template", template=STORY, assign_to="story"),
node("out", "output", fields=[{"name": "r", "value": "{{ vars.story }}"}],
contract={"metrics_from": ["caliber"], "narrative": "{{ vars.story }}"}))
async def test_legacy_matches_name_their_sealed_metric(client):
row = await finish(narrated())
assert row.status == "succeeded", row.error
[card] = [e for e in await events(row.id, "node.finished", "caliber")]
sealed_card = next(e["artifact"] for e in card.data["evidence"] if e["kind"] == "metric_set")
# 人为插一张同名指标的「口径卡」:值不一样,不能被拿来当出处
await artifact_store.put_json({"kind": "metric_set", "caliber": "别的口径", "caliber_version": "v9",
"metrics": [{"id": "gmv", "name": "销售额", "value": 1.0}]},
kind="metric_set", run_id=row.id, node_id="caliber")
body = (await client.get(f"/api/runs/{row.id}/evidence")).json()
assert body["mode"] == "legacy_contract"
gmv = next(m for m in body["legacy"]["matched"] if m["metric"] == "gmv")
[source] = gmv["sources"]
assert source["kind"] == "metric" and source["ref"] == "m:gmv" and source["artifact"] == sealed_card
assert source["value"] == 45678.5 and source["rendered"] == "45,678.5元" and source["name"] == "销售额"
assert (source["caliber"], source["version"], source["node_id"]) == ("周报口径", "v2", "caliber")
assert source["sealed"] is True
audit = (await client.get(f"/api/runs/{row.id}/evidence/audit")).json()
assert audit["mode"] == "legacy_contract"
guessed = next(g for g in audit["groups"] if g["key"] == "candidate")["rows"]
assert [r["text"] for r in guessed] == ["45678.5", "1234"]
assert guessed[0]["evidence"].startswith("m:gmv") and "按数值匹配" in guessed[0]["note"]
bare = next(g for g in audit["groups"] if g["key"] == "none")["rows"]
assert [r["text"] for r in bare] == ["999"]
# 句子一栏取成果字段里这个数前后的原文(对上的数 trace_numbers 不给 context,以前整列是空的)
for r in guessed + bare:
assert r["field"] == "r" and r["text"] in r["sentence"] and r["sentence"] in STORY, r
assert "销售额 45678.5 元" in guessed[0]["sentence"]
async def test_legacy_matches_without_a_sealed_card_have_no_source(client):
row = await finish(narrated())
async with SessionLocal() as session:
finished = (await session.execute(select(RunEvent).where(
RunEvent.run_id == row.id, RunEvent.type == "node.finished", RunEvent.node_id == "caliber"))).scalar_one()
finished.data = {k: v for k, v in finished.data.items() if k not in ("evidence", "artifact")}
await session.commit()
await reseal(row.id)
body = (await client.get(f"/api/runs/{row.id}/evidence")).json()
assert body["seal"]["ok"] is True
assert all(m["sources"] == [] for m in body["legacy"]["matched"])
+187
View File
@@ -0,0 +1,187 @@
"""旧运行的「猜测来源」:没有契约、没有报告文档的答案,按数值去已封存的证据里找可能的来源。
这不是证据:一期核对真实反例时,168 个数里有 131 个能在快照里找到相同的值,大量是 0、1、2
这种巧合。所以它只做降级展示,而且规矩很死:
- 只从调用方传进来的已封存条目里找(查询快照的单元格、口径卡的指标值),自己不读库——
人为插进工件表的行进不了这个函数
- 每个数字最多 3 个候选,按接近程度排,距离相同的按传入顺序;同样的输入永远同样的输出
- 片段化返回,有候选的数字 state 记 candidate,一个都没有的记 none
"""
from __future__ import annotations
import copy
import pytest
from app.engine.evidence import (
GUESS_NOTE,
GUESS_SCHEMA,
MAX_CANDIDATES,
cell_eid,
guess_sources,
make_eid,
)
Q_ART, M_ART, OTHER = "a" * 64, "d" * 64, "e" * 64
METRICS = {"kind": "metric_set", "artifact": M_ART, "node_id": "caliber", "content": {
"caliber": "周报口径", "caliber_version": "v2", "metrics": [
{"id": "gmv", "name": "销售额", "value": 45678.5, "unit": "元", "decimals": 1},
{"id": "wow", "name": "环比", "value": 8.7, "unit": "%", "decimals": 1, "format": "plain"},
{"id": "gone", "name": "缺值", "value": None, "unit": ""}]}}
QUERY = {"kind": "query", "artifact": Q_ART, "alias": "Q1", "node_id": "fetch", "tool": "db_query__warehouse",
"content": {
"columns": ["region", "gmv", "rate", "code"],
"rows": [["east", 45678.5, 0.087, "00123"], ["west", "12004.0", 0.02, "7"], ["north", 45678.2, None, "x"]],
"column_types": {"region": "text", "gmv": "number", "rate": "number", "code": "text"}}}
TEXT = "本周销售额 45,678.5 元,环比 8.7%,西区 12,004,比率 2%,另有 999 单,编号 123,日期 2026-09-01。"
def numbers(result):
return [s for s in result["segments"] if s["kind"] == "number"]
def refs(seg):
return [c.get("ref") for c in seg.get("candidates") or []]
def test_segments_tile_the_text_and_carry_candidates():
result = guess_sources(TEXT, [METRICS, QUERY])
assert result["schema"] == GUESS_SCHEMA and result["mode"] == "legacy_text" and result["note"] == GUESS_NOTE
assert "猜测" in GUESS_NOTE and "不能当证据" in GUESS_NOTE
assert "".join(s["text"] for s in result["segments"]) == TEXT
assert [s["id"] for s in result["segments"]] == [f"s{i}" for i in range(len(result["segments"]))]
for s in result["segments"]:
assert TEXT[s["span"][0]:s["span"][1]] == s["text"]
found = {s["text"]: (s["state"], refs(s)) for s in numbers(result)}
assert found == {
"45,678.5": ("candidate", ["m:gmv", "Q1.r0.gmv"]),
"8.7%": ("candidate", ["m:wow", "Q1.r0.rate"]),
"12,004": ("candidate", ["Q1.r1.gmv"]),
"2%": ("candidate", ["Q1.r1.rate"]),
"999": ("none", []),
"123": ("none", []), # 文本列里的 "00123" 是编号,不按数比
}
assert result["stats"] == {"numbers": 6, "guessed": 4, "unguessed": 2, "candidates": 6}
assert all(s["state"] == "neutral" for s in result["segments"] if s["kind"] == "text")
def test_candidate_shapes():
result = guess_sources("销售额 45,678.5", [METRICS, QUERY])
metric, cell = numbers(result)[0]["candidates"]
assert metric == {"kind": "metric", "ref": "m:gmv", "artifact": M_ART, "locator": {"metric": "gmv"},
"eid": make_eid("metric", M_ART, {"metric": "gmv"}), "value": 45678.5,
"rendered": "45,678.5元", "name": "销售额", "caliber": "周报口径", "version": "v2",
"node_id": "caliber", "diff": 0.0}
assert cell == {"kind": "cell", "ref": "Q1.r0.gmv", "alias": "Q1", "artifact": Q_ART,
"locator": {"row": 0, "column": "gmv"}, "eid": cell_eid(Q_ART, 0, "gmv"), "value": 45678.5,
"rendered": "45,678.5", "node_id": "fetch", "tool": "db_query__warehouse", "diff": 0.0}
def test_closer_values_come_first_and_ties_keep_the_given_order():
result = guess_sources("约 45,678", [QUERY, METRICS])
[seg] = numbers(result)
# 写的是整数,容差 0.5:45678.2 比 45678.5 更近;两个 45678.5 按传入顺序(查询在前)
assert refs(seg) == ["Q1.r2.gmv", "Q1.r0.gmv", "m:gmv"]
assert [c["diff"] for c in seg["candidates"]] == pytest.approx([0.2, 0.5, 0.5])
def test_at_most_three_candidates_per_number():
zeros = {"kind": "query", "artifact": OTHER, "alias": "Q2", "content": {
"columns": ["a", "b"], "rows": [[0, 0], [0, 0], [0, 0]]}}
[seg] = numbers(guess_sources("退款 0 单", [zeros]))
assert MAX_CANDIDATES == 3 and len(seg["candidates"]) == 3
assert refs(seg) == ["Q2.r0.a", "Q2.r0.b", "Q2.r1.a"]
[seg] = numbers(guess_sources("退款 0 单", [zeros], limit=1))
assert refs(seg) == ["Q2.r0.a"]
def test_only_the_entries_passed_in_are_searched():
"""函数自己不读库:一份没传进来的快照里就算有这个值,也不会成为候选。"""
result = guess_sources(TEXT, [QUERY])
assert refs(numbers(result)[0]) == ["Q1.r0.gmv"]
assert all(not r.startswith("m:") for s in numbers(result) for r in refs(s))
# 不认识的种类、坏掉的内容一律跳过,不崩
junk = [{"kind": "tool", "artifact": OTHER, "content": {"columns": ["x"], "rows": [[45678.5]]}},
{"kind": "retrieval", "artifact": OTHER, "content": {"hits": [{"content": "45678.5"}]}},
{"kind": "query", "artifact": OTHER, "content": None},
{"kind": "query", "artifact": OTHER, "content": {"columns": ["x"], "rows": "oops"}},
{"kind": "metric_set", "content": {"metrics": "oops"}},
"not a dict"]
result = guess_sources(TEXT, junk)
assert all(s["state"] == "none" for s in numbers(result))
def test_same_input_same_output_and_inputs_untouched():
entries = [METRICS, QUERY]
before = copy.deepcopy(entries)
first = guess_sources(TEXT, entries)
assert guess_sources(TEXT, entries) == first
assert entries == before
def test_text_without_numbers_and_empty_text():
assert guess_sources("", [QUERY])["segments"] == []
result = guess_sources("没有数字,只有 v2 这种标识符。", [QUERY])
assert numbers(result) == [] and result["stats"]["numbers"] == 0
# --------------------------------------------------------------------------
# 遮罩的列:原始值一律不露,候选里也不行
# --------------------------------------------------------------------------
SALARY = {"kind": "query", "artifact": OTHER, "alias": "Q2", "content": {
"columns": ["region", "salary", "headcount"],
"rows": [["east", 8123.45, 12], ["west", 9001.0, 8]],
"column_types": {"region": "text", "salary": "number", "headcount": "number"},
"mask_columns": ["salary"]}}
@pytest.mark.parametrize("text", ["东区人均 8,123.45 元", "东区人均 8,123 元", "西区人均 9,001 元"])
def test_columns_masked_at_query_time_are_never_candidates(text):
"""快照记下的遮罩(查询当时数据源的 mask_columns):按容差对得上也不给候选——候选的 value、
rendered、diff 都会把原值带出去,写成 8,123 的也会露出 8,123.45。"""
result = guess_sources(text, [SALARY])
[seg] = numbers(result)
assert seg["state"] == "none" and "candidates" not in seg
assert result["stats"]["candidates"] == 0
# 没遮的列照样能猜
[seg] = numbers(guess_sources("东区 12 人", [SALARY]))
assert refs(seg) == ["Q2.r0.headcount"]
def test_columns_masked_now_are_never_candidates():
"""数据源现在设的遮罩(调用方按 artifact 交进来,不分大小写):事后加的遮罩同样生效。"""
[seg] = numbers(guess_sources("销售额 45,678.5", [QUERY], masked={Q_ART: ["GMV"]}))
assert refs(seg) == []
[seg] = numbers(guess_sources("销售额 45,678.5", [QUERY, METRICS], masked={Q_ART: ["gmv"]}))
assert refs(seg) == ["m:gmv"] # 口径卡的指标是公开口径,不受列遮罩影响
[seg] = numbers(guess_sources("比率 2%", [QUERY], masked={OTHER: ["rate"]}))
assert refs(seg) == ["Q1.r1.rate"] # 遮罩只管它那一份快照
[seg] = numbers(guess_sources("比率 2%", [QUERY], masked={Q_ART: "code, Rate"}))
assert refs(seg) == [] # 写成「a, b」的遮罩也认:宁可多遮
[seg] = numbers(guess_sources("比率 2%", [QUERY], masked="rate"))
assert refs(seg) == ["Q1.r1.rate"] # 形状不对的整个参数不当真,也不崩
def test_a_value_that_cannot_be_rendered_is_skipped_for_the_next_one():
"""渲染不出来的格(非零却显示成 0)不当候选,名额让给下一个:和逐格先渲染的结果一样。"""
tiny = {"kind": "query", "artifact": OTHER, "alias": "Q2", "content": {
"columns": ["a"], "rows": [[1e-12], [0.3], [0.2], [0.4]]}}
# 写的是整数 0,容差 0.5:最近的 1e-12 显示不出来,第四近的 0.4 顶上
[seg] = numbers(guess_sources("退款 0 单", [tiny]))
assert refs(seg) == ["Q2.r2.a", "Q2.r1.a", "Q2.r3.a"]
assert all(c["rendered"] != "—" for c in seg["candidates"])
def test_big_snapshots_are_searched_quickly():
import time
cols = [f"c{i}" for i in range(12)]
rows = [[(r * 12 + c) * 1.25 for c in range(12)] for r in range(5000)]
sealed = [{"kind": "query", "artifact": f"{i:064x}", "alias": f"Q{i + 1}",
"content": {"columns": cols, "rows": rows}} for i in range(4)]
started = time.perf_counter()
result = guess_sources("数 " + ",".join(str(v * 1.25) for v in range(0, 60000, 997)), sealed)
assert time.perf_counter() - started < 1.0
assert result["stats"]["guessed"] == result["stats"]["numbers"] > 0
+267
View File
@@ -0,0 +1,267 @@
"""引文:[[q:K1|原话]] 必须在那次检索命中的片段里逐字出现。
知识库的结论以前只能写成 [[see:K1]]:读者知道依据是哪次检索,看不到是哪一句。引文把
原话写进报告,系统拿检索快照(取回时复验哈希)逐字核对:
- 空白归一化后逐字出现才算有出处;改一个字就是没有出处(state none,issue unresolved_ref)
- 正文里显示引文本身;目录里根本没有这次检索时显示占位,和别的解析不了的引用一样
- eid 带上命中片段的定位:第几条命中、原文里的起止
- 出口复核重新取快照、重新比对,不信文档里记的结论
"""
from __future__ import annotations
import asyncio
import random
import pytest
from langchain_core.messages import AIMessage
from sqlalchemy import select
from app.core import artifact_store
from app.db.base import SessionLocal
from app.db.models import Run, RunEvent
from app.engine.evidence import (
LATER_REASON,
StreamRenderer,
build_catalog,
catalog_prompt,
compose_doc,
iter_segments,
make_eid,
render_markers,
verify_doc,
)
from app.engine.runner import run_manager
K_ART = "c" * 64
HITS = [
{"chunk_id": "ch-1", "document_id": "doc-1", "title": "售后月报", "ordinal": 3,
"content": "九月退款主要集中在\n东区,原因是物流延误。退款率 2.35%,高于上月。"},
{"chunk_id": "ch-2", "document_id": "doc-2", "title": "Ops notes", "ordinal": 0,
"content": "Net revenue\nrose after the promotion ended. 物流延误已经解决。"},
]
SNAPSHOT = {"query": "退款", "collection": "kb", "hits": HITS}
def loader_of(content):
def load(artifact):
return content if artifact == K_ART else None
return load
LOAD = loader_of(SNAPSHOT)
@pytest.fixture
def catalog():
return build_catalog(nodes={}, ledger=[{"kind": "retrieval", "node_id": "kb", "exec": 1, "artifact": K_ART,
"source": "kb", "rows": len(HITS)}])
def quotes(doc):
return [s for s in iter_segments(doc) if s["kind"] == "quote"]
def test_verbatim_quote_resolves_with_its_location(catalog):
doc = compose_doc("售后说「[[q:K1|退款主要集中在东区]]」。[[see:K1]]", catalog, loader=LOAD)
[seg] = quotes(doc)
assert doc["markdown"] == "售后说「退款主要集中在东区」。"
assert seg["text"] == "退款主要集中在东区" and seg["state"] == "deterministic"
cite = seg["cite"]
assert cite["kind"] == "quote" and cite["role"] == "quote" and cite["alias"] == "K1"
loc = cite["locator"]
assert loc["hit"] == 0 and HITS[0]["content"][loc["start"]:loc["end"]] == "退款主要集中在\n东区"
assert cite["eid"] == make_eid("quote", K_ART, loc)
assert cite["source"] == {"artifact": K_ART, "document": "doc-1", "chunk": "ch-1", "title": "售后月报",
"ordinal": 3}
assert doc["violations"] == [] and doc["stats"]["quotes"] == 1
assert doc["blocks"][0]["units"][0]["cites"] == ["K1"]
def test_one_changed_character_is_not_a_quote(catalog):
doc = compose_doc("售后说「[[q:K1|退款主要集中在西区]]」。", catalog, loader=LOAD)
[seg] = quotes(doc)
# 正文照样显示引文本身,只是标成没有出处
assert seg["text"] == "退款主要集中在西区" and seg["state"] == "none" and seg["issue"] == "unresolved_ref"
assert "逐字" in seg["cite"]["reason"]
assert [v["code"] for v in doc["violations"]] == ["unresolved_ref"]
assert doc["stats"]["quotes"] == 0
@pytest.mark.parametrize("quote", [
"Net revenue rose after the promotion ended.",
"Net revenue rose",
"退款率 2.35%,高于上月",
"退款主要集中在 东区",
])
def test_whitespace_is_normalised(catalog, quote):
doc = compose_doc(f"原话:[[q:K1|{quote}]]", catalog, loader=LOAD)
[seg] = quotes(doc)
assert seg["state"] == "deterministic", seg["cite"].get("reason")
loc = seg["cite"]["locator"]
assert " ".join(HITS[loc["hit"]]["content"][loc["start"]:loc["end"]].split()).replace(" ", "") \
== quote.replace(" ", "")
@pytest.mark.parametrize("quote", ["Netrevenue rose", "net revenue rose", "退款主要集中在东区。九月"])
def test_normalising_is_not_fuzzy(catalog, quote):
doc = compose_doc(f"原话:[[q:K1|{quote}]]", catalog, loader=LOAD)
assert quotes(doc)[0]["state"] == "none"
def test_numbers_inside_a_verified_quote_are_not_bare(catalog):
doc = compose_doc("[[q:K1|退款率 2.35%,高于上月]]", catalog, loader=LOAD)
assert doc["violations"] == [] and doc["stats"]["uncited_numbers"] == 0
@pytest.mark.parametrize("text, words", [
("[[q:K9|退款主要集中在东区]]", "目录里没有 K9"),
("[[q:K1|]]", "引文是空的"),
("[[q:K1|东区]]", "太短"),
])
def test_quotes_that_cannot_be_checked(catalog, text, words):
doc = compose_doc(text, catalog, loader=LOAD)
[seg] = quotes(doc)
assert seg["state"] == "none" and words in seg["cite"]["reason"], seg["cite"]["reason"]
def test_missing_retrieval_shows_a_placeholder(catalog):
doc = compose_doc("[[q:K9|退款主要集中在东区]]。", catalog, loader=LOAD)
assert doc["markdown"] == "⟦?q:K9|退款主要集中在东区⟧。"
def test_tampered_snapshot_is_not_evidence(catalog):
def tampered(artifact):
raise ValueError("hash mismatch")
doc = compose_doc("[[q:K1|退款主要集中在东区]]", catalog, loader=tampered)
[seg] = quotes(doc)
assert seg["state"] == "none" and "哈希" in seg["cite"]["reason"]
def test_exit_check_rereads_the_snapshot(catalog):
doc = compose_doc("[[q:K1|物流延误已经解决]]。", catalog, loader=LOAD)
assert doc["violations"] == [] and quotes(doc)[0]["cite"]["locator"]["hit"] == 1
changed = {**SNAPSHOT, "hits": [HITS[0], {**HITS[1], "content": "物流延误仍在持续。"}]}
checked = verify_doc(doc, catalog, loader=loader_of(changed))
assert [v["code"] for v in checked["violations"]] == ["unresolved_ref"]
assert verify_doc(doc, catalog, loader=LOAD)["ok"] is True
def phase_two(doc):
"""把文档改回二期组装的样子:那时 q / t / c 一律判为解析不了,正文是占位,原因是 LATER_REASON。
占位的字和现在「目录里没有 K9」时一样,只有记下的原因不同。"""
for seg in iter_segments(doc):
if seg.get("issue") == "unresolved_ref" and seg["ref"][:2] in ("q:", "t:", "c:"):
seg["cite"]["reason"] = LATER_REASON
return doc
def test_quotes_from_before_the_upgrade_are_rechecked_the_old_way(catalog):
"""二期的报告文档(升级前组装,停在报告和出口之间)升级后被复核:原话今天查得到也不能算
「渲染对不上」——那会被出口当成完整性问题记缺口。按当时的规矩:占位,解析不了。"""
doc = phase_two(compose_doc("售后说[[q:K1|退款主要集中在东区]]。", {}, loader=LOAD))
[seg] = quotes(doc)
assert seg["text"] == "⟦?q:K1|退款主要集中在东区⟧"
checked = verify_doc(doc, catalog, loader=LOAD)
assert [v["code"] for v in checked["violations"]] == ["unresolved_ref"]
assert LATER_REASON in checked["violations"][0]["message"]
# 只认占位:正文被改成了引文本身、又自称是二期的,照样按现在的规矩查
seg["text"] = "退款主要集中在东区"
doc["markdown"] = "售后说退款主要集中在东区。"
tail = [x for x in iter_segments(doc)][-1]
seg["span"] = [3, 3 + len(seg["text"])]
tail["span"] = [seg["span"][1], seg["span"][1] + len(tail["text"])]
[unit] = [u for b in doc["blocks"] for u in b["units"]]
unit["span"] = [0, tail["span"][1]]
assert [v["code"] for v in verify_doc(doc, catalog, loader=LOAD)["violations"]] == ["state_mismatch"]
def test_entity_markers_from_before_the_upgrade_are_rechecked_the_old_way():
ledger = [{"kind": "query", "node_id": "f", "exec": 1, "artifact": "a" * 64, "tool": "db_query__w", "source": "w",
"columns": ["gmv"], "rows": 1, "truncated": False, "schema_artifact": "5" * 64, "tables": ["orders"]},
{"kind": "schema", "node_id": "f", "exec": 1, "artifact": "5" * 64, "source": "w",
"tables": {"orders": ["id", "gmv"]}}]
doc = phase_two(compose_doc("看 [[t:orders]] 和 [[c:orders.gmv]]。", {}))
assert [s["text"] for s in iter_segments(doc) if s.get("ref")] == ["⟦?t:orders⟧", "⟦?c:orders.gmv⟧"]
checked = verify_doc(doc, build_catalog(nodes={}, ledger=ledger))
assert [v["code"] for v in checked["violations"]] == ["unresolved_ref", "unresolved_ref"]
def test_streaming_equals_full_render(catalog):
text = "开头 [[q:K1|退款主要集中在东区]],又说 [[q:K1|退款主要集中在西区]];[[q:K9|无此检索]] 完。[[see:K1]]"
whole = render_markers(text, catalog, loader=LOAD)
assert whole == "开头 退款主要集中在东区,又说 退款主要集中在西区;⟦?q:K9|无此检索⟧ 完。"
rng = random.Random(11)
for _ in range(200):
renderer = StreamRenderer(catalog, loader=LOAD)
out, i = [], 0
while i < len(text):
step = rng.randint(1, 7)
out.append(renderer.feed(text[i:i + step]))
i += step
out.append(renderer.flush())
assert "".join(out) == whole
def test_catalog_prompt_shows_hits_to_quote_from(catalog):
prompt = catalog_prompt(catalog, loader=LOAD)
assert "[[q:K1|" in prompt and "逐字" in prompt and "[[see:K1]]" in prompt
assert "售后月报" in prompt and "退款主要集中在" in prompt and "Ops notes" in prompt
# --------------------------------------------------------------------------
# 跑一次:检索 → 报告撰写,引错了原话会被要求重写
# --------------------------------------------------------------------------
@pytest.fixture
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
async def test_report_node_checks_quotes_against_the_retrieval(engine_up, monkeypatch):
from app.memory import kb
from app.providers import mock_model
async def fake_search(session, **kw):
return [dict(HITS[0], score=0.9)]
monkeypatch.setattr(kb, "search", fake_search)
replies = ["售后说「[[q:K1|退款主要集中在西区]]」。[[see:K1]]", "售后说「[[q:K1|退款主要集中在东区]]」。[[see:K1]]"]
seen: list = []
def decide(self, messages):
seen.append(list(messages))
return AIMessage(content=replies[min(len(seen), len(replies)) - 1])
monkeypatch.setattr(mock_model.MockChatModel, "_decide", decide)
nodes = [{"id": nid, "type": t, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": cfg}} for nid, t, cfg in [
("start", "input", {}), ("docs", "retrieve", {"query": "退款", "collection": "kb"}),
("write", "report", {"instructions": "引用售后的原话"}),
("out", "output", {"fields": [{"name": "r", "value": "{{ nodes.write.text }}"}]})]]
graph = {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
run = await run_manager.start(graph=graph, input_payload={})
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run.id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
break
assert row.status == "succeeded", row.error
assert row.output["r"] == "售后说「退款主要集中在东区」。"
first = "\n".join(str(m.content) for m in seen[0])
assert "[[q:K1|" in first and "退款主要集中在" in first and "售后月报" in first
assert len(seen) == 2 and "找不到这句原话" in str(seen[1][-1].content)
async with SessionLocal() as session:
[checked] = (await session.execute(select(RunEvent).where(
RunEvent.run_id == run.id, RunEvent.type == "report.checked"))).scalars().all()
assert checked.data["ok"] is True and checked.data["repairs"] == 1 and checked.data["stats"]["quotes"] == 1
doc = artifact_store.load(checked.data["doc_artifact"])
[seg] = quotes(doc)
retrieval = doc["catalog"]["K1"]["artifact"]
assert seg["cite"]["eid"] == make_eid("quote", retrieval, seg["cite"]["locator"])
assert seg["cite"]["source"]["chunk"] == "ch-1"
+215
View File
@@ -0,0 +1,215 @@
"""证据门禁 G1–G5 的收尾:验收留下的小问题和 B 段的交叉请求。
- G1 和 G2 对开了 cite_fields 的 agent 字段看法一致:逐格核对过的字段是数据,不是模型写的话;
- 选项里只有标了 handoff 的那一项才是「交给 Copilot」,节点 id 恰好叫 copilot 的报告照常应用;
- 自动修复接口读不懂图时也带 handoff 键;
- 已发布级别的 G2 对每个出口都给警告(用户拍板:G 规则受管挡、已发布警告,范围一样);
- G4、G5 挡住的地方,validate 那条同义的提示不再重复;
- 删掉开了 cite_fields 的 agent 的 output_schema 算降低要求;
- Copilot 自查带上全图默认,和真正的门禁口径一致;
- 受管契约的 claims 写 on_uncited: ignore 给警告,并能一键改成 degrade。
"""
from __future__ import annotations
import copy
import json
import pytest
from httpx import ASGITransport, AsyncClient
from app.engine.autofix import HANDOFF, apply_fixes, forbidden_changes
from app.engine.governance import publish_issues
from app.engine.schema import GraphSpec, validate_graph
from app.main import app
from tests.test_autofix_governance import (
SCHEMA, STRICT, codes, compute_feeding, config, create, errors, insert, node, plan, script, traceable,
with_story,
)
def issues(graph: dict, level: str = "governed") -> list:
return publish_issues(GraphSpec.model_validate(graph), level=level)
def cited_agent(graph: dict, *, cite: bool = True) -> dict:
ask = node("ask", "agent", "查数助手", prompt="查一下", tools=["db_query__shop"], approval="dangerous",
output_schema=SCHEMA, cite_fields=cite, assign_to="facts")
return insert(graph, ask, before="done", after="write")
# ---------------------------------------------------------------- G1 / G2 一致
@pytest.mark.parametrize("ref", ["{{ vars.facts.gmv }}", "{{ nodes.ask.data.gmv }}"])
def test_a_cited_agent_field_in_a_contract_exit_is_data(ref):
graph = cited_agent(traceable())
config(graph, "done")["fields"].append({"name": "GMV", "value": ref})
found = codes(errors(graph))
assert "governed.exit_text_source" not in found and "governed.text_bypass" not in found, found
assert not [f for f in plan(graph) if f.startswith("governed.exit_text_source")]
def test_an_uncited_agent_field_in_a_contract_exit_is_still_text():
graph = cited_agent(traceable(), cite=False)
config(graph, "done")["fields"].append({"name": "GMV", "value": "{{ vars.facts.gmv }}"})
assert "governed.exit_text_source" in codes(errors(graph))
# ---------------------------------------------------------------- handoff 只认标记
def renamed(graph: dict, old: str, new: str) -> dict:
text = json.dumps(graph, ensure_ascii=False)
return json.loads(text.replace(f'"{old}"', f'"{new}"').replace(f"nodes.{old}.", f"nodes.{new}."))
def test_the_handoff_option_is_marked():
fix = plan(compute_feeding())["governed.caliber_compute_input:calc"]
marked = [o for o in fix["options"] if o.get("handoff")]
assert [o["value"] for o in marked] == [HANDOFF]
def test_a_report_whose_id_is_copilot_is_applied_not_handed_off():
graph = renamed(with_story(traceable()), "write", HANDOFF)
fid = "governed.exit_text_source:done"
assert [o["value"] for o in plan(graph)[fid]["options"]] == [HANDOFF]
out = apply_fixes(graph, [fid], {fid: HANDOFF}, level="governed")
assert out["applied"] == [fid] and out["handoff"] == [], out
assert config(out["graph"], "done")["fields"][2]["value"] == "{{ nodes.copilot.text }}"
# ---------------------------------------------------------------- 接口的早退回包
@pytest.fixture
async def client():
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
async def test_autofix_on_an_unreadable_graph_still_has_handoff(client):
wf = await create(client, traceable())
body = (await client.post(f"/api/workflows/{wf}/autofix", json={
"level": "governed", "graph": {"nodes": "坏的"}, "apply": ["x"]})).json()
assert body["ok"] is False and body["handoff"] == []
# ---------------------------------------------------------------- G2 已发布级别的范围
def test_published_g2_warns_on_every_exit():
graph = {"nodes": [node("start", "input", "问题"), node("chat", "llm", "回答", prompt="{{ input.q }}"),
node("out", "output", "回答出口", fields=[{"name": "回答", "value": "{{ nodes.chat.text }}"}])],
"edges": [{"source": "start", "target": "chat"}, {"source": "chat", "target": "out"}]}
published = [i for i in issues(graph, "published") if i.code == "governed.text_bypass"]
assert [i.level for i in published] == ["warning"] and published[0].node_id == "chat"
governed = [i for i in issues(graph, "governed") if i.code == "governed.text_bypass"]
assert [i.level for i in governed] == ["error"]
# ---------------------------------------------------------------- G4 / G5 不重复
def test_the_g4_error_is_not_repeated_by_a_validate_warning():
graph = compute_feeding()
spec = GraphSpec.model_validate(graph)
# validate 自己照样提示(画布上没发布也看得到),带编号
assert "evidence.caliber_compute_input" in {i.code for i in validate_graph(spec).issues}
for level in ("governed", "published"):
found = issues(graph, level)
assert "governed.caliber_compute_input" in {i.code for i in found}
assert "evidence.caliber_compute_input" not in {i.code for i in found}, level
def test_the_g5_error_is_not_repeated_by_a_validate_warning():
graph = insert(traceable(), node("ask", "agent", "查数助手", prompt="查", tools=["db_query__shop"],
approval="dangerous", output_schema=SCHEMA, assign_to="facts"),
before="card", after="fetch")
config(graph, "card")["metrics"].append({"id": "n", "expression": "vars.facts.gmv"})
spec = GraphSpec.model_validate(graph)
assert "evidence.cite_fields_off" in {i.code for i in validate_graph(spec).issues}
found = issues(graph, "governed")
assert "governed.caliber_agent_cite_fields" in {i.code for i in found}
assert "evidence.cite_fields_off" not in {i.code for i in found}
def test_the_validate_warning_stays_where_no_gate_rule_covers_it():
graph = cited_agent(traceable(), cite=False) # 不给口径卡供数:G5 不管,validate 的提示留着
assert "evidence.cite_fields_off" in {i.code for i in issues(graph, "governed")}
# ---------------------------------------------------------------- 删 output_schema 算降低要求
def test_removing_the_schema_of_a_cited_agent_is_refused():
before = cited_agent(traceable())
after = copy.deepcopy(before)
config(after, "ask").pop("output_schema")
[reason] = forbidden_changes(before, after)
assert "「查数助手」" in reason and "output_schema" in reason
config(after, "ask")["output_schema"] = {}
assert forbidden_changes(before, after)
# ---------------------------------------------------------------- Copilot 自查带全图默认
async def test_copilot_self_check_sees_graph_defaults(client, monkeypatch):
graph = compute_feeding()
config(graph, "write").clear()
config(graph, "write")["instructions"] = "写周报"
graph["defaults"] = dict(STRICT)
assert "governed.report_policy" not in codes(errors(graph)) # 真正的门禁认全图默认
model = script(monkeypatch, [])
wf = await create(client, graph)
await client.post(f"/api/workflows/{wf}/autofix", json={"level": "governed", "apply": [], "assist": True})
human = next(text for role, text in model.calls[0] if role == "human")
assert "「计算」" in human and "要写明" not in human
async def test_copilot_self_check_still_sees_report_policy_without_defaults(client, monkeypatch):
graph = compute_feeding()
config(graph, "write").clear()
config(graph, "write")["instructions"] = "写周报"
model = script(monkeypatch, [])
wf = await create(client, graph)
await client.post(f"/api/workflows/{wf}/autofix", json={"level": "governed", "apply": [], "assist": True})
human = next(text for role, text in model.calls[0] if role == "human")
assert "要写明" in human
def test_the_assist_rules_forbid_marking_sources_and_loosening_reports():
from app.api.copilot import _ASSIST_RULES
assert "evidence_role" in _ASSIST_RULES and "source" in _ASSIST_RULES
for key in ("numbers", "on_violation", "claims", "cite_fields", "output_schema"):
assert key in _ASSIST_RULES, key
# ---------------------------------------------------------------- 契约 claims 写 ignore
def ignoring() -> dict:
return traceable(claims={"policy": "require_citation", "on_uncited": "ignore"})
def test_contract_ignore_warns_under_governed_only():
found = [i for i in issues(ignoring()) if i.code == "contract.claims_ignored"]
assert [(i.level, i.node_id, i.field) for i in found] == [("warning", "done", "contract.claims")]
assert not [i for i in issues(ignoring(), "published") if i.code == "contract.claims_ignored"]
assert not [i for i in issues(traceable(claims={"policy": "require_citation", "on_uncited": "withhold"}))
if i.code == "contract.claims_ignored"]
def test_contract_ignore_is_fixed_to_degrade():
fid = "contract.claims_ignored:done"
fix = plan(ignoring())[fid]
assert fix["kind"] == "auto" and fix["preview"] == {"field": "contract.claims.on_uncited",
"before": "ignore", "after": "degrade"}
out = apply_fixes(ignoring(), [fid], level="governed")
assert out["applied"] == [fid], out["rejected"]
assert config(out["graph"], "done")["contract"]["claims"] == {"policy": "require_citation",
"on_uncited": "degrade"}
assert "contract.claims_ignored" not in {i["code"] for i in out["remaining"]}
# 反方向(degrade 改回 ignore)是放宽,要拒
assert forbidden_changes(out["graph"], ignoring())
@@ -0,0 +1,475 @@
"""受管出具的证据门禁 G1–G5(governance.lint_for_publish)。
报告里的每个数都点得开出处,前提是这张图在结构上走得通:出口核对报告撰写节点的文档(G1)、
模型写的话不绕过报告直接进出口(G2)、报告节点按最严的规则自查(G3)、口径卡的输入不是沙箱里
算出来的(G4)、给口径卡供数的 agent 交的是逐格核对过的字段(G5)。
受管级别是硬错误、挡住发布;已发布级别只给警告。判定只看图和配置,不看运行结果;已经发布的
版本照常运行,下次发布时按新规则查。
"""
from __future__ import annotations
import asyncio
import copy
import pytest
from httpx import ASGITransport, AsyncClient
from langchain_core.messages import AIMessage
from app.db.base import SessionLocal
from app.db.models import Run, Workflow, WorkflowVersion
from app.engine.governance import lint_for_publish, publish_issues
from app.engine.runner import run_manager
from app.engine.schema import CLAIMS_LATER, GraphSpec, validate_graph
from app.main import app
from app.seed import TEMPLATES
#: 报告撰写节点在受管模板里要写明的三项
STRICT = {"numbers": "strict", "on_violation": "fail", "claims": "require_citation"}
G_CODES = {
"governed.report_from_required", "governed.exit_text_source", "governed.text_bypass",
"governed.report_policy", "governed.caliber_compute_input", "governed.caliber_agent_schema",
"governed.caliber_agent_cite_fields", "governed.caliber_model_input",
}
def node(nid, ntype, label=None, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": label or nid, "config": config}}
def traceable(*, report=None, fields=None, **contract_overrides) -> dict:
"""input → 查库 → 口径卡 → 报告撰写 → 出具(引用模式契约):受管要求全部满足,包括 G1–G5。"""
contract = {"report_from": "write", "metrics_from": ["card"], "required": ["gmv"], "strict": True}
contract.update(contract_overrides)
nodes = [
node("start", "input", "周期", fields=[{"name": "week", "default": "2026-W37"}]),
node("fetch", "tool", "取数", tool="db_query__shop",
args={"sql": "SELECT SUM(amount) AS gmv, COUNT(*) AS orders FROM orders"}),
node("card", "metrics", "周报口径卡", caliber="周报口径", metrics=[
{"id": "gmv", "name": "销售额", "unit": "元", "expression": "cell(nodes.fetch, 0, 'gmv')"},
{"id": "orders", "name": "订单数", "unit": "单", "expression": "cell(nodes.fetch, 0, 'orders')"}]),
node("write", "report", "报告撰写", instructions="写周报", **(STRICT if report is None else report)),
node("done", "output", "出具",
fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}] if fields is None else fields,
contract={k: v for k, v in contract.items() if v is not None}),
]
edges = [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]
return {"nodes": nodes, "edges": edges}
def find(graph: dict, nid: str) -> dict:
return next(n for n in graph["nodes"] if n["id"] == nid)
def config(graph: dict, nid: str) -> dict:
return find(graph, nid)["data"]["config"]
def insert(graph: dict, new: dict, *, before: str, after: str) -> dict:
"""把 new 接在 after 和 before 之间(原来的直连保留)。"""
graph["nodes"].insert(len(graph["nodes"]) - 1, new)
graph["edges"] += [{"source": after, "target": new["id"]}, {"source": new["id"], "target": before}]
return graph
def lint(graph: dict, level: str = "governed") -> list:
return lint_for_publish(GraphSpec.model_validate(graph), level=level).issues
def g_issues(graph: dict, level: str = "governed") -> list:
return [i for i in lint(graph, level) if i.code in G_CODES]
def by_code(graph: dict, code: str, level: str = "governed") -> list:
return [i for i in lint(graph, level) if i.code == code]
@pytest.fixture
async def client():
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
async def publish(client, graph: dict, level: str = "governed") -> dict:
wf = (await client.post("/api/workflows", json={"name": "证据门禁-受管", "graph": graph})).json()
body = (await client.post(f"/api/workflows/{wf['id']}/publish", json={"level": level})).json()
await client.delete(f"/api/workflows/{wf['id']}")
return body
def test_the_baseline_passes_every_rule():
for level in ("governed", "published"):
issues = publish_issues(GraphSpec.model_validate(traceable()), level=level)
assert [i for i in issues if i.level == "error"] == [], [i.message for i in issues]
assert not {i.code for i in issues} & G_CODES
def test_every_rule_carries_a_code_and_points_at_a_node():
graph = traceable(report={}, narrative="{{ nodes.write.text }}", report_from=None)
graph = insert(graph, node("calc", "code", "计算", code="print(1)", assign_to="calc"), before="card",
after="fetch")
config(graph, "card")["metrics"].append({"id": "extra", "expression": "vars.calc"})
issues = g_issues(graph)
assert {i.code for i in issues} >= {"governed.report_from_required", "governed.report_policy",
"governed.caliber_compute_input"}
assert all(i.node_id and i.field for i in issues), [(i.code, i.node_id, i.field) for i in issues]
# --------------------------------------------------------------------------
# G1:带契约的出口核对报告撰写节点的文档,成果字段的文字只能来自报告撰写节点
# --------------------------------------------------------------------------
async def test_a_governed_contract_without_report_from_is_refused(client):
# 旧的叙述模式:数字按数值回头匹配,出处可能不唯一
graph = traceable(report_from=None, narrative="{{ nodes.write.text }}")
[issue] = by_code(graph, "governed.report_from_required")
assert issue.level == "error" and issue.node_id == "done" and issue.field == "contract.report_from"
assert "report_from" in issue.message and "narrative" in issue.message
body = await publish(client, graph)
assert body["ok"] is False
assert "governed.report_from_required" in {i["code"] for i in body["issues"] if i["level"] == "error"}
# 两样都没写的,照旧由 contract.report_from_missing 报,不重复报一条
bare = traceable(report_from=None)
assert by_code(bare, "governed.report_from_required") == []
assert (await publish(client, bare))["ok"] is False
@pytest.mark.parametrize("producer, label", [
(node("narrate", "llm", "叙述", prompt="写周报", assign_to="story"), "模型调用"),
(node("shape", "transform", "整形", mode="template", template="{{ nodes.card.text }}", assign_to="story"),
"数据整形"),
(node("calc", "code", "计算", code="print('x')", assign_to="story"), "沙箱代码"),
])
@pytest.mark.parametrize("ref", ["{{ nodes.%s.text }}", "{{ vars.story }}"])
def test_an_exit_field_taking_text_from_a_non_report_node_is_named(producer, label, ref):
graph = insert(traceable(), producer, before="done", after="write")
config(graph, "done")["fields"].append({"name": "摘要", "value": "本周:" + ref % producer["id"]
if "%s" in ref else "本周:" + ref})
[issue] = by_code(graph, "governed.exit_text_source")
assert issue.level == "error" and issue.node_id == "done" and issue.field == "fields[1].value"
assert "「摘要」" in issue.message and f"「{producer['data']['label']}」({label})" in issue.message
assert [i.level for i in by_code(graph, "governed.exit_text_source", "published")] == ["warning"]
@pytest.mark.parametrize("value", [
"{{ nodes.write.text }}", "{{ vars.report }}", "{{ nodes.write.stats | json }}", "{{ nodes.card.text }}",
"{{ input.week }}", "{{ vars.week }}", "{{ nodes.fetch }}", "{{ usage.total_tokens }}", "由系统生成,仅供内部参考",
"{{ last_message }}",
])
def test_deterministic_sources_and_report_text_are_fine(value):
graph = traceable()
config(graph, "write")["assign_to"] = "report"
config(graph, "done")["fields"].append({"name": "附", "value": value})
assert g_issues(graph) == [], [i.message for i in g_issues(graph)]
def test_a_source_code_node_is_a_data_source_but_a_compute_one_is_not():
graph = insert(traceable(), node("raw", "code", "取原始数据", code="print(1)", evidence_role="source"),
before="done", after="fetch")
config(graph, "done")["fields"].append({"name": "原始数据", "value": "{{ nodes.raw }}"})
assert by_code(graph, "governed.exit_text_source") == []
config(graph, "raw")["evidence_role"] = "compute"
assert len(by_code(graph, "governed.exit_text_source")) == 1
def test_an_exit_without_fields_hands_over_the_last_message():
# 没配字段就把最后一条消息当成果:最后一条是报告的,没问题;是模型调用写的,就绕过了报告
graph = traceable(fields=[])
assert g_issues(graph) == []
graph = insert(graph, node("polish", "llm", "润色", prompt="{{ nodes.write.text }}"), before="done",
after="write")
graph["edges"] = [e for e in graph["edges"] if (e["source"], e["target"]) != ("write", "done")]
[issue] = by_code(graph, "governed.exit_text_source")
assert issue.field == "fields" and "最后一条消息" in issue.message and "「润色」" in issue.message
# 出口没有契约时 G1 不管,G2 照样追到这段绕过报告的文字
config(graph, "done").pop("contract")
assert [i.node_id for i in by_code(graph, "governed.text_bypass")] == ["polish"]
# --------------------------------------------------------------------------
# G2:模型写的文字没经过报告撰写节点,从图上追到出口
# --------------------------------------------------------------------------
def test_an_llm_bypassing_the_report_is_caught_through_intermediate_nodes():
graph = traceable()
graph = insert(graph, node("narrate", "llm", "叙述", prompt="写周报", assign_to="story"), before="done",
after="card")
graph = insert(graph, node("shape", "transform", "整形", mode="template",
template="【周报】{{ vars.story }}", assign_to="shaped"), before="done", after="narrate")
config(graph, "done")["fields"].append({"name": "摘要", "value": "{{ vars.shaped }}"})
[bypass] = by_code(graph, "governed.text_bypass")
assert bypass.level == "error" and bypass.node_id == "narrate"
assert "「叙述」(模型调用)" in bypass.message and "「出具」" in bypass.message and "「整形」" in bypass.message
# 出口字段点的是整形节点:G1 点名字段,G2 点名写文字的那个模型节点,两处各报各的
assert [i.field for i in by_code(graph, "governed.exit_text_source")] == ["fields[1].value"]
def test_a_direct_reference_in_a_contract_exit_is_left_to_g1():
graph = insert(traceable(), node("narrate", "llm", "叙述", prompt="写周报"), before="done", after="card")
config(graph, "done")["fields"].append({"name": "摘要", "value": "{{ nodes.narrate.text }}"})
assert len(by_code(graph, "governed.exit_text_source")) == 1
assert by_code(graph, "governed.text_bypass") == []
def test_every_exit_counts_at_both_levels():
graph = traceable()
graph["nodes"] += [node("chat", "llm", "闲聊", prompt="你好"),
node("side", "output", "附带成果", fields=[{"name": "回答", "value": "{{ nodes.chat.text }}"}])]
graph["edges"] += [{"source": "start", "target": "chat"}, {"source": "chat", "target": "side"}]
[issue] = by_code(graph, "governed.text_bypass")
assert issue.node_id == "chat" and "「附带成果」" in issue.message and issue.level == "error"
# 已发布级别范围一样,只是轻重不同:没带契约的出口也给一条警告(用户拍板:G 规则受管挡、已发布警告)
[issue] = by_code(graph, "governed.text_bypass", "published")
assert issue.node_id == "chat" and issue.level == "warning"
def test_text_that_goes_through_a_report_does_not_bypass_it():
graph = insert(traceable(), node("notes", "agent", "查数员", tools=["db_query__shop"], prompt="查 orders"),
before="write", after="fetch")
config(graph, "write")["notes_from"] = ["notes"]
assert g_issues(graph) == []
@pytest.mark.parametrize("value, flagged", [
("{{ vars.facts.gmv }}", False), # 开了 cite_fields:assign_to 拿到的是逐格核对过的字段
("{{ nodes.ask.data.gmv }}", False),
("{{ nodes.ask.text }}", True), # 它的原话照样是模型写的
])
def test_cited_agent_fields_are_data_not_text(value, flagged):
graph = insert(traceable(), node("ask", "agent", "查数员", tools=["db_query__shop"], prompt="查 orders",
output_schema={"type": "object", "properties": {"gmv": {"type": "number"}}},
cite_fields=True, assign_to="facts"), before="done", after="fetch")
graph = insert(graph, node("shape", "transform", "整形", mode="template", template=value, assign_to="x"),
before="done", after="ask")
graph["nodes"].append(node("side", "output", "附带成果", fields=[{"name": "值", "value": "{{ vars.x }}"}]))
graph["edges"].append({"source": "shape", "target": "side"})
assert bool(by_code(graph, "governed.text_bypass")) is flagged
# --------------------------------------------------------------------------
# G3:报告撰写节点按最严的规则自查,并显式声明结论句策略
# --------------------------------------------------------------------------
def test_a_report_node_must_spell_out_the_strict_settings():
graph = traceable(report={})
[issue] = by_code(graph, "governed.report_policy")
assert issue.level == "error" and issue.node_id == "write" and issue.field == "numbers"
for piece in ("numbers: strict", "on_violation: fail", "claims: require_citation"):
assert piece in issue.message
assert [i.level for i in by_code(graph, "governed.report_policy", "published")] == ["warning"]
@pytest.mark.parametrize("key, value", [("numbers", "off"), ("on_violation", "flag"), ("claims", "off")])
def test_a_looser_setting_is_named(key, value):
graph = traceable(report={**STRICT, key: value})
[issue] = by_code(graph, "governed.report_policy")
assert issue.field == key and f"现在是 {value}" in issue.message
for other in set(STRICT) - {key}:
assert f"{other}:" not in issue.message
def test_graph_defaults_count_like_they_do_at_run_time():
graph = traceable(report={"claims": "require_citation"})
graph["defaults"] = {"numbers": "strict", "on_violation": "fail"}
assert by_code(graph, "governed.report_policy") == []
graph["defaults"]["on_violation"] = "flag"
assert [i.field for i in by_code(graph, "governed.report_policy")] == ["on_violation"]
def test_an_unsupported_claims_value_is_left_to_validate():
graph = traceable(report={**STRICT, "claims": "judge"})
assert by_code(graph, "governed.report_policy") == []
[bad] = [i for i in validate_graph(GraphSpec.model_validate(graph)).issues if i.code == "report.claims_invalid"]
assert bad.level == "error" and CLAIMS_LATER in bad.message
# --------------------------------------------------------------------------
# G4:口径卡的输入不能来自计算角色的沙箱代码
# --------------------------------------------------------------------------
@pytest.mark.parametrize("role", [None, "compute"])
def test_compute_code_feeding_a_caliber_card_is_refused(role):
graph = traceable()
extra = {} if role is None else {"evidence_role": role}
graph = insert(graph, node("calc", "code", "计算", code="print(1)", assign_to="calc", **extra), before="card",
after="fetch")
config(graph, "card")["metrics"].append({"id": "ratio", "expression": "vars.calc.ratio"})
[issue] = by_code(graph, "governed.caliber_compute_input")
assert issue.level == "error" and issue.node_id == "calc" and issue.field == "evidence_role"
assert "「周报口径卡」" in issue.message and "source" in issue.message
assert [i.level for i in by_code(graph, "governed.caliber_compute_input", "published")] == ["warning"]
config(graph, "calc")["evidence_role"] = "source"
assert by_code(graph, "governed.caliber_compute_input") == []
def test_compute_code_is_caught_behind_a_transform():
graph = insert(traceable(), node("calc", "code", "计算", code="print(1)"), before="card", after="fetch")
graph = insert(graph, node("shape", "transform", "整形", mode="json", template="{{ nodes.calc.stdout }}",
assign_to="shaped"), before="card", after="calc")
config(graph, "card")["metrics"].append({"id": "ratio", "expression": "vars.shaped.ratio"})
assert [i.node_id for i in by_code(graph, "governed.caliber_compute_input")] == ["calc"]
# --------------------------------------------------------------------------
# G5:给口径卡供数的 agent 配 output_schema 并开 cite_fields
# --------------------------------------------------------------------------
SCHEMA = {"type": "object", "properties": {"gmv": {"type": "number"}}}
def feeding_agent(**config_) -> dict:
graph = insert(traceable(), node("ask", "agent", "查数员", tools=["db_query__shop"], prompt="查 orders",
assign_to="facts", **config_), before="card", after="start")
config(graph, "card")["metrics"].append({"id": "gmv2", "expression": "vars.facts.gmv"})
return graph
def test_an_agent_feeding_a_card_needs_an_output_schema():
[issue] = by_code(feeding_agent(), "governed.caliber_agent_schema")
assert issue.level == "error" and issue.node_id == "ask" and issue.field == "output_schema"
assert "「周报口径卡」" in issue.message
def test_an_agent_with_a_schema_needs_cite_fields():
graph = feeding_agent(output_schema=SCHEMA)
[issue] = by_code(graph, "governed.caliber_agent_cite_fields")
assert issue.node_id == "ask" and issue.field == "cite_fields"
assert by_code(graph, "governed.caliber_agent_schema") == []
assert g_issues(feeding_agent(output_schema=SCHEMA, cite_fields=True)) == []
def test_an_agent_that_only_feeds_the_report_is_not_a_card_input():
graph = insert(traceable(), node("ask", "agent", "查数员", tools=["db_query__shop"], prompt="查 orders"),
before="write", after="fetch")
assert by_code(graph, "governed.caliber_agent_schema") == []
def test_a_model_call_feeding_a_card_is_refused():
graph = insert(traceable(), node("guess", "llm", "估算", prompt="估一个数", assign_to="g"), before="card",
after="start")
config(graph, "card")["metrics"].append({"id": "g", "expression": "vars.g"})
[issue] = by_code(graph, "governed.caliber_model_input")
assert issue.node_id == "guess" and "「估算」(模型调用)" in issue.message
# --------------------------------------------------------------------------
# 已发布级别只给警告;已经发布的版本照常运行
# --------------------------------------------------------------------------
def everything_wrong() -> dict:
graph = feeding_agent()
graph = insert(graph, node("calc", "code", "计算", code="print(1)", assign_to="calc"), before="card",
after="fetch")
config(graph, "card")["metrics"].append({"id": "ratio", "expression": "vars.calc.ratio"})
graph = insert(graph, node("narrate", "llm", "叙述", prompt="写周报", assign_to="story"), before="done",
after="card")
graph = insert(graph, node("shape", "transform", "整形", mode="template", template="{{ vars.story }}",
assign_to="shaped"), before="done", after="narrate")
config(graph, "done")["fields"].append({"name": "摘要", "value": "{{ vars.shaped }}"})
config(graph, "done")["contract"].update(report_from=None, narrative="{{ nodes.write.text }}")
config(graph, "write").clear()
return graph
async def test_the_published_level_only_warns(client):
graph = everything_wrong()
assert {i.code for i in g_issues(graph)} == G_CODES - {"governed.caliber_agent_cite_fields",
"governed.caliber_model_input"}
assert all(i.level == "error" for i in g_issues(graph))
warned = g_issues(graph, "published")
assert {i.code for i in warned} == {i.code for i in g_issues(graph)}
assert all(i.level == "warning" for i in warned)
assert (await publish(client, graph, "published"))["ok"] is True
assert (await publish(client, graph, "governed"))["ok"] is False
def test_template_nine_passes_once_completed():
tpl = next(t for t in TEMPLATES if t["name"].startswith("⑨"))
graph = copy.deepcopy({"nodes": tpl["nodes"], "edges": tpl["edges"]})
# 模板原样:G 规则里至多差报告撰写节点那三项
assert {i.code for i in g_issues(graph)} <= {"governed.report_policy"}
next(n for n in graph["nodes"] if n["type"] == "report")["data"]["config"].update(STRICT)
issues = publish_issues(GraphSpec.model_validate(graph), level="governed")
assert [i for i in issues if i.level == "error"] == [], [i.message for i in issues if i.level == "error"]
assert not {i.code for i in issues} & G_CODES
@pytest.fixture
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
async def test_an_already_governed_version_keeps_running_formally(client, engine_up, monkeypatch):
"""升级前按受管发布的版本(报告节点没写那三项)照常发起正式运行;再发布时才按新规则查。"""
from app.providers import mock_model
monkeypatch.setattr(mock_model.MockChatModel, "_decide",
lambda self, messages: AIMessage(content="本周销售额 [[m:gmv]]。[[see:m:gmv]]"))
graph = traceable(report={})
card = config(graph, "card")
card["metrics"] = [{"id": "gmv", "name": "销售额", "unit": "元", "expression": "vars.kpi.gmv"}]
graph["nodes"][1] = node("fetch", "transform", "取数", mode="expression", expression="{'gmv': 300.5}",
assign_to="kpi")
async with SessionLocal() as session:
wf = Workflow(name="证据门禁-老受管版本", graph=graph, status="governed", published_version=1)
session.add(wf)
await session.flush()
session.add(WorkflowVersion(workflow_id=wf.id, version=1, graph=graph, level="governed"))
await session.commit()
wf_id = wf.id
r = await client.post("/api/runs", json={"workflow_id": wf_id, "run_class": "formal"})
assert r.status_code == 201, r.text
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, r.json()["id"])
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
break
assert row.status == "succeeded", row.error
# 下次发布按新规则:报告节点那三项没写明,受管发布被拦
check = (await client.post(f"/api/workflows/{wf_id}/publish-check", json={"level": "governed"})).json()
assert check["ok"] is False
assert "governed.report_policy" in {i["code"] for i in check["issues"] if i["level"] == "error"}
# --------------------------------------------------------------------------
# validate:报告撰写节点的 claims 只接受 off / require_citation
# --------------------------------------------------------------------------
def claims_issues(value, *, on_defaults: bool = False) -> list:
graph = traceable(report={} if on_defaults else {"claims": value})
if on_defaults:
graph["defaults"] = {"claims": value}
return [i for i in validate_graph(GraphSpec.model_validate(graph)).issues if i.code == "report.claims_invalid"]
@pytest.mark.parametrize("value", [None, "", "off", "require_citation"])
def test_supported_claims_values_pass(value):
assert claims_issues(value) == []
def test_judge_is_a_later_version():
[issue] = claims_issues("judge")
assert issue.level == "error" and issue.node_id == "write" and issue.field == "claims"
assert CLAIMS_LATER in issue.message and "off" in issue.message and "require_citation" in issue.message
@pytest.mark.parametrize("value", ["strict", "Require_Citation", 1, ["off"], {"policy": "judge"}])
def test_other_claims_values_are_errors(value):
[issue] = claims_issues(value)
assert issue.level == "error" and "off / require_citation" in issue.message and repr(value) in issue.message
def test_claims_from_graph_defaults_are_checked_too():
[issue] = claims_issues("judge", on_defaults=True)
assert issue.node_id == "write"
@@ -100,4 +100,6 @@ def test_a_narrative_contract_is_unchanged():
contract = contract_of(graph)
del contract["report_from"]
contract["narrative"] = "{{ nodes.write.text }}"
assert [i for i in lint(graph) if i.level == "error"] == []
# _lint_contract 对叙述模式照旧;受管级别另有门禁 G1 要求核对报告撰写节点的文档
assert [i for i in lint(graph, "published") if i.level == "error"] == []
assert [i.code for i in lint(graph) if i.level == "error"] == ["governed.report_from_required"]
+2 -1
View File
@@ -37,7 +37,8 @@ def weekly(**contract_overrides) -> dict:
node("card", "metrics", "周报口径卡", caliber="周报口径", metrics=[
{"id": "gmv", "name": "销售额", "unit": "元", "expression": "cell(nodes.fetch, 0, 'gmv')"},
{"id": "orders", "name": "订单数", "unit": "单", "expression": "cell(nodes.fetch, 0, 'orders')"}]),
node("write", "report", "报告撰写", instructions="写周报"),
node("write", "report", "报告撰写", instructions="写周报",
numbers="strict", on_violation="fail", claims="require_citation"),
node("done", "output", "出具", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}],
contract={k: v for k, v in contract.items() if v is not None}),
]
+191
View File
@@ -0,0 +1,191 @@
"""报告节点的结论句策略 claims:off(默认,和以前一样)/ require_citation(没挂依据的结论句计入缺口)。
结论句要靠裁判模型判断支不支持,那是后续版本的事;这一版只做确定性的一半:
- require_citation 时写作提示明确要求每句结论挂 [[see:…]];没挂的照常产出、计数,
出口按出具档位降档(io.py),报告节点自己不为它失败、不为它重写
- report.checked 事件和节点产出都带上 claims,出口据此判档
- 写 judge 时节点直接报错「结论句裁判在后续版本支持」,不能悄悄当成 off
- 升级前发起的运行一个字都不变
"""
from __future__ import annotations
import asyncio
import pytest
from langchain_core.messages import AIMessage
from sqlalchemy import select
from app.core import artifact_store
from app.db.base import SessionLocal
from app.db.models import Run, RunEvent
from app.engine import runner as runner_mod
from app.engine.evidence import CLAIMS_RULE, build_catalog, compose_doc, uncited_claims
from app.engine.runner import run_manager
from app.engine.schema import CLAIMS_LATER, REPORT_CLAIMS, claims_problem
@pytest.fixture(autouse=True)
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
def node(nid, ntype, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
METRICS = [{"id": "gmv", "name": "销售额", "unit": "元", "decimals": 1, "expression": "vars.kpi.gmv"}]
#: 第一句挂了依据,第二句是没挂依据的结论(有方向词),第三句是连接性的话
TEXT = "本周销售额 [[m:gmv]]。[[see:m:gmv]]增长主要来自新客。下面看细节。"
def weekly(**report):
nodes = [
node("start", "input", fields=[{"name": "week", "default": "2026-W37"}]),
node("fetch", "transform", mode="expression", expression="{'gmv': 45678.5}", assign_to="kpi"),
node("caliber", "metrics", caliber="周报口径", caliber_version="v2", metrics=METRICS),
node("write", "report", instructions="写周报", **report),
node("out", "output", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}]),
]
ids = [n["id"] for n in nodes]
return {"nodes": nodes, "edges": [{"source": a, "target": b} for a, b in zip(ids, ids[1:])]}
def script(monkeypatch, reply=TEXT) -> list:
from app.providers import mock_model
seen: list = []
def _decide(self, messages):
seen.append(list(messages))
return AIMessage(content=reply)
monkeypatch.setattr(mock_model.MockChatModel, "_decide", _decide)
return seen
def lenient(monkeypatch):
"""装作画布校验没拦住(校验以后放宽、或者绕过校验发起的运行):节点执行时自己也要守住这一关,
报的是同一句话。"""
original = runner_mod.validate_graph
def permissive(spec):
report = original(spec)
report.issues = [i for i in report.issues if i.field != "claims"]
report.ok = not any(i.level == "error" for i in report.issues)
return report
monkeypatch.setattr(runner_mod, "validate_graph", permissive)
async def finish(graph) -> Run:
run = await run_manager.start(graph=graph, input_payload={})
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run.id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
return row
raise AssertionError(f"run {run.id} 没跑完:{row.status}")
async def events(run_id: str, etype: str) -> list[RunEvent]:
async with SessionLocal() as session:
q = select(RunEvent).where(RunEvent.run_id == run_id, RunEvent.type == etype).order_by(RunEvent.seq)
return list((await session.execute(q)).scalars())
async def checked_and_output(run_id: str) -> tuple[dict, dict]:
[checked] = await events(run_id, "report.checked")
finished = [e for e in await events(run_id, "node.finished") if e.node_id == "write"][0]
return checked.data, artifact_store.load(finished.data["artifact"])
def prompt_of(seen: list) -> str:
return "\n".join(str(m.content) for m in seen[0])
async def test_default_is_off_and_changes_nothing(monkeypatch):
seen = script(monkeypatch)
row = await finish(weekly())
assert row.status == "succeeded", row.error
checked, output = await checked_and_output(row.id)
assert checked["claims"] == "off" and output["claims"] == "off"
assert CLAIMS_RULE not in prompt_of(seen)
assert checked["stats"]["uncited_claims"] == 1
async def test_require_citation_asks_for_support_but_does_not_block(monkeypatch):
seen = script(monkeypatch)
row = await finish(weekly(claims="require_citation", on_violation="fail"))
assert row.status == "succeeded", row.error
assert len(seen) == 1, "没挂依据的结论句不触发重写"
assert CLAIMS_RULE in prompt_of(seen)
checked, output = await checked_and_output(row.id)
assert checked["claims"] == "require_citation" and output["claims"] == "require_citation"
assert checked["ok"] is True and checked["violations"] == []
assert checked["stats"]["uncited_claims"] == output["stats"]["uncited_claims"] == 1
doc = artifact_store.load(checked["doc_artifact"])
[flagged] = uncited_claims(doc)
assert flagged["text"] == "增长主要来自新客。" and doc["markdown"][slice(*flagged["span"])] == flagged["text"]
async def test_judge_is_refused_until_it_exists(monkeypatch):
seen = script(monkeypatch)
with pytest.raises(ValueError) as info: # 画布校验先拦(E3-gov 的 validate)
await run_manager.start(graph=weekly(claims="judge"), input_payload={})
assert CLAIMS_LATER in str(info.value)
lenient(monkeypatch)
row = await finish(weekly(claims="judge"))
assert row.status == "failed" and claims_problem("judge") in (row.error or ""), row.error
assert seen == [], "不支持的策略不该先花一次模型调用"
@pytest.mark.parametrize("value", ["sometimes", 1, ["require_citation"]])
async def test_other_values_are_refused(monkeypatch, value):
script(monkeypatch)
lenient(monkeypatch)
row = await finish(weekly(claims=value))
assert row.status == "failed", row.error
assert claims_problem(value) in row.error and "require_citation" in row.error
async def test_runs_from_before_the_upgrade_are_untouched(monkeypatch):
async def no_snapshot(session):
return None
monkeypatch.setattr(runner_mod, "_agent_limits", no_snapshot)
seen = script(monkeypatch)
row = await finish(weekly())
assert row.status == "succeeded", row.error
checked, output = await checked_and_output(row.id)
assert "claims" not in checked and "claims" not in output
assert CLAIMS_RULE not in prompt_of(seen)
# --------------------------------------------------------------------------
# 出口和界面共用的纯函数
# --------------------------------------------------------------------------
def test_the_supported_policies_match_the_validator():
from app.engine.nodes.report import CLAIMS, report_claims
from app.engine.schema import GraphSpec
assert CLAIMS == REPORT_CLAIMS == ("off", "require_citation")
spec = GraphSpec.model_validate(weekly(claims="require_citation"))
assert report_claims(spec, spec.node_map()["write"]) == "require_citation"
spec = GraphSpec.model_validate(weekly())
assert report_claims(spec, spec.node_map()["write"]) == "off"
spec.defaults["claims"] = "require_citation" # 图级 defaults 同样生效,和执行时一个规则
assert report_claims(spec, spec.node_map()["write"]) == "require_citation"
def test_uncited_claims_lists_claim_units_without_support():
catalog = build_catalog(nodes={"caliber": {"kind": "metric_set", "caliber": "c", "caliber_version": "v1",
"artifact": "f" * 64,
"metrics": [{"id": "gmv", "name": "销售额", "value": 1, "unit": ""}]}})
doc = compose_doc("销售额 [[m:gmv]]。环比下降。[[see:m:gmv]]主要来自东区。连接的话。", catalog)
assert [u["text"] for u in uncited_claims(doc)] == ["主要来自东区。"]
assert all(set(u) == {"unit", "span", "text"} for u in uncited_claims(doc))
+167
View File
@@ -0,0 +1,167 @@
"""SQL 守卫的单遍扫描:去注释要认得字符串和标识符引号。
以前先去注释、后认字符串:字符串里的 `--`、`/*` 被当成注释删掉,执行的 SQL 和写的不一样
(run_query 执行的正是 guard.check 的返回值)。只改成执行原文又会让守卫看到的和数据库执行的
不一致,所以扫描必须一遍走完,同时认得注释、字符串、标识符。
"""
from __future__ import annotations
import sqlite3
import uuid
from types import SimpleNamespace
import pytest
from app.data.engine import engines, run_query
from app.data.guard import SqlRejected, check, is_write, split_statements, strip_comments
def rejected(sql: str, *, readonly: bool = True, dialect: str | None = None) -> str | None:
try:
check(sql, readonly=readonly, source_name="测试库", dialect=dialect)
return None
except SqlRejected as e:
return str(e)
# ---------------------------------------------------------------- 字符串原样保留
@pytest.mark.parametrize("dialect", [None, "sqlite", "postgresql", "mysql", "oracle"])
@pytest.mark.parametrize("sql", [
"SELECT '--' AS a",
"SELECT '/* x */' AS a",
"SELECT 'a -- b', 'c /* d' AS a",
'SELECT "col--x" FROM t',
])
def test_comment_markers_inside_quotes_are_kept(sql, dialect):
assert check(sql, readonly=True, dialect=dialect) == sql
assert strip_comments(sql, dialect=dialect) == sql
def test_real_comments_are_still_removed():
assert check("SELECT 1 -- 行注释", readonly=True).strip() == "SELECT 1"
assert check("SELECT /* 块注释 */ 1", readonly=True).split() == ["SELECT", "1"]
assert check("SELECT '--' /* 注释 */ AS a", readonly=True).split() == ["SELECT", "'--'", "AS", "a"]
# 删掉的注释要留一个空格:'x'/**/AS 删成 'x'AS 就粘成一个词了
assert check("SELECT 'x'/**/AS a", readonly=True).split() == ["SELECT", "'x'", "AS", "a"]
@pytest.mark.parametrize("sql,dialect,kept", [
("SELECT 1 /* a /* b */ c */ AS n", "postgresql", "SELECT 1 AS n"), # PG 的块注释可以嵌套
("SELECT E'it\\'s -- x' AS a", "postgresql", None), # E'' 里认反斜杠
("SELECT [a--b] FROM t", "sqlite", None), # SQLite 的方括号标识符
("SELECT q'[it's -- x]' AS a FROM dual", "oracle", None), # Oracle 的 q'[…]'
("SELECT a$b$ AS c -- $b$", "postgresql", "SELECT a$b$ AS c"), # 标识符里的 $ 不开字符串
])
def test_dialect_specific_quoting(sql, dialect, kept):
assert check(sql, readonly=True, dialect=dialect) == (kept if kept is not None else sql)
def test_doubled_quote_escape_is_one_string():
sql = "SELECT 'it''s -- fine' AS a"
assert check(sql, readonly=True) == sql
def test_dollar_quoted_body_is_a_string_on_postgres():
sql = "SELECT $$ -- not a comment $$ AS a, $tag$ /* nor this */ $tag$ AS b"
assert check(sql, readonly=True, dialect="postgresql") == sql
def test_mysql_backslash_escape_without_ambiguity_is_allowed():
sql = r"SELECT 'O\'Brien' AS s"
assert check(sql, readonly=True, dialect="mysql") == sql
def test_mysql_backslash_that_moves_a_comment_boundary_is_rejected():
"""默认 sql_mode 下是一个字符串,NO_BACKSLASH_ESCAPES 下后半截是注释:两种读法不一致就拒。"""
reason = rejected(r"SELECT 'a\'b -- c' AS s", dialect="mysql")
assert reason and "反斜杠" in reason
@pytest.fixture
async def data_source(tmp_path):
path = tmp_path / "scan.db"
conn = sqlite3.connect(path)
conn.execute("CREATE TABLE t (id INTEGER, note TEXT)")
conn.executemany("INSERT INTO t VALUES (?, ?)",
[(1, "a -- b"), (2, "c /* d */"), (3, "plain")])
conn.commit()
conn.close()
src = SimpleNamespace(
id=f"scan-{uuid.uuid4().hex[:8]}", kind="sqlite", database=str(path), readonly=True,
name="测试源", options={}, password=None, username="u", host="h", port=None,
)
yield src
await engines.invalidate(src.id)
async def test_query_with_comment_markers_in_strings_runs_and_returns_right_rows(data_source):
"""字符串里的 -- 和 /* 要原样到达数据库,结果才对。"""
result = await run_query(data_source, "SELECT id FROM t WHERE note = 'a -- b'")
assert result.rows == [[1]]
result = await run_query(data_source, "SELECT id FROM t WHERE note = 'c /* d */' -- 真注释")
assert result.rows == [[2]]
result = await run_query(data_source, "SELECT '--' AS a, '/*' AS b")
assert result.rows == [["--", "/*"]]
result = await run_query(data_source, "SELECT id FROM t WHERE note LIKE '%--%' /* ' */ ORDER BY id")
assert result.rows == [[1]]
# ---------------------------------------------------------------- 拦截
@pytest.mark.parametrize("sql,dialect", [
("SELECT '--'; DROP TABLE t", None),
("SELECT '--'; DROP TABLE t", "sqlite"),
("SELECT '/*'; DELETE FROM t", None),
("SELECT 1 /* ' */; DELETE FROM t", None),
("SELECT 1 -- '\n; DELETE FROM t", None),
("SELECT $$--$$; DROP TABLE t", "postgresql"),
("SELECT $x$ ; $x$; DELETE FROM t", "postgresql"),
("SELECT '\\'; DELETE FROM t", "sqlite"),
("SELECT '\\'; DELETE FROM t", "postgresql"),
("SELECT '\\\\'; DELETE FROM t", "mysql"),
("SELECT 1 # '\n; DELETE FROM t", "mysql"),
("SELECT \"--\"; DELETE FROM t", None),
("SELECT `--`; DELETE FROM t", None),
])
def test_statement_smuggling_is_rejected(sql, dialect):
reason = rejected(sql, dialect=dialect)
assert reason is not None, f"放过了:{sql!r} ({dialect})"
reason_rw = rejected(sql, readonly=False, dialect=dialect)
assert reason_rw is not None, f"可写源上放过了:{sql!r} ({dialect})"
@pytest.mark.parametrize("sql,dialect", [
("SELECT '--' AS a FROM t WHERE 1=1 /* ' */ AND 0 = 0 UNION SELECT 1 FROM t FOR UPDATE", None),
("SELECT $$ ' $$ AS a FROM t FOR UPDATE", "postgresql"),
])
def test_hidden_writes_are_seen_through_quotes_and_comments(sql, dialect):
reason = rejected(sql, dialect=dialect)
assert reason and "只读" in reason, f"漏了:{sql!r}"
@pytest.mark.parametrize("sql,dialect", [
("SELECT '--'; DELETE FROM t", None),
("SELECT 1 /* ' */; DELETE FROM t", "sqlite"),
("SELECT $$--$$; DELETE FROM t", "postgresql"),
])
def test_is_write_counts_smuggled_statements(sql, dialect):
"""审批这道门:看不清的一律当写,等人批。"""
assert is_write(sql, dialect=dialect)
def test_split_statements_respects_quotes_before_comments():
assert len(split_statements("SELECT '--'; SELECT 2")) == 2
assert len(split_statements("SELECT '-- ; x'")) == 1
assert len(split_statements("SELECT $$;$$", dialect="postgresql")) == 1
def test_a_write_word_seen_under_another_mysql_setting_counts():
"""默认设置下是字符串,关了反斜杠转义就成了代码:哪种读法下会写,就按会写处理。"""
sql = r"SELECT 'can\'t delete' AS a"
reason = rejected(sql, dialect="mysql")
assert reason and "只读" in reason and "反斜杠" in reason
assert is_write(sql, dialect="mysql")
assert rejected(sql, readonly=False, dialect="mysql") is None
+2 -1
View File
@@ -51,7 +51,8 @@ def weekly() -> dict:
node("fetch", "transform", mode="expression", expression="{'gmv': 300.5}", assign_to="kpi"),
node("card", "metrics", caliber="周报口径", metrics=[
{"id": "gmv", "name": "销售额", "unit": "元", "expression": "vars.kpi.gmv"}]),
node("write", "report", instructions="写周报"),
node("write", "report", instructions="写周报",
numbers="strict", on_violation="fail", claims="require_citation"),
node("out", "output", fields=[{"name": "周报", "value": "{{ nodes.write.text }}"}],
contract={"report_from": "write", "metrics_from": ["card"], "required": ["gmv"], "strict": True}),
]
+26 -1
View File
@@ -1,6 +1,6 @@
import type {
Approval, AutofixResult, Conversation, ConversationDetail, ConversationTurn, CustomTool, DataSource, EvidenceGraph,
EvidenceSegmentDetail, GraphSpec, KbDocument, MemoryItem, Provider, PublishCheck, PublishLevel, ReviewResult, Run,
EvidenceAudit, EvidenceSegmentDetail, GraphSpec, KbDocument, MemoryItem, Provider, PublishCheck, PublishLevel, ReviewResult, Run,
RunEvent, RunStatus, Skill, ToolChange, ToolInfo, ToolTrust, ValidationIssue, VarIssue, Variable, Workflow,
WorkflowVersion,
} from '../types'
@@ -702,6 +702,31 @@ export const api = {
get<EvidenceSegmentDetail>(
`/runs/${encodeURIComponent(runId)}/evidence/segments/${encodeURIComponent(segmentId)}${qs({ report: opts?.report })}`,
opts),
/**
* 审计表:每个有状态的片段按状态分组(有出处 / 无证据 / 可疑实体 / 旧运行猜测),附上封存核对的结果。
* groups 只要这几组(none、suspicious…)
*/
audit: (runId: string, opts?: RequestOptions & { groups?: string[] }) =>
get<EvidenceAudit>(`/runs/${encodeURIComponent(runId)}/evidence/audit${qs({ groups: opts?.groups?.join(',') || undefined })}`, opts),
/**
* 导出审计表:?format=json / csv 作为附件下载。回来的是文件本身和后端给的文件名
* (Content-Disposition;没给就用 evidence-<运行号前 8 位>.<格式>)
*/
auditExport: async (runId: string, format: 'json' | 'csv', groups?: string[]): Promise<{ blob: Blob; name: string }> => {
const path = `/runs/${encodeURIComponent(runId)}/evidence/audit${qs({ format, groups: groups?.join(',') || undefined })}`
let res: Response
try {
res = await fetch(BASE + path, { headers: actorHeader() })
} catch (e: any) {
const err = new ApiError(0, NETWORK_MESSAGE, { kind: 'network', raw: `${e?.name ?? 'Error'}: ${e?.message ?? e}` })
report(false, err)
throw err
}
if (!res.ok) throw failure(res.status, res.statusText, await res.text().catch(() => ''), path)
report(true)
const named = /filename\*?=(?:UTF-8'')?"?([^";]+)"?/i.exec(res.headers.get('Content-Disposition') ?? '')?.[1]
return { blob: await res.blob(), name: named ? decodeURIComponent(named) : `evidence-${runId.slice(0, 8)}.${format}` }
},
},
// ---- 会话 ----
+4
View File
@@ -522,6 +522,10 @@ function FieldInput({ field, id, nodeId, syntax, value, config, invalid, describ
{(field.options ?? []).map((o) => (
<option key={o.value} value={o.value}>{o.label}</option>
))}
{/* config 里是个不在选项里的值:照实列出来(不能选回去),不让它看着像第一个选项 */}
{field.unknownLabel && typeof value === 'string' && value && !(field.options ?? []).some((o) => o.value === value) && (
<option value={value} disabled>{field.unknownLabel(value)}</option>
)}
</select>
)
+33 -2
View File
@@ -14,9 +14,10 @@ import { api } from '../api/client'
import { isComposing, StatusBadge, toast } from '../components/ui'
import { matchedSource } from '../lib/evidence'
import { explainRunError } from '../lib/explain'
import { formatDuration, formatLapse, formatNumber, formatTokens, NONE } from '../lib/format'
import { formatDuration, formatLapse, formatNumber, formatTokens, NONE, shortId } from '../lib/format'
import { statusMeta } from '../lib/status'
import { issuanceLabel, RUN_CLASS_LABEL } from '../lib/terms'
import { issuanceLabel, reportStampText, RUN_CLASS_LABEL } from '../lib/terms'
import { useReportStamp, type ReportStamp } from './reportStamp'
import { ApprovalCard } from '../run/RunPanel'
import type { NodeState, NodeTrace } from '../run/trace'
import { useRunClock } from '../run/useRunClock'
@@ -187,6 +188,23 @@ function matchedLines(matched: unknown): string {
return `回指明细:\n${lines.join('\n')}${more}`
}
/** 报告节点章的悬停说明:两个数怎么来的,缺口里各有什么 */
function reportStampTitle(r: ReportStamp): string {
const st = r.stats
const n = (v: unknown) => (typeof v === 'number' && Number.isFinite(v) ? v : 0)
const uncited = n(st.uncited_claims)
return [
`报告核对(这张图最近一次运行${r.runId ? ` ${shortId(r.runId)}` : ''}):${reportStampText(r.cited, r.none)}`,
typeof st.numbers === 'number' ? `数字 ${formatNumber(n(st.numbers_cited))}/${formatNumber(st.numbers)} 有出处` : '',
n(st.entities) || n(st.quotes) ? `表名字段名 ${formatNumber(n(st.entities))} 处、引文 ${formatNumber(n(st.quotes))} 处有出处` : '',
n(st.unresolved) ? `引用解析不了 ${formatNumber(n(st.unresolved))} 处` : '',
n(st.unknown_entities) ? `可能是编造的名字 ${formatNumber(n(st.unknown_entities))} 个` : '',
n(st.unverified_entities) ? `核对不了的名字 ${formatNumber(n(st.unverified_entities))} 个(只标注,不计入)` : '',
uncited ? `没挂依据的结论句 ${formatNumber(uncited)} 句${r.claims === 'require_citation' ? '(计入缺口)' : '(结论句策略不要求,不计入)'}` : '',
'完整清单在记录页这次运行的「证据」页签',
].filter(Boolean).join('\n')
}
/**
* 每个节点有几路上游(不算循环的回边)。排队中的节点据此说清在等什么:
* 汇合点是「等其余上游」,单入口的只是轮到它之前那一拍。按拓扑缓存
@@ -724,6 +742,8 @@ function NodeCardImpl({ id, data, selected, isConnectable }: NodeProps<FlowNode>
const issuanceDetail = useStudio((s) =>
(type === 'output' ? (s.run?.output as any)?._issuance as Record<string, any> | undefined
?? (issuanceOf(s.events).id === id ? issuanceOf(s.events).data : undefined) : undefined))
// 报告节点:这张图最近一次运行的核对统计,画成卡上的章(没有就不画)
const reportStamp = useReportStamp(id, type === 'report')
// 只有排队中的卡片要知道自己有几路上游,别的卡片不必为改图重跑这一趟
const fanIn = useStudio((s) => (state === 'queued' ? fanInOf(s.nodes, s.edges)[id] ?? 0 : 0))
@@ -1075,6 +1095,17 @@ function NodeCardImpl({ id, data, selected, isConnectable }: NodeProps<FlowNode>
</span>
)}
{/* 报告节点的章:「引用 N · 无证据 M」,这张图最近一次运行的 report.checked。回放时游标还没走到
那一刻不画。章不是按钮:完整清单在记录页的「证据」页签 */}
{reportStamp && (reportStamp.at == null || view.at == null || view.at >= reportStamp.at) && (
<span role="img" aria-label={reportStampTitle(reportStamp)} title={reportStampTitle(reportStamp)}
className={clsx('nc-stamp is-evidence', reportStamp.none ? 'is-degraded' : 'is-formal')}
data-report-stamp="" data-stamp-cited={reportStamp.cited} data-stamp-none={reportStamp.none}>
<StatusBadge status={reportStamp.none ? 'waiting' : 'done'} size={11} animate={false} decorative />
<span aria-hidden className="tnum">{reportStampText(reportStamp.cited, reportStamp.none)}</span>
</span>
)}
{/* 出口:标签画在出线第一段的上方,不压在线上。跑过之后命中的那条亮起来、
落空的压暗——六出口的分支节点跑完,不这样的话谁也说不清它到底选了哪条。
运行前一律中性色:绿色的「通过」挂在还没跑的分支上,读起来像已经通过了 */}
+6 -2
View File
@@ -581,6 +581,8 @@ function FixControl({ fix, pf }: { fix: PublishFix; pf: PreflightState }) {
const picked = (v: unknown) => (fix.multiple
? Array.isArray(value) && value.some((x) => same(x, v))
: value !== undefined && same(value, v))
// 选的是「交给 Copilot」那一项:和点「交给 Copilot」一回事,请求带 assist,等待时说 Copilot 在修
const handoff = !fix.multiple && (fix.options ?? []).some((o) => o.handoff === true && picked(o.value))
return (
<fieldset className="mb-1 ml-[22px] mt-0.5 rounded border px-2 py-1.5" data-fix-choice={fix.id}>
<legend className="px-1 text-faint">{fix.label}{fix.multiple ? ` · ${T.multiple}` : ''}</legend>
@@ -603,8 +605,10 @@ function FixControl({ fix, pf }: { fix: PublishFix; pf: PreflightState }) {
</div>
<div className="mt-1.5 flex items-center gap-2">
<button type="button" className="btn btn-xs" data-fix-action="choice" disabled={!ready || pf.working}
onClick={() => void pf.requestPreview({ apply: [fix.id], choices: { [fix.id]: value } })}>
<Wand2 size={10} aria-hidden /> {T.choose}
data-fix-handoff={handoff ? '' : undefined} title={handoff ? T.assistHint : undefined}
onClick={() => void pf.requestPreview({
apply: [fix.id], choices: { [fix.id]: value }, ...(handoff ? { assist: true } : {}) })}>
{handoff ? <Sparkles size={10} aria-hidden /> : <Wand2 size={10} aria-hidden />} {handoff ? T.assist : T.choose}
</button>
{!ready && <span className="text-faint">{T.chooseFirst}</span>}
</div>
+10 -2
View File
@@ -220,7 +220,8 @@ function normalizeFix(raw: any): PublishFix | null {
if (!kind) return null
const options = asArray<any>(raw.options)
.filter((o) => o && typeof o === 'object' && 'value' in o)
.map((o) => ({ value: o.value, label: asText(o.label ?? o.value), hint: o.hint ? asText(o.hint) : null }))
.map((o) => ({ value: o.value, label: asText(o.label ?? o.value), hint: o.hint ? asText(o.hint) : null,
...(o.handoff === true ? { handoff: true } : {}) }))
// 选项类没有候选就没法选:不给控件,比给一个空下拉强
if (kind === 'choice' && !options.length) return null
return {
@@ -353,7 +354,13 @@ const EXTRA_FIELDS: Record<string, string> = {
workflow_version: '钉住版本', upgrade_policy: '上游发了新版本时', defaults: '全图默认',
}
/** 点号后面那一段的叫法:契约里的键,全图默认里的审批策略 */
const SUB_KEYS: Record<string, string> = { ...CONTRACT_KEYS, approval: '审批策略' }
const SUB_KEYS: Record<string, string> = {
...CONTRACT_KEYS, approval: '审批策略', claims: '没挂依据的结论句', on_uncited: '没挂依据时',
}
/** 契约 claims 的 on_uncited:没挂依据的结论句怎么算 */
const ON_UNCITED_LABEL: Record<string, string> = {
ignore: '只标出来,不算缺口', degrade: '计入缺口、出具降档', withhold: '不予出具',
}
/** 值是节点 id 的几个键(报告、契约的 metrics_from,契约的 report_from):预览里写节点名,不写 id */
const NODE_REF_KEYS = new Set(['metrics_from', 'report_from'])
@@ -386,6 +393,7 @@ export function fixValueText(value: unknown, field?: string | null, nameOf?: Nod
return APPROVAL_POLICY_LABEL[value as keyof typeof APPROVAL_POLICY_LABEL]
}
if (typeof value === 'boolean') return value ? '是' : '否'
if (key === 'on_uncited' && typeof value === 'string' && value in ON_UNCITED_LABEL) return ON_UNCITED_LABEL[value]
if (key === 'workflow_version' && (typeof value === 'number' || /^\d+$/.test(String(value)))) return `v${value}`
if (key && NODE_REF_KEYS.has(key)) {
const ids = Array.isArray(value) ? value : [value]
+29
View File
@@ -43,6 +43,11 @@ export interface FieldDef {
disabled?: (config: Record<string, any>) => string | null
/** nodeRefs:只能选这一类的上游节点(报告的 metrics_from 只收口径卡) */
refType?: NodeType
/**
* select:config 里写着一个不在选项里的值(手写的、Copilot 写的、后续版本才支持的)时,
* 下拉里照实多列一项、写明为什么不认。不给的话浏览器会把它显示成第一个选项——看着像默认值
*/
unknownLabel?: (value: string) => string
advanced?: boolean
}
@@ -558,6 +563,30 @@ export const NODE_DEFS: Record<NodeType, NodeDef> = {
key: 'max_repairs', label: '最多重写几次', type: 'number', min: 0, max: 3, placeholder: '1',
help: '有违规时把清单交回去让模型重写。每次都是一整篇的调用,写不对的模型多给几次也大多写不对',
},
{
// 后端只认 off / require_citation(schema.REPORT_CLAIMS);judge 是四期的模型裁判,写了会被校验拦下。
// 没写等于 off:select 显示第一项,正好是默认
key: 'claims', label: '没挂依据的结论句', type: 'select',
options: [
{ value: 'off', label: '不管(默认):不参与出具判档' },
{ value: 'require_citation', label: '计入缺口:出具按档位降档' },
],
help: '结论句是带数字、方向词或因果词的句子。选「计入缺口」时,写作提示要求每句结论挂上 [[see:…]] 依据,'
+ '没挂的记为缺口、出具降档(系统自动链接的表名字段名不算依据)。受管级别发布必须显式选它。'
+ '结论句裁判在后续版本支持',
unknownLabel: (v) => (v === 'judge' ? 'judge:结论句裁判在后续版本支持,现在不能用' : `${v}:不认识的值,只能选上面两项`),
},
{
// 后端只认 link / off(report.py 的 ENTITIES)
key: 'entities', label: '表名、字段名', type: 'select', advanced: true,
options: [
{ value: 'link', label: '核对,对得上的可以点开(默认)' },
{ value: 'off', label: '不核对' },
],
help: '报告里写到的表名、字段名按本次运行的表结构快照和查询核对:对得上的能点开看出处,'
+ '反引号里写了哪里都没有的名字标成「可能是编造的名字」',
unknownLabel: (v) => `${v}:不认识的值,只能选 link 或 off`,
},
...MODEL_FIELDS,
{ key: 'system', label: '角色设定', type: 'textarea', advanced: true, placeholder: '你是…' },
...COMMON_TAIL,
+73
View File
@@ -0,0 +1,73 @@
import { useEffect } from 'react'
import { stampCounts } from '../lib/evidence'
import { useEvidence } from '../store/evidence'
import { useStudio } from '../store/studio'
import type { EvidenceStats, RunEvent } from '../types'
/**
* 报告节点卡上的章:「引用 N · 无证据 M」,数据取这张图最近一次运行的 report.checked。
*
* - 画布上挂着一次运行(刚跑的、从记录页回放的):就用它的事件,跟着 report.checked 实时落下;回放时
* 游标还没走到那一刻不画
* - 没挂运行:取这张工作流最近一次运行的证据图(只认封存范围内的 report.checked)
* - 两边都没有这个节点的核对(没跑过、跑到一半、老后端):不画章
*/
export interface ReportStamp {
cited: number
none: number
stats: EvidenceStats
claims?: string | null
/** report.checked 落下的时刻(毫秒,和航迹同一条时间轴);取自证据图的没有 */
at: number | null
runId: string | null
}
interface Checked { stats: EvidenceStats; claims?: string | null; at: number | null; event: RunEvent }
const checkedCache = new WeakMap<RunEvent[], Map<string, Checked>>()
/** 同一条事件给出同一个对象:选择器返回稳定引用,别的事件进来时报告卡不跟着重渲染 */
const byEvent = new WeakMap<RunEvent, Checked>()
/** 这次运行里每个报告节点最后一次核对,按事件数组缓存(每条事件都会把所有卡片的选择器跑一遍) */
function checkedOf(events: RunEvent[]): Map<string, Checked> {
const hit = checkedCache.get(events)
if (hit) return hit
const out = new Map<string, Checked>()
for (const e of events) {
if (e.type !== 'report.checked' || !e.node_id || !e.data?.stats || typeof e.data.stats !== 'object') continue
let c = byEvent.get(e)
if (!c) {
c = { stats: e.data.stats, claims: e.data.claims, at: typeof e.ts === 'number' ? e.ts * 1000 : null, event: e }
byEvent.set(e, c)
}
out.set(e.node_id, c)
}
checkedCache.set(events, out)
return out
}
export function useReportStamp(nodeId: string, enabled: boolean): ReportStamp | null {
const workflowId = useStudio((s) => s.workflow?.id ?? null)
const runId = useStudio((s) => s.run?.id ?? null)
const live = useStudio((s) => (enabled && s.run ? checkedOf(s.events).get(nodeId) ?? null : null))
const latest = useEvidence((s) => (enabled && workflowId ? s.latest[workflowId] : undefined))
const loadLatest = useEvidence((s) => s.loadLatest)
const dropLatest = useEvidence((s) => s.dropLatest)
// 没挂运行时才去取最近一次运行;挂着运行时它就是「最近一次」,之前取的那份随之作废——
// 「清除」把运行摘下来(或者换一张图再回来)时重新取,不拿这次运行之前的旧统计冒充「最近一次」
useEffect(() => {
if (!enabled || !workflowId) return
if (runId) dropLatest(workflowId)
else void loadLatest(workflowId)
}, [enabled, workflowId, runId, loadLatest, dropLatest])
if (!enabled) return null
if (runId) {
if (!live) return null
const counts = stampCounts(live.stats, live.claims)
return counts ? { ...counts, stats: live.stats, claims: live.claims, at: live.at, runId } : null
}
const data = latest?.status === 'ok' ? latest.data : undefined
const report = data?.reports[nodeId]
if (!report?.stats) return null
const counts = stampCounts(report.stats, report.claims)
return counts ? { ...counts, stats: report.stats, claims: report.claims, at: null, runId: data?.runId ?? null } : null
}
+39 -1
View File
@@ -26,8 +26,11 @@ import { EvidenceDoc, EvidenceField } from '../run/EvidenceDoc'
import { Markdown } from '../run/Markdown'
import evidenceFixture from '../run/__tests__/evidence-doc.json'
import evidenceQuery from '../run/__tests__/evidence-query.json'
import evidenceEntity from '../run/__tests__/evidence-entity.json'
import { EvidenceGuessView } from '../run/EvidenceGuess'
import { IssuanceBanner } from '../run/AssistantStream'
import mdPinned from '../run/__tests__/markdown-pinned.json'
import type { EvidenceDocData } from '../types'
import type { EvidenceDocData, EvidenceGraph } from '../types'
import '../index.css'
/**
@@ -189,6 +192,8 @@ const EV_DOC = evidenceFixture.doc as unknown as EvidenceDocData
/** 第二期:查询单元格、整表、口径卡来源(evidence-query.json,compose_doc 真跑出来的) */
const EV_QUERY = evidenceQuery.doc as unknown as EvidenceDocData
const EV_LEGACY = evidenceFixture.legacy
/** 第三期:表名字段名、可疑实体、核对不了的名字、逐字引文(evidence-entity.json,compose_doc 真跑出来的) */
const EV_ENTITY = evidenceEntity.doc as unknown as EvidenceDocData
/** 同一份旧契约的出具,去掉位置:老运行就是这样,只能按字符串标 */
const LEGACY_LOOSE = {
matched: EV_LEGACY.matched.map(({ start: _s, end: _e, ...m }) => m),
@@ -269,6 +274,38 @@ function EvidenceOnly() {
<EvidenceDoc doc={EV_QUERY} artifact={evidenceQuery.doc_artifact} runId={evidenceQuery.run_id} label="answer" dense tally />
</div>
</div>
{/*
第三期:表名字段名(反引号里的、标记的、自动链接的)、可疑实体、核对不了的名字、逐字引文。
片段接口由 check-evidence 用 page.route 伪造(按 runId 认出这一份夹具)
*/}
<div className="flex flex-col gap-4" id="evidence-entity-demo">
<div id="evidence-entity" className="rounded-lg border p-3">
<div className="mb-1 text-2xs text-faint">表名字段名、可疑实体、逐字引文</div>
<EvidenceDoc doc={EV_ENTITY} artifact={evidenceEntity.doc_artifact} runId={evidenceEntity.run_id} label="answer" tally />
</div>
<div id="evidence-entity-narrow" className="rounded-lg border bg-panel p-2.5" style={{ width: 360, maxWidth: '100%' }}>
<EvidenceDoc doc={EV_ENTITY} artifact={evidenceEntity.doc_artifact} runId={evidenceEntity.run_id} label="answer" dense tally />
</div>
{/* 没有契约的旧运行:按数值猜的候选(guess_sources 真跑出来的),默认折叠 */}
<div id="evidence-guess" className="rounded-lg border p-3">
<div className="mb-1 text-2xs text-faint">旧运行 · 按数值猜的可能来源</div>
<EvidenceGuessView guesses={evidence.guessesOf(evidenceEntity.legacy.graph as unknown as EvidenceGraph)} />
</div>
{/* 出具横幅那一行:数字都有出处,可疑名字、没挂依据的结论句也要写出来,可疑名字也给「定位下一处」 */}
<div id="issuance-line" className="flex flex-col gap-2 rounded-lg border p-3">
{([['suspect', 3, 2, 1], ['clean', 3, 0, 0], ['names', 0, 1, 0]] as const).map(([key, total, suspect, claims]) => (
<div key={key} data-issuance-case={key}>
<IssuanceBanner
issuance={{ tier: 'degraded', matched_numbers: 3, metrics_checked: 2,
...(claims ? { claims: { policy: 'require_citation', uncited_claims: claims } } : {}) }}
evidence={{
counts: { total, cited: total, none: 0, other: 0, hidden: 0, structural: 0, noSegment: 0, suspect, unverified: 0 },
onNext: () => { const w = window as any; w.__issuanceNext = (w.__issuanceNext ?? 0) + 1 },
}} />
</div>
))}
</div>
</div>
<div id="evidence-long" className="rounded-lg border p-3">
<div className="mb-1 text-2xs text-faint">长报告:{EV_LONG.blocks.length} 块,块级 content-visibility,超过 3000 字先折叠</div>
<EvidenceDoc doc={EV_LONG} label="长报告" tally />
@@ -561,6 +598,7 @@ function Harness() {
lib: { format, status, keys, terms, errors, explain, health, actor, validation, evidence },
evidenceFixture,
evidenceQuery,
evidenceEntity,
}
createRoot(document.getElementById('root')!).render(
+27
View File
@@ -976,6 +976,14 @@
font-weight: 500;
letter-spacing: 0;
}
/* 报告节点的章(引用 N · 无证据 M)放在卡片下沿:上沿左边是血缘标签(产出 / 引用),右边是校验角标 */
.nc-stamp.is-evidence {
top: auto;
/* 只压住卡片下沿 4px:卡片最后一行(校验提示、耗时)不被章挡住 */
bottom: -17px;
right: 14px;
letter-spacing: 0;
}
.nc-stamp.is-formal { color: var(--st-done); }
.nc-stamp.is-degraded { color: var(--st-waiting); }
.nc-stamp.is-withheld { color: var(--st-failed); }
@@ -1596,6 +1604,25 @@
.ev-seg[data-ev-line="solid"] { text-decoration-style: solid; }
/* 点状线比实线细碎,同样 1px 在小字号下几乎看不见 */
.ev-seg[data-ev-line="dotted"] { text-decoration-style: dotted; text-decoration-thickness: 1.5px; }
/* 引文:前面挂一个引号(字形那一条通道),颜色随状态——有出处的绿、原话对不上的琥珀。
引号是装饰,读屏从 aria-label 里听到「引文」,这里的替代文字给空串 */
.ev-seg[data-ev-kind="quote"]::before {
content: "“";
content: "“" / "";
margin-right: 1px;
color: var(--ev-glyph, currentColor);
font-weight: 600;
}
/* 旧运行的候选:比无证据的点状线更淡、更细,绝不像确定性的实线。正文里的候选是 Markdown 的标记(.ev-mark),
不是按钮:旧答案保留原来的版式,候选的明细在下面的清单里逐条列出 */
.ev-seg[data-ev-state="candidate"] { text-decoration-thickness: 1px; }
.ev-mark {
text-decoration-line: underline;
text-decoration-style: dotted;
text-decoration-thickness: 1px;
text-decoration-color: var(--ev-line, currentColor);
text-underline-offset: 3px;
}
.ev-seg:hover,
.ev-seg[aria-expanded="true"] { background: var(--ev-soft, transparent); }
.ev-seg:focus-visible,
+427 -13
View File
@@ -7,16 +7,21 @@
* 「未裁判」借 --st-cancelled(暗字色),「猜测」借 --st-idle(最淡的字色)——
* 绝不用确定性的绿,模型的判断不能长得像系统核对过的事实。
*
* 本期(数字层)实际只会出现两种:确定性(引用解析成功)和无证据(裸数字、引用
* 不存在)。其余几种先把外观定下,后续期接上时不用再各处补。
* 一期(数字层)只会出现两种:确定性(引用解析成功)和无证据(裸数字、引用不存在)。
* 三期多了表名字段名、逐字引文:有出处的实体、引文和数字同一套「有出处」线型(状态就是
* 确定性,另有 EVIDENCE_KIND_STYLE 给它们各自的字形和说法);可疑实体(可能是编造的
* 名字)和核对不了的名字是无证据的两个变体,线型同无证据、字形和文字各不相同;旧运行
* 按数值猜的候选用最淡的点状线。概率性的几种先把外观定下,四期接上时不用再各处补。
*/
import { EVIDENCE_KIND_LABEL, EVIDENCE_STATE_LABEL, EVIDENCE_TEXT, UPGRADE_POLICY_LABEL, evidenceTally } from './terms'
import {
EVIDENCE_AUDIT_TEXT, EVIDENCE_KIND_LABEL, EVIDENCE_STATE_LABEL, EVIDENCE_TEXT, UPGRADE_POLICY_LABEL, evidenceTally,
} from './terms'
import { NONE, formatNumber, shortId } from './format'
import type {
EvidenceBlock, EvidenceCaliberSource, EvidenceCaliberUpgrade, EvidenceDocData, EvidenceFieldRef, EvidenceGraph,
EvidenceInput, EvidenceLocator, EvidenceSeal, EvidenceSegment, EvidenceSegmentDetail, EvidenceStep, EvidenceUnit,
EvidenceViolation, ReviewResult,
EvidenceBlock, EvidenceCaliberSource, EvidenceCaliberUpgrade, EvidenceCandidate, EvidenceDocData, EvidenceFieldRef,
EvidenceGraph, EvidenceGuess, EvidenceInput, EvidenceLocator, EvidenceSeal, EvidenceSegment, EvidenceSegmentDetail,
EvidenceStats, EvidenceStep, EvidenceUnit, EvidenceViolation, ReviewResult,
} from '../types'
import type { MarkSpec } from '../run/Markdown'
@@ -100,6 +105,20 @@ export const EVIDENCE_STATE: Record<EvidenceStateCode, EvidenceStateMeta> = {
alert: false, phase: 3,
hint: '旧运行按数值猜的可能来源,不能当证据',
},
// 可疑实体:线型同无证据(点状),字形和文字另起,一眼分得出「没写出处」和「名字可能是编的」
suspect: {
code: 'suspect', label: EVIDENCE_STATE_LABEL.suspect, line: 'dotted', glyph: '?!',
color: 'var(--st-waiting)', decoration: 'var(--st-waiting)', soft: 'var(--st-waiting-soft)',
alert: true, phase: 3,
hint: '本次运行的表结构快照、查询用到的表、查询结果列里都没有这个名字,可能是编造的',
},
// 核对不了:表结构快照不全,找不到不等于不存在。只是标注,不进 n / N 的跳转,颜色压低
unverified: {
code: 'unverified', label: EVIDENCE_STATE_LABEL.unverified, line: 'dotted', glyph: '…',
color: 'var(--st-cancelled)', decoration: mix('--st-cancelled', 80), soft: 'var(--st-cancelled-soft)',
alert: false, phase: 3,
hint: '这个数据源的表太多,表结构快照只存了一部分:找不到这个名字,也说不准它不存在',
},
}
/** 图例、检查脚本按这个顺序列 */
@@ -109,17 +128,58 @@ export const EVIDENCE_STATES = Object.keys(EVIDENCE_STATE) as EvidenceStateCode[
* 片段在正文里的状态。文字、结构片段(行首符号、表格竖线)是 null:不画线、不进键盘顺序。
* 后端的 probabilistic 在裁判给出结论之前按「未裁判」画
*/
export function segmentState(seg: Pick<EvidenceSegment, 'kind' | 'state'>): EvidenceStateCode | null {
export function segmentState(
seg: Pick<EvidenceSegment, 'kind' | 'state'> & Partial<Pick<EvidenceSegment, 'issue' | 'cite'>>,
): EvidenceStateCode | null {
if (seg.kind === 'structural') return null
switch (seg.state) {
case 'deterministic': return 'deterministic'
case 'none': return 'none'
case 'none':
// 无证据的两个变体:可疑实体(反引号里发现的没有 cite,[[t:编造]] 的 cite 带 unknown)、核对不了
if (seg.issue === 'unknown_entity' || seg.cite?.unknown) return 'suspect'
if (seg.issue === 'unverified_entity' || seg.cite?.unverified) return 'unverified'
return 'none'
case 'probabilistic': return 'unjudged'
case 'candidate': return 'candidate'
default: return null
}
}
/**
* 有出处的片段里,实体和引文另有一层外观:线型、颜色和数字同一套「有出处」,字形和文字
* 各自不同——引文前面挂一个引号,实体写在反引号里的照行内代码画。四个通道同 EVIDENCE_STATE
*/
export type EvidenceKindCode = 'entity' | 'quote'
export interface EvidenceKindStyle {
code: EvidenceKindCode
line: EvidenceLine
glyph: string
color: string
label: string
hint: string
}
export const EVIDENCE_KIND_STYLE: Record<EvidenceKindCode, EvidenceKindStyle> = {
entity: {
code: 'entity', line: 'solid', glyph: '', color: 'var(--st-done)', label: '有出处 · 表或字段',
hint: '本次运行的表结构快照、查询用到的表或查询结果列里真实存在的名字',
},
quote: {
code: 'quote', line: 'solid', glyph: '“', color: 'var(--st-done)', label: '有出处 · 逐字引文',
hint: '在知识库检索命中的原文里逐字出现(空白归一化后比对)',
},
}
/** 片段是实体还是引文(状态另看 segmentState):其余返回 null */
export function segmentKind(seg: Pick<EvidenceSegment, 'kind'>): EvidenceKindCode | null {
return seg.kind === 'entity' ? 'entity' : seg.kind === 'quote' ? 'quote' : null
}
/** 片段给人看的名字:反引号里的实体去掉两个反引号 */
export function segName(seg: Pick<EvidenceSegment, 'text' | 'kind' | 'code' | 'name'>): string {
if (seg.kind === 'entity' && (seg.code || /^`[^`]+`$/.test(seg.text))) return seg.name ?? seg.text.replace(/^`|`$/g, '')
return seg.text
}
/** 一个句子(单元格、列表项)的正文:结构片段不算 */
export function unitText(unit: Pick<EvidenceUnit, 'segments'>): string {
return (unit.segments ?? []).filter((s) => s.kind !== 'structural').map((s) => s.text).join('')
@@ -144,10 +204,30 @@ export function sourceOf(seg: EvidenceSegment, doc?: Pick<EvidenceDocData, 'cata
if (cite.kind === 'input' || entry?.kind === 'input') {
return `${EVIDENCE_KIND_LABEL.input} ${entry?.locator?.field ?? cite.locator?.field ?? cite.ref ?? ''}`.trim()
}
if (cite.kind === 'table' || cite.kind === 'column' || entry?.kind === 'table' || entry?.kind === 'column') {
// 「表 orders · 出现在查询 Q1、Q2」:名字从目录取(大小写照库里的),查询编号是报告里的全局编号
const kind = (cite.kind ?? entry?.kind) === 'table' ? EVIDENCE_TEXT.table : EVIDENCE_TEXT.column
const name = entry?.alias ? String(entry.alias).replace(/^[tc]:/, '') : cite.ref ?? ''
const queries: string[] = Array.isArray(entry?.queries) ? entry.queries.map(String) : []
return [`${kind} ${name}`.trim(), queries.length ? EVIDENCE_TEXT.entityQueries(queries) : ''].filter(Boolean).join(' · ')
}
if (cite.kind === 'quote') {
const where = quoteWhere(cite.source)
return [`${EVIDENCE_KIND_LABEL.quote} ${cite.alias ?? ''}`.trim(), where].filter(Boolean).join(' · ')
}
const kind = cite.kind ? EVIDENCE_KIND_LABEL[cite.kind] : ''
return [kind, cite.alias ?? cite.ref].filter(Boolean).join(' ')
}
/** 引文出自哪:「出自「运营手册 · 退款」第 3 段」;只有文档 id 时写 id 的前几位 */
export function quoteWhere(src: { title?: string; document?: string; ordinal?: number } | null | undefined): string {
if (!src || typeof src !== 'object') return ''
const title = typeof src.title === 'string' && src.title.trim() ? src.title.trim()
: typeof src.document === 'string' && src.document ? shortId(src.document, 12) : ''
const ordinal = typeof src.ordinal === 'number' && Number.isFinite(src.ordinal) ? EVIDENCE_TEXT.quoteChunk(src.ordinal) : ''
return [title ? EVIDENCE_TEXT.quoteFrom(title) : '', ordinal].filter(Boolean).join(' ')
}
/**
* 查询快照里的位置怎么说:「第 6 行 · amount」「第 1–5 行 · amount」「第 1–5 行 · week、amount」。
* 行号给人看从 1 数(引用原文 Q3.r5 里的 r5 从 0 数,面板另把原文摆出来)
@@ -331,10 +411,16 @@ export function caliberUpgradeText(up: EvidenceCaliberUpgrade | null | undefined
return EVIDENCE_TEXT.caliberUpgrade(latest, policy)
}
/** 无证据的原因:裸数字,还是引用解析不了(带后端给的人话原因) */
/**
* 无证据的原因:裸数字、引用解析不了(带后端给的人话原因)、可疑实体、核对不了。
* 反引号里自动发现的可疑名字没有 cite,用固定的说法
*/
export function reasonOf(seg: EvidenceSegment): string {
if (seg.issue === 'uncited_number' || (!seg.cite && seg.kind === 'number')) return EVIDENCE_TEXT.uncited
return seg.cite?.reason ?? (seg.ref ? `引用 ${seg.ref} 解析不了` : '')
if (seg.cite?.reason) return seg.cite.reason
if (seg.issue === 'unknown_entity') return EVIDENCE_TEXT.suspect
if (seg.issue === 'unverified_entity') return EVIDENCE_TEXT.unverified
return seg.ref ? `引用 ${seg.ref} 解析不了` : ''
}
/**
@@ -344,8 +430,10 @@ export function reasonOf(seg: EvidenceSegment): string {
export function segmentLabel(seg: EvidenceSegment, doc?: Pick<EvidenceDocData, 'catalog'> | null): string {
const state = segmentState(seg)
if (!state) return seg.text
const why = state === 'none' ? reasonOf(seg) : sourceOf(seg, doc)
return `${seg.text},${EVIDENCE_STATE[state].label}${why ? `:${why}` : ''}`
const why = state === 'none' || state === 'suspect' || state === 'unverified' ? reasonOf(seg) : sourceOf(seg, doc)
// 可疑实体的原话本身就以「可能是编造的名字」收尾,状态那几个字不再重复一遍
const label = state === 'suspect' && why.includes(EVIDENCE_STATE.suspect.label) ? '' : EVIDENCE_STATE[state].label
return `${segName(seg)},${[label, why].filter(Boolean).join(':')}`
}
/** 在正文里画得出线的违规:指向的片段存在,而且不是结构片段 */
@@ -364,6 +452,9 @@ export function locatable(doc: EvidenceDocData, v: EvidenceViolation): EvidenceS
export interface EvidenceTally {
/** 数字总数:带引用的数字片段 + 裸数字(含列表序号、代码块标签里的) */
total: number
/** 可疑实体(可能是编造的名字)、核对不了的名字:数字那半句之外另说 */
suspect?: number
unverified?: number
cited: number
/** 没有出处的数字:总数 − 有出处(裸数字、解析不了的数字引用)。横幅上「无证据 M」的 M */
none: number
@@ -409,7 +500,11 @@ export function docTally(doc: EvidenceDocData): EvidenceTally {
: uncited != null && unresolved != null
? Math.max(0, uncited + unresolved - none)
: segs.filter((s) => segmentState(s) === 'none' && s.kind !== 'number').length
return { total, cited, none, other, hidden: hidden.length, structural, noSegment: hidden.length - structural }
// 三期:可疑实体、核对不了的名字。stats 缺这两个键(老文档、没开实体层)时按违规清单数
const suspect = num(st.unknown_entities) ?? all.filter((v) => v.code === 'unknown_entity').length
const unverified = num(st.unverified_entities) ?? all.filter((v) => v.code === 'unverified_entity').length
return { total, cited, none, other, hidden: hidden.length, structural, noSegment: hidden.length - structural,
suspect, unverified }
}
/**
@@ -431,6 +526,8 @@ export function statsTally(stats: EvidenceDocData['stats'] | null | undefined):
export function tallySummary(t: EvidenceTally, keys: boolean): string {
return [
evidenceTally(t.cited, t.total, t.other),
t.suspect ? `${formatNumber(t.suspect)} 个${EVIDENCE_STATE.suspect.label}` : '',
t.unverified ? `${formatNumber(t.unverified)} 个名字${EVIDENCE_STATE.unverified.label}` : '',
t.structural ? EVIDENCE_TEXT.structuralCount(t.structural) : '',
t.noSegment ? EVIDENCE_TEXT.noSegmentCount(t.noSegment) : '',
keys ? EVIDENCE_TEXT.keysHint : '',
@@ -619,3 +716,320 @@ export function codePointIndex(text: string): (cp: number) => number {
export function segmentsOf(blocks: EvidenceBlock[]): EvidenceSegment[] {
return blocks.flatMap((b) => (b.units ?? []).flatMap((u) => u.segments ?? []))
}
// -------------------------------------------------------------------------
// 三期:实体步骤、引文步骤、画布上的证据路径
// -------------------------------------------------------------------------
/**
* 可疑实体的「最接近的已知名字」:接口给目录键(c:orders.amount),或对象 {alias, kind, name, table?}
* (字段的 name 只是字段名,带着 table 时写成 表.字段)。统一成去掉 t: / c: 前缀的名字,最多 3 个
*/
export function closestNames(list: unknown): string[] {
if (!Array.isArray(list)) return []
const names = list.map((x) => {
if (typeof x === 'string') return x
if (!x || typeof x !== 'object') return ''
const o = x as { alias?: unknown; name?: unknown; table?: unknown }
if (typeof o.alias === 'string' && o.alias) return o.alias
const name = typeof o.name === 'string' ? o.name : ''
return typeof o.table === 'string' && o.table && name && !name.includes('.') ? `${o.table}.${name}` : name
}).map((n) => n.replace(/^[tc]:/, '').trim()).filter(Boolean)
return [...new Set(names)].slice(0, 3)
}
/** 实体的来历一行一句:「表结构快照」「查询 Q2 · 查询 SQL 用到的表」「查询 Q1 · 查询结果列」 */
export function entitySources(list: unknown): { kind: string; text: string; truncated: boolean }[] {
if (!Array.isArray(list)) return []
const out: { kind: string; text: string; truncated: boolean }[] = []
for (const src of list) {
if (!src || typeof src !== 'object') continue
const kind = String((src as any).kind ?? '')
const alias = typeof (src as any).alias === 'string' ? (src as any).alias : ''
const what = EVIDENCE_TEXT.entitySource[kind] ?? kind
const text = [alias ? EVIDENCE_TEXT.query(alias) : '', what].filter(Boolean).join(' · ')
if (text && !out.some((o) => o.text === text)) out.push({ kind, text, truncated: !!(src as any).truncated })
}
return out
}
/**
* 引文在原文里的位置,切成前、中、后三截给面板高亮。match 是码点偏移(后端 Python 的下标),
* 先换成 JS 的 UTF-16 下标。对不上(越界、那一截的字和引文对不上)时按引文在原文里找一次,
* 再找不到就不高亮——宁可不标,不标错地方。原文很长时前后各留 context 个字,多的写省略号
*/
export function quoteWindow(original: string, match: { start?: number; end?: number } | null | undefined,
quote: string, context = 120): { before: string; hit: string; after: string; cutBefore: boolean; cutAfter: boolean } | null {
if (!original) return null
const norm = (t: string) => t.replace(/\s+/g, '')
const at = codePointIndex(original)
let a = typeof match?.start === 'number' ? at(match.start) : -1
let b = typeof match?.end === 'number' ? at(match.end) : -1
const fits = a >= 0 && b > a && b <= original.length && (!quote || norm(original.slice(a, b)) === norm(quote))
if (!fits) {
const i = quote ? original.indexOf(quote) : -1
if (i < 0) return null
a = i
b = i + quote.length
}
const from = Math.max(0, a - context)
const to = Math.min(original.length, b + context)
return { before: original.slice(from, a), hit: original.slice(a, b), after: original.slice(b, to),
cutBefore: from > 0, cutAfter: to < original.length }
}
/**
* 点开一个片段时,画布上该亮哪些节点:产出证据的(查询、口径卡、检索)和用到它的(报告)。
* 先按文档目录认(打开就有),证据链取回来后再补上链里每一步的节点(口径卡的输入、查询)
*/
export function evidenceTrace(
seg: EvidenceSegment, doc: Pick<EvidenceDocData, 'catalog' | 'node_id'>, chain: EvidenceStep[] = [],
): { producers: string[]; consumers: string[] } {
const catalog = doc.catalog ?? {}
const producers = new Set<string>()
const add = (id: unknown) => { if (typeof id === 'string' && id) producers.add(id) }
const entry = seg.cite?.alias ? catalog[seg.cite.alias] : undefined
add(entry?.node_id)
// 实体:它出现在哪几次查询里,那几次查询是谁跑的
const aliases = [
...(Array.isArray(entry?.queries) ? entry.queries : []),
...(Array.isArray(entry?.sources) ? entry.sources.map((s: any) => s?.alias) : []),
]
for (const a of aliases) if (typeof a === 'string') add(catalog[a]?.node_id)
for (const step of chain) {
add(step.node_id)
for (const inp of step.inputs ?? []) add(inp.node_id)
for (const q of step.queries ?? []) if (typeof q === 'string') add(catalog[q]?.node_id)
}
const consumers = doc.node_id ? [doc.node_id] : []
for (const c of consumers) producers.delete(c)
return { producers: [...producers], consumers }
}
// -------------------------------------------------------------------------
// 报告节点卡上的章
// -------------------------------------------------------------------------
/**
* 章上的两个数:「引用 N」是有出处的片段(数字、值、表名字段名、引文),「无证据 M」是没出处的
* 数字、解析不了的引用、可疑实体,结论句策略为 require_citation 时再加上没挂依据的结论句。
* 统计缺了数字那两项(别的版本的后端)返回 null,不画章
*/
export function stampCounts(stats: EvidenceStats | null | undefined, claims?: string | null):
{ cited: number; none: number } | null {
const t = statsTally(stats)
if (!t) return null
const cited = t.cited + (num(stats?.values) ?? 0) + (num(stats?.entities) ?? 0) + (num(stats?.quotes) ?? 0)
const none = t.none + t.other + (num(stats?.unknown_entities) ?? 0)
+ (claims === 'require_citation' ? num(stats?.uncited_claims) ?? 0 : 0)
return { cited, none }
}
// -------------------------------------------------------------------------
// 旧运行的猜测
// -------------------------------------------------------------------------
/** 证据图里的猜测(legacy_text):可能是一份、一个列表、或按字段分的对象。认不出的丢掉 */
export function guessesOf(graph: EvidenceGraph | null | undefined): EvidenceGuess[] {
if (!graph) return []
const isGuess = (g: unknown): g is EvidenceGuess =>
!!g && typeof g === 'object' && Array.isArray((g as EvidenceGuess).segments)
const from = graph.guess ?? (graph.mode === 'legacy_text' ? graph.legacy : null)
if (!from) return []
if (isGuess(from)) return [from]
if (Array.isArray(from)) return from.filter(isGuess)
if (typeof from === 'object') {
// {字段名: guess} 或 {fields: {…}}
// {note, fields: [{field, markdown, segments, stats}]}(后端 legacy_text 的样子),或 {字段名: guess}
const note = typeof (from as any).note === 'string' ? (from as any).note : undefined
const inner = (from as any).fields && typeof (from as any).fields === 'object' ? (from as any).fields : from
if (Array.isArray(inner)) return inner.filter(isGuess).map((g) => ({ note, ...g }))
return Object.entries(inner).filter(([, g]) => isGuess(g)).map(([field, g]) => ({ note, field, ...(g as EvidenceGuess) }))
}
return []
}
/** 一个候选说一句:「查询 Q1 · 第 1 行 · gmv」「口径卡指标 销售额」,差值不为 0 时补一句 */
export function candidateText(c: EvidenceCandidate): string {
const loc = c.locator ?? {}
let where = ''
if (c.kind === 'metric' || typeof loc.metric === 'string') {
const name = c.name ?? loc.metric ?? c.ref?.replace(/^m:/, '') ?? ''
where = `${EVIDENCE_KIND_LABEL.metric} ${name}`.trim()
} else {
const alias = c.alias ?? (typeof c.ref === 'string' ? c.ref.split('.')[0] : '')
where = [alias ? EVIDENCE_TEXT.query(alias) : EVIDENCE_KIND_LABEL.cell, locatorText(loc)].filter(Boolean).join(' · ')
}
const diff = typeof c.diff === 'number' && Number.isFinite(c.diff) && c.diff !== 0
? EVIDENCE_TEXT.guessDiff(evidenceValue(Math.abs(c.diff))) : ''
return [where, c.rendered && c.rendered !== NONE ? `= ${c.rendered}` : '', diff].filter(Boolean).join(' ')
}
// -------------------------------------------------------------------------
// 记录页的审计表
// -------------------------------------------------------------------------
/** 分组的键和后端 /evidence/audit 一致:有出处 / 无证据 / 可疑实体 / 旧运行猜测 */
export type AuditGroup = 'cited' | 'none' | 'suspicious' | 'candidate'
/** 表里分组的顺序:有问题的在前(正常态安静,异常态醒目)。导出的文件按后端的顺序 */
export const AUDIT_GROUPS: readonly AuditGroup[] = ['none', 'suspicious', 'cited', 'candidate']
export type AuditFilter = 'all' | 'problems'
/** 「只看无证据 / 可疑实体」对应的分组,也是导出时 ?groups= 的值 */
export const PROBLEM_GROUPS: readonly AuditGroup[] = ['none', 'suspicious']
export interface AuditRow {
/**
* 行的键,一张表里各不相同(React 的 key,也是键盘走表时找行的依据):片段的行 `报告:片段`;
* 违规、没挂依据的结论句 `报告:片段:问题:位置`——同一段文字里可以有好几条同样的违规(粗体、链接里
* 两个可疑名字),片段和问题都一样,只有位置不同。万一还撞上,后来的带 `#序号`
*/
key: string
/** 报告节点 id;旧运行的猜测是成果字段名 */
report: string
/** 能在正文里打开的片段 id */
seg?: string
text: string
state: EvidenceStateCode
group: AuditGroup
/** number / value / entity / quote / claim / violation */
kind: string
/** 出处(有出处的)或原因(没有的) */
source: string
ref?: string
/** 所在的句子 */
sentence: string
/** 这件证据在不在封存范围内(封存核对也通过);没有证据的是 null */
sealed: boolean | null
}
const GROUP_OF: Record<EvidenceStateCode, AuditGroup> = {
deterministic: 'cited', none: 'none', suspect: 'suspicious', unverified: 'suspicious', candidate: 'candidate',
supported: 'cited', partial: 'none', unsupported: 'none', unjudged: 'cited', connective: 'cited',
}
const AUDIT_GROUP_SET = new Set<string>(['cited', 'none', 'suspicious', 'candidate'])
/** 反引号里的名字去掉反引号(后端审计行的 text 照正文原样带着) */
const bareName = (kind: string, text: string) => (kind === 'entity' && /^`[^`]+`$/.test(text) ? text.slice(1, -1) : text)
/**
* 后端 /evidence/audit 的 JSON → 表里的行。形状照 api/evidence.py(groups[].rows[]),缺字段的宽容:
* 状态按 state + issue 认(可疑实体、核对不了是无证据的两个变体),出处写 evidence,原因写 note
*/
export function auditFromApi(body: unknown): AuditRow[] {
const groups = body && typeof body === 'object' && Array.isArray((body as any).groups) ? (body as any).groups : []
const rows: AuditRow[] = []
const taken = new Map<string, number>()
/** 后端每条违规一行:同一片段、同一个 code 的违规可以有好几条,键带上位置;还撞的按出现顺序编号 */
const unique = (base: string) => {
const n = taken.get(base) ?? 0
taken.set(base, n + 1)
return n ? `${base}#${n}` : base
}
groups.forEach((g: any, gi: number) => {
for (const [ri, r] of (Array.isArray(g?.rows) ? g.rows : []).entries()) {
if (!r || typeof r !== 'object') continue
const kind = String(r.kind ?? '')
const state = segmentState({ kind, state: r.state ?? 'none', issue: r.issue ?? undefined })
?? (r.group === 'candidate' ? 'candidate' : 'none')
const group: AuditGroup = AUDIT_GROUP_SET.has(r.group) ? r.group : AUDIT_GROUP_SET.has(g?.key) ? g.key : GROUP_OF[state]
const report = String(r.report ?? r.field ?? '')
const seg = typeof r.segment === 'string' && r.segment ? r.segment : undefined
const at = Array.isArray(r.span) && Number.isInteger(r.span[0]) ? String(r.span[0]) : `${gi}.${ri}`
rows.push({
key: unique(seg && kind !== 'violation' && kind !== 'claim' ? `${report}:${seg}`
: `${report}:${seg ?? '-'}:${r.issue ?? kind}:${at}`),
report, seg: kind === 'claim' ? undefined : seg,
text: bareName(kind, String(r.text ?? '')) || NONE, state, group, kind,
source: String((group === 'cited' || group === 'candidate') && r.evidence ? r.evidence : r.note ?? r.evidence ?? ''),
ref: typeof r.ref === 'string' ? r.ref : undefined,
sentence: String(r.sentence ?? '').trim(),
sealed: typeof r.sealed === 'boolean' ? r.sealed : null,
})
}
})
return rows
}
/**
* 后端没有审计接口时(老后端),按正文这份文档自己拼:每个有状态的片段一行(连接性文字、结构片段不进),
* 另加画不了线的违规(列表序号、代码块标签里的数字,句末依据里写错的引用)——键盘用户在表里能看全每一处。
* 封存状态按证据图里同一份报告、同一个别名的 sealed 取;没有证据图时是 null
*/
export function auditRows(doc: EvidenceDocData, opts: { report?: string; graph?: EvidenceGraph | null } = {}): AuditRow[] {
const report = opts.report ?? doc.node_id ?? ''
const sealedOf = (alias?: string): boolean | null => {
if (!alias || !opts.graph?.evidence) return null
const hit = opts.graph.evidence.find((e) => e?.alias === alias && (!e.report || !report || e.report === report))
return typeof hit?.sealed === 'boolean' ? hit.sealed : null
}
const rows: AuditRow[] = []
const shown = new Set<string>()
for (const block of doc.blocks ?? []) {
for (const unit of block.units ?? []) {
const sentence = unitText(unit).trim()
for (const seg of unit.segments ?? []) {
const state = segmentState(seg)
if (!state) continue
shown.add(seg.id)
const bad = state === 'none' || state === 'suspect' || state === 'unverified'
rows.push({
key: `${report}:${seg.id}`, report, seg: seg.id, text: segName(seg), state, group: GROUP_OF[state],
kind: seg.kind, source: bad ? reasonOf(seg) : sourceOf(seg, doc), ref: seg.ref, sentence,
sealed: bad ? null : sealedOf(seg.cite?.alias),
})
}
}
}
;(doc.violations ?? []).forEach((v, i) => {
if (!GAP_CODES.has(v.code) && v.code !== 'unknown_entity' && v.code !== 'unverified_entity') return
// 画得出线的违规已经是上面的一行;粗体、链接里的可疑名字没切片段(指着一段文字),也在这里补上
if (v.segment && shown.has(v.segment)) return
const state: EvidenceStateCode = v.code === 'unknown_entity' ? 'suspect' : v.code === 'unverified_entity' ? 'unverified' : 'none'
rows.push({
key: `${report}:v${i}`, report, text: v.text ?? (v.ref ? `[[${v.ref}]]` : NONE), state, group: GROUP_OF[state],
// 「正文里画不了线」由表格按行能不能打开来写(后端的行也一样),这里只放违规本身的话
kind: 'violation', source: v.message ?? '', ref: v.ref,
sentence: (v.context ?? '').trim(), sealed: null,
})
})
return rows
}
/** 旧运行的猜测 → 表里的行:有候选的数字进「旧运行猜测」,没有的进「无证据」(和后端一致) */
export function guessRows(guess: EvidenceGuess): AuditRow[] {
const field = guess.field ?? ''
return (guess.segments ?? []).filter((s) => s.kind === 'number').map((s) => {
const hit = s.state === 'candidate' && !!s.candidates?.length
return {
key: `${field}:${s.id}`, report: field, text: s.text, state: hit ? 'candidate' as const : 'none' as const,
group: hit ? 'candidate' as const : 'none' as const, kind: 'number',
source: hit ? (s.candidates ?? []).map(candidateText).join(';') : EVIDENCE_TEXT.guessNone,
sentence: '', sealed: null,
}
})
}
/** 按组归拢;problems 只留无证据和可疑实体两组。空组不返回 */
export function groupRows(rows: AuditRow[], filter: AuditFilter = 'all'): { group: AuditGroup; rows: AuditRow[] }[] {
const keep = filter === 'problems' ? new Set<AuditGroup>(PROBLEM_GROUPS) : null
return AUDIT_GROUPS.filter((g) => !keep || keep.has(g))
.map((group) => ({ group, rows: rows.filter((r) => r.group === group) }))
.filter((g) => g.rows.length > 0)
}
/** 后端没有导出接口时按表里的行导出的 CSV:中文表头,逗号、引号、换行转义,公式字符开头的格子前面加 ' */
export function auditCsv(rows: AuditRow[]): string {
const cell = (v: unknown) => {
let t = v == null ? '' : String(v)
// 电子表格会把 = + - @ 开头的格子当公式算:报告里的字来自模型,不能照原样写进去(纯数字除外)
if (/^[=+\-@\t\r]/.test(t) && !/^[+-]?[\d,.]+%?$/.test(t)) t = `'${t}`
return /[",\n\r]/.test(t) ? `"${t.replace(/"/g, '""')}"` : t
}
const c = EVIDENCE_AUDIT_TEXT.cols
const head = ['分组', c.report, '片段', c.text, c.state, c.source, '引用', c.sentence, c.seal]
const seal = (v: boolean | null) => (v == null ? '' : v ? '是' : '否')
return [head, ...rows.map((r) => [
EVIDENCE_AUDIT_TEXT.groups[r.group], r.report, r.seg ?? '', r.text, EVIDENCE_STATE[r.state].label, r.source,
r.ref ?? '', r.sentence, seal(r.sealed),
])].map((r) => r.map(cell).join(',')).join('\r\n')
}
+111 -3
View File
@@ -144,6 +144,8 @@ export const EVIDENCE_STATE_LABEL = {
none: '无证据',
connective: '连接性文字',
candidate: '猜测的来源',
suspect: '可能是编造的名字',
unverified: '核对不了',
} as const
/**
@@ -188,9 +190,9 @@ export const EVIDENCE_TEXT = {
input: '运行输入',
seal: '封存',
sealOk: '已封存 · 核对一致',
sealDocOk: '报告文档已封存 · 核对一致(这个数字本身没有证据可封存)',
/** 片段接口取不到、封存状态是从整次运行的证据图查的:只核对了文档在封存范围内,这个数的链没逐项核对 */
sealDocChain: '报告文档已封存 · 核对一致(这个数字的证据链这次没取到,没有逐项核对)',
sealDocOk: '报告文档已封存 · 核对一致(这一段本身没有证据可封存)',
/** 片段接口取不到、封存状态是从整次运行的证据图查的:只核对了文档在封存范围内,这一段的链没逐项核对 */
sealDocChain: '报告文档已封存 · 核对一致(这一段的证据链这次没取到,没有逐项核对)',
sealBad: '已封存 · 核对不一致',
sealOpen: '尚未封存',
sealOutside: '不在封存范围内',
@@ -264,8 +266,114 @@ export const EVIDENCE_TEXT = {
`口径卡「${caliber || '—'}」${version},来自「${workflow}」${wfVersion}`,
caliberUpgrade: (latest: string | null, policy: string) =>
`${latest ? `上游已有 ${latest}` : '上游有新版本'},按「${policy}」处置`,
// ---- 第三期:表名字段名、逐字引文、旧运行的猜测 ----
entity: '表结构',
table: '表',
column: '字段',
entityQueries: (aliases: string[]) => `出现在查询 ${aliases.join('、')}`,
entityNoQuery: '这次的查询里没有用到它,只在表结构快照里',
syncedAt: (at: string) => `表结构快照同步于 ${at}`,
snapshotPartial: '库里的表太多,表结构快照只存了一部分',
columnType: '字段类型',
ownerTables: (tables: string[]) => `好几张表都有这个字段:${tables.join('、')}`,
entitySource: { schema: '表结构快照', sql: '查询 SQL 用到的表', result: '查询结果列' } as Record<string, string>,
entityPending: '正在取表结构…',
entityMissing: '字段类型、同步时间要从证据接口取,这一次没取到',
/** 反引号里写了个名字,哪里都没有(没有 cite 时的原因;有 cite 的用后端给的原话) */
suspect: '本次运行的表结构快照、查询用到的表、查询结果列里都没有这个名字,可能是编造的名字',
unverified: '表结构快照只存了一部分,查询用到的表、查询结果列里也没有这个名字,核对不了它存不存在',
suspectTag: '可疑名字',
/** 出具横幅那一行:报告要求结论句挂依据(claims: require_citation)时,没挂的有几句 */
uncitedClaimsTag: '没挂依据的结论句',
closest: '最接近的已知名字',
checked: (schemas: number, queries: number) =>
`查过 ${formatNumber(schemas)} 份表结构快照、${formatNumber(queries)} 次查询`,
tableColumns: (n: number, view: boolean) => `${view ? '视图' : '表'} · ${formatNumber(n)} 个字段`,
columnTypes: (types: Record<string, string>) =>
`各表的类型:${Object.entries(types).map(([t, v]) => `${t} ${v}`).join('、')}`,
quoteBad: '按记下的位置从原文里切出来,和这句引文对不上:原文可能被改过',
quoteCollection: (c: string) => `知识库「${c}」`,
closestNone: '没有相近的已知名字',
quote: '逐字引文',
quoteFrom: (title: string) => `出自「${title}」`,
quoteChunk: (n: number) => `第 ${formatNumber(n)} 段`,
quoteHit: '引文在原文里的位置(高亮)',
quotePending: '正在取原文…',
quoteMissing: '原文要从证据接口取,这一次没取到',
quoteMiss: '原文里找不到这句话:写作者写的不是一字不差的原话',
quoteHash: '检索快照取回时哈希对不上,疑似被改过:原文不能当证据',
/** 旧运行:按数值猜的候选 */
guessTitle: '猜测的来源',
guessNote: '猜测的来源,不能当证据:按数值在这次运行已封存的查询结果和口径卡里找相同的值,同值的巧合很多',
guessCount: (guessed: number, numbers: number) =>
numbers ? `${formatNumber(numbers)} 个数字里 ${formatNumber(guessed)} 个找到了可能的来源` : '答案里没有数字',
guessNone: '没找到数值相同的格子',
guessDiff: (d: string) => `相差 ${d}`,
guessCandidates: '可能来自',
} as const
/**
* 记录页「证据」页签:报告、常驻面板、审计表。审计表按状态分组,四组的说法和片段状态同一套
*/
export const EVIDENCE_AUDIT_TEXT = {
tab: '证据',
tabTitle: '报告里每一段的出处:左边报告、右边证据、下面整张清单,可以只看有问题的、导出',
title: '证据清单',
groups: {
none: '无证据',
suspicious: '可疑实体',
cited: '有出处',
candidate: '旧运行猜测',
} as Record<string, string>,
groupHint: {
none: '裸数字、解析不了的引用、没挂依据的结论句',
suspicious: '表名、字段名在本次运行的表结构和查询里找不到',
cited: '系统从证据里取值、核对过的片段',
candidate: '按数值猜的可能来源,不能当证据,默认收起',
} as Record<string, string>,
filterLabel: '清单筛选',
filterAll: '全部',
filterProblems: '只看无证据 / 可疑实体',
cols: { text: '片段', state: '状态', source: '出处 / 原因', sentence: '所在的句子', report: '报告', seal: '封存' },
sealIn: '在封存范围内',
sealOut: '不在封存范围内',
sealNa: '—',
exportJson: '导出 JSON',
exportCsv: '导出 CSV',
exported: (name: string) => `已导出 ${name}`,
exportLocal: '后端没有导出接口:按页面上这张清单导出(没有经过后端的封存核对)',
fallbackUnsupported: '后端还没有审计接口:清单按页面上的报告拼出来,每行的封存状态取自证据图',
fallbackError: (why: string) => `审计接口没取到(${why}):清单按页面上的报告拼出来`,
empty: '这一组没有片段',
emptyProblems: '没有无证据、可疑实体的片段',
hidden: '正文里画不了线,只在这里列出',
hiddenWhy: '这一处在列表序号、代码块标签、粗体或链接里,或者在句末依据里:正文里没有能画线、能点开的片段,只在这张清单里列出',
keys: '在清单里用上下方向键逐行走,回车在右边打开这一段的证据',
panelIdle: '点报告里的片段,或清单里的一行,这里显示它的出处',
sealTitle: '封存核对',
sealOk: (seq?: number) => `已封存 · 核对一致${seq != null ? `(封存到第 ${formatNumber(seq)} 条事件)` : ''}`,
sealBad: '已封存 · 核对不一致:封存之后有事件被改过、删过或插过',
sealOpen: '尚未封存:运行还没收尾,证据还可能变',
sealUnknown: '封存状态拿不到',
docOk: (node: string) => `报告「${node}」的文档哈希一致`,
docBad: (node: string) => `报告「${node}」的文档和哈希对不上,疑似被改过`,
docGone: (node: string) => `报告「${node}」的文档在工件库里取不回来`,
mode: {
none: '这次运行没有报告文档,也没有出具契约:没有可以逐段展示的证据',
legacy_contract: '旧版出具:按数值匹配口径卡,不是显式引用。同一个值对得上好几个指标时,出处不唯一',
legacy_text: '这次运行没有出具契约:下面是按数值猜的可能来源,默认收起',
} as Record<string, string>,
modeLabel: { cited: '逐段引用', none: '没有证据', legacy_contract: '旧版出具', legacy_text: '没有契约的旧运行' } as Record<string, string>,
legacyMatched: (n: number) => `按数值回指上 ${formatNumber(n)} 个数字`,
legacyUnmatched: (n: number) => `${formatNumber(n)} 个回指不上`,
legacyNoField: '回指的位置对不上成果里的任何一个字段,只列出数字,不在正文上标',
} as const
/** 报告节点卡上的章:这张图最近一次运行的报告核对统计 */
export const reportStampText = (cited: number, none: number) =>
`引用 ${formatNumber(cited)} · 无证据 ${formatNumber(none)}`
/**
* 口径升版的三种处置,和后端 governance.UPGRADE_POLICY_LABELS 同一套说法。子工作流和
* 钉版本的口径卡(caliber_from)共用
+532
View File
@@ -0,0 +1,532 @@
import {
useCallback, useEffect, useId, useLayoutEffect, useMemo, useRef, useState, type KeyboardEvent as ReactKeyboardEvent,
} from 'react'
import { ChevronRight, Download, FileText } from 'lucide-react'
import clsx from 'clsx'
import { ApiError, api } from '../../api/client'
import { ErrorState, Skeleton, Spinner, StatusBadge, rovingTarget, toast, useRadioGroup } from '../../components/ui'
import {
EVIDENCE_STATE, PROBLEM_GROUPS, auditCsv, auditFromApi, auditRows, groupRows, guessRows, guessesOf, issuanceMarks,
segmentState, type AuditFilter, type AuditGroup, type AuditRow,
} from '../../lib/evidence'
import { formatNumber } from '../../lib/format'
import { EVIDENCE_AUDIT_TEXT as T } from '../../lib/terms'
import { EvidenceDoc, type EvidenceDocHandle } from '../../run/EvidenceDoc'
import { EvidenceGuessView } from '../../run/EvidenceGuess'
import { Markdown, inlineStyles } from '../../run/Markdown'
import { useEvidence } from '../../store/evidence'
import type { EvidenceAudit, EvidenceDocData, EvidenceGraph, Run } from '../../types'
/**
* 记录页的「证据」页签(方案 6.4):左边报告,右边常驻的证据面板,报告下面是整张审计表。
*
* - 报告按证据图里封存的那几份画(片段接口也只认它们):文档按工件 id 取、取回时后端复验哈希;
* 证据图说哈希对不上的,不画成可点的报告,醒目地说它疑似被改过。
* - 审计表按状态分组(无证据、可疑实体在前),可以只看有问题的两组,可以导出 JSON / CSV
* (后端 /evidence/audit?format=…;老后端没有这个接口时按页面上的清单导出,并照实说)。
* 键盘用户在表里能看全每一处:一个 Tab 位,↑/↓ 逐行走,回车在右边打开那一段。
* - 没有报告的运行照实说是哪种:没有证据(none)、旧版出具按数值匹配(legacy_contract)、
* 没有契约按数值猜的候选(legacy_text,默认收起,写明不能当证据)。
*
* 窄的时候(详情栏不到 880px)不留右栏:面板从底部抽出,和问数据页一样。
*/
export function EvidencePane({ run, output, labelOf, refreshKey }: {
run: Run
/** 这次运行的成果(旧契约按位置标数字时要取那个字段的原文) */
output: Record<string, any> | null
labelOf: (id?: string | null) => string | undefined
/** 运行又往前走了(事件数、状态变了):证据图和审计表重取 */
refreshKey: string
}) {
const graphSlot = useEvidence((s) => s.graphs[run.id])
const loadGraph = useEvidence((s) => s.loadGraph)
const reloadGraph = useEvidence((s) => s.reloadGraph)
const first = useRef(true)
useEffect(() => {
if (first.current) { first.current = false; void loadGraph(run.id); return }
void reloadGraph(run.id)
}, [run.id, refreshKey, loadGraph, reloadGraph])
const audit = useAudit(run.id, refreshKey)
const graph = graphSlot?.data
const [root, setRoot] = useState<HTMLDivElement | null>(null)
const wide = useWide(root, 880)
const [dock, setDock] = useState<HTMLElement | null>(null)
const [opened, setOpened] = useState<Set<string>>(() => new Set())
const handles = useRef(new Map<string, EvidenceDocHandle | null>())
// 报告文档:证据图里封存的那几份,按工件 id 取(store 永久缓存)
const reports = useMemo(() => (graph?.mode === 'cited' ? graph.reports ?? [] : [])
.filter((r) => typeof r?.doc_artifact === 'string' && !!r.doc_artifact && r.hash_ok !== false), [graph])
const docs = useEvidence((s) => s.docs)
const loadDoc = useEvidence((s) => s.loadDoc)
useEffect(() => { for (const r of reports) void loadDoc(r.doc_artifact!) }, [reports, loadDoc])
const docOf = useCallback((node?: string): EvidenceDocData | undefined => {
const r = reports.find((x) => x.node_id === node)
const slot = r?.doc_artifact ? docs[r.doc_artifact] : undefined
return slot?.status === 'ok' ? slot.data : undefined
}, [reports, docs])
const guesses = useMemo(() => guessesOf(graph), [graph])
// 审计表的行:后端的审计接口优先(每行的封存核对是后端做的);老后端没有它时按页面上的文档、猜测自己拼
const fallback = audit.status === 'unsupported' || audit.status === 'error'
const rows = useMemo<AuditRow[]>(() => {
if (audit.status === 'ok') return auditFromApi(audit.data)
if (!fallback || !graph) return []
if (graph.mode === 'legacy_text') return guesses.flatMap(guessRows)
return reports.flatMap((r) => {
const doc = docOf(r.node_id)
return doc ? auditRows(doc, { report: r.node_id, graph }) : []
})
}, [audit, fallback, graph, guesses, reports, docOf])
const [filter, setFilter] = useState<AuditFilter>('all')
// 哪些行能在正文里打开:`报告:片段` 的集合,一份文档算一次(长报告几千个片段,不能每行各扫一遍)。
// 只收有状态的片段,和 EvidenceDoc 能打开的是同一批:违规行指着的文字片段(粗体、链接里的可疑名字)、
// 结构片段(列表序号 100. 里的数字)在正文里没有能点的地方,画成按钮就是点了没反应
const openable = useMemo(() => {
const keys = new Set<string>()
for (const r of reports) {
const doc = docOf(r.node_id)
for (const b of doc?.blocks ?? []) for (const u of b.units ?? []) for (const seg of u.segments ?? []) {
if (segmentState(seg)) keys.add(`${r.node_id}:${seg.id}`)
}
}
return keys
}, [reports, docOf])
const canOpen = useCallback((row: AuditRow) => !!row.seg && openable.has(`${row.report}:${row.seg}`), [openable])
const openRow = useCallback((row: AuditRow) => {
if (row.seg) handles.current.get(row.report)?.open(row.seg)
}, [])
const onView = useCallback((report: string, open: boolean) => {
setOpened((prev) => {
if (prev.has(report) === open) return prev
const next = new Set(prev)
if (open) next.add(report)
else next.delete(report)
return next
})
}, [])
const mode = graph?.mode ?? audit.data?.mode
const seal = audit.data?.seal ?? graph?.seal
if (!graph && graphSlot?.status !== 'error') {
return (
<div className="p-4" aria-busy="true" data-evidence-pane="loading">
<Skeleton rows={6} height={16} />
</div>
)
}
if (!graph) {
return <ErrorState error={graphSlot?.error} onRetry={() => void reloadGraph(run.id)} className="flex-1" />
}
return (
<div ref={setRoot} className="flex h-full min-h-0 min-w-0" data-evidence-pane={mode ?? ''} data-evidence-wide={wide ? '1' : '0'}>
<div className="min-h-0 min-w-0 flex-1 overflow-y-auto" data-evidence-main="">
<div className="space-y-4 px-4 py-3">
<SummaryBar mode={mode} seal={seal} message={audit.data?.seal?.message}
reports={graph.reports ?? []} labelOf={labelOf} />
{mode === 'cited' && reports.map((r) => {
const doc = docOf(r.node_id)
const slot = r.doc_artifact ? docs[r.doc_artifact] : undefined
return (
<section key={r.node_id} className="rounded-lg border bg-panel p-3" data-evidence-report={r.node_id}>
<header className="mb-2 flex min-w-0 items-center gap-1.5 text-2xs text-faint">
<FileText size={11} aria-hidden />
<span className="min-w-0 truncate">
报告「{labelOf(r.node_id) ?? r.node_id}」{r.fields?.length ? ` · 成果字段 ${r.fields.join('、')}` : ''}
</span>
</header>
{doc ? (
<EvidenceDoc ref={(h) => { handles.current.set(r.node_id!, h) }} doc={doc} artifact={r.doc_artifact}
runId={run.id} label={r.fields?.join('、') || r.node_id} tally
panel={wide ? 'dock' : 'auto'} dock={dock} onView={(o) => onView(r.node_id!, o)} />
) : slot?.status === 'error' ? (
<p className="text-2xs" style={{ color: 'var(--st-failed)' }}>{T.docGone(labelOf(r.node_id) ?? r.node_id ?? '')}</p>
) : <Skeleton rows={4} height={14} />}
</section>
)
})}
{/* 证据图说哈希对不上的报告:不画成可点的,醒目地说 */}
{mode === 'cited' && (graph.reports ?? []).filter((r) => r.hash_ok === false).map((r) => (
<p key={r.node_id} className="rounded border px-2 py-1.5 text-2xs" data-evidence-report-bad={r.node_id}
style={{ color: 'var(--st-failed)', borderColor: 'var(--st-failed)', background: 'var(--st-failed-soft)' }}>
{T.docBad(labelOf(r.node_id) ?? r.node_id ?? '')}
</p>
))}
{mode !== 'cited' && (
<ModeNote mode={mode} graph={graph} output={output} guesses={guesses} />
)}
<AuditTable runId={run.id} rows={rows} filter={filter} onFilter={setFilter} canOpen={canOpen} onOpen={openRow}
loading={audit.status === 'loading' && !rows.length} fallback={fallback && rows.length > 0}
unsupported={audit.status === 'unsupported'} error={audit.status === 'error' ? audit.error : undefined}
multiReport={reports.length > 1} labelOf={labelOf} mode={mode} />
</div>
</div>
{wide && (
<aside ref={setDock} className="relative flex w-[380px] min-w-0 shrink-0 flex-col border-l" data-evidence-dock=""
aria-label="证据面板">
{!opened.size && (
<p className="m-auto max-w-[260px] px-4 text-center text-2xs leading-relaxed text-faint" data-evidence-dock-idle="">
{mode === 'cited' ? T.panelIdle : T.mode[mode ?? 'none'] ?? T.mode.none}
</p>
)}
</aside>
)}
</div>
)
}
/** 详情栏宽到能放下右栏吗:按容器量,不按屏幕(左边还有运行列表) */
function useWide(el: HTMLElement | null, min: number): boolean {
const [wide, setWide] = useState(true)
useLayoutEffect(() => {
if (!el || typeof ResizeObserver !== 'function') return
const on = () => setWide(el.getBoundingClientRect().width >= min)
on()
const ro = new ResizeObserver(on)
ro.observe(el)
return () => ro.disconnect()
}, [el, min])
return wide
}
type AuditState =
| { status: 'loading'; data?: EvidenceAudit; error?: undefined }
| { status: 'ok'; data: EvidenceAudit; error?: undefined }
| { status: 'unsupported'; data?: undefined; error?: undefined }
| { status: 'error'; data?: undefined; error: unknown }
/** 审计表的数据。接口不存在(老后端:404 且没有机读码)算「不支持」,退回页面自己拼 */
function useAudit(runId: string, key: string): AuditState {
const [state, setState] = useState<AuditState>({ status: 'loading' })
useEffect(() => {
const ctrl = new AbortController()
setState((s) => (s.status === 'ok' ? { status: 'loading', data: s.data } : { status: 'loading' }))
api.evidence.audit(runId, { signal: ctrl.signal }).then(
(data) => setState({ status: 'ok', data }),
(error) => {
if (ctrl.signal.aborted) return
const missing = error instanceof ApiError && (error.status === 404 || error.status === 405) && !error.code
setState(missing ? { status: 'unsupported' } : { status: 'error', error })
},
)
return () => ctrl.abort()
}, [runId, key])
return state
}
/** 顶上一行:这次运行按哪种方式展示证据、封存核对的结果、每份报告文档的哈希 */
function SummaryBar({ mode, seal, message, reports, labelOf }: {
mode?: string
seal?: EvidenceGraph['seal'] & { events?: number; message?: string }
message?: string
reports: NonNullable<EvidenceGraph['reports']>
labelOf: (id?: string | null) => string | undefined
}) {
const verdict: { status: 'done' | 'failed' | 'idle'; text: string } = !seal
? { status: 'idle', text: T.sealUnknown }
: !seal.sealed ? { status: 'idle', text: T.sealOpen }
: seal.ok === false ? { status: 'failed', text: T.sealBad }
: { status: 'done', text: T.sealOk(seal.manifest_seq ?? seal.events) }
return (
<div className="flex flex-wrap items-center gap-x-3 gap-y-1 text-2xs" data-evidence-summary-bar="">
<span className="chip" data-evidence-mode={mode ?? ''}>{T.modeLabel[mode ?? 'none'] ?? mode}</span>
<span className="flex min-w-0 items-center gap-1.5" data-evidence-seal={verdict.status} title={message}>
<StatusBadge status={verdict.status} size={12} decorative animate={false} />
<span className="text-faint">{T.sealTitle}:</span>
<span className={verdict.status === 'failed' ? undefined : 'text-dim'}
style={verdict.status === 'failed' ? { color: 'var(--st-failed)' } : undefined}>{verdict.text}</span>
</span>
{reports.map((r) => {
const name = labelOf(r.node_id) ?? r.node_id ?? ''
const bad = r.hash_ok === false
const gone = r.hash_ok == null
return (
<span key={r.node_id} className="min-w-0" data-evidence-doc-hash={bad ? 'bad' : gone ? 'gone' : 'ok'}
style={{ color: bad ? 'var(--st-failed)' : gone ? 'var(--st-waiting)' : 'var(--text-faint)' }}>
{bad ? T.docBad(name) : gone ? T.docGone(name) : T.docOk(name)}
</span>
)
})}
</div>
)
}
/** 没有逐段引用的运行:照实说是哪一种,旧契约把回指的数字按位置标在原文上,没有契约的给收起的猜测 */
function ModeNote({ mode, graph, output, guesses }: {
mode?: string; graph: EvidenceGraph; output: Record<string, any> | null; guesses: ReturnType<typeof guessesOf>
}) {
const legacy = graph.mode === 'legacy_contract' && graph.legacy && !Array.isArray(graph.legacy) ? graph.legacy : null
const field = typeof legacy?.field === 'string' ? legacy.field : null
const text = field && typeof output?.[field] === 'string' ? output[field] as string : null
const matched: any[] = Array.isArray(legacy?.matched) ? legacy!.matched : []
const unmatched: any[] = Array.isArray(legacy?.unmatched) ? legacy!.unmatched : []
return (
<section className="space-y-2" data-evidence-mode-note={mode ?? 'none'}>
<p className="rounded border px-2.5 py-1.5 text-xs leading-relaxed text-dim">
{T.mode[mode ?? 'none'] ?? T.mode.none}
{graph.note && graph.note !== T.mode[mode ?? 'none'] && <span className="mt-0.5 block text-2xs text-faint">{graph.note}</span>}
</p>
{legacy && (
<div className="rounded-lg border bg-panel p-3" data-evidence-legacy-contract="">
<div className="mb-1.5 flex flex-wrap gap-x-2 text-2xs text-faint">
<span>{legacy.note ?? T.mode.legacy_contract}</span>
<span>· {T.legacyMatched(matched.length)}</span>
{unmatched.length > 0 && <span style={{ color: 'var(--st-waiting)' }}>· {T.legacyUnmatched(unmatched.length)}</span>}
</div>
{text
? <Markdown text={text} marks={issuanceMarks({ matched, unmatched_numbers: unmatched })} />
: <p className="text-2xs text-dim">{T.legacyNoField}</p>}
</div>
)}
{mode === 'legacy_text' && guesses.length > 0 && (
<div className="rounded-lg border bg-panel p-3" data-evidence-legacy-text="">
<EvidenceGuessView guesses={guesses} />
</div>
)}
</section>
)
}
/**
* 审计表:按状态分组,一组一个 tbody。「旧运行猜测」那组默认收起(猜的,不能当证据)。
* 键盘:整张表一个 Tab 位(roving),↑/↓ 逐行走、Home/End 到头尾,回车或点击在右边打开那一段
*/
function AuditTable({
runId, rows, filter, onFilter, canOpen, onOpen, loading, fallback, unsupported, error, multiReport, labelOf, mode,
}: {
runId: string
rows: AuditRow[]
filter: AuditFilter
onFilter: (f: AuditFilter) => void
canOpen: (row: AuditRow) => boolean
onOpen: (row: AuditRow) => void
loading: boolean
/** 行是页面自己拼的(审计接口不支持或没取到) */
fallback: boolean
unsupported: boolean
error?: unknown
multiReport: boolean
labelOf: (id?: string | null) => string | undefined
mode?: string
}) {
const groups = useMemo(() => groupRows(rows, filter), [rows, filter])
const [collapsed, setCollapsed] = useState<Set<AuditGroup>>(() => new Set(['candidate']))
const visible = useMemo(() => groups.flatMap((g) => (collapsed.has(g.group) ? [] : g.rows)), [groups, collapsed])
const [current, setCurrent] = useState<string | null>(null)
const cur = current && visible.some((r) => r.key === current) ? current : visible[0]?.key ?? null
const tableRef = useRef<HTMLTableElement>(null)
const keysId = useId()
const radio = useRadioGroup<AuditFilter>(['all', 'problems'], filter, onFilter)
const [exporting, setExporting] = useState<null | 'json' | 'csv'>(null)
const problems = rows.filter((r) => r.group === 'none' || r.group === 'suspicious').length
// 焦点当场挪过去(每一行都画着,只是 tabIndex=-1):等下一帧再挪的话,连按几下方向键会落在同一行上
const focusRow = (key: string) => {
setCurrent(key)
tableRef.current?.querySelector<HTMLElement>(`[data-audit-focus="${CSS.escape(key)}"]`)?.focus()
}
const onKey = (e: ReactKeyboardEvent, key: string) => {
if (e.key !== 'ArrowDown' && e.key !== 'ArrowUp' && e.key !== 'Home' && e.key !== 'End') return
const i = visible.findIndex((r) => r.key === key)
// ↑/↓ 当作一维走(rovingTarget 按 both 认上下),到头停住、不首尾相接:表格读到底就是底
const to = e.key === 'ArrowDown' ? Math.min(visible.length - 1, i + 1)
: e.key === 'ArrowUp' ? Math.max(0, i - 1)
: rovingTarget(e, i, visible.length, 'both')
if (to < 0 || to === i) { e.preventDefault(); return }
e.preventDefault()
focusRow(visible[to].key)
}
const exportAs = async (format: 'json' | 'csv') => {
setExporting(format)
const wanted = filter === 'problems' ? [...PROBLEM_GROUPS] : undefined
try {
const { blob, name } = await api.evidence.auditExport(runId, format, wanted)
save(name, blob)
toast.ok(T.exported(name))
} catch (e) {
// 老后端没有导出接口:按页面上这张清单导出,并照实说没经过后端的封存核对
if (e instanceof ApiError && (e.status === 404 || e.status === 405) && !e.code) {
const shown = groupRows(rows, filter).flatMap((g) => g.rows)
const name = `evidence-${runId.slice(0, 8)}.${format}`
save(name, new Blob([format === 'csv' ? `${auditCsv(shown)}` : JSON.stringify({
run_id: runId, mode, source: 'page', groups: groupRows(rows, filter).map((g) => ({ key: g.group, rows: g.rows })),
}, null, 2)], { type: format === 'csv' ? 'text/csv;charset=utf-8' : 'application/json' }))
toast.info(T.exportLocal)
} else {
toast.error(e, { key: 'evidence:export' })
}
} finally {
setExporting(null)
}
}
return (
<section className="rounded-lg border bg-panel" data-evidence-audit="" aria-labelledby={`${keysId}-title`}>
<header className="flex flex-wrap items-center gap-2 border-b px-3 py-2">
<h3 id={`${keysId}-title`} className="text-xs font-semibold">{T.title}</h3>
<span className="tnum text-2xs text-faint">{formatNumber(rows.length)} 处</span>
<div role="radiogroup" aria-label={T.filterLabel} className="inline-flex rounded-md border p-px text-2xs" data-audit-filter={filter}>
{(['all', 'problems'] as const).map((f) => (
<button key={f} type="button" {...radio(f)} onClick={() => onFilter(f)} data-audit-filter-option={f}
className={clsx('rounded px-2 py-0.5', filter === f ? 'bg-accent-soft text-fg' : 'text-faint hover:text-dim')}>
{f === 'all' ? T.filterAll : `${T.filterProblems}(${formatNumber(problems)})`}
</button>
))}
</div>
<span className="flex-1" />
{(['json', 'csv'] as const).map((f) => (
<button key={f} type="button" className="btn btn-xs" disabled={!!exporting} onClick={() => void exportAs(f)}
data-audit-export={f} title={filter === 'problems' ? `只导出${T.filterProblems.replace('只看', '')}` : undefined}>
{exporting === f ? <Spinner size={10} /> : <Download size={10} aria-hidden />} {f === 'json' ? T.exportJson : T.exportCsv}
</button>
))}
</header>
{fallback && (
<p className="border-b px-3 py-1 text-2xs text-faint" data-audit-fallback={unsupported ? 'unsupported' : 'error'}>
{unsupported ? T.fallbackUnsupported : T.fallbackError(error instanceof Error ? error.message : '未知原因')}
</p>
)}
<p id={keysId} className="sr-only">{T.keys}</p>
{loading ? <div className="p-3"><Skeleton rows={4} height={14} /></div>
: !groups.length ? (
<p className="px-3 py-3 text-2xs text-faint" data-audit-empty="">{filter === 'problems' ? T.emptyProblems : T.empty}</p>
) : (
<div className="overflow-x-auto">
<table ref={tableRef} className="w-full min-w-[560px] table-fixed border-collapse text-xs" aria-describedby={keysId}
aria-labelledby={`${keysId}-title`} data-audit-table="">
{/* 定宽:出处是长文字,按内容分宽的话会被挤成一个字宽。所在的句子放在出处下面一行,不另占一列 */}
<colgroup>
<col style={{ width: '24%' }} />
<col style={{ width: '7rem' }} />
<col />
{multiReport && <col style={{ width: '6rem' }} />}
<col style={{ width: '5.5rem' }} />
</colgroup>
<thead>
<tr className="text-left text-2xs text-faint">
<th scope="col" className="px-3 py-1.5 font-medium">{T.cols.text}</th>
<th scope="col" className="px-2 py-1.5 font-medium">{T.cols.state}</th>
<th scope="col" className="px-2 py-1.5 font-medium">{T.cols.source}<span className="font-normal">({T.cols.sentence})</span></th>
{multiReport && <th scope="col" className="px-2 py-1.5 font-medium">{T.cols.report}</th>}
<th scope="col" className="px-3 py-1.5 text-right font-medium">{T.cols.seal}</th>
</tr>
</thead>
{groups.map((g) => {
const shut = collapsed.has(g.group)
return (
<tbody key={g.group} data-audit-group={g.group} data-audit-collapsed={shut ? '1' : '0'}>
<tr className="border-t">
<th scope="colgroup" colSpan={multiReport ? 5 : 4} className="px-2 py-1 text-left">
<button type="button" className="inline-flex items-center gap-1.5 rounded px-1 py-0.5 text-2xs transition-colors hover:bg-hover"
aria-expanded={!shut} data-audit-group-toggle={g.group}
onClick={() => setCollapsed((prev) => {
const next = new Set(prev)
if (shut) next.delete(g.group)
else next.add(g.group)
return next
})}>
<ChevronRight size={11} aria-hidden className={clsx('text-faint', !shut && 'rotate-90')} />
<span className="font-semibold" style={{ color: groupColor(g.group) }}>{T.groups[g.group]}</span>
<span className="tnum text-faint">{formatNumber(g.rows.length)}</span>
<span className="font-normal text-faint">· {T.groupHint[g.group]}</span>
</button>
</th>
</tr>
{!shut && g.rows.map((r) => (
<AuditLine key={r.key} row={r} current={cur === r.key} openable={canOpen(r)} multiReport={multiReport}
labelOf={labelOf} onOpen={() => { setCurrent(r.key); onOpen(r) }}
onFocus={() => setCurrent(r.key)} onKey={(e) => onKey(e, r.key)} />
))}
</tbody>
)
})}
</table>
</div>
)}
</section>
)
}
/** 表格里的字按正文显示的样子写:去掉行内 Markdown 的语法字符(反引号、星号、链接的括号) */
function plain(text: string): string {
if (!/[`*_~[]/.test(text)) return text
const { hide } = inlineStyles(text)
let out = ''
for (let i = 0; i < text.length; i++) if (!hide[i]) out += text[i]
return out
}
const groupColor = (g: AuditGroup): string =>
g === 'none' || g === 'suspicious' ? 'var(--st-waiting)' : g === 'cited' ? 'var(--st-done)' : EVIDENCE_STATE.candidate.color
function AuditLine({ row, current, openable, multiReport, labelOf, onOpen, onFocus, onKey }: {
row: AuditRow; current: boolean; openable: boolean; multiReport: boolean
labelOf: (id?: string | null) => string | undefined
onOpen: () => void; onFocus: () => void; onKey: (e: ReactKeyboardEvent) => void
}) {
const meta = EVIDENCE_STATE[row.state]
const text = plain(row.text)
const sentence = plain(row.sentence)
const label = `${text},${meta.label}${row.source ? `:${row.source}` : ''}`
const common = {
tabIndex: current ? 0 : -1, 'data-audit-focus': row.key, onFocus, onKeyDown: onKey,
className: clsx('mono max-w-full truncate rounded-sm text-left outline-none focus-visible:outline focus-visible:outline-2 focus-visible:outline-offset-1 focus-visible:outline-[color:var(--accent)]',
row.kind === 'claim' && 'font-sans'),
}
return (
<tr className="border-t align-top hover:bg-hover" data-audit-row={row.key} data-audit-state={row.state}
data-audit-seg={row.seg ?? ''}>
<td className="px-3 py-1.5">
{openable ? (
<button type="button" {...common} aria-label={label} title={label} onClick={onOpen} data-audit-open={row.seg}
className={clsx(common.className, 'block underline decoration-dotted underline-offset-2 hover:text-[var(--accent)]')}>
{text}
</button>
) : (
<span {...common} className={clsx(common.className, 'block')} title={text}>{text}</span>
)}
</td>
<td className="px-2 py-1.5" style={{ color: meta.color }}>
{meta.glyph && <span aria-hidden className="mr-0.5">{meta.glyph}</span>}{meta.label}
</td>
<td className="px-2 py-1.5">
<span className="block text-dim [overflow-wrap:anywhere]" data-audit-source="">{row.source || '—'}</span>
{/* 打不开的违规行(列表序号里的数字、粗体和链接里的可疑名字):说清为什么只在表里 */}
{row.kind === 'violation' && !openable && (
<span className="mt-0.5 block text-2xs text-faint" title={T.hiddenWhy} data-audit-hidden="">{T.hidden}</span>
)}
{sentence && sentence !== text && (
<span className="mt-0.5 block text-2xs text-faint" title={sentence} data-audit-sentence=""
style={{ display: '-webkit-box', WebkitLineClamp: 1, WebkitBoxOrient: 'vertical', overflow: 'hidden' }}>
「{sentence}」
</span>
)}
</td>
{multiReport && <td className="whitespace-nowrap px-2 py-1.5 text-faint">{labelOf(row.report) ?? row.report}</td>}
<td className="whitespace-nowrap px-3 py-1.5 text-right text-2xs" data-audit-sealed={row.sealed == null ? 'na' : row.sealed ? 'yes' : 'no'}>
{row.sealed == null ? <span className="text-faint">{T.sealNa}</span>
: row.sealed ? <span style={{ color: 'var(--st-done)' }}>{T.sealIn}</span>
: <span style={{ color: 'var(--st-waiting)' }}>{T.sealOut}</span>}
</td>
</tr>
)
}
/** 存成文件:同 AssistantStream 的下载,CSV 由后端带了 BOM(Excel 按 GBK 猜编码,不带的话中文列名乱码) */
function save(name: string, blob: Blob) {
const url = URL.createObjectURL(blob)
const a = document.createElement('a')
a.href = url
a.download = name
a.click()
setTimeout(() => URL.revokeObjectURL(url), 1000)
}
@@ -24,6 +24,8 @@ import {
asView, duplicateNames, graphShape, idTail, isLiveRun, runScope, type DetailView,
} from './model'
import { ArtifactsPane, useRunArtifacts } from './ArtifactsPane'
import { EvidencePane } from './EvidencePane'
import { EVIDENCE_AUDIT_TEXT } from '../../lib/terms'
import { ClassChip, MoreMenu, RunTabs, TierChip, copyText, type MenuItem, type TabItem } from './parts'
import { ProvenanceBar, Telemetry, runClocks } from './Telemetry'
import { TracePane } from './TracePane'
@@ -523,6 +525,7 @@ export function RunDetailView({ runId, onChange, onDeleted }: {
const viewTabs: TabItem<DetailView>[] = [
{ key: 'stream', label: '时间线', title: '逐步看每个节点做了什么、说了什么' },
{ key: 'trace', label: '航迹', title: '按时间摊开:慢在哪、卡在哪、谁和谁同时;拖到任意一刻回放' },
{ key: 'evidence', label: EVIDENCE_AUDIT_TEXT.tab, title: EVIDENCE_AUDIT_TEXT.tabTitle },
{
key: 'artifacts', label: '工件', count: artifactCount, unit: '件', tone: 'quiet',
title: artifactCount == null ? '这次运行存下的证据和产出' : `${artifactCount} 件:查询和工具调用的原始结果、每个节点每一次的产出`,
@@ -651,6 +654,11 @@ export function RunDetailView({ runId, onChange, onDeleted }: {
labelOf={labelOf} nodeHref={hrefFor} goneFromCanvas={goneFromCanvas} onRevealStep={revealStep} />
</div>
)}
{view === 'evidence' && (
<div className="absolute inset-0" data-view-pane="evidence">
<EvidencePane run={run} output={turn.output ?? null} labelOf={labelOf} refreshKey={artifactKey} />
</div>
)}
{view === 'artifacts' && (
<div className="absolute inset-0" data-view-pane="artifacts">
<ArtifactsPane list={artifacts} graph={d.graph} trace={d.trace} labelOf={labelOf}
+10 -3
View File
@@ -291,9 +291,12 @@ export function cancelReason(error: string | null | undefined): string | null {
// 详情的几个视图
// -------------------------------------------------------------------------
/** stream:时间线(逐步的文字);trace:航迹(按时间摊开、可回放);artifacts:工件 */
export type DetailView = 'stream' | 'trace' | 'artifacts'
export const DETAIL_VIEWS: readonly DetailView[] = ['stream', 'trace', 'artifacts']
/**
* stream:时间线(逐步的文字);trace:航迹(按时间摊开、可回放);evidence:证据(报告、常驻面板、
* 审计表);artifacts:工件
*/
export type DetailView = 'stream' | 'trace' | 'evidence' | 'artifacts'
export const DETAIL_VIEWS: readonly DetailView[] = ['stream', 'trace', 'evidence', 'artifacts']
export const asView = (v: string | null | undefined): DetailView =>
(DETAIL_VIEWS as readonly string[]).includes(v ?? '') ? (v as DetailView) : 'stream'
@@ -328,6 +331,7 @@ export const ARTIFACT_KIND_LABEL: Record<string, string> = {
node_output: '节点产出',
metric_set: '口径卡指标集',
report_doc: '报告文档',
schema_snapshot: '表结构快照',
}
export const artifactKindLabel = (kind: string): string => ARTIFACT_KIND_LABEL[kind] ?? kind
@@ -342,6 +346,9 @@ export function artifactDescription(kind: string, meta?: Record<string, any> | n
case 'metric_set': return meta?.caliber
? `口径卡「${meta.caliber}」${meta.version ?? ''}` : '口径卡的指标、算式和输入'
case 'report_doc': return '报告:逐段的片段和每个数字的出处'
// 查库时冻结的表结构(工件行的 meta 里有 {source, tables: 表数量}):报告里的表名、字段名按它核对
case 'schema_snapshot': return meta?.source
? `表结构:数据源「${meta.source}」${typeof meta.tables === 'number' ? ` · ${meta.tables} 张表` : ''}` : '查库时冻结的表结构'
default: return '工件'
}
}
+38
View File
@@ -18,6 +18,7 @@ import { runClassLabel } from '../lib/terms'
import type { RunUsage, ToolChange } from '../types'
import { AssistantStream, type StreamTurn } from './AssistantStream'
import { Composer } from './Composer'
import { EvidenceHostContext, type EvidenceHost } from './evidenceHost'
import {
copilotOutcome, decodeCopilot, decodeRun, exitLabels, issueLine, SELF_CHECK_ROUNDS, type CopilotIssue, type CopilotOutcome,
} from './decode'
@@ -541,6 +542,39 @@ function useExitLabels() {
), [sig])
}
/**
* 画布右栏里的证据面板能做的两件事(evidenceHost):
* - 点开片段时把证据路径交给血缘层(setLineage):产出证据的节点(查询、口径卡、检索)实线描边、
* 报告虚线描边,两者之间的走线提亮。画布上已经没有的节点不交(对不到任何卡片)。面板关掉、换走时只
* 收回自己画的那一条——悬停变量画的血缘不去动它
* - 面板里的节点名点一下:选中并对准画布上的节点,和步骤行的「在画布上看」同一个动作
*/
function useCanvasEvidenceHost(): EvidenceHost {
const select = useStudio((s) => s.select)
const focusNode = useStudio((s) => s.focusNode)
const drawn = useRef<string | null>(null)
return useMemo<EvidenceHost>(() => {
const exists = (id: string) => useStudio.getState().nodes.some((n) => n.id === id)
return {
onTrace: (trace) => {
const s = useStudio.getState()
if (trace) {
drawn.current = trace.label
s.setLineage({ var: trace.label, producers: trace.producers.filter(exists), consumers: trace.consumers.filter(exists) })
} else if (drawn.current && s.lineage?.var === drawn.current) {
drawn.current = null
s.setLineage(null)
}
},
onNode: (id) => { select(id); focusNode(id) },
nodeLabel: (id) => {
const node = useStudio.getState().nodes.find((n) => n.id === id)
return node ? node.data.label || id : undefined
},
}
}, [select, focusNode])
}
function RunView({ onBack, reveal, onRevealed }: {
onBack: () => void; reveal: Reveal | null; onRevealed: () => void
}) {
@@ -563,6 +597,7 @@ function RunView({ onBack, reveal, onRevealed }: {
const [stopping, setStopping] = useState(false)
const bodyRef = useRef<HTMLDivElement>(null)
const exitLabelOf = useExitLabels()
const evidenceHost = useCanvasEvidenceHost()
const turns = useMemo<StreamTurn[]>(() => {
const finished = [...events].reverse().find((e) => e.type === 'run.finished')
@@ -734,6 +769,8 @@ function RunView({ onBack, reveal, onRevealed }: {
<div ref={bodyRef} className="min-h-0 flex-1">
{raw ? <RawEvents /> : (
// 成果里的报告点开片段时:证据路径画到画布上,面板里的节点名点一下就对准那个节点
<EvidenceHostContext.Provider value={evidenceHost}>
<AssistantStream
turns={turns}
dense
@@ -767,6 +804,7 @@ function RunView({ onBack, reveal, onRevealed }: {
</div>
) : null)}
/>
</EvidenceHostContext.Provider>
)}
</div>
+24 -8
View File
@@ -23,7 +23,7 @@ import {
} from './decode'
import { CODE_COLUMN, CopyChip, LEADING_ZERO, Markdown, type MarkSpec } from './Markdown'
import { EvidenceField, type EvidenceDocHandle } from './EvidenceDoc'
import { docTally, evidenceFields, issuanceMarks, type EvidenceTally } from '../lib/evidence'
import { EVIDENCE_STATE, docTally, evidenceFields, issuanceMarks, type EvidenceTally } from '../lib/evidence'
import type { EvidenceDocData } from '../types'
import { useRunClock } from './useRunClock'
import type { ReviewResult } from '../types'
@@ -2256,6 +2256,12 @@ export function IssuanceBanner({ issuance, runClass, evidence }: {
}
const hitsOf = (c: any): number | undefined =>
perCaliber.get([c?.caliber, c?.version].filter(Boolean).join(' @ '))
// 数字之外另说的两样:可疑名字(按报告文档数,和文档自己的证据条同一个数)、没挂依据的结论句(只在要求挂依据时有)
const counts = evidence?.counts
const suspect = counts?.suspect ?? 0
const uncitedClaims = Number(issuance?.claims?.uncited_claims ?? 0) || 0
const numbers = counts ? evidenceTally(counts.cited, counts.total, counts.other) : ''
const numbersOk = !counts?.none && !counts?.other
return (
// data-issuance-banner:画布上成果节点的出具印章据此滚过来;tabIndex 让「去看出具」能把焦点带到这里
@@ -2275,14 +2281,24 @@ export function IssuanceBanner({ issuance, runClass, evidence }: {
</span>
</div>
{meta.hint && <div className="mt-1 text-2xs leading-relaxed text-dim">{meta.hint}</div>}
{evidence?.counts && (evidence.counts.total > 0 || evidence.counts.other > 0) && (
// 逐段证据的计数:和报告核对那一行、证据条同一种说法。有无证据的地方才给「定位下一处」
{counts && (counts.total > 0 || counts.other > 0 || suspect > 0 || uncitedClaims > 0) && (
// 逐段证据的计数:和报告核对那一行、证据条同一种说法。可疑名字、没挂依据的结论句不是数字,
// 各自另起一句。有无证据的数字、可疑名字时才给「定位下一处」(结论句不是片段,定位不到)
<div className="mt-1 flex flex-wrap items-center gap-2 text-2xs" data-evidence-line="">
<span className="tnum" style={{
color: evidence.counts.none || evidence.counts.other ? 'var(--st-waiting)' : 'var(--st-done)' }}>
{evidenceTally(evidence.counts.cited, evidence.counts.total, evidence.counts.other)}
</span>
{(evidence.counts.none > 0 || evidence.counts.other > 0) && evidence.onNext && (
{numbers && (
<span className="tnum" style={{ color: numbersOk ? 'var(--st-done)' : 'var(--st-waiting)' }}>{numbers}</span>
)}
{suspect > 0 && (
<span className="tnum" style={{ color: 'var(--st-waiting)' }} data-evidence-line-suspect="">
{numbers ? '· ' : ''}{EVIDENCE_STATE.suspect.glyph} {EVIDENCE_TEXT.suspectTag} {formatNumber(suspect)}
</span>
)}
{uncitedClaims > 0 && (
<span className="tnum" style={{ color: 'var(--st-waiting)' }} data-evidence-line-claims="">
{numbers || suspect > 0 ? '· ' : ''}{EVIDENCE_TEXT.uncitedClaimsTag} {formatNumber(uncitedClaims)}
</span>
)}
{(counts.none > 0 || counts.other > 0 || suspect > 0) && evidence?.onNext && (
<button type="button" className="btn btn-xs" data-evidence-next="" onClick={evidence.onNext}>
{EVIDENCE_TEXT.locateNext}
</button>
+59 -19
View File
@@ -8,7 +8,7 @@ import { ListChecks } from 'lucide-react'
import clsx from 'clsx'
import { rovingTarget } from '../components/ui'
import {
EVIDENCE_STATE, docTally, segmentLabel, segmentState, tallySummary, unitText, type EvidenceStateCode,
EVIDENCE_STATE, docTally, segmentKind, segmentLabel, segmentState, tallySummary, unitText, type EvidenceStateCode,
type EvidenceTally as Tally,
} from '../lib/evidence'
import { formatNumber } from '../lib/format'
@@ -40,6 +40,8 @@ export interface EvidenceDocHandle {
next: (dir?: 1 | -1) => boolean
/** 打开违规清单。没有违规返回 false */
list: () => boolean
/** 打开某个片段的面板(记录页的审计表点一行)。focus 为 true 时焦点也挪到正文里那一段;没有这个片段返回 false */
open: (segId: string, opts?: { focus?: boolean }) => boolean
}
/** 和 AssistantStream 的长文本折叠同一个量级:先显示这么多字,后面的折起来 */
@@ -131,10 +133,16 @@ function verticalTarget(model: Model, id: string, down: boolean): string | null
return model.unitFirst[(u + (down ? 1 : -1) + n) % n] ?? null
}
/** 片段的外观:线型走 data 属性(CSS 里只管线型),颜色写成两个变量交给 index.css */
/** 片段的外观:线型走 data 属性(CSS 里只管线型),颜色写成变量交给 index.css(引文前的引号用 --ev-glyph) */
function segStyle(state: EvidenceStateCode): CSSProperties {
const meta = EVIDENCE_STATE[state]
return { '--ev-line': meta.decoration, '--ev-soft': meta.soft } as CSSProperties
return { '--ev-line': meta.decoration, '--ev-soft': meta.soft, '--ev-glyph': meta.color } as CSSProperties
}
/** 句末小标签:哪几种异常态在句末挂字形加文字(去掉颜色也读得出)。可疑名字和无证据分开挂 */
const TAG_TEXT: Partial<Record<EvidenceStateCode, string>> = {
none: EVIDENCE_STATE_LABEL.none,
suspect: EVIDENCE_TEXT.suspectTag,
}
export const EvidenceDoc = forwardRef<EvidenceDocHandle, {
@@ -149,11 +157,18 @@ export const EvidenceDoc = forwardRef<EvidenceDocHandle, {
dense?: boolean
/** 成果字段名,读屏用 */
label?: string
/** 面板放哪:窄栏里栏内展开,宽屏从右侧弹出,窄屏从底部抽出。auto 按 dense 和屏宽定 */
/**
* 面板放哪:窄栏里栏内展开,宽屏从右侧弹出,窄屏从底部抽出,dock 放进页面给的 dock 元素里(记录页
* 的常驻面板)。auto 按 dense 和屏宽定
*/
panel?: 'auto' | PanelMode
/** panel 为 dock 时面板挂在这个元素里 */
dock?: HTMLElement | null
/** 面板开了、关了(dock 的空位据此显示提示) */
onView?: (open: boolean) => void
/** 自己显示「N/N 数字有出处」那一条。上面已经有出具横幅(它会写这一行)时不用 */
tally?: boolean
}>(function EvidenceDoc({ doc, artifact, runId, dense = false, label, panel = 'auto', tally = false }, ref) {
}>(function EvidenceDoc({ doc, artifact, runId, dense = false, label, panel = 'auto', dock, onView, tally = false }, ref) {
const blocks = useMemo(() => doc.blocks ?? [], [doc])
const model = useMemo(() => buildModel(blocks), [blocks])
const counts = useMemo(() => docTally(doc), [doc])
@@ -246,7 +261,18 @@ export const EvidenceDoc = forwardRef<EvidenceDocHandle, {
return list[list.length - 1]
}, [model])
// 面板开关告诉页面(dock 的空位据此显示「点片段看出处」)
const open = !!view
useEffect(() => { onView?.(open) }, [open, onView])
useImperativeHandle(ref, () => ({
open: (id, opts) => {
if (!model.seg.has(id)) return false
if (opts?.focus) focusSeg(id, { flash: true })
else setCurrent(id)
openSeg(id)
return true
},
next: (dir = 1) => {
const target = step(model.alerts, view?.kind === 'seg' ? view.id : current, dir)
if (target) {
@@ -300,7 +326,7 @@ export const EvidenceDoc = forwardRef<EvidenceDocHandle, {
close()
return
} else if (e.key === 'Tab' && !e.shiftKey && view && mode !== 'inline') {
// 面板在页面末尾(portal):Tab 直接进面板,不让人穿过整页去找
// 面板在页面末尾(portal)或页面另一栏(dock):Tab 直接进面板,不让人穿过整页去找
const heading = document.getElementById(`${panelId}-title`)
if (heading) { e.preventDefault(); heading.focus() }
return
@@ -341,7 +367,7 @@ export const EvidenceDoc = forwardRef<EvidenceDocHandle, {
<div ref={rootRef} data-evidence-doc="" className="min-w-0">
<p id={summaryId} className="sr-only" data-evidence-summary="">{summary}</p>
<div className="sr-only" aria-live="polite" aria-atomic="true">{announce}</div>
{tally && (counts.total > 0 || counts.other > 0) && (
{tally && (counts.total > 0 || counts.other > 0 || !!counts.suspect) && (
<EvidenceTally counts={counts} onNext={() => {
const target = step(model.alerts, current, 1)
if (target) { focusSeg(target, { flash: true }); openSeg(target) } else openViolations()
@@ -381,7 +407,8 @@ export const EvidenceDoc = forwardRef<EvidenceDocHandle, {
<button type="button" className="text-[var(--accent)] hover:underline" onClick={() => setExpanded(false)}>收起</button>
</div>
)}
{mode !== 'inline' && panelNode && createPortal(panelNode, document.body)}
{mode === 'dock' && panelNode && dock && createPortal(panelNode, dock)}
{mode !== 'inline' && mode !== 'dock' && panelNode && createPortal(panelNode, document.body)}
</div>
)
})
@@ -394,6 +421,7 @@ function flashOnce(el: HTMLElement) {
/** 面板放哪。auto:窄栏(dense)里栏内展开,宽屏侧边,窄屏底部抽屉;屏宽变了跟着变 */
function usePanelMode(panel: 'auto' | PanelMode, dense: boolean): PanelMode {
// dock 由页面自己定宽窄(记录页窄的时候换成 drawer 再传进来),这里原样用
const query = '(min-width: 900px)'
const [wide, setWide] = useState(() => typeof matchMedia !== 'function' || matchMedia(query).matches)
useEffect(() => {
@@ -411,16 +439,26 @@ function usePanelMode(panel: 'auto' | PanelMode, dense: boolean): PanelMode {
/** 证据条:没有出具横幅时由文档自己说「7/12 数字有出处 · 无证据 5 · 另有 1 处引用解析不了」 */
export function EvidenceTally({ counts, onNext, onList }: {
counts: Pick<Tally, 'total' | 'cited' | 'none' | 'other'>
counts: Pick<Tally, 'total' | 'cited' | 'none' | 'other' | 'suspect'>
onNext?: () => void
onList?: () => void
}) {
const clean = !counts.none && !counts.other
const suspect = counts.suspect ?? 0
const clean = !counts.none && !counts.other && !suspect
const numbers = evidenceTally(counts.cited, counts.total, counts.other)
return (
<div className="mb-1.5 flex flex-wrap items-center gap-x-2 gap-y-1 text-2xs" data-evidence-tally="">
<span className="tnum" style={{ color: clean ? 'var(--st-done)' : 'var(--st-waiting)' }}>
{evidenceTally(counts.cited, counts.total, counts.other)}
</span>
{numbers && (
<span className="tnum" style={{ color: !counts.none && !counts.other ? 'var(--st-done)' : 'var(--st-waiting)' }}>
{numbers}
</span>
)}
{/* 可疑名字另起一句:它不是数字,混进「无证据 M」就算不平了 */}
{suspect > 0 && (
<span className="tnum" style={{ color: 'var(--st-waiting)' }} data-evidence-tally-suspect="">
{numbers ? '· ' : ''}{EVIDENCE_STATE.suspect.glyph} {EVIDENCE_TEXT.suspectTag} {formatNumber(suspect)}
</span>
)}
{!clean && onNext && (
<button type="button" className="btn btn-xs" data-evidence-next="" onClick={onNext}>
{EVIDENCE_TEXT.locateNext}
@@ -541,7 +579,7 @@ function UnitView({ unit, doc, current, active, panelId, code, tag }: {
const text = segs.map((s) => s.text).join('')
const styles = useMemo<InlineStyles | null>(() => (code ? null : inlineStyles(text)), [text, code])
let pos = 0
let alert = false
const alerts = new Set<EvidenceStateCode>()
const nodes = segs.map((s) => {
const from = pos
pos += s.text.length
@@ -551,7 +589,7 @@ function UnitView({ unit, doc, current, active, panelId, code, tag }: {
: s.text
if (!state) return <span key={s.id}>{s.strong ? <strong className="font-semibold">{body}</strong> : body}</span>
const meta = EVIDENCE_STATE[state]
if (meta.alert) alert = true
if (meta.alert && TAG_TEXT[state]) alerts.add(state)
const open = active === s.id
return (
<button
@@ -561,6 +599,8 @@ function UnitView({ unit, doc, current, active, panelId, code, tag }: {
data-seg={s.id}
data-ev-state={state}
data-ev-line={meta.line}
// 实体、引文:线型和颜色照状态,另有自己的样子(引文前挂引号,见 index.css)
data-ev-kind={segmentKind(s) ?? undefined}
tabIndex={current === s.id ? 0 : -1}
aria-label={segmentLabel(s, doc)}
aria-expanded={open}
@@ -574,11 +614,11 @@ function UnitView({ unit, doc, current, active, panelId, code, tag }: {
return (
<>
{nodes}
{alert && tag && (
<span className="ev-tag" aria-hidden="true" title={EVIDENCE_STATE.none.hint}>
{EVIDENCE_STATE.none.glyph}{EVIDENCE_STATE_LABEL.none}
{tag && [...alerts].map((state) => (
<span key={state} className="ev-tag" aria-hidden="true" title={EVIDENCE_STATE[state].hint} data-ev-tag={state}>
{EVIDENCE_STATE[state].glyph}{TAG_TEXT[state]}
</span>
)}
))}
</>
)
}
+102
View File
@@ -0,0 +1,102 @@
import { useId, useMemo, useState } from 'react'
import { ChevronRight } from 'lucide-react'
import clsx from 'clsx'
import { EVIDENCE_STATE, candidateText } from '../lib/evidence'
import { formatNumber } from '../lib/format'
import { EVIDENCE_TEXT } from '../lib/terms'
import type { EvidenceGuess } from '../types'
import { Markdown, type MarkSpec } from './Markdown'
/**
* 没有契约的旧运行:按数值猜的可能来源(用户拍板第 8 条)。
*
* 默认折叠:收着的时候就是原来的答案,一个字不多;标题写明这是「猜测」。展开后同一段答案里
* 有候选的数字画最淡的点状线(证据状态表里「旧运行候选」那一档,绝不用确定性的实线和绿),
* 下面逐个列出候选,开头先写「猜测的来源,不能当证据」——同值的巧合很多,这不是证据。
*
* 答案照原来的 Markdown 版式画(旧答案常有表格、列表),候选是行内的标记,不是按钮:键盘和读屏
* 用户在候选清单里看全每一个,标记上也带着读屏专用的一句「猜测的来源:…」。
*/
export function EvidenceGuessView({ guesses, dense = false, defaultOpen = false }: {
guesses: EvidenceGuess[]
dense?: boolean
defaultOpen?: boolean
}) {
const [open, setOpen] = useState(defaultOpen)
const bodyId = useId()
const stats = useMemo(() => guesses.reduce((acc, g) => ({
numbers: acc.numbers + (g.stats?.numbers ?? 0),
guessed: acc.guessed + (g.stats?.guessed ?? 0),
}), { numbers: 0, guessed: 0 }), [guesses])
const note = guesses.find((g) => g.note)?.note || EVIDENCE_TEXT.guessNote
const many = guesses.length > 1
if (!guesses.length) return null
return (
<section data-ev-guess="" data-open={open ? 'true' : 'false'} className="min-w-0">
<button type="button" className="flex w-full min-w-0 items-center gap-1.5 rounded px-1 py-0.5 text-left text-2xs transition-colors hover:bg-hover"
aria-expanded={open} aria-controls={bodyId} onClick={() => setOpen((v) => !v)} data-ev-guess-toggle="">
<ChevronRight size={11} aria-hidden className={clsx('shrink-0 text-faint', open && 'rotate-90')} />
<span className="shrink-0 font-medium" style={{ color: EVIDENCE_STATE.candidate.color }}>
<span aria-hidden>{EVIDENCE_STATE.candidate.glyph} </span>{EVIDENCE_TEXT.guessTitle}
</span>
<span className="min-w-0 truncate text-faint">
· {EVIDENCE_TEXT.guessCount(stats.guessed, stats.numbers)} · 不能当证据
</span>
</button>
<div id={bodyId} className="mt-1.5 space-y-3" data-ev-guess-body={open ? '' : undefined}>
{open && (
<p className="rounded border border-dashed px-2 py-1 text-2xs leading-relaxed text-dim" data-ev-guess-note="">
{note}
</p>
)}
{guesses.map((g, i) => (
<GuessField key={g.field ?? i} guess={g} open={open} dense={dense} title={many ? g.field : undefined} />
))}
</div>
</section>
)
}
function GuessField({ guess, open, dense, title }: {
guess: EvidenceGuess; open: boolean; dense: boolean; title?: string
}) {
const segs = useMemo(() => (guess.segments ?? []).filter((s) => s.kind === 'number'), [guess])
const guessed = segs.filter((s) => s.state === 'candidate' && s.candidates?.length)
const missed = segs.filter((s) => !(s.state === 'candidate' && s.candidates?.length))
// 候选按位置标在原文上(码点偏移,Markdown 自己换算、核对切出来的字);收着的时候不标,就是原来的答案
const marks = useMemo<MarkSpec[] | undefined>(() => (open ? guessed.map((s) => ({
token: s.text, tone: 'candidate' as const, start: s.span?.[0], end: s.span?.[1],
title: (s.candidates ?? []).map(candidateText).join(';'),
})) : undefined), [open, guessed])
const text = guess.markdown ?? ''
return (
<div className="min-w-0" data-ev-guess-field={guess.field ?? ''}>
{title && <div className="mb-0.5 text-2xs font-medium text-faint">{title}</div>}
<Markdown text={text} dense={dense} marks={marks} />
{open && (
<div className="mt-2 text-2xs">
<div className="mb-1 font-medium text-faint">{EVIDENCE_TEXT.guessCandidates}</div>
{guessed.length ? (
<ol className="space-y-1" data-ev-candidates="">
{guessed.map((s) => (
<li key={s.id} className="flex min-w-0 items-baseline gap-2" data-ev-candidate={s.id}>
<span className="mono tnum shrink-0 font-medium">
<span aria-hidden style={{ color: EVIDENCE_STATE.candidate.color }}>{EVIDENCE_STATE.candidate.glyph}</span>{s.text}
</span>
<span className="min-w-0 text-dim [overflow-wrap:anywhere]">
{(s.candidates ?? []).map(candidateText).join(';')}
</span>
</li>
))}
</ol>
) : <p className="text-dim">{EVIDENCE_TEXT.guessNone}</p>}
{missed.length > 0 && guessed.length > 0 && (
<p className="mt-1 text-faint" data-ev-guess-missed="">
{EVIDENCE_TEXT.guessNone}:{missed.map((s) => s.text).join('、')}({formatNumber(missed.length)} 个)
</p>
)}
</div>
)}
</div>
)
}
+278 -26
View File
@@ -7,18 +7,21 @@ import { ArrowLeft, CornerDownRight, ListChecks, ShieldCheck, X } from 'lucide-r
import clsx from 'clsx'
import { StatusBadge, isComposing } from '../components/ui'
import {
EVIDENCE_STATE, caliberSourceText, caliberUpgradeText, docForeign, evidenceValue as valueText, graphDoc, inputSource,
integrityFailures, locatable, locatorText, notableInput, queryOf, queryWindow, reasonOf, sealVerdict, segmentState,
sourceOf, type SealStatus, type SourceTone,
EVIDENCE_KIND_STYLE, EVIDENCE_STATE, caliberSourceText, caliberUpgradeText, closestNames, docForeign, entitySources,
evidenceTrace, evidenceValue as valueText, graphDoc, inputSource, integrityFailures, locatable, locatorText, notableInput,
queryOf, queryWindow, quoteWhere, quoteWindow, reasonOf, sealVerdict, segName, segmentKind, segmentState, sourceOf,
type SealStatus, type SourceTone,
} from '../lib/evidence'
import { formatNumber, NONE, shortId } from '../lib/format'
import { formatDateTime, formatNumber, NONE, shortId } from '../lib/format'
import { EVIDENCE_TEXT } from '../lib/terms'
import { useCatalog } from '../store/catalog'
import { segmentKey, useEvidence } from '../store/evidence'
import type {
EvidenceDocData, EvidenceInput, EvidenceSeal, EvidenceSegment, EvidenceStep, EvidenceUnit, EvidenceViolation,
EvidenceDocData, EvidenceInput, EvidenceQuoteSource, EvidenceSeal, EvidenceSegment, EvidenceStep, EvidenceUnit,
EvidenceViolation,
} from '../types'
import { ArtifactViewer, ResultTable } from './AssistantStream'
import { useEvidenceHost } from './evidenceHost'
import { CopyChip } from './Markdown'
/**
@@ -39,11 +42,16 @@ import { CopyChip } from './Markdown'
* 事后插行),接口是从封存范围内的事件找的文档。两边不是同一份时,接口的封存状态和
* 证据链说的是另一份报告——不能挂到正文上画成绿的,要醒目地说正文不是封存的那一份。
*
* 三种摆法:side 从右侧弹出(宽屏),drawer 从底部抽出(窄屏),inline 在 360px 的
* 画布右栏里直接栏内展开。都不是模态的:开着面板照样能在正文里走。
* 三期多了两种步骤:表或字段(出现在哪几次查询里、表结构快照的同步时间、字段类型;可疑的
* 名字给出最接近的已知名字),逐字引文(原文所在的文档和片段,引文在原文里的位置高亮)。
* 画布右栏里点开片段时,把证据路径交给画布(evidenceHost),节点名点一下就对准那个节点。
*
* 四种摆法:side 从右侧弹出(宽屏),drawer 从底部抽出(窄屏),inline 在 360px 的
* 画布右栏里直接栏内展开,dock 放进页面给的一块常驻位置(记录页的「证据」页签)。
* 都不是模态的:开着面板照样能在正文里走。
*/
export type PanelMode = 'side' | 'drawer' | 'inline'
export type PanelMode = 'side' | 'drawer' | 'inline' | 'dock'
export type PanelView = { kind: 'seg'; id: string } | { kind: 'violations' }
export function EvidencePanel({ id, mode, doc, artifact, runId, view, onClose, onLocate, onViolations }: {
@@ -72,12 +80,18 @@ export function EvidencePanel({ id, mode, doc, artifact, runId, view, onClose, o
onClose()
}
const title = view.kind === 'violations' ? EVIDENCE_TEXT.violations : found?.seg.text ?? EVIDENCE_TEXT.panelTitle
const title = view.kind === 'violations' ? EVIDENCE_TEXT.violations
: found ? segName(found.seg) : EVIDENCE_TEXT.panelTitle
// 有出处的实体、引文:徽标写得更具体(「有出处 · 逐字引文」),颜色和字形照同一套
const kind = found && state === 'deterministic' ? segmentKind(found.seg) : null
const badge = kind ? { ...meta!, label: EVIDENCE_KIND_STYLE[kind].label, glyph: EVIDENCE_KIND_STYLE[kind].glyph } : meta
const frame = mode === 'side'
? 'ev-panel-side fixed bottom-0 right-0 top-0 z-40 flex flex-col border-l bg-panel shadow-elev-3'
: mode === 'drawer'
? 'ev-panel-rise fixed bottom-0 left-0 right-0 z-40 flex flex-col rounded-t-lg border-t bg-panel shadow-elev-3'
: 'ev-panel-rise my-2 flex flex-col rounded-lg border bg-panel'
: mode === 'dock'
? 'flex h-full min-h-0 flex-col bg-panel'
: 'ev-panel-rise my-2 flex flex-col rounded-lg border bg-panel'
return (
<section
@@ -95,9 +109,9 @@ export function EvidencePanel({ id, mode, doc, artifact, runId, view, onClose, o
<ArrowLeft size={12} aria-hidden /> {EVIDENCE_TEXT.back}
</button>
)}
{meta && (
<span className="chip shrink-0" style={{ color: meta.color, borderColor: meta.color }} data-ev-badge={meta.code}>
{meta.glyph && <span aria-hidden>{meta.glyph}</span>}{meta.label}
{badge && (
<span className="chip shrink-0" style={{ color: badge.color, borderColor: badge.color }} data-ev-badge={badge.code}>
{badge.glyph && <span aria-hidden>{badge.glyph}</span>}{badge.label}
</span>
)}
<h3 id={titleId} tabIndex={-1} className="mono min-w-0 flex-1 truncate text-sm font-semibold outline-none">
@@ -110,7 +124,8 @@ export function EvidencePanel({ id, mode, doc, artifact, runId, view, onClose, o
</button>
)}
</header>
<div className={clsx('space-y-3 px-3 py-2.5 leading-relaxed', mode !== 'inline' && 'min-h-0 flex-1 overflow-y-auto')}>
<div className={clsx('space-y-3 px-3 py-2.5 leading-relaxed', mode !== 'inline' && 'min-h-0 flex-1 overflow-y-auto')}
data-ev-body="">
{view.kind === 'violations'
? <ViolationList doc={doc} onLocate={onLocate} />
: found
@@ -180,14 +195,37 @@ function SegmentBody({ panelId, doc, artifact, runId, seg, unit, onViolations }:
const state = segmentState(seg)
const cite = seg.cite
const entry = cite?.alias ? doc.catalog?.[cite.alias] : undefined
const chain: EvidenceStep[] = Array.isArray(detail?.chain) ? detail.chain.filter((s) => s && typeof s === 'object') : []
const chain: EvidenceStep[] = useMemo(() => (Array.isArray(detail?.chain)
? detail.chain.filter((s) => s && typeof s === 'object') : []), [detail])
const metricStep = chain.find((s) => s.step === 'metric')
const inputStep = chain.find((s) => s.step === 'run_input')
const entityStep = chain.find((s) => s.step === 'entity')
const quoteStep = chain.find((s) => s.step === 'quote')
const queries = chain.filter((s) => s.step === 'query')
const isMetric = cite?.kind === 'metric' || entry?.kind === 'metric' || !!metricStep
const isInput = cite?.kind === 'input' || entry?.kind === 'input' || !!inputStep
// 三期:有出处的表名字段名、引文(原话对不上的引文也画这一节:说清原文在哪、对不上)
const isEntity = !isMetric && !isInput && seg.kind === 'entity' && state === 'deterministic'
const isQuote = !isMetric && !isInput && seg.kind === 'quote'
// 报告直接引用的查询单元格([[v:Q3.r5.amount]]、整表里的一格):证据链就是那一次查询
const isCell = !isMetric && !isInput && (cite?.kind === 'cell' || entry?.kind === 'query' || !!queries.length)
const isCell = !isMetric && !isInput && !isEntity && !isQuote
&& (cite?.kind === 'cell' || entry?.kind === 'query' || !!queries.length)
const bad = state === 'none' || state === 'suspect' || state === 'unverified'
const closest = closestNames(detail?.closest ?? entityStep?.closest)
// 画布右栏:点开片段时把证据路径交给画布(产出证据的节点实线、报告虚线),链取回来后再补全
const host = useEvidenceHost()
const onTrace = host.onTrace
const trace = useMemo(() => evidenceTrace(seg, doc, chain), [seg, doc, chain])
const traceKey = `${trace.producers.join(',')}|${trace.consumers.join(',')}`
useEffect(() => {
if (!onTrace) return
const text = segName(seg)
onTrace({ label: `证据 ${text.length > 16 ? `${text.slice(0, 15)}…` : text}`, ...trace })
return () => onTrace(null)
// trace 按内容比(traceKey):同一条路径不重画
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [onTrace, seg.id, traceKey])
const inputs: EvidenceInput[] = metricStep?.inputs?.length
? metricStep.inputs
: chain.filter((s) => s.step === 'input').map((s) => s as EvidenceInput)
@@ -218,6 +256,7 @@ function SegmentBody({ panelId, doc, artifact, runId, seg, unit, onViolations }:
docOnly: state !== 'deterministic',
viaGraph: slot?.status === 'error',
})
const pending = !!runId && !docIssue && (!slot || slot.status === 'loading')
const chainFailed = !runId ? EVIDENCE_TEXT.noRun
: docIssue ? EVIDENCE_TEXT.chainForeign
: slot?.status === 'error' ? EVIDENCE_TEXT.chainMissing : ''
@@ -236,20 +275,49 @@ function SegmentBody({ panelId, doc, artifact, runId, seg, unit, onViolations }:
<Sentence unit={unit} active={seg.id} />
</Part>
{state === 'none' && (
<Part title="为什么没有证据" data-ev-reason="">
<p style={{ color: 'var(--st-waiting)' }}>{reason}</p>
{bad && (
<Part title={state === 'suspect' ? '为什么标成可疑' : state === 'unverified' ? '为什么核对不了' : '为什么没有证据'}
data-ev-reason="">
<p style={{ color: state === 'unverified' ? 'var(--text-dim)' : 'var(--st-waiting)' }}>{reason}</p>
{/* 违规的原话多半是「引用 … 解析不了:<原因>」,和上面那句重复的不再列;裸数字那条说的是怎么改,留着 */}
{violations.filter((v) => v.message && !(reason && v.message.includes(reason))).map((v, i) => (
<p key={i} className="mt-1 text-dim">{v.message}</p>
))}
{seg.ref && <p className="mono mt-1 text-2xs text-faint">[[{seg.ref}]]</p>}
{(state === 'suspect' || state === 'unverified') && runId && !docIssue && (
// 可疑的名字:给最接近的已知名字,多半是写错了一个字(接口按封存的台账重建目录找的)
<div className="mt-1.5" data-ev-closest={closest.length ? '' : 'none'}>
<div className="mb-0.5 text-2xs text-faint">{EVIDENCE_TEXT.closest}</div>
{closest.length ? (
<ul className="flex flex-wrap gap-1">
{closest.map((n) => <li key={n} className="chip mono" data-ev-closest-name="">{n}</li>)}
</ul>
) : (
<p className="text-2xs text-dim">{pending ? EVIDENCE_TEXT.entityPending : EVIDENCE_TEXT.closestNone}</p>
)}
{entityStep?.checked && (
<p className="mt-1 text-2xs text-faint" data-ev-checked="">
{EVIDENCE_TEXT.checked(entityStep.checked.schemas ?? 0, entityStep.checked.queries ?? 0)}
</p>
)}
</div>
)}
</Part>
)}
{isEntity && (
<EntityPart step={entityStep} entry={entry} cite={cite} seal={seal} pending={pending}
failed={chainFailed} catalog={doc.catalog} />
)}
{isQuote && (
<QuotePart step={quoteStep} cite={cite} quote={seg.text} resolved={state === 'deterministic'} seal={seal}
pending={pending} failed={chainFailed} node={entry?.node_id} label={entry?.label} />
)}
{isMetric && (
<MetricPart step={metricStep} entry={entry} inputs={inputs} runId={runId} seal={seal}
pending={!!runId && !docIssue && (!slot || slot.status === 'loading')}
pending={pending}
failed={chainFailed}
resolved={state === 'deterministic'}
linkOf={linkOf}
@@ -275,7 +343,7 @@ function SegmentBody({ panelId, doc, artifact, runId, seg, unit, onViolations }:
// 解析不了的单元格引用(行越界、没有这一列、受管没声明 cells)接口本来就不给链,上面已经说了原因,
// 再摆一块「这一次没取到」就像是接口出了错
<QueryFallback entry={entry} alias={cite?.alias} where={locatorText(cite?.locator)}
pending={!!runId && !docIssue && (!slot || slot.status === 'loading')}
pending={pending}
failed={chainFailed} />
)}
@@ -283,10 +351,21 @@ function SegmentBody({ panelId, doc, artifact, runId, seg, unit, onViolations }:
<QueryPart key={`${q.artifact ?? ''}-${i}`} id={queryId(i)} step={q} seal={seal} masked={masked} />
))}
{state === 'deterministic' && !isMetric && !isInput && !isCell && (
{state === 'deterministic' && !isMetric && !isInput && !isCell && !isEntity && !isQuote && (
<p className="text-dim">{sourceOf(seg, doc) || detail?.note}</p>
)}
{host.onNode && (trace.producers.length > 0 || trace.consumers.length > 0) && (
// 画布右栏:这段证据经过的节点,点一下在画布上选中并对准
<Part title="画布上的节点" data-ev-nodes="">
<div className="flex flex-wrap items-center gap-1">
{trace.producers.map((n) => <NodeChip key={n} id={n} />)}
{trace.consumers.length > 0 && trace.producers.length > 0 && <span className="text-faint" aria-hidden>→</span>}
{trace.consumers.map((n) => <NodeChip key={n} id={n} />)}
</div>
</Part>
)}
<Part title={EVIDENCE_TEXT.seal} data-ev-seal="">
<SealLine status={verdict.status} label={verdict.label} />
</Part>
@@ -309,6 +388,167 @@ function SegmentBody({ panelId, doc, artifact, runId, seg, unit, onViolations }:
type InputLink = { index: number; alias: string; go: () => void } | undefined
/**
* 面板里提到的节点。画布右栏里是按钮:点一下在画布上选中并对准它(画布上已经没有这个节点
* 时不给按钮,点了会落空);别处照旧写节点 id
*/
function NodeChip({ id, prefix }: { id?: string | null; prefix?: string }) {
const host = useEvidenceHost()
if (!id) return null
const label = host.nodeLabel?.(id)
if (host.onNode && label !== undefined) {
return (
<button type="button" className="chip max-w-full truncate transition-colors hover:bg-hover" data-ev-node={id}
title={`在画布上选中并对准「${label || id}」`} onClick={() => host.onNode!(id)}>
{prefix}{label || id}
</button>
)
}
return <span className="mono" data-ev-node={id}>{prefix}{id}</span>
}
/**
* 表或字段:是什么、字段类型、出现在哪几次查询里、从哪来的(表结构快照、查询 SQL、查询结果列)、
* 表结构快照什么时候同步的。证据接口没取到时照文档目录里记着的来历写,缺的照实说
*/
function EntityPart({ step, entry, cite, seal, pending, failed, catalog }: {
step?: EvidenceStep
entry?: Record<string, any>
cite?: EvidenceSegment['cite']
seal?: EvidenceSeal
pending: boolean
failed: string
catalog?: EvidenceDocData['catalog']
}) {
const kind = step?.kind ?? entry?.kind ?? cite?.kind
const table = step?.table ?? entry?.table ?? entry?.locator?.table ?? cite?.locator?.table
const column = step?.column ?? entry?.locator?.column ?? cite?.locator?.column
const name = kind === 'table' ? (step?.name ?? entry?.name ?? table ?? cite?.ref)
: table && column ? `${table}.${column}` : (step?.name ?? entry?.name ?? column ?? cite?.ref)
const queries: string[] = (Array.isArray(step?.queries) ? step.queries : Array.isArray(entry?.queries) ? entry.queries : [])
.map(String).filter(Boolean)
const sources = entitySources(step?.sources ?? entry?.sources)
const truncated = sources.some((s) => s.truncated) || step?.snapshot_truncated === true || (step as any)?.truncated === true
const owners: string[] = Array.isArray(step?.tables) ? step.tables : Array.isArray(entry?.tables) ? entry.tables : []
const nodes = [...new Set(queries.map((q) => catalog?.[q]?.node_id).filter((n): n is string => typeof n === 'string'))]
return (
<Part title={EVIDENCE_TEXT.entity} data-ev-entity={kind ?? ''}>
<div className="flex flex-wrap items-baseline gap-x-2 gap-y-0.5">
<span className="chip shrink-0">{kind === 'table' ? EVIDENCE_TEXT.table : EVIDENCE_TEXT.column}</span>
<span className="mono min-w-0 font-medium [overflow-wrap:anywhere]">{name || NONE}</span>
{step?.type && (
<span className="mono text-2xs text-dim" data-ev-entity-type="">{EVIDENCE_TEXT.columnType} {step.type}</span>
)}
{kind === 'table' && typeof (step as any)?.columns === 'number' && (
<span className="text-2xs text-dim">{EVIDENCE_TEXT.tableColumns((step as any).columns, !!step?.is_view)}</span>
)}
</div>
{step?.comment && <p className="mt-0.5 text-2xs text-dim">{step.comment}</p>}
{owners.length > 1 && <p className="mt-0.5 text-2xs text-dim">{EVIDENCE_TEXT.ownerTables(owners)}</p>}
{step?.types && !step.type && Object.keys(step.types).length > 0 && (
<p className="mono mt-0.5 text-2xs text-dim" data-ev-entity-type="">{EVIDENCE_TEXT.columnTypes(step.types)}</p>
)}
<p className="mt-1" data-ev-entity-queries="">
{queries.length ? EVIDENCE_TEXT.entityQueries(queries) : EVIDENCE_TEXT.entityNoQuery}
</p>
{nodes.length > 0 && (
<div className="mt-0.5 flex flex-wrap items-center gap-1 text-2xs text-faint">
{nodes.map((n) => <NodeChip key={n} id={n} prefix="节点 " />)}
</div>
)}
{sources.length > 0 && (
<ul className="mt-1 space-y-0.5 text-2xs text-dim">
{sources.map((src) => <li key={src.text} data-ev-entity-source={src.kind}>· {src.text}</li>)}
</ul>
)}
{step?.synced_at && (
<p className="mt-1 text-2xs text-faint" data-ev-entity-synced="">{EVIDENCE_TEXT.syncedAt(formatDateTime(step.synced_at))}</p>
)}
{truncated && (
<p className="mt-0.5 text-2xs" style={{ color: 'var(--st-waiting)' }} data-ev-entity-partial="">{EVIDENCE_TEXT.snapshotPartial}</p>
)}
<Integrity step={step} seal={seal} />
{!step && (
<p className="mt-1.5 text-2xs text-faint" data-ev-chain={pending ? 'loading' : 'missing'}>
{pending ? EVIDENCE_TEXT.entityPending : failed && failed !== EVIDENCE_TEXT.chainMissing ? failed : EVIDENCE_TEXT.entityMissing}
</p>
)}
</Part>
)
}
/**
* 逐字引文:原文所在的文档和片段,引文在原文里的位置高亮(前后带一截上下文)。检索快照哈希对不上时
* 不摆原文——那一段不能当证据。原话对不上的引文也画这一节,只写目录里记着的是哪次检索
*/
function QuotePart({ step, cite, quote, resolved, seal, pending, failed, node, label }: {
step?: EvidenceStep
cite?: EvidenceSegment['cite']
quote: string
resolved: boolean
seal?: EvidenceSeal
pending: boolean
failed: string
node?: string
/** 目录里这次检索的说明:「知识库「运营手册」 · 2 条」 */
label?: string
}) {
const source = (step?.source && typeof step.source === 'object' ? step.source as EvidenceQuoteSource : undefined)
?? cite?.source
const where = quoteWhere(source)
const bad = integrityFailures(step, seal)
// 原文:接口给在 content 里(text 是引文本身);content 为 null 是快照不在封存范围里、取不回来——不拿引文冒充原文
const original = step && 'content' in step ? step.content ?? undefined : step?.text
const loc = cite?.locator
const mismatch = step?.match_ok === false
const win = original && !bad.includes('hash') && !mismatch
? quoteWindow(original, step?.match ?? (loc ? { start: loc.start, end: loc.end } : null), step?.quote ?? quote)
: null
const collection = step?.collection
return (
<Part title={EVIDENCE_TEXT.quote} data-ev-quote-step="">
<div className="flex flex-wrap items-baseline gap-x-2 gap-y-0.5" data-ev-quote-source="">
<span className="mono font-medium">{step?.alias ?? cite?.alias ?? NONE}</span>
{where ? <span>{where}</span> : label && <span className="text-dim">{label}</span>}
{collection && where && <span className="text-2xs text-faint">{EVIDENCE_TEXT.quoteCollection(collection)}</span>}
{source?.document && (
<span className="mono text-2xs text-faint" title={`文档 ${source.document}${source.chunk ? ` · 片段 ${source.chunk}` : ''}`}>
{shortId(source.document, 12)}
</span>
)}
<NodeChip id={node ?? step?.node_id} prefix="节点 " />
</div>
{(bad.length > 0 || mismatch) && (
<IntegrityList code={[...bad.map((k) => (k === 'hash' ? 'quote-hash' : k)), ...(mismatch ? ['quote-match'] : [])].join(' ')}
items={[...bad.map((k) => (k === 'hash' ? EVIDENCE_TEXT.quoteHash : EVIDENCE_TEXT.integrity[k])),
...(mismatch ? [EVIDENCE_TEXT.quoteBad] : [])]} />
)}
{win && (
<>
<blockquote className="mt-1 whitespace-pre-wrap rounded border-l-2 bg-bg px-2 py-1 text-2xs leading-relaxed text-dim [overflow-wrap:anywhere]"
data-ev-quote="">
{win.cutBefore && '…'}{win.before}
<mark className="rounded-sm px-0.5" data-ev-quote-hit=""
style={{ background: resolved ? 'var(--st-done-soft)' : 'var(--st-waiting-soft)', color: 'var(--text)' }}>
{win.hit}
</mark>
{win.after}{win.cutAfter && '…'}
</blockquote>
<p className="mt-0.5 text-2xs text-faint">{EVIDENCE_TEXT.quoteHit}</p>
</>
)}
{resolved && !win && !bad.length && !mismatch && (
// 接口说了为什么没有原文(不在封存范围里、取不回来)就照原话
<p className="mt-1.5 text-2xs text-faint" data-ev-chain={pending ? 'loading' : 'missing'}>
{pending ? EVIDENCE_TEXT.quotePending : step?.note
|| (failed && failed !== EVIDENCE_TEXT.chainMissing ? failed : EVIDENCE_TEXT.quoteMissing)}
</p>
)}
{!resolved && <p className="mt-1 text-2xs" style={{ color: 'var(--st-waiting)' }}>{EVIDENCE_TEXT.quoteMiss}</p>}
</Part>
)
}
/** 口径卡钉住的上游工作流 id:方案写在 caliber_from,接口给在 source 上 */
function pinnedWorkflow(step?: EvidenceStep): string | undefined {
const from = step?.caliber_from ?? (step?.source && typeof step.source === 'object' ? step.source : undefined)
@@ -323,12 +563,14 @@ function Sentence({ unit, active }: { unit: EvidenceUnit; active: string }) {
const state = segmentState(s)
const on = s.id === active
const number = s.kind === 'number' || s.kind === 'value'
// 反引号里的表名字段名照行内代码写,不露反引号
const code = s.kind === 'entity' && !!s.code
return (
<span key={s.id}
className={clsx(number && 'font-semibold tnum', on && 'rounded-sm px-0.5')}
className={clsx(number && 'font-semibold tnum', code && 'mono', on && 'rounded-sm px-0.5')}
style={on && state ? { background: EVIDENCE_STATE[state].soft, color: 'var(--text)' } : undefined}
data-ev-here={on ? '' : undefined}>
{s.text}
{code ? segName(s) : s.text}
</span>
)
})}
@@ -354,6 +596,7 @@ function MetricPart({ step, entry, inputs, runId, seal, pending, failed, resolve
/** 钉住的上游工作流在目录里的名字(接口没给名字时用) */
workflowName?: string
}) {
const host = useEvidenceHost()
const name = step?.name ?? entry?.name ?? step?.metric ?? entry?.locator?.metric ?? NONE
const value = step?.value !== undefined ? step.value : entry?.value
const rendered = step?.rendered ?? entry?.rendered
@@ -382,6 +625,7 @@ function MetricPart({ step, entry, inputs, runId, seal, pending, failed, resolve
? <span data-ev-caliber-from="">{from}</span>
: (caliber || version) && <span>口径卡「{caliber || NONE}」<span className="mono">{version}</span></span>}
{artifact && <span className="mono text-faint" title={`口径卡工件 ${artifact}`}>工件 {shortId(artifact)}</span>}
{host.onNode && <NodeChip id={step?.node_id ?? entry?.node_id} />}
</div>
)}
{upgrade && (
@@ -508,7 +752,6 @@ function SourcesPart({ inputs, linkOf }: { inputs: EvidenceInput[]; linkOf: (inp
// 那个节点内部的编号,和面板里的「查询 Qn」不是一回事,不拿来写
const alias = link?.alias || inp.query || (typeof inp.cell === 'string' ? inp.cell.split('.')[0] : '')
const where = [
inp.node_id && `节点 ${inp.node_id}`,
inp.field && (inp.via === 'agent_field' ? `字段 ${inp.field}` : inp.field),
[alias, locatorText(inp.locator)].filter(Boolean).join(' · '),
].filter(Boolean).join(' · ')
@@ -525,7 +768,12 @@ function SourcesPart({ inputs, linkOf }: { inputs: EvidenceInput[]; linkOf: (inp
</button>
)}
</div>
{where && <div className="mono text-2xs text-faint [overflow-wrap:anywhere]">{where}</div>}
{(inp.node_id || where) && (
<div className="mono text-2xs text-faint [overflow-wrap:anywhere]">
{inp.node_id && <NodeChip id={inp.node_id} prefix="节点 " />}
{inp.node_id && where && ' · '}{where}
</div>
)}
{said.text && <p className="mt-0.5" style={{ color: tone.color }}>{said.text}</p>}
{said.reason && <p className="mt-0.5 text-2xs text-dim">{said.reason}</p>}
</li>
@@ -544,6 +792,7 @@ function SourcesPart({ inputs, linkOf }: { inputs: EvidenceInput[]; linkOf: (inp
* 这时候这些行不能当证据,比「有出处」重要得多
*/
function QueryPart({ id, step, seal, masked }: { id: string; step: EvidenceStep; seal?: EvidenceSeal; masked: string[] }) {
const host = useEvidenceHost()
const win = useMemo(() => queryWindow(step), [step])
const [open, setOpen] = useState(false)
const scroller = useRef<HTMLDivElement>(null)
@@ -580,6 +829,7 @@ function QueryPart({ id, step, seal, masked }: { id: string; step: EvidenceStep;
{EVIDENCE_TEXT.query(alias)}
{step.tool && <span className="mono ml-1.5 font-normal">{step.tool}</span>}
</span>
{host.onNode && <NodeChip id={step.node_id} />}
{step.sql && <CopyChip label={EVIDENCE_TEXT.copySql} text={() => step.sql ?? ''} />}
</div>
{step.sql && (
@@ -671,6 +921,8 @@ function SealLine({ status, label }: { status: SealStatus; label: string }) {
const CODE_LABEL: Record<string, string> = {
uncited_number: '裸数字',
unresolved_ref: '引用解析不了',
unknown_entity: '可能是编造的名字',
unverified_entity: '核对不了的名字',
}
/** 违规清单:报告撰写节点核对出来的全部问题,画得出线的能定位,画不出的说清在哪 */
+17 -4
View File
@@ -1,10 +1,10 @@
import {
createContext, Fragment, useContext, useMemo, useState, type HTMLAttributes, type ReactNode,
createContext, Fragment, useContext, useMemo, useState, type CSSProperties, type HTMLAttributes, type ReactNode,
} from 'react'
import { Check, Copy } from 'lucide-react'
import clsx from 'clsx'
import { toast } from '../components/ui'
import { codePointIndex } from '../lib/evidence'
import { EVIDENCE_STATE, codePointIndex } from '../lib/evidence'
/**
* 模型输出的 Markdown 渲染。
@@ -60,8 +60,11 @@ export function Markdown({ text, dense, marks }: {
export interface MarkSpec {
token: string
title: string
/** ambiguous:回指上了,但同一个值对得上好几个指标,出处不唯一 */
tone: 'warn' | 'ok' | 'ambiguous'
/**
* ambiguous:回指上了,但同一个值对得上好几个指标,出处不唯一;
* candidate:没有契约的旧运行按数值猜的候选(最淡的点状线,读屏在字后面听到「猜测的来源:…」)
*/
tone: 'warn' | 'ok' | 'ambiguous' | 'candidate'
/** 在原文里的码点偏移 [start, end) */
start?: number
end?: number
@@ -108,6 +111,16 @@ const shiftAt = (at: At | undefined, i: number): At | undefined =>
at == null ? undefined : typeof at === 'number' ? at + i : at.slice(i)
function markSpan(spec: MarkSpec, text: string, key: string): ReactNode {
if (spec.tone === 'candidate') {
// 猜测:外观取证据状态表里「旧运行候选」那一档,绝不用确定性的实线和绿
const meta = EVIDENCE_STATE.candidate
return (
<span key={key} title={spec.title} className="ev-mark cursor-help" data-mark="candidate"
data-ev-state={meta.code} data-ev-line={meta.line} style={{ '--ev-line': meta.decoration } as CSSProperties}>
{text}<span className="sr-only">({meta.label}:{spec.title})</span>
</span>
)
}
const color = spec.tone === 'ok' ? 'var(--st-done)' : 'var(--st-waiting)'
return (
<span key={key} title={spec.title}
File diff suppressed because one or more lines are too long
+31
View File
@@ -0,0 +1,31 @@
import { createContext, useContext } from 'react'
/**
* 证据面板所在的地方能做什么。
*
* 同一个面板在问数据页、记录页、画布右栏里各出现一次,只有画布右栏旁边有一张图:
* 点开片段时要把证据路径画到图上,面板里的节点名点一下要选中并对准那个节点。
* 这些靠上下文交给面板,不顺着 AssistantStream → Output → EvidenceDoc 一层层传 props;
* 别处不提供,面板就照旧(节点名是文字,不画路径)。
*/
export interface EvidenceTrace {
/** 画布上血缘层的标签:「证据 8.7%」 */
label: string
/** 产出证据的节点:查询、口径卡、检索 */
producers: string[]
/** 用到它的节点:报告 */
consumers: string[]
}
export interface EvidenceHost {
/** 片段打开时交出证据路径,关掉时交 null */
onTrace?: (trace: EvidenceTrace | null) => void
/** 面板里的节点名点一下:选中并对准画布上的这个节点 */
onNode?: (nodeId: string) => void
/** 节点 id → 画布上的名字;画布上没有这个节点时返回 undefined(不给按钮,点了会落空) */
nodeLabel?: (nodeId: string) => string | undefined
}
export const EvidenceHostContext = createContext<EvidenceHost>({})
export const useEvidenceHost = (): EvidenceHost => useContext(EvidenceHostContext)
+57 -10
View File
@@ -1,6 +1,6 @@
import { create } from 'zustand'
import { api } from '../api/client'
import type { EvidenceDocData, EvidenceGraph, EvidenceSegmentDetail } from '../types'
import { ApiError, api } from '../api/client'
import type { EvidenceDocData, EvidenceGraph, EvidenceSegmentDetail, EvidenceStats } from '../types'
/**
* 可点击证据的缓存:报告文档、证据图、点开过的片段。
@@ -20,6 +20,12 @@ export interface Slot<T> {
error?: unknown
}
/** 一张图最近一次运行里每个报告节点的核对统计(画布上报告节点卡的章用) */
export interface LatestReports {
runId: string | null
reports: Record<string, { stats?: EvidenceStats; claims?: string | null }>
}
interface EvidenceState {
/**
* 哪一份报告的面板开着。同一屏上可能有好几份报告(问数据页的几轮、改写前的原件),
@@ -30,9 +36,19 @@ interface EvidenceState {
docs: Record<string, Slot<EvidenceDocData>>
graphs: Record<string, Slot<EvidenceGraph>>
segments: Record<string, Slot<EvidenceSegmentDetail>>
/** 按工作流:最近一次运行的报告统计。画布没挂着运行时,报告节点卡的章从这里取 */
latest: Record<string, Slot<LatestReports>>
loadDoc: (artifact: string) => Promise<void>
loadGraph: (runId: string) => Promise<void>
/** 证据图重新取一次:运行还在往前走(记录页接着流)时,封存状态和报告都还会变 */
reloadGraph: (runId: string) => Promise<void>
loadSegment: (runId: string, segmentId: string, report?: string) => Promise<void>
loadLatest: (workflowId: string) => Promise<void>
/**
* 这张图的「最近一次运行」作废:画布上挂着这张图的一次运行(刚跑的、回放的)时,之前取的那份
* 不一定还是最近的。清掉以后,运行摘下来(「清除」、换一张图再回来)时 loadLatest 重新取
*/
dropLatest: (workflowId: string) => void
}
/**
@@ -57,18 +73,24 @@ export class NotADocError extends Error {
}
export const useEvidence = create<EvidenceState>((set, get) => {
/** 取一次、记进 table[key];正在取或已经取到的不再发 */
async function fill<K extends 'docs' | 'graphs' | 'segments'>(
table: K, key: string, fetcher: () => Promise<NonNullable<EvidenceState[K][string]['data']>>,
/**
* 取一次、记进 table[key];正在取或已经取到的不再发。取的途中这一格被清掉或换了(dropLatest 之后
* 又起了一次),回来的结果作废:不能让早先发出的请求把后来那份盖掉
*/
async function fill<K extends 'docs' | 'graphs' | 'segments' | 'latest'>(
table: K, key: string, fetcher: () => Promise<NonNullable<EvidenceState[K][string]['data']>>, force = false,
) {
const cur = get()[table][key]
if (cur && cur.status !== 'error') return
set((s) => ({ [table]: { ...s[table], [key]: { status: 'loading' } } }) as Partial<EvidenceState>)
if (cur && (cur.status === 'loading' || (cur.status === 'ok' && !force))) return
// 重取时留着上一份:界面不闪回「正在取」
const pending = (force && cur?.data ? { ...cur, status: 'loading' } : { status: 'loading' }) as Slot<unknown>
set((s) => ({ [table]: { ...s[table], [key]: pending } }) as Partial<EvidenceState>)
const settle = (slot: Slot<unknown>) => set((s) => (s[table][key] === pending
? { [table]: { ...s[table], [key]: slot } } as Partial<EvidenceState> : s))
try {
const data = await fetcher()
set((s) => ({ [table]: { ...s[table], [key]: { status: 'ok', data } } }) as Partial<EvidenceState>)
settle({ status: 'ok', data: await fetcher() })
} catch (error) {
set((s) => ({ [table]: { ...s[table], [key]: { status: 'error', error } } }) as Partial<EvidenceState>)
settle({ status: 'error', error })
}
}
@@ -78,6 +100,7 @@ export const useEvidence = create<EvidenceState>((set, get) => {
docs: {},
graphs: {},
segments: {},
latest: {},
loadDoc: (artifact) => fill('docs', artifact, async () => {
const res = await api.artifact(artifact)
const doc = asDoc(res?.content)
@@ -85,7 +108,31 @@ export const useEvidence = create<EvidenceState>((set, get) => {
return doc
}),
loadGraph: (runId) => fill('graphs', runId, () => api.evidence.graph(runId)),
reloadGraph: (runId) => fill('graphs', runId, () => api.evidence.graph(runId), true),
loadSegment: (runId, segmentId, report) =>
fill('segments', segmentKey(runId, segmentId, report), () => api.evidence.segment(runId, segmentId, { report })),
// 最近一次运行 → 它封存范围内的报告核对(report.checked)。审计接口的 reports 带着结论句策略(claims,
// 章上「无证据」要不要算没挂依据的结论句靠它);只要 reports,所以只取最小的「可疑实体」那一组行。
// 老后端没有审计接口时退回证据图(没有 claims,没挂依据的结论句不计入)。没有运行、没有报告都是空表,不画章
loadLatest: (workflowId) => fill('latest', workflowId, async () => {
const [run] = await api.runs.list({ workflow_id: workflowId, limit: 1 })
if (!run) return { runId: null, reports: {} }
const list = await api.evidence.audit(run.id, { groups: ['suspicious'] }).then((a) => a.reports ?? [], async (e) => {
if (!(e instanceof ApiError && (e.status === 404 || e.status === 405) && !e.code)) throw e
await get().loadGraph(run.id)
return get().graphs[run.id]?.data?.reports ?? []
})
const reports: LatestReports['reports'] = {}
for (const r of list) if (r?.node_id) reports[r.node_id] = { stats: r.stats, claims: r.claims }
return { runId: run.id, reports }
}),
dropLatest: (workflowId) => {
if (!(workflowId in get().latest)) return
set((s) => {
const latest = { ...s.latest }
delete latest[workflowId]
return { latest }
})
},
}
})
+179 -5
View File
@@ -325,7 +325,9 @@ export interface PublishFix {
node_id?: string | null
label: string
preview?: { field?: string | null; before?: unknown; after?: unknown } | null
options?: { value: unknown; label: string; hint?: string | null }[]
/** handoff:选了这一项就是「交给 Copilot」(G4 的「它在做计算」)。只认这个标记,不认值——
* 别的选项拿节点 id 当值,节点 id 恰好叫 copilot 的报告撰写节点照样是普通候选 */
options?: { value: unknown; label: string; hint?: string | null; handoff?: boolean }[]
multiple?: boolean
/** 建议值:界面上标「建议」,仍然要人选 */
default?: unknown
@@ -599,7 +601,9 @@ export interface EvidenceCitation {
ref?: string
/** 目录里的别名,如 m:gmv、i:week、Q1 */
alias?: string
locator?: { metric?: string; field?: string; row?: number; column?: string; table?: string }
/** 引文:hit 是检索快照里第几条命中,start / end 是引文在那条原文里的码点位置 */
locator?: { metric?: string; field?: string; row?: number; column?: string; table?: string
hit?: number; start?: number; end?: number }
eid?: string
/** metric / input / cell / table / column / quote / query */
kind?: string
@@ -610,6 +614,21 @@ export interface EvidenceCitation {
conv?: string
value?: unknown
rendered?: string
/** 引文的原文出处:检索快照、文档、片段 */
source?: EvidenceQuoteSource
/** [[t:]] / [[c:]] 写了一个哪里都找不到的名字:可能是编造的 */
unknown?: boolean
/** 表结构快照不全,找不到的名字只能说核对不了 */
unverified?: boolean
}
/** 引文出自哪:检索快照工件、知识库里的文档和片段 */
export interface EvidenceQuoteSource {
artifact?: string
document?: string
chunk?: string
title?: string
ordinal?: number
}
/** 可点的最小片段 */
@@ -625,8 +644,53 @@ export interface EvidenceSegment {
ref?: string
cite?: EvidenceCitation
strong?: boolean
/** uncited_number / unresolved_ref */
/** uncited_number / unresolved_ref / unknown_entity(可能是编造的名字)/ unverified_entity(核对不了) */
issue?: string
/** 实体写在反引号里:text 自带两个反引号,按行内代码画 */
code?: boolean
/** 系统自动链接的名字,不是写作者写的标记(不算句子的依据) */
auto?: boolean
/** 反引号里自动发现的可疑名字(没有 ref 时名字在这里) */
name?: string
/** 旧运行按数值猜的候选(state 为 candidate) */
candidates?: EvidenceCandidate[]
}
/**
* 旧运行(没有契约)按数值猜的一个候选:查询快照的一格,或口径卡的一个指标。
* 只做展示,不是证据
*/
export interface EvidenceCandidate {
kind?: 'cell' | 'metric' | string
/** Q1.r0.gmv / m:gmv */
ref?: string
alias?: string
artifact?: string
locator?: EvidenceLocator & { metric?: string }
eid?: string
value?: unknown
rendered?: string
/** 指标名 */
name?: string
caliber?: string
version?: string
node_id?: string
tool?: string
/** 和答案里那个数差多少 */
diff?: number
}
/** 旧运行的猜测:guess_sources 的输出(agentlab.guess/1) */
export interface EvidenceGuess {
schema?: string
mode?: string
/** 「猜测的来源,不能当证据:…」 */
note?: string
markdown?: string
segments?: EvidenceSegment[]
stats?: { numbers?: number; guessed?: number; unguessed?: number; candidates?: number }
/** 猜的是成果里哪个字段 */
field?: string
}
/** 句子、列表项或表格单元格 */
@@ -681,6 +745,11 @@ export interface EvidenceStats {
see?: number
uncited_claims?: number
violations?: number
/** 三期:有出处的实体、引文片段数;可疑实体、核对不了的名字数。老文档没有这几个键,缺省当 0 */
entities?: number
quotes?: number
unknown_entities?: number
unverified_entities?: number
}
/** report_doc 工件的内容 */
@@ -724,12 +793,23 @@ export interface EvidenceGraph {
schema?: string
/** cited / legacy_contract / legacy_text / none */
mode?: string
/** 没有报告时为什么(none)、旧版出具怎么认的(legacy_*):一句人话 */
note?: string
/**
* legacy_contract:{note, tier, matched, unmatched, field},按旧契约的位置标出的数字;
* legacy_text:guess_sources 的输出(EvidenceGuess),可能按字段分成几份
*/
legacy?: Record<string, any> | Record<string, any>[] | null
/** legacy_text 另一种放法:猜测单独放在这里 */
guess?: EvidenceGuess | EvidenceGuess[] | Record<string, EvidenceGuess> | null
seal?: EvidenceSeal
reports?: { node_id?: string; doc_artifact?: string; doc_sealed?: boolean; fields?: string[]; stats?: EvidenceStats
/** 文档取回时哈希复验:false 对不上(或不是这次运行这个节点写的),null 文件不在了 */
hash_ok?: boolean | null }[]
hash_ok?: boolean | null
/** 写这份报告时的结论句策略(report.checked 里记的;老后端、升级前的运行没有) */
claims?: string | null }[]
evidence?: { alias?: string; eid?: string; kind?: string; label?: string; node_id?: string; artifact?: string
sealed?: boolean; cited_by?: string[] }[]
sealed?: boolean; cited_by?: string[]; report?: string }[]
edges?: { from: string; to: string; rel: string }[]
}
@@ -864,6 +944,98 @@ export interface EvidenceStep extends Omit<EvidenceInput, 'status'> {
note?: string
/** 数据源改名或删掉了、按查询当时记下的遮罩处理时的那句说明。行照样有 */
mask_note?: string
// ---- entity:表或字段(三期)----
/** entity:table / column */
kind?: string
table?: string
column?: string
/** 字段类型(表结构快照里记的) */
type?: string
/** 同名字段在好几张表里时,是哪几张 */
tables?: string[]
/** 这个名字的来历:表结构快照、查询 SQL、查询结果列 */
sources?: EvidenceEntitySource[]
/** 出现在哪几次查询里(报告目录的全局编号) */
queries?: string[]
/** 表结构快照什么时候同步的 */
synced_at?: string
/** 可疑实体:最接近的已知名字 */
closest?: (string | { alias?: string; name?: string; table?: string; kind?: string })[]
/** 可疑实体:查过几份表结构快照、几次查询 */
checked?: { schemas?: number; queries?: number }
/** 表结构快照只存了一部分(库里表太多) */
snapshot_truncated?: boolean
/** 好几张表都有的同名字段:各表的类型 */
types?: Record<string, string>
/** 表:几个字段、是不是视图;表或字段的注释 */
is_view?: boolean
comment?: string
nullable?: boolean
primary_key?: boolean
// ---- quote:逐字引文(三期)----
/** quote:引文所在那条命中的原文(也可能放在 content 里) */
text?: string
/** quote:引文所在那条命中的全文;快照不在封存范围里、取不回来时是 null */
content?: string | null
quote?: string
match?: { hit?: number; start?: number; end?: number }
/** quote:按记下的位置把原文切出来,和引文逐字比对的结果 */
match_ok?: boolean | null
/** quote:哪个知识库、检索的是什么 */
collection?: string
query?: string | null
}
/** 实体的一处来历 */
export interface EvidenceEntitySource {
/** schema(表结构快照)/ sql(查询 SQL 用到的表)/ result(查询结果列) */
kind?: string
artifact?: string
alias?: string
source?: string
truncated?: boolean
}
/** GET /runs/{id}/evidence/audit 的一行(形状照 api/evidence.py;前端经 lib/evidence 的 auditFromApi 读) */
export interface EvidenceAuditRow {
/** cited / none / suspicious / candidate */
group?: string
report?: string | null
field?: string | null
segment?: string | null
unit?: string | null
/** number / value / entity / quote / claim / violation */
kind?: string
text?: string
state?: string
issue?: string | null
ref?: string | null
alias?: string | null
evidence_kind?: string | null
evidence?: string | null
eid?: string | null
artifact?: string | null
node_id?: string | null
sealed?: boolean | null
span?: [number, number] | null
sentence?: string
note?: string | null
candidates?: EvidenceCandidate[]
}
/** GET /runs/{id}/evidence/audit:记录页的审计表 */
export interface EvidenceAudit {
run_id?: string
schema?: string
mode?: string
seal?: EvidenceSeal & { events?: number; message?: string }
reports?: { node_id?: string; doc_artifact?: string; hash_ok?: boolean | null; doc_sealed?: boolean
fields?: string[]; claims?: string | null; stats?: EvidenceStats }[]
groups?: { key?: string; label?: string; count?: number; rows?: EvidenceAuditRow[] }[]
counts?: Record<string, number>
total?: number
note?: string
legacy_note?: string
}
/** GET /runs/{id}/evidence/segments/{sid}:点开一个片段 */
@@ -880,4 +1052,6 @@ export interface EvidenceSegmentDetail {
seal?: EvidenceSeal
/** 查询步骤里被遮罩的列(数据源 options.mask_columns)。遮罩只减少暴露,不是安全边界 */
redacted?: { columns?: string[] }
/** 可疑实体:最接近的已知名字(最多 3 个),给「是不是想写…」 */
closest?: (string | { alias?: string; name?: string; table?: string; kind?: string })[]
}
+3 -2
View File
@@ -52,9 +52,10 @@ const ORDER = [
]
// 最近一次全量里各项的耗时(秒)。只决定排队顺序:长的先开跑,短的填空,结果不受影响。
// 新加的检查不在表里就按 60 秒算
// 2026-09-29 三期收尾那一次四道并行的实测(秒):只用来排先后,长的先跑
const ESTIMATE = {
'run-states': 220, 'canvas-fx': 181, stream: 162, studio: 125, chat: 124, manage: 115, runs: 91,
shell: 84, 'ui-kit': 69, ui: 61, evidence: 56, publish: 31, guards: 10, 'canvas-layout': 4, tokens: 1, trace: 1, decode: 1,
'run-states': 299, 'canvas-fx': 264, stream: 176, chat: 164, studio: 153, manage: 142, runs: 105,
shell: 85, 'ui-kit': 74, evidence: 67, ui: 67, publish: 54, guards: 17, 'canvas-layout': 8, tokens: 1, trace: 1, decode: 1,
}
// 量时间最多、以前并行最容易挂的:同一时刻最多跑其中一项
const EXCLUSIVE = new Set(['run-states', 'canvas-fx'])
+280 -3
View File
@@ -27,6 +27,11 @@ const fx = JSON.parse(readFileSync(`${root}frontend/src/run/__tests__/evidence-d
// 第二期:报告直接引用查询单元格、整表,口径卡的输入核对到 agent 字段和查询快照。文档同样是
// compose_doc 真跑出来的(查询快照走 loader),片段接口的答复按方案第 5 节和 NOTES-A2 的形状拼
const fxq = JSON.parse(readFileSync(`${root}frontend/src/run/__tests__/evidence-query.json`, 'utf8'))
// 第三期:表名字段名(反引号里的、标记的、自动链接的)、可疑实体、核对不了的名字、逐字引文,
// 以及没有契约的旧运行按数值猜的候选。文档和猜测都是后端真跑出来的(compose_doc / guess_sources)
const fxe = JSON.parse(readFileSync(`${root}frontend/src/run/__tests__/evidence-entity.json`, 'utf8'))
/** 夹具里某段文字是哪个片段:检查按文字认片段,重新生成夹具时编号变了也不用改 */
const segOf = (doc, text) => doc.blocks.flatMap((b) => b.units.flatMap((u) => u.segments)).find((s) => s.text === text)?.id
let failed = 0
const check = (name, cond, detail = '') => {
@@ -55,10 +60,10 @@ async function routed(page) {
const url = new URL(req.url())
if (req.method() !== 'GET') return r.abort()
const seg = url.pathname.match(/\/api\/runs\/([^/]+)\/evidence\/segments\/([^/]+)$/)
// 按运行认夹具:第二期那份(查询链)和第一期那份各有自己的片段
const set = url.pathname.includes(`/runs/${fxq.run_id}/`) ? fxq : fx
// 按运行认夹具:第一期、第二期(查询链)、第三期(实体、引文)各有自己的片段
const set = url.pathname.includes(`/runs/${fxq.run_id}/`) ? fxq : url.pathname.includes(`/runs/${fxe.run_id}/`) ? fxe : fx
if (seg) {
hits.push(`${set === fxq ? 'q:' : ''}${decodeURIComponent(seg[2])}`)
hits.push(`${set === fxq ? 'q:' : set === fxe ? 'e:' : ''}${decodeURIComponent(seg[2])}`)
const body = set.segments[decodeURIComponent(seg[2])]
return body ? r.fulfill({ json: body })
: r.fulfill({ status: 404, json: { detail: '报告里没有这个片段', code: 'evidence_segment_not_found' } })
@@ -1125,6 +1130,48 @@ await section('studio', '报告撰写节点:画布认得、检查器能配(R
check('「指标来自」只列上游的口径卡(下游那张不列)', opts.length === 1 && opts[0].includes('周报口径'), opts.join(' | '))
const onViolation = await p.locator('[data-field="on_violation"] option').allInnerTexts()
check('违规时的默认写明按运行类别', onViolation[0]?.includes('探索运行照常产出'), onViolation.join(' | '))
// 三期:结论句策略 claims(off / require_citation,judge 是四期的,不出现在选项里)和表名字段名核对 entities
// 表名字段名核对收在「高级选项」里(很少要改),先展开
await p.locator('button[aria-expanded]', { hasText: '高级选项' }).click().catch(() => {})
await p.waitForTimeout(100)
const all = await p.evaluate(() => [...document.querySelectorAll('[data-field]')].map((el) => el.getAttribute('data-field')))
check('检查器有结论句策略、表名字段名核对(高级选项里)两项', fields.includes('claims') && all.includes('entities')
&& !fields.includes('entities'), all.join(','))
const claimVals = await p.locator('[data-field="claims"] option').evaluateAll((els) => els.map((e) => e.value))
check('结论句策略只有 off / require_citation,没有 judge', claimVals.join(',') === 'off,require_citation', claimVals.join(','))
const claimsLabel = await p.locator('[data-field="claims"] label').first().innerText().catch(() => '')
check('结论句策略有中文标签(发布前修复的预览也用它)', /结论句/.test(claimsLabel) && !/claims/.test(claimsLabel), claimsLabel)
const claimsHelp = await p.locator('[data-field="claims"]').innerText()
check('……说明写明「结论句裁判在后续版本支持」、受管级别要显式选', claimsHelp.includes('结论句裁判在后续版本支持')
&& claimsHelp.includes('受管'), claimsHelp.replace(/\s+/g, ' ').slice(0, 160))
check('没写 claims 时显示默认的 off', await p.locator('[data-field="claims"] select').inputValue() === 'off'
|| (await p.locator('[data-field="claims"] select').evaluate((el) => el.options[el.selectedIndex]?.value)) === 'off')
const entityVals = await p.locator('[data-field="entities"] option').evaluateAll((els) => els.map((e) => e.value))
check('表名字段名核对:link / off', entityVals.join(',') === 'link,off', entityVals.join(','))
await p.locator('[data-field="claims"] select').selectOption('require_citation')
const cfgOf = () => p.evaluate(() => window.__studio.getState().nodes.find((n) => n.id === 'write')?.data.config)
check('选「要求挂依据」写进 config.claims', (await cfgOf())?.claims === 'require_citation')
// 手写或别处写进来的 judge:不能悄悄显示成 off,要照实写出来并说后续版本支持
await p.evaluate(() => {
const st = window.__studio.getState()
const n = st.nodes.find((x) => x.id === 'write')
st.updateNode('write', { config: { ...n.data.config, claims: 'judge' } })
})
await p.waitForTimeout(150)
const shown = await p.locator('[data-field="claims"] select').evaluate((el) => el.options[el.selectedIndex]?.textContent ?? '')
check('config 里写着 judge:下拉照实显示 judge 并说后续版本支持,不装成 off', shown.includes('judge') && shown.includes('后续版本'), shown)
const fixLabel = await p.evaluate(async () => {
// 按页面自己加载时的地址 import:改过的模块地址带 ?t=,直接写 /src/… 会拿到另一份实例
const url = performance.getEntriesByType('resource').map((e) => e.name)
.find((n) => { try { return new URL(n).pathname === '/src/canvas/issues.ts' } catch { return false } })
const m = await import(url ?? '/src/canvas/issues.ts')
return [m.fixFieldLabel('claims', 'report'), m.fixFieldLabel('contract.claims.on_uncited', 'output'),
m.fixValueText('ignore', 'contract.claims.on_uncited'), m.fixValueText('degrade', 'contract.claims.on_uncited')]
}).catch((e) => [String(e)])
check('发布前修复预览里 claims 这个字段写中文名', fixLabel[0] === claimsLabel.trim() && !/claims/.test(fixLabel[0]), fixLabel[0])
check('契约 claims 的 on_uncited(ignore 改成 degrade 那条修复):字段和值都写中文',
!/claims|on_uncited/.test(fixLabel[1] ?? 'claims') && fixLabel[2] === '只标出来,不算缺口' && fixLabel[3] === '计入缺口、出具降档',
JSON.stringify(fixLabel.slice(1)))
if (SHOTS) {
for (const theme of ['dark', 'light']) {
await p.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme)
@@ -1309,6 +1356,236 @@ await section('config', '配置项:agent 的 cite_fields、code 的证据角
await ctx.close()
})
// ------------------------------------------------------------------ 第三期:实体、引文、旧运行的猜测
const E = {
num: segOf(fxe.doc, '45,678.5'), orders: segOf(fxe.doc, 'orders'), week: segOf(fxe.doc, 'orders.week'),
refunds: segOf(fxe.doc, '`refunds`'), column: segOf(fxe.doc, '`refunds.refund_amount`'),
coupon: segOf(fxe.doc, '`orders.coupon_code`'), discount: segOf(fxe.doc, 'orders.discount'),
promo: segOf(fxe.doc, '`promo_events`'), customers: segOf(fxe.doc, 'customers'), bare: segOf(fxe.doc, '30'),
quote: segOf(fxe.doc, '退款金额以财务确认日为准,未确认的退款不计入当周'), miss: segOf(fxe.doc, '周报一律按自然周统计'),
}
const ent = '#evidence-entity'
await section('entity', '三期:表名字段名、可疑实体、逐字引文——四通道分得开', async () => {
check('夹具里认得出每一种片段', Object.values(E).every(Boolean), JSON.stringify(E))
const look = await page.evaluate(([sel, E]) => Object.fromEntries(Object.entries(E).map(([k, id]) => {
const el = document.querySelector(`${sel} [data-seg="${id}"]`)
if (!el) return [k, null]
const cs = getComputedStyle(el)
return [k, { style: cs.textDecorationStyle, color: cs.textDecorationColor, state: el.getAttribute('data-ev-state'),
kind: el.getAttribute('data-ev-kind'), label: el.getAttribute('aria-label') ?? '', text: el.innerText,
code: !!el.querySelector('code'), before: getComputedStyle(el, '::before').content }]
})), [ent, E])
check('有出处的实体和数字同一套「有出处」线型:实线、同一个颜色', look.orders?.state === 'deterministic'
&& look.orders.style === 'solid' && look.orders.color === look.num?.color, JSON.stringify([look.orders, look.num?.color]))
check('可疑实体用「无证据」的线型(点状),和有出处的实体线型不同', look.coupon?.state === 'suspect'
&& look.coupon.style === 'dotted' && look.coupon.style !== look.orders?.style, JSON.stringify(look.coupon))
check('标记写的可疑实体([[c:…]] 编造的字段)同样标成可疑', look.discount?.state === 'suspect' && look.discount.style === 'dotted',
JSON.stringify(look.discount))
check('可疑实体的 aria-label 写「可能是编造的名字」,名字不带反引号', look.coupon?.label.startsWith('orders.coupon_code,')
&& look.coupon.label.includes('可能是编造的名字'), look.coupon?.label)
check('核对不了的名字:点状线、自己的说法,不说「编造」', look.promo?.state === 'unverified' && look.promo.style === 'dotted'
&& look.promo.label.includes('核对不了') && !look.promo.label.includes('编造'), look.promo?.label)
check('核对不了和可疑实体颜色不同(它只是标注,不醒目)', look.promo?.color !== look.coupon?.color, `${look.promo?.color} / ${look.coupon?.color}`)
check('反引号里的实体按行内代码画,不露反引号', look.refunds?.code && !look.refunds.text.includes('`')
&& look.column?.code, JSON.stringify([look.refunds, look.column?.text]))
check('自动链接的名字(正文里的已知表名)也是有出处的实体', look.customers?.state === 'deterministic' && look.customers.kind === 'entity')
check('aria-label:实体说清是表还是字段、出现在哪几次查询', look.orders?.label === 'orders,有出处:表 orders · 出现在查询 Q1、Q2',
look.orders?.label)
check('引文片段用引用样式:前面挂引号、实线', look.quote?.kind === 'quote' && look.quote.before.includes('“')
&& look.quote.style === 'solid' && look.quote.state === 'deterministic', JSON.stringify(look.quote))
check('原话对不上的引文照样是引用样式,但线型换成点状(无证据)', look.miss?.kind === 'quote' && look.miss.before.includes('“')
&& look.miss.style === 'dotted' && look.miss.state === 'none', JSON.stringify(look.miss))
check('引文的 aria-label 写出出处', look.quote?.label.includes('有出处') && look.quote.label.includes('运营手册 · 退款'), look.quote?.label)
const tags = await page.locator(`${ent} .ev-tag`).allInnerTexts()
check('句末小标签:可疑名字挂「?!可疑名字」,和「?无证据」分开', tags.includes('?!可疑名字') && tags.includes('?无证据'), tags.join('|'))
const tally = await page.locator(`${ent} [data-evidence-tally]`).innerText().catch(() => '')
check('证据条另说可疑名字的个数', tally.includes('可疑名字 2'), tally)
const summary = await page.locator(`${ent} [data-evidence-summary]`).innerText().catch(() => '')
check('读屏摘要写出可疑实体、核对不了的个数', summary.includes('2 个可能是编造的名字') && summary.includes('1 个名字核对不了'), summary)
// n / N:可疑实体进跳转(异常态),核对不了的不进(只是标注)
await page.locator(`${ent} [data-seg="${E.num}"]`).focus()
const jumps = []
for (let i = 0; i < 5; i++) {
await page.keyboard.press('n')
jumps.push(await active(page))
}
check('n 依次跳到裸数字、可疑实体、原话对不上的引文,跳过核对不了的名字',
jumps.join(',') === [E.bare, E.coupon, E.discount, E.miss, E.bare].join(','), jumps.join(','))
})
await section('e-panel', '三期:面板里的实体步骤、可疑实体的提示、引文步骤', async () => {
const p = panel(page)
await page.locator(`${ent} [data-seg="${E.week}"]`).click()
await page.waitForSelector('[data-evidence-panel] [data-ev-entity-type]', { timeout: 4000 }).catch(() => {})
check('面板标题是名字本身', (await p.locator('h3').innerText().catch(() => '')) === 'orders.week')
check('实体步骤写字段类型', (await p.locator('[data-ev-entity-type]').innerText().catch(() => '')).includes('VARCHAR(8)'))
check('实体步骤写表结构快照的同步时间', (await p.locator('[data-ev-entity-synced]').innerText().catch(() => '')).includes('2026-09-20'),
await p.locator('[data-ev-entity-synced]').innerText().catch(() => ''))
check('快照不全时照实说只存了一部分', (await p.innerText()).includes('只存了一部分'))
check('实体的来历:表结构快照', (await p.locator('[data-ev-entity-source]').allInnerTexts()).some((t) => t.includes('表结构快照')))
await page.locator(`${ent} [data-seg="${E.orders}"]`).click()
await page.waitForTimeout(250)
const queries = await p.locator('[data-ev-entity-queries]').innerText().catch(() => '')
check('表:出现在哪几次查询里', queries.includes('Q1') && queries.includes('Q2'), queries)
const srcs = await p.locator('[data-ev-entity-source]').allInnerTexts()
check('表的来历逐条列:表结构快照、查询 SQL', srcs.some((t) => t.includes('查询 Q2') && t.includes('SQL')), srcs.join(' | '))
check('实体步骤取自证据接口(按 runId:segId 取了一次)', harness.hits.includes(`e:${E.orders}`), harness.hits.filter((h) => h.startsWith('e:')).join(','))
await page.locator(`${ent} [data-seg="${E.discount}"]`).click()
// 提示要等片段接口回来(先显示「正在取」):等到名字出来再读
await page.waitForSelector('[data-evidence-panel] [data-ev-closest-name]', { timeout: 4000 }).catch(() => {})
const reason = await p.locator('[data-ev-reason]').innerText().catch(() => '')
check('可疑实体说清为什么:本次运行里哪里都没有这个名字', reason.includes('可能是编造的名字'), reason.slice(0, 80))
const closest = await p.locator('[data-ev-closest-name]').allInnerTexts()
check('可疑实体给出最接近的已知名字(最多 3 个,不带 c: 前缀)', closest.length >= 1 && closest.length <= 3
&& closest.includes('orders.amount') && closest.every((n) => !n.startsWith('c:')), closest.join(','))
check('可疑实体的面板徽标:?! 可能是编造的名字', (await p.locator('[data-ev-badge]').innerText().catch(() => '')).includes('可能是编造的名字'))
await page.locator(`${ent} [data-seg="${E.promo}"]`).click()
await page.waitForTimeout(250)
const promo = await p.innerText()
check('核对不了的名字:说快照不全、核对不了,不说编造', promo.includes('核对不了') && !promo.includes('编造'), promo.slice(0, 120))
await page.locator(`${ent} [data-seg="${E.quote}"]`).click()
await page.waitForSelector('[data-evidence-panel] [data-ev-quote-hit]', { timeout: 4000 }).catch(() => {})
const src = await p.locator('[data-ev-quote-source]').innerText().catch(() => '')
check('引文步骤:原文所在的文档和片段', src.includes('运营手册 · 退款') && src.includes('第 3 段'), src)
const hit = await p.locator('[data-ev-quote-hit]').innerText().catch(() => '')
check('引文在原文里的位置高亮出来,就是那句原话', hit === '退款金额以财务确认日为准,未确认的退款不计入当周', hit)
const ctx = await p.locator('[data-ev-quote]').innerText().catch(() => '')
check('高亮前后带着原文的上下文', ctx.includes('第三章 退款口径') && ctx.includes('部分退款'), ctx.slice(0, 80))
await page.locator(`${ent} [data-seg="${E.miss}"]`).click()
await page.waitForTimeout(250)
const miss = await p.innerText()
check('原话对不上的引文:说清找不到这句原话', miss.includes('找不到这句原话'), miss.slice(0, 120))
await page.keyboard.press('Escape')
if (SHOTS) {
for (const theme of ['dark', 'light']) {
await page.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme)
for (const [name, id] of [['entity', E.orders], ['suspect', E.discount], ['quote', E.quote]]) {
await page.locator(`${ent} [data-seg="${id}"]`).click()
await page.waitForTimeout(350)
await page.screenshot({ path: `${SHOTS}/evidence3-${name}-${theme}.png` })
}
await page.keyboard.press('Escape')
}
await page.evaluate(() => document.documentElement.setAttribute('data-theme', 'dark'))
}
})
await section('e-narrow', '三期在 360px:栏内展开不横向滚动;减少动效', async () => {
const narrow = '#evidence-entity-narrow'
const overflow = () => page.evaluate((sel) => {
const el = document.querySelector(sel)
return { sw: el.scrollWidth, cw: el.clientWidth }
}, narrow)
for (const id of [E.quote, E.discount, E.orders]) {
await page.locator(`${narrow} [data-seg="${id}"]`).click()
await page.waitForSelector(`${narrow} [data-evidence-panel="inline"]`, { timeout: 4000 }).catch(() => {})
await page.waitForTimeout(250)
const o = await overflow()
check(`360px 栏内面板(${id})不出横向滚动`, o.sw <= o.cw, JSON.stringify(o))
}
await page.keyboard.press('Escape')
const r = await open('/ui-harness.html?evidence=1', { reduced: true, w: 360, h: 780 })
const scroll = () => r.page.evaluate(() => ({ sw: document.documentElement.scrollWidth, vw: innerWidth }))
await r.page.locator(`${ent} [data-seg="${E.quote}"]`).click()
await r.page.waitForSelector('[data-evidence-panel] [data-ev-quote]', { timeout: 4000 }).catch(() => {})
const s1 = await scroll()
check('360px 屏幕:引文面板(底部抽屉)整页不横向滚动', s1.sw <= s1.vw, JSON.stringify(s1))
const anim = await r.page.evaluate(() => document.querySelector('[data-evidence-panel]')?.getAnimations({ subtree: true })
.filter((a) => !(a instanceof CSSTransition && Number(a.effect?.getTiming().duration) <= 0.01)).length ?? -1)
check('减少动效:引文面板没有动画', anim === 0, String(anim))
await r.page.keyboard.press('Escape')
await r.page.locator('#evidence-guess [data-ev-guess-toggle]').click().catch(() => {})
await r.page.waitForTimeout(150)
const s2 = await scroll()
check('360px 屏幕:展开猜测之后整页不横向滚动', s2.sw <= s2.vw
&& await r.page.locator('#evidence-guess [data-ev-guess]').getAttribute('data-open').catch(() => null) === 'true', JSON.stringify(s2))
const guessAnim = await r.page.evaluate(() => document.querySelector('#evidence-guess [data-ev-guess]')?.getAnimations({ subtree: true })
.filter((a) => !(a instanceof CSSTransition && Number(a.effect?.getTiming().duration) <= 0.01)).length ?? -1)
check('减少动效:展开猜测没有动画', guessAnim === 0, String(guessAnim))
check('没有运行时报错(三期 360px)', r.errors.length === 0, r.errors.join(' | '))
await r.ctx.close()
})
await section('banner', '出具横幅那一行:数字都有出处时,可疑名字、没挂依据的结论句也写出来', async () => {
const at = (key) => page.locator(`#issuance-line [data-issuance-case="${key}"]`)
const line = await at('suspect').locator('[data-evidence-line]').innerText().catch(() => '')
check('数字那半句照旧:3 个数字都有出处', line.includes('3 个数字都有出处'), line)
check('可疑名字另起一句:?! 可疑名字 2', await at('suspect').locator('[data-evidence-line-suspect]').count() === 1
&& line.includes('?! 可疑名字 2'), line)
check('没挂依据的结论句另起一句:没挂依据的结论句 1', await at('suspect').locator('[data-evidence-line-claims]').count() === 1
&& line.includes('没挂依据的结论句 1'), line)
const next = at('suspect').locator('[data-evidence-next]')
check('数字都有出处、只有可疑名字时也给「定位下一处」', await next.count() === 1)
await page.evaluate(() => { window.__issuanceNext = 0 })
await next.click()
check('点了它就去找下一处', await page.evaluate(() => window.__issuanceNext) === 1)
const clean = await at('clean').locator('[data-evidence-line]').innerText().catch(() => '')
check('都干净的那一份:只有数字那半句,不给「定位下一处」', clean.includes('3 个数字都有出处')
&& await at('clean').locator('[data-evidence-line-suspect], [data-evidence-line-claims], [data-evidence-next]').count() === 0, clean)
const names = await at('names').locator('[data-evidence-line]').innerText().catch(() => '')
check('一个数字都没有、只有可疑名字:这一行照样出来,开头不带「·」,也给「定位下一处」',
names.trim().startsWith('?! 可疑名字 1') && await at('names').locator('[data-evidence-next]').count() === 1, names)
})
await section('guess', '旧运行按数值猜的候选:默认折叠,展开写明「猜测」,淡点状线', async () => {
const g = '#evidence-guess'
const box = page.locator(`${g} [data-ev-guess]`)
check('默认折叠', await box.getAttribute('data-open').catch(() => null) === 'false')
check('折叠时看不到候选(不在页面上,不只是藏起来)', await page.locator(`${g} [data-ev-state="candidate"]`).count() === 0)
const head = await page.locator(`${g} [data-ev-guess-toggle]`).innerText().catch(() => '')
check('折叠的标题就写明「猜测」', head.includes('猜测'), head)
const toggle = page.locator(`${g} [data-ev-guess-toggle]`)
check('展开按钮是 button,带 aria-expanded=false', await toggle.evaluate((el) => el.tagName).catch(() => '') === 'BUTTON'
&& await toggle.getAttribute('aria-expanded').catch(() => null) === 'false')
const answer = await page.locator(`${g}`).innerText().catch(() => '')
check('折叠时答案本身照常显示', answer.includes('本周销售额 45,678.5 元'), answer.slice(0, 60))
await toggle.click()
await page.waitForTimeout(150)
check('展开后 aria-expanded=true', await toggle.getAttribute('aria-expanded').catch(() => null) === 'true')
const note = await page.locator(`${g} [data-ev-guess-note]`).innerText().catch(() => '')
check('展开后写明「猜测的来源,不能当证据」', note.includes('猜测的来源,不能当证据'), note)
const cands = await page.evaluate((sel) => [...document.querySelectorAll(`${sel} [data-ev-state="candidate"]`)].map((el) => {
const cs = getComputedStyle(el)
// 候选是 Markdown 里的行内标记(保留旧答案的版式),读屏靠标记里那句 sr-only 的「猜测的来源:…」
return { style: cs.textDecorationStyle, color: cs.textDecorationColor, line: el.getAttribute('data-ev-line'),
label: el.textContent ?? '', title: el.getAttribute('title') ?? '', text: el.firstChild?.textContent ?? '' }
}), g)
check('有候选的数字画成候选(4 个)', cands.length === 4, cands.map((c) => c.text).join(','))
check('候选用淡点状线,不用确定性的实线', cands.length > 0 && cands.every((c) => c.style === 'dotted' && c.line === 'dotted'),
JSON.stringify(cands[0]))
const detColor = await page.evaluate((sel) => {
const el = document.querySelector(`${sel} [data-ev-state="deterministic"]`)
return el ? getComputedStyle(el).textDecorationColor : ''
}, ent)
check('候选的线和「有出处」不是一个颜色', cands.length > 0 && cands.every((c) => c.color !== detColor), `${cands[0]?.color} / ${detColor}`)
check('候选给读屏写「猜测的来源」和候选出处,悬停也看得到', cands[0]?.label.includes('猜测的来源') && cands[0].label.includes('Q1')
&& cands[0].title.includes('Q1'), cands[0]?.label)
const list = await page.locator(`${g} [data-ev-candidates]`).innerText().catch(() => '')
check('候选清单:每个数字可能来自哪一格', list.includes('查询 Q1') && list.includes('gmv'), list.slice(0, 120))
const plain = await page.evaluate((sel) => [...document.querySelectorAll(`${sel} [data-ev-guess-body] [data-ev-state]`)]
.map((el) => el.firstChild?.textContent ?? ''), g)
check('没猜到候选的数字不画线(40%、50,000)', !plain.includes('40%') && !plain.includes('50,000'), plain.join(','))
await toggle.click()
await page.waitForTimeout(100)
check('再点一次收起', await box.getAttribute('data-open').catch(() => null) === 'false')
if (SHOTS) {
await toggle.click()
for (const theme of ['dark', 'light']) {
await page.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme)
await page.waitForTimeout(200)
await page.locator(g).screenshot({ path: `${SHOTS}/evidence3-guess-${theme}.png` })
}
await page.evaluate(() => document.documentElement.setAttribute('data-theme', 'dark'))
await toggle.click()
}
})
if (SHOTS) {
for (const theme of ['dark', 'light']) {
await page.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme)
+87 -3
View File
@@ -100,6 +100,23 @@ const DUP = {
edges: chain('in', 'ask1', 'ask2', 'caliber', 'done'),
}
/**
* 两个 choice:沙箱代码喂口径卡(G4),候选里「交给 Copilot」那一项带 handoff 标记;成果字段接哪个报告
* (G1),唯一的候选是一个 id 恰好叫 copilot 的报告撰写节点——它只是普通候选,不是「交给 Copilot」
*/
const HANDOFF = {
nodes: [
node('in', 'input', 0, '入口', { fields: [{ name: 'week', required: true }] }),
node('calc', 'code', 240, '计算', { code: 'print(1)', assign_to: 'calc' }),
node('caliber', 'metrics', 480, '周报口径', { caliber: '周报口径', caliber_version: 'v1',
metrics: [{ id: 'ratio', name: '比例', expression: 'vars.calc.ratio' }] }),
node('copilot', 'report', 720, '报告撰写', { instructions: '写周报', numbers: 'strict', on_violation: 'fail', claims: 'require_citation' }),
node('done', 'output', 960, '成果', { fields: [{ name: 'report', value: '{{ vars.story }}' }],
contract: { report_from: 'copilot', metrics_from: ['caliber'], required: ['ratio'], strict: true } }),
],
edges: chain('in', 'calc', 'caliber', 'copilot', 'done'),
}
const wf = (id, name, graph, extra = {}) => ({
id, name, description: '检查脚本伪造的工作流', graph, tags: [], version: 3, is_template: false,
status: 'draft', published_version: null, published_by: null, run_count: 0,
@@ -111,6 +128,7 @@ const FAKES = {
'pf-lib': wf('pf-lib', '__publish_check_lib__', { nodes: [], edges: [] }, { status: 'published', published_version: 4 }),
'pf-bare': wf('pf-bare', '__publish_check_bare__', BARE, { status: 'governed', published_version: 2 }),
'pf-dup': wf('pf-dup', '__publish_check_dup__', DUP, { status: 'governed', published_version: 2 }),
'pf-handoff': wf('pf-handoff', '__publish_check_handoff__', HANDOFF, { status: 'governed', published_version: 2 }),
// 带全图默认的图:画布只管节点和连线,保存、检查、修复都得把 defaults 原样带着
'pf-defs': wf('pf-defs', '__publish_check_defaults__', { ...EASY, defaults: { approval: 'always', model: 'demo-model' } }),
}
@@ -162,6 +180,19 @@ function lint(graph, level, { followsNever = false } = {}) {
add(n, 'governed.subgraph_unpinned', `${who(n)}(子工作流)没有钉住版本,口径会随上游最新版漂移`,
{ kind: 'auto', label: '钉到它最新的已发布版本 v4', preview: { field: 'workflow_version', before: null, after: 4 } })
}
if (n.type === 'code' && c.evidence_role !== 'source') {
add(n, 'governed.caliber_compute_input', `${who(n)}(沙箱代码)的产出喂给了口径卡,而它的角色是计算`, {
kind: 'choice', label: `${who(n)}的产出喂给了口径卡:它是在取数,还是在做计算?`,
options: [{ value: 'source', label: '它在取数:标成 source(取数)' },
{ value: 'copilot', label: '它在做计算:交给 Copilot 把计算挪进口径卡', handoff: true }],
})
}
if (n.type === 'output' && c.contract?.report_from && !(c.fields ?? []).every((f) => f.value.includes(`nodes.${c.contract.report_from}.`))) {
add(n, 'governed.exit_text_source', `${who(n)}的成果字段取的不是报告撰写节点的正文`, {
kind: 'choice', label: '把成果字段改成取哪个报告撰写节点的正文',
options: graph.nodes.filter((r) => r.type === 'report').map((r) => ({ value: r.id, label: who(r), hint: '报告撰写' })),
})
}
if (n.type === 'supervisor') {
add(n, 'governed.supervisor', `${who(n)}(多 Agent 协作):受管模板不允许全动态规划的节点`,
{ kind: 'assist', label: '换成固定编排是结构性的改动,交给 Copilot' })
@@ -202,6 +233,7 @@ function autofix(body, { unpublished = false, followsNever = false } = {}) {
const changes = []
const rejected = []
const applied = []
const handoff = []
const byId = (id) => graph.nodes.find((n) => n.id === id)
for (const id of body.apply ?? []) {
const fix = before.fixes.find((f) => f.id === id)
@@ -230,6 +262,15 @@ function autofix(body, { unpublished = false, followsNever = false } = {}) {
if (!Array.isArray(v) || !v.length) { rejected.push({ fix_id: id, reason: '要人选:没给选择' }); continue }
const contract = { report_from: 'write', metrics_from: ['caliber'], strict: true, required: v }
change('contract', null, contract); c.contract = contract
} else if (fix.code === 'governed.caliber_compute_input') {
const v = body.choices?.[id]
// 只认候选上的 handoff 标记,和后端 autofix._handoff 一样
if (fix.options.some((o) => o.handoff && o.value === v)) { handoff.push(id); continue }
change('evidence_role', c.evidence_role ?? null, v); c.evidence_role = v
} else if (fix.code === 'governed.exit_text_source') {
const v = body.choices?.[id]
const to = `{{ nodes.${v}.text }}`
change('fields[0].value', c.fields[0].value, to); c.fields[0].value = to
} else if (fix.code === 'contract.required_missing') {
const v = body.choices?.[id]
if (!Array.isArray(v) || !v.length) { rejected.push({ fix_id: id, reason: '要人选:没给选择' }); continue }
@@ -238,7 +279,7 @@ function autofix(body, { unpublished = false, followsNever = false } = {}) {
applied.push(id)
}
let assist = null
if (body.assist) {
if (body.assist || handoff.length) {
const team = byId('team')
if (team) {
changes.push({ fix_id: 'assist', node_id: 'team', node_title: team.data.label, field: 'max_rounds', before: team.data.config.max_rounds, after: 2,
@@ -249,7 +290,7 @@ function autofix(body, { unpublished = false, followsNever = false } = {}) {
questions: ['复核团队要换成哪一个固定编排的 Agent?它现在只有一个成员 checker,保留它的职责吗?'] }
}
const remaining = lint(graph, body.level, { followsNever })
return { graph, changes, applied, rejected, remaining: remaining.issues, assist, ok: remaining.ok, ops: [] }
return { graph, changes, applied, rejected, handoff, remaining: remaining.issues, assist, ok: remaining.ok, ops: [] }
}
// ---------------------------------------------------------------- 浏览器
@@ -261,7 +302,7 @@ const writes = []
* backend:'new'(两个接口都有)| 'old'(两个接口 404)| 'nofix'(有检查、没有 autofix)
* save:'ok' | 'fail'(保存接口 500)| 'slow'(1.5 秒后才回,看保存进行中的样子)
*/
async function open({ id = 'pf-hard', width = 1440, height = 900, backend = 'new', save = 'ok', unpublished = false } = {}) {
async function open({ id = 'pf-hard', width = 1440, height = 900, backend = 'new', save = 'ok', unpublished = false, fixDelay = 0 } = {}) {
const followsNever = id === 'pf-dup'
const ctx = await browser.newContext({ viewport: { width, height } })
opened.add(ctx)
@@ -316,6 +357,7 @@ async function open({ id = 'pf-hard', width = 1440, height = 900, backend = 'new
const body = req.postDataJSON()
state.autofixes.push(body)
if (state.backend !== 'new') return json(route, { detail: 'Not Found' }, 404)
if (fixDelay) await new Promise((r) => setTimeout(r, fixDelay))
return json(route, autofix(body, { unpublished, followsNever }))
}
if (method === 'POST' && sub === 'publish') {
@@ -793,6 +835,48 @@ await section('全图默认不会被画布弄丢:保存、发布前检查、
void first
})
await section('choice 里选「交给 Copilot」:请求带 assist、等待时说 Copilot 在修;id 叫 copilot 的报告只是普通候选', async () => {
const { page, state, errors } = await open({ id: 'pf-handoff', fixDelay: 1200 })
await openDialog(page)
const g4 = dialog(page).locator('[data-fix-choice="governed.caliber_compute_input:calc"]')
const g1 = dialog(page).locator('[data-fix-choice="governed.exit_text_source:done"]')
await g4.waitFor()
check('两个 choice 都画出来了', await g1.count() === 1)
check('没选之前,按钮不是「交给 Copilot」的样子', await g4.locator('[data-fix-handoff]').count() === 0)
await g4.getByText('它在做计算', { exact: false }).click()
const button = g4.locator('[data-fix-action="choice"]')
check('选了「交给 Copilot」那一项,按钮换成「交给 Copilot」', await button.getAttribute('data-fix-handoff') === ''
&& (await button.innerText()).includes('交给 Copilot'), await button.innerText())
await button.click()
const loading = dialog(page).locator('[data-fix-preview="loading"]')
await loading.waitFor()
check('等待时写「Copilot 正在试着修」,不是「正在生成修复预览」', (await loading.innerText()).includes('Copilot 正在试着修'), await loading.innerText())
const body = state.autofixes.at(-1)
check('请求带 assist: true,选的值原样带上', body?.assist === true && body?.choices?.['governed.caliber_compute_input:calc'] === 'copilot'
&& JSON.stringify(body?.apply) === JSON.stringify(['governed.caliber_compute_input:calc']), JSON.stringify(body))
await dialog(page).locator('[data-fix-preview="ready"], [data-fix-preview]:not([data-fix-preview="loading"])').first().waitFor()
check('Copilot 的总结摆出来了', await waitFor(page, async () => (await dialog(page).locator('[data-fix-assist]').count()) > 0))
await shoot(page, 'publish-dialog-handoff', dialog(page))
// 另一个 choice:唯一的候选是 id 叫 copilot 的报告撰写节点——普通候选,不交给 Copilot
const { page: page2, state: state2, errors: errors2 } = await open({ id: 'pf-handoff', fixDelay: 1200 })
await openDialog(page2)
const exit = dialog(page2).locator('[data-fix-choice="governed.exit_text_source:done"]')
await exit.waitFor()
await exit.locator('input[type="radio"]').first().check()
const pick = exit.locator('[data-fix-action="choice"]')
check('选了 id 叫 copilot 的报告:按钮不变成「交给 Copilot」', await pick.getAttribute('data-fix-handoff') === null
&& !(await pick.innerText()).includes('交给 Copilot'), await pick.innerText())
await pick.click()
const wait2 = dialog(page2).locator('[data-fix-preview="loading"]')
await wait2.waitFor()
check('等待时写「正在生成修复预览」', (await wait2.innerText()).includes('正在生成修复预览'), await wait2.innerText())
const body2 = state2.autofixes.at(-1)
check('请求不带 assist,选的值是 copilot(节点 id)', !body2?.assist && body2?.choices?.['governed.exit_text_source:done'] === 'copilot',
JSON.stringify(body2))
check('没有页面错误', !errors.length && !errors2.length, [...errors, ...errors2].join(' | '))
})
await section('360px 下不横向滚动', async () => {
const { page } = await open({ width: 360, height: 780 })
// 窄屏上「发布」收在「更多操作」里
+296
View File
@@ -16,6 +16,8 @@ import { chromium } from '../frontend/node_modules/playwright-core/index.mjs'
const WEB = process.env.AGENTLAB_WEB ?? 'http://localhost:5273'
const API = process.env.AGENTLAB_API ?? 'http://localhost:8000/api'
// RUNS_SHOTS=<目录>:证据页签亮暗各截几张,供人眼复核
const SHOTS = process.env.RUNS_SHOTS ?? ''
const CHROME = process.env.CHROME_PATH
?? '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
@@ -47,7 +49,11 @@ const check = (name, cond, detail = '') => {
* 不让一处卡住把后面的检查一起吞掉。页面是各节共用的:
* 出错那一节停在哪,下一节就从哪接着
*/
// RUNS_ONLY=证据页签 只跑段名里含这些字的段(逗号分隔;改坏验证时省时间)。前面几段给后面留的数据
// (航迹那一节的时刻、工件)只在整本跑时有,单跑依赖它们的段会报「中途出错」;check-all 不传它
const ONLY = (process.env.RUNS_ONLY ?? '').split(',').filter(Boolean)
async function section(name, fn) {
if (ONLY.length && !ONLY.some((k) => name.includes(k)) && name !== '收尾') return
console.log(`\n=== ${name} ===`)
try {
await fn()
@@ -1698,6 +1704,296 @@ await section('屏幕矮、节点多:读数区不被航迹坞压住', async ()
fakes.delete('GET /api/approvals')
})
// ------------------------------------------------------------------ 证据页签(可点击证据三期)
// 夹具是后端真跑出来的(compose_doc、guess_sources、api/evidence.py 的审计函数),只用通用名
const fxe = JSON.parse(readFileSync(new URL('../frontend/src/run/__tests__/evidence-entity.json', import.meta.url), 'utf8'))
const EV = 'fake0evidence0000000000000000'
const EV_LEGACY = 'fake0evlegacy0000000000000000'
const EV_NONE = 'fake0evnone000000000000000000'
const EV_OLD = 'fake0evoldback000000000000000'
const evGraph = { nodes: [
{ id: 'in', type: 'input', position: { x: 0, y: 0 }, data: { label: '输入', config: {} } },
{ id: 'fetch', type: 'agent', position: { x: 200, y: 0 }, data: { label: '取数', config: { tools: ['db_query__shop'] } } },
{ id: 'manual', type: 'retrieve', position: { x: 200, y: 150 }, data: { label: '查手册', config: {} } },
{ id: 'write', type: 'report', position: { x: 400, y: 0 }, data: { label: '写周报', config: {} } },
{ id: 'out', type: 'output', position: { x: 600, y: 0 }, data: { label: '成果', config: {} } },
], edges: [{ source: 'in', target: 'fetch' }, { source: 'fetch', target: 'write' }, { source: 'manual', target: 'write' },
{ source: 'write', target: 'out' }] }
const evRun = (id, output) => ({
...base, id, workflow_id: null, workflow_name: '证据检查', status: 'succeeded', run_class: 'formal',
version: 3, version_hash: null, manifest_hash: 'f'.repeat(64), manifest_seq: 88, error: null, error_node_id: null,
input: { week: '2026-W37' }, output, usage: { input_tokens: 10, output_tokens: 10, cost_usd: 0, wall_ms: 4000, active_ms: 4000, wait_ms: 0 },
created_at: new Date(Date.now() - 7200e3).toISOString(), started_at: new Date(Date.now() - 7200e3).toISOString(),
finished_at: new Date(Date.now() - 7196e3).toISOString(),
})
const evEvents = (output, report = true) => {
const t0 = Date.now() / 1000 - 7200
return [
{ seq: 1, type: 'run.started', node_id: null, ts: t0, data: { nodes: 5 } },
{ seq: 2, type: 'node.started', node_id: 'write', ts: t0 + 1, data: { node_type: 'report', label: '写周报' } },
...(report ? [{ seq: 3, type: 'report.checked', node_id: 'write', ts: t0 + 3, data: fxe.report_checked }] : []),
{ seq: 4, type: 'node.finished', node_id: 'write', ts: t0 + 3.1, data: { duration_ms: 2100 } },
{ seq: 5, type: 'run.finished', node_id: null, ts: t0 + 4, data: { output, usage: {}, timing: { wall_ms: 4000, active_ms: 4000, wait_ms: 0 } } },
]
}
/** 证据相关的请求(审计、导出、片段)记下来:导出发出的请求要带对 format 和 groups */
const evRequests = []
page.on('request', (r) => {
const u = new URL(r.url())
if (/\/api\/runs\/fake0ev[^/]*\/evidence/.test(u.pathname)) evRequests.push(`${u.pathname}${u.search}`)
})
const auditFake = (body, csv) => (url) => {
const format = url.searchParams.get('format')
const groups = url.searchParams.get('groups')?.split(',')
const pick = groups ? { ...body, groups: body.groups.filter((g) => groups.includes(g.key)) } : body
if (format === 'csv') {
return { status: 200, body: csv, headers: { 'Content-Type': 'text/csv; charset=utf-8',
'Content-Disposition': `attachment; filename="evidence-${body.run_id.slice(0, 8)}.csv"` } }
}
if (format === 'json') {
return { status: 200, body: JSON.stringify(pick), headers: { 'Content-Type': 'application/json',
'Content-Disposition': `attachment; filename="evidence-${body.run_id.slice(0, 8)}.json"` } }
}
return { status: 200, json: pick }
}
const evFakes = (id, { output, graph, audit, csv, report = true }) => {
fakes.set(`GET /api/runs/${id}`, () => ({ status: 200, json: evRun(id, output) }))
fakes.set(`GET /api/runs/${id}/events`, () => ({ status: 200, json: evEvents(output, report) }))
fakes.set(`GET /api/runs/${id}/graph`, () => ({ status: 200, json: { graph: evGraph, workflow_id: null, version: 3 } }))
fakes.set(`GET /api/runs/${id}/artifacts`, () => ({ status: 200, json: [] }))
fakes.set(`GET /api/runs/${id}/evidence`, () => ({ status: 200, json: { ...graph, run_id: id } }))
fakes.set(`GET /api/runs/${id}/evidence/audit`, audit ? auditFake({ ...audit, run_id: id }, csv ?? '')
: () => ({ status: 404, json: { detail: 'Not Found' } }))
}
const openEvidence = async (id) => {
await page.goto(`${WEB}/runs/${id}?view=evidence`, { waitUntil: 'networkidle' })
await page.locator('[data-view-pane=evidence] [data-evidence-pane]:not([data-evidence-pane=loading])').waitFor({ timeout: 8000 })
await page.waitForTimeout(300)
}
const auditGroups = () => page.locator('[data-audit-table] tbody[data-audit-group]').evaluateAll((els) => els.map((e) => e.getAttribute('data-audit-group')))
const auditRowsShown = () => page.locator('[data-audit-table] tr[data-audit-row]').count()
await section('证据页签:左边报告、右边常驻面板、下方审计表', async () => {
evFakes(EV, { output: fxe.output, graph: fxe.graph, audit: fxe.audit, csv: fxe.audit_csv })
fakes.set(`GET /api/artifacts/${fxe.doc_artifact}`, () => ({ status: 200, json: { id: fxe.doc_artifact, content: fxe.doc } }))
fakes.set(`GET /api/runs/${EV}/evidence/segments/`, () => ({ status: 404, json: {} }))
await page.route(new RegExp(`/api/runs/${EV}/evidence/segments/`), (r) => {
const sid = decodeURIComponent(new URL(r.request().url()).pathname.split('/').pop())
return fxe.segments[sid] ? r.fulfill({ json: fxe.segments[sid] }) : r.fulfill({ status: 404, json: { detail: '没有这个片段' } })
})
await page.goto(`${WEB}/runs/${EV}`, { waitUntil: 'networkidle' })
await page.locator('[data-run-detail]').waitFor()
check('详情多了「证据」页签(在航迹和工件之间)', await viewTab('evidence').count() === 1
&& (await page.locator('[data-run-detail] [role=tab]').evaluateAll((els) => els.map((e) => e.getAttribute('data-tab')))).join(',') === 'stream,trace,evidence,artifacts')
check('没打开证据页签之前不取证据图和审计表', !evRequests.some((u) => u.includes(EV)), evRequests.join(' '))
await viewTab('evidence').click()
await page.locator('[data-view-pane=evidence] [data-evidence-pane=cited]').waitFor({ timeout: 8000 }).catch(() => {})
await page.waitForTimeout(400)
check('证据页签能打开,地址记着 ?view=evidence', await viewTab('evidence').getAttribute('aria-selected') === 'true'
&& new URL(page.url()).searchParams.get('view') === 'evidence')
check('左边是报告(逐段可点的文档)', await page.locator('[data-evidence-report=write] [data-evidence-doc] [data-seg]').count() > 5)
check('右边是常驻面板的位置,没点片段时说怎么用', await page.locator('[data-evidence-dock] [data-evidence-dock-idle]').count() === 1)
const seal = await page.locator('[data-evidence-seal]').innerText().catch(() => '')
check('显示封存核对的结果', await page.locator('[data-evidence-seal]').getAttribute('data-evidence-seal') === 'done'
&& seal.includes('已封存 · 核对一致') && seal.includes('88'), seal)
check('……也说报告文档的哈希一致', (await page.locator('[data-evidence-doc-hash=ok]').innerText().catch(() => '')).includes('写周报'))
check('审计表按状态分组:无证据、可疑实体在前,有出处在后', (await auditGroups()).join(',') === 'none,suspicious,cited',
(await auditGroups()).join(','))
const counts = await page.locator('[data-audit-group-toggle]').allInnerTexts()
check('每组写着条数(无证据 3、可疑实体 3、有出处 10)', counts.join('|').includes('无证据\n3') || (counts[0].includes('3') && counts[1].includes('3') && counts[2].includes('10')),
counts.join(' | ').replace(/\n/g, ' '))
check('一共 16 行', await auditRowsShown() === 16, String(await auditRowsShown()))
// 这张图的出口没有出具契约:后端说「要求了,但没有契约按它判档」,不说「出具时计入缺口」。表里照后端的原话写
const claim = page.locator('[data-audit-table] tbody[data-audit-group=none] tr[data-audit-row]', { hasText: '另外参考了' })
const claimNote = fxe.audit.groups.find((g) => g.key === 'none')?.rows.find((r) => r.kind === 'claim')?.note ?? '(夹具里没有结论句那行)'
check('没挂依据的结论句也列在无证据里,原因照后端的原话(claims: require_citation、没有出具契约)', await claim.count() === 1
&& claimNote.includes('require_citation') && (await claim.innerText()).includes(claimNote), claimNote)
check('可疑实体那组有「可能是编造的名字」和「核对不了」', (await page.locator('[data-audit-table] tbody[data-audit-group=suspicious]').innerText())
.includes('可能是编造的名字') && (await page.locator('[data-audit-table] tbody[data-audit-group=suspicious]').innerText()).includes('核对不了'))
check('有出处的行写着在封存范围内', await page.locator('[data-audit-table] tbody[data-audit-group=cited] [data-audit-sealed=yes]').count() === 10)
check('反引号里的名字在表里不带反引号', !(await page.locator('[data-audit-table]').innerText()).includes('`'))
// 只看无证据 / 可疑实体
await page.locator('[data-audit-filter-option=problems]').click()
await page.waitForTimeout(200)
check('筛选「只看无证据 / 可疑实体」:只剩这两组', (await auditGroups()).join(',') === 'none,suspicious' && await auditRowsShown() === 6,
`${(await auditGroups()).join(',')} ${await auditRowsShown()}`)
check('筛选是一组单选(radio),选中的那项 aria-checked', await page.locator('[data-audit-filter-option=problems]').getAttribute('aria-checked') === 'true')
// 导出:发出的请求要带对 format,筛选了就带 groups
evRequests.length = 0
const dl1 = page.waitForEvent('download', { timeout: 5000 }).catch(() => null)
await page.locator('[data-audit-export=csv]').click()
const d1 = await dl1
check('导出 CSV:请求 /evidence/audit?format=csv,只要筛出来的两组',
evRequests.some((u) => u === `/api/runs/${EV}/evidence/audit?format=csv&groups=none%2Csuspicious`), evRequests.join(' '))
check('……下载的文件名是后端给的', d1?.suggestedFilename() === `evidence-${EV.slice(0, 8)}.csv`, d1?.suggestedFilename() ?? '没有下载')
await page.locator('[data-audit-filter-option=all]').click()
evRequests.length = 0
const dl2 = page.waitForEvent('download', { timeout: 5000 }).catch(() => null)
await page.locator('[data-audit-export=json]').click()
const d2 = await dl2
check('导出 JSON:请求 ?format=json,不筛就不带 groups', evRequests.some((u) => u === `/api/runs/${EV}/evidence/audit?format=json`),
evRequests.join(' '))
check('……下载了 .json', d2?.suggestedFilename()?.endsWith('.json'), d2?.suggestedFilename() ?? '没有下载')
// 键盘:整张表一个 Tab 位,↑/↓ 逐行走完每一行,回车在右边打开
const focusables = await page.locator('[data-audit-table] [data-audit-focus]').evaluateAll((els) => els.map((e) => e.tabIndex))
check('表里只有一个 Tab 位(roving)', focusables.filter((t) => t === 0).length === 1 && focusables.length === 16, focusables.join(','))
await page.locator('[data-audit-table] [data-audit-focus][tabindex="0"]').focus()
const seen = [await page.evaluate(() => document.activeElement?.getAttribute('data-audit-focus'))]
for (let i = 0; i < 20; i++) {
await page.keyboard.press('ArrowDown')
seen.push(await page.evaluate(() => document.activeElement?.getAttribute('data-audit-focus')))
}
const uniq = [...new Set(seen.filter(Boolean))]
check('↓ 能逐行走完全部 16 行(包括没挂依据的结论句这种打不开的行)', uniq.length === 16, `${uniq.length} 行`)
check('走到底停住,不首尾相接', seen[seen.length - 1] === seen[seen.length - 2])
await page.keyboard.press('Home')
const home = await page.evaluate(() => document.activeElement?.getAttribute('data-audit-focus'))
await page.keyboard.press('End')
const end = await page.evaluate(() => document.activeElement?.getAttribute('data-audit-focus'))
check('Home / End 到第一行、最后一行', home === uniq[0] && end === uniq[uniq.length - 1], `${home} / ${end}`)
// 找到「orders.week」那一行,回车打开
const target = await page.locator('[data-audit-table] [data-audit-open]', { hasText: /^orders\.week$/ }).getAttribute('data-audit-focus')
await page.keyboard.press('Home')
for (let i = 0; i < 20; i++) {
if (await page.evaluate(() => document.activeElement?.getAttribute('data-audit-focus')) === target) break
await page.keyboard.press('ArrowDown')
}
await page.keyboard.press('Enter')
await page.locator('[data-evidence-dock] [data-evidence-panel=dock] [data-ev-entity-type]').waitFor({ timeout: 5000 }).catch(() => {})
check('回车在右边的常驻面板里打开这一段:实体步骤、字段类型', (await page.locator('[data-evidence-dock] [data-evidence-panel] h3').innerText().catch(() => '')) === 'orders.week'
&& (await page.locator('[data-evidence-dock] [data-ev-entity-type]').innerText().catch(() => '')).includes('VARCHAR'))
check('打开之后焦点还在表里(接着往下走)', await page.evaluate(() => document.activeElement?.getAttribute('data-audit-focus')) === target)
check('面板开着时空位提示收起', await page.locator('[data-evidence-dock] [data-evidence-dock-idle]').count() === 0)
check('正文里对应的片段标着展开', await page.locator('[data-evidence-report=write] [data-seg][aria-expanded=true]').innerText().catch(() => '') === 'orders.week')
// 点正文里的片段:面板跟着换
await page.locator('[data-evidence-report=write] [data-seg]', { hasText: '退款金额以财务确认日为准' }).click()
await page.locator('[data-evidence-dock] [data-ev-quote-hit]').waitFor({ timeout: 5000 }).catch(() => {})
check('点正文里的引文:右边换成引文步骤,原文里高亮', (await page.locator('[data-evidence-dock] [data-ev-quote-hit]').innerText().catch(() => ''))
.includes('退款金额以财务确认日为准'))
if (SHOTS) {
for (const theme of ['dark', 'light']) {
await page.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme)
await page.waitForTimeout(250)
await page.screenshot({ path: `${SHOTS}/runs-evidence-${theme}.png` })
await page.locator('[data-evidence-audit]').scrollIntoViewIfNeeded()
await page.waitForTimeout(150)
await page.screenshot({ path: `${SHOTS}/runs-evidence-audit-${theme}.png` })
}
await page.evaluate(() => document.documentElement.removeAttribute('data-theme'))
}
await page.keyboard.press('Escape')
})
await section('证据页签:同一段文字里好几条同样的违规、正文里点不开的行', async () => {
// 夹具同样是后端真跑的:粗体、链接里反引号写的可疑名字只记违规、不切片段——同一段文字上两条 unknown_entity、
// 两条 unverified_entity(片段和 code 都相同),列表序号 100. 是结构片段上的裸数字。这几行正文里都点不开
const dup = fxe.dup
const EV_DUP = 'fake0evdupviol000000000000000'
evFakes(EV_DUP, { output: dup.output, graph: dup.graph, audit: dup.audit })
fakes.set(`GET /api/artifacts/${dup.doc_artifact}`, () => ({ status: 200, json: { id: dup.doc_artifact, content: dup.doc } }))
await page.route(new RegExp(`/api/runs/${EV_DUP}/evidence/segments/`), (r) => {
const sid = decodeURIComponent(new URL(r.request().url()).pathname.split('/').pop())
return dup.segments[sid] ? r.fulfill({ json: dup.segments[sid] }) : r.fulfill({ status: 404, json: { detail: '没有这个片段' } })
})
const consoleErrors = []
const onConsole = (m) => { if (m.type() === 'error') consoleErrors.push(m.text()) }
page.on('console', onConsole)
try {
await openEvidence(EV_DUP)
await page.locator('[data-audit-table]').waitFor({ timeout: 5000 }).catch(() => {})
const texts = await page.locator('[data-audit-table] [data-audit-focus]').allInnerTexts()
const keys = await page.locator('[data-audit-table] tr[data-audit-row]').evaluateAll((els) => els.map((e) => e.getAttribute('data-audit-row')))
const total = dup.audit.groups.reduce((n, g) => n + g.rows.length, 0)
check(`每条违规一行(${total} 行),同一片段同一种问题的几行键各不相同`, keys.length === total && new Set(keys).size === keys.length,
keys.join(' '))
check('React 没有报「两个子元素键相同」', !consoleErrors.some((t) => /same key/i.test(t)), consoleErrors.find((t) => /same key/i.test(t))?.slice(0, 120) ?? '')
await page.locator('[data-audit-table] [data-audit-focus][tabindex="0"]').focus()
const seen = [await page.evaluate(() => document.activeElement?.getAttribute('data-audit-focus'))]
for (let i = 0; i < total + 2; i++) {
await page.keyboard.press('ArrowDown')
seen.push(await page.evaluate(() => document.activeElement?.getAttribute('data-audit-focus')))
}
const walked = [...new Set(seen.filter(Boolean))]
check(`↓ 逐行走完全部 ${total} 行,不卡在同一片段的第一条违规上`, walked.length === total
&& seen.slice(0, total - 1).every((k, i) => k !== seen[i + 1]), `${walked.length} 行:${seen.join(' → ')}`)
const reached = await page.evaluate((ks) => ks.map((k) => document.querySelector(`[data-audit-focus="${CSS.escape(k)}"]`)?.textContent), walked)
check('……粗体、链接里的四个可疑名字都走得到', ['orders.coupon_id', 'orders.promo_code', 'campaign_tags', 'promo_rules'].every((n) => reached.includes(n)),
reached.join('、'))
// 点不开的行不画成按钮:只有有状态的片段(这里是那个有出处的数字)能在右边打开
const openers = await page.locator('[data-audit-table] [data-audit-open]').evaluateAll((els) => els.map((e) => e.getAttribute('data-audit-open')))
check('只有正文里点得开的片段画成按钮(文字片段、结构片段上的违规不是)', openers.join(',') === 's1', openers.join(',') || '一个都没有')
const hidden = page.locator('[data-audit-table] tr[data-audit-row]:has([data-audit-hidden])')
check('……点不开的五行都说清只在表里列出', await hidden.count() === 5
&& (await hidden.first().innerText()).includes('只在这里列出'), String(await hidden.count()))
const deadKey = keys.find((k, i) => texts[i] === 'orders.promo_code')
await page.locator(`[data-audit-table] [data-audit-focus="${deadKey}"]`).focus()
await page.keyboard.press('Enter')
await page.waitForTimeout(300)
check('……在点不开的行上按回车:右边不开一个空面板,也不报错', await page.locator('[data-evidence-dock] [data-evidence-dock-idle]').count() === 1)
await page.locator('[data-audit-table] [data-audit-open="s1"]').click()
await page.locator('[data-evidence-dock] [data-evidence-panel]').waitFor({ timeout: 5000 }).catch(() => {})
check('点得开的那行:右边打开这一段', (await page.locator('[data-evidence-dock] [data-evidence-panel] h3').innerText().catch(() => '')) === '45,678.5')
await page.keyboard.press('Escape')
} finally {
page.off('console', onConsole)
for (const k of [...fakes.keys()]) if (k.includes(EV_DUP)) fakes.delete(k)
fakes.delete(`GET /api/artifacts/${dup.doc_artifact}`)
}
})
await section('证据页签:没有报告的运行照实说明(none / 旧运行猜测 / 老后端)', async () => {
evFakes(EV_LEGACY, { output: fxe.legacy.output, graph: fxe.legacy.graph, audit: fxe.legacy.audit, report: false })
await openEvidence(EV_LEGACY)
check('没有契约的旧运行:mode 是 legacy_text,照实说', await page.locator('[data-evidence-pane=legacy_text]').count() === 1
&& (await page.locator('[data-evidence-mode-note=legacy_text]').innerText()).includes('没有出具契约'))
check('猜测默认收起', await page.locator('[data-evidence-legacy-text] [data-ev-guess]').getAttribute('data-open') === 'false')
check('审计表里「旧运行猜测」那组默认收起', await page.locator('tbody[data-audit-group=candidate]').getAttribute('data-audit-collapsed') === '1'
&& await page.locator('tbody[data-audit-group=candidate] tr[data-audit-row]').count() === 0)
await page.locator('[data-evidence-legacy-text] [data-ev-guess-toggle]').click()
check('展开猜测写明「猜测的来源,不能当证据」', (await page.locator('[data-evidence-legacy-text] [data-ev-guess-note]').innerText().catch(() => ''))
.includes('猜测的来源,不能当证据'))
await page.locator('[data-audit-group-toggle=candidate]').click()
check('展开那一组:有候选的 4 个数字,线型是候选那一档', await page.locator('tbody[data-audit-group=candidate] tr[data-audit-row]').count() === 4
&& (await page.locator('tbody[data-audit-group=candidate] tr[data-audit-row]').evaluateAll((els) => els.every((e) => e.getAttribute('data-audit-state') === 'candidate'))))
if (SHOTS) {
for (const theme of ['dark', 'light']) {
await page.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme)
await page.waitForTimeout(200)
await page.screenshot({ path: `${SHOTS}/runs-evidence-legacy-${theme}.png` })
}
await page.evaluate(() => document.documentElement.removeAttribute('data-theme'))
}
evFakes(EV_NONE, { output: { answer: '这次没有数字' }, report: false,
graph: { schema: 'agentlab.evidence/1', mode: 'none', note: '这次运行没有报告文档,也没有出具契约,没有可以展示的证据', seal: { sealed: true, ok: true },
reports: [], evidence: [], edges: [] },
audit: { schema: 'agentlab.evidence.audit/1', mode: 'none', seal: { sealed: true, ok: true }, reports: [], groups: [], counts: {}, total: 0 } })
await openEvidence(EV_NONE)
check('mode 为 none:照实说没有证据', await page.locator('[data-evidence-pane=none] [data-evidence-mode-note=none]').count() === 1
&& (await page.locator('[data-evidence-mode-note=none]').innerText()).includes('没有'))
// 老后端:审计接口不存在(404 没有机读码)——清单按页面上的报告自己拼,导出也按页面上的
evFakes(EV_OLD, { output: fxe.output, graph: fxe.graph, audit: null })
await openEvidence(EV_OLD)
await page.locator('[data-audit-fallback]').waitFor({ timeout: 5000 }).catch(() => {})
check('老后端没有审计接口:照实说,清单按页面上的报告拼', await page.locator('[data-audit-fallback=unsupported]').count() === 1
&& await auditRowsShown() > 10, String(await auditRowsShown()))
const dl = page.waitForEvent('download', { timeout: 5000 }).catch(() => null)
await page.locator('[data-audit-export=csv]').click()
const d = await dl
check('……导出退回按页面上的清单,也照实说', !!d && (await page.locator('[data-toast], [role=status], [role=alert]').allInnerTexts()).join(' ').includes('后端没有导出接口'))
for (const id of [EV, EV_LEGACY, EV_NONE, EV_OLD]) {
for (const k of [...fakes.keys()]) if (k.includes(id)) fakes.delete(k)
}
fakes.delete(`GET /api/artifacts/${fxe.doc_artifact}`)
})
await section('收尾', async () => {
// 提取模板之后会跳进画布:画布一打开就拿图去做校验、变量分析,这两个是带图的只读 POST
// (同样被探针拦下,不落库)。赶上它们发出来之前就离开了画布的话就没有
+234 -4
View File
@@ -7,6 +7,7 @@
//
// 跑之前前端得起着(./scripts/dev.sh),默认连 5273。对别的实例(比如一份沙箱拷贝)跑时
// 带上地址:AGENTLAB_WEB=http://localhost:<前端端口> node scripts/check-studio.mjs
import { readFileSync } from 'node:fs'
import { chromium } from '../frontend/node_modules/playwright-core/index.mjs'
const WEB = process.env.AGENTLAB_WEB ?? 'http://localhost:5273'
@@ -100,6 +101,19 @@ const TEAM_BOUND = {
? { ...n, data: { ...n.data, config: { ...n.data.config, tools: [...QUERY_TOOLS] } } } : n)),
}
// 可疑实体、引文、表名字段名的报告(可点击证据三期):夹具是后端真跑出来的,只用通用名
const fxe = JSON.parse(readFileSync(new URL('../frontend/src/run/__tests__/evidence-entity.json', import.meta.url), 'utf8'))
const EV_GRAPH = {
nodes: [
node('start', 'input', 0, 200, '输入', { fields: [{ name: 'week', required: true }] }),
node('fetch', 'agent', 300, 120, '取数', { prompt: '查 {{ input.week }} 的订单', tools: ['db_query__shop'], assign_to: 'rows' }),
node('manual', 'retrieve', 300, 320, '查手册', { query: '退款口径', collection: 'ops', assign_to: 'manual' }),
node('write', 'report', 640, 200, '写周报', { instructions: '写本周周报', numbers: 'strict', claims: 'require_citation' }),
node('done', 'output', 980, 200, '成果', { fields: [{ name: 'answer', value: '{{ nodes.write.text }}' }] }),
],
edges: [edge('start', 'fetch'), edge('start', 'manual'), edge('fetch', 'write'), edge('manual', 'write'), edge('write', 'done')],
}
const wf = (id, extra) => ({
id, name: extra.name, description: extra.description ?? '检查脚本伪造的工作流', graph: extra.graph ?? GRAPH,
tags: extra.tags ?? [], version: extra.version ?? 3, is_template: !!extra.is_template,
@@ -113,6 +127,7 @@ const FAKES = {
'st-tpl': wf('st-tpl', { name: '__studio_check_tpl__', is_template: true, tags: ['extracted'] }),
'st-team': wf('st-team', { name: '__studio_check_team__', graph: TEAM_BOUND, version: 2, published_version: 2, status: 'published' }),
'st-lint': wf('st-lint', { name: '__studio_check_lint__', graph: TEAM_GRAPH, version: 1 }),
'st-ev': wf('st-ev', { name: '__studio_check_ev__', graph: EV_GRAPH, version: 2 }),
}
const VERSIONS = [
{ id: 'v3', version: 3, note: '', created_at: '2026-09-26T02:00:00Z' },
@@ -168,7 +183,7 @@ async function ready(page, fn, what, { arg = null, timeout = 15000, retry = true
* only:目录里只放这几张伪造的(不混真实的),用来造「删完一张不剩」
* platform:伪装成别的平台('Windows'),看快捷键提示是不是跟着平台走
*/
async function open({ width = 1440, height = 900, path = '/studio/st-main', prefs = {}, stream, only, platform, init } = {}) {
async function open({ width = 1440, height = 900, path = '/studio/st-main', prefs = {}, stream, only, platform, init, before } = {}) {
const ctx = await browser.newContext({ viewport: { width, height } })
opened.add(ctx)
// init:[函数, 参数],在页面脚本之前跑(比如装数渲染次数的钩子)
@@ -277,6 +292,8 @@ async function open({ width = 1440, height = 900, path = '/studio/st-main', pref
input: {}, output: {}, error: '', usage: {}, created_at: '2026-09-26T02:00:00Z',
}))
// before:页面第一次加载之前再装几条路由(画布一打开就发的请求,比如报告节点卡去取最近一次运行)
if (before) await before(page)
await page.goto(`${WEB}${path}`, { waitUntil: 'networkidle' })
await ready(page, () => window.__studio?.getState().workflow != null, '画布的 store(window.__studio)')
await page.waitForTimeout(500)
@@ -1779,9 +1796,14 @@ await section('自动排版:请求在路上时画布锁了或改了,不套
const okToast = page.locator('[role="status"] > div').filter({ hasText: '已重新排版' })
const pos = () => S(page, () => JSON.stringify(window.__studio.getState().nodes.map((n) => [n.id, n.position.x, n.position.y])))
const layoutBtn = page.getByRole('button', { name: '自动排版' })
// 等排版接口真的回来再看结果:固定等 1100ms 在四道并行、后端忙的时候不够(接口本身就压了 700ms)
const laidOut = () => page.waitForResponse((r) => new URL(r.url()).pathname.endsWith('/api/copilot/layout'),
{ timeout: 15000 }).catch(() => null)
const settle = async (done) => { await done; await page.waitForTimeout(250) }
const p0 = await pos()
const past0 = (await st(page)).past
let done = laidOut()
await layoutBtn.click()
await page.waitForTimeout(150)
await page.evaluate(() => window.__studio.setState((s) => ({
@@ -1789,7 +1811,7 @@ await section('自动排版:请求在路上时画布锁了或改了,不套
input: {}, output: {}, error: '', usage: {}, created_at: '2026-09-26T02:00:00Z' },
runPhase: 'running', trace: { ...s.trace, phase: 'running', runClass: 'formal' },
})))
await page.waitForTimeout(1100)
await settle(done)
check('正式运行半路开始:排版不套用', await pos() === p0 && (await st(page)).past === past0)
check('也不报「已重新排版」(撤销按钮会撤掉别的改动)', await okToast.count() === 0)
check('说清为什么没排', await page.locator('[role="status"] > div').filter({ hasText: '正式运行进行中' }).count() === 1)
@@ -1798,17 +1820,19 @@ await section('自动排版:请求在路上时画布锁了或改了,不套
run: { ...s.run, status: 'succeeded' }, runPhase: 'succeeded', trace: { ...s.trace, phase: 'succeeded' },
})))
await page.waitForTimeout(150)
done = laidOut()
await layoutBtn.click()
await page.waitForTimeout(150)
await S(page, () => window.__studio.getState().updateNode('lookup', { label: '背景检索(排版时改的)' }))
await page.waitForTimeout(1100)
await settle(done)
const kept = await S(page, () => window.__studio.getState().nodes.find((n) => n.id === 'lookup')?.data.label)
check('排版期间又改了画布:不拿旧图的排版盖掉刚才的改动', kept === '背景检索(排版时改的)', kept)
check('说清这次排版没套用', await okToast.count() === 0
&& await page.locator('[role="status"] > div').filter({ hasText: '排版期间画布又改过了' }).count() === 1)
done = laidOut()
await layoutBtn.click()
await page.waitForTimeout(1100)
await settle(done)
check('正常情况照样排版、给撤销', await okToast.count() === 1 && await okToast.locator('button:has-text("撤销")').count() === 1)
check('没有运行时报错', errors.length === 0, errors.slice(0, 2).join(' | '))
await ctx.close()
@@ -2120,6 +2144,212 @@ await section('发起就被拒(工具不存在)给修复入口;整形节
await run.ctx.close()
})
await section('证据路径:点开报告片段时画布高亮、节点名能对准;报告卡上的章', async () => {
// 运行号用夹具里文档记着的那个:成果字段取文档时核对「是不是这次运行写的」,对不上就按普通文本画
const EV_RUN = fxe.run_id
const json = (route, body, status = 200) => route.fulfill({ status, contentType: 'application/json', body: JSON.stringify(body) })
/** 伪造这张图的「最近一次运行」:latest 为 null 时一次都没跑过;传函数时每次请求现取(中途换成新跑的那次) */
const routes = (latest) => async (page) => {
await page.route((u) => { const x = new URL(u); return x.pathname === '/api/runs' && x.searchParams.get('workflow_id') === 'st-ev' },
(r) => {
const id = typeof latest === 'function' ? latest() : latest
return json(r, id ? [{ id, workflow_id: 'st-ev', status: 'succeeded', run_class: 'exploratory' }] : [])
})
await page.route(new RegExp(`/api/runs/${fxe.run_id}/evidence$`), (r) => json(r, fxe.graph))
await page.route(new RegExp(`/api/runs/${fxe.run_id}/evidence/audit(\\?.*)?$`), (r) => json(r, fxe.audit))
await page.route(new RegExp(`/api/runs/${fxe.dup.run_id}/evidence/audit(\\?.*)?$`), (r) => json(r, fxe.dup.audit))
await page.route(new RegExp(`/api/artifacts/${fxe.doc_artifact}$`), (r) => json(r, { id: fxe.doc_artifact, content: fxe.doc }))
await page.route(new RegExp(`/api/artifacts/${fxe.dup.doc_artifact}$`), (r) => json(r, { id: fxe.dup.doc_artifact, content: fxe.dup.doc }))
await page.route(new RegExp(`/api/runs/${fxe.run_id}/evidence/segments/`), (r) => {
const sid = decodeURIComponent(new URL(r.request().url()).pathname.split('/').pop())
return fxe.segments[sid] ? json(r, fxe.segments[sid]) : json(r, { detail: '没有这个片段' }, 404)
})
}
const stamp = (page) => page.locator('.react-flow__node[data-id="write"] [data-report-stamp]')
const st = fxe.report_checked.stats
const cited = st.numbers_cited + st.values + st.entities + st.quotes
const none = (st.numbers - st.numbers_cited) + Math.max(0, st.uncited_numbers + st.unresolved - (st.numbers - st.numbers_cited))
+ st.unknown_entities + st.uncited_claims
// 另一次运行(结论句策略 off):章上的数照 stampCounts 的口径算
const sd = fxe.dup.report_checked.stats
const dupText = `引用 ${sd.numbers_cited + sd.values + (sd.entities ?? 0) + (sd.quotes ?? 0)} · 无证据 ${(sd.numbers - sd.numbers_cited)
+ Math.max(0, sd.uncited_numbers + sd.unresolved - (sd.numbers - sd.numbers_cited)) + (sd.unknown_entities ?? 0)}`
/** 画布上把运行摘下来(RunHud「清除」):等它重新去取这张图的最近一次运行(取回来、画完再读章) */
const clearAndRefetch = async (pg) => {
const refetch = pg.waitForRequest((r) => { const x = new URL(r.url()); return x.pathname === '/api/runs' && x.searchParams.get('workflow_id') === 'st-ev' },
{ timeout: 3000 }).catch(() => null)
await S(pg, () => window.__studio.getState().clearRun())
const req = await refetch
await pg.waitForLoadState('networkidle').catch(() => {})
await pg.waitForTimeout(400)
return !!req
}
// 一次都没跑过:不画章
let latestRun = null
const { page, errors } = await open({ path: '/studio/st-ev', before: routes(() => latestRun) })
await page.waitForTimeout(400)
check('这张图没有运行:报告节点卡上不画章', await stamp(page).count() === 0)
// 挂上一次运行:report.checked 落下就盖章,数字取它的统计(结论句策略为 require_citation,没挂依据的结论句计入)
await S(page, ({ id, rc, output }) => {
const s = window.__studio
const t = Date.now() / 1000 - 5
s.setState({ run: { id, workflow_id: 'st-ev', workflow_name: '__studio_check_ev__', status: 'queued', input: { week: '2026-W37' },
output: {}, error: null, usage: {}, run_class: 'exploratory', version: null } })
const list = [['run.started', null, {}], ['node.started', 'write', { label: '写周报', node_type: 'report' }],
['report.checked', 'write', rc], ['node.finished', 'write', { duration_ms: 1200 }],
['run.finished', null, { output, usage: {}, timing: { wall_ms: 1500, active_ms: 1500, wait_ms: 0 } }]]
list.forEach((e, i) => s.getState().applyEvent({ seq: i + 1, type: e[0], node_id: e[1], data: e[2], ts: t + i * 0.2 }))
}, { id: EV_RUN, rc: fxe.report_checked, output: fxe.output })
await stamp(page).waitFor({ timeout: 4000 }).catch(() => {})
const text = await stamp(page).innerText().catch(() => '')
check(`报告卡上盖章「引用 ${cited} · 无证据 ${none}」`, text === `引用 ${cited} · 无证据 ${none}`, text)
const title = await stamp(page).getAttribute('title').catch(() => '')
check('章的悬停说明写出可疑名字、没挂依据的结论句计入缺口', (title ?? '').includes('可能是编造的名字 2')
&& (title ?? '').includes('计入缺口'), (title ?? '').replace(/\n/g, ' / '))
check('有缺口的章用提醒色,不是绿的', (await stamp(page).getAttribute('class')).includes('is-degraded'))
check('别的节点卡上没有章', await page.locator('[data-report-stamp]').count() === 1)
// 右栏的报告点开片段:证据路径画到画布上(LineageLayer 的 style[data-lineage])
const doc = page.locator('[data-assistant-panel] [data-evidence-doc]')
await doc.waitFor({ timeout: 6000 }).catch(() => {})
check('右栏的成果是逐段可点的报告', await doc.locator('[data-seg]').count() > 5)
await doc.locator('[data-seg]', { hasText: /^orders$/ }).first().click()
await page.waitForSelector('[data-assistant-panel] [data-evidence-panel="inline"]', { timeout: 4000 }).catch(() => {})
await page.waitForTimeout(300)
const lineage = await page.evaluate(() => {
const el = document.querySelector('style[data-lineage]')
return el ? { var: el.getAttribute('data-lineage'), css: el.textContent } : null
})
check('点开片段:画布上画出证据路径(LineageLayer 的 data-lineage 是这段证据)', lineage?.var === '证据 orders', lineage?.var ?? '没有')
check('……产出证据的查询节点实线描边、报告节点虚线描边', !!lineage && /\[data-id="fetch"\][^{]*> \.nc \{\s*box-shadow/.test(lineage.css)
&& /\[data-id="write"\][^{]*> \.nc \{\s*outline: 1\.5px dashed/.test(lineage.css), (lineage?.css ?? '').slice(0, 160))
const state = await S(page, () => window.__studio.getState().lineage)
check('……用的是 setLineage 的原形状 {var, producers, consumers}', JSON.stringify(state) === JSON.stringify({ var: '证据 orders', producers: ['fetch'], consumers: ['write'] }),
JSON.stringify(state))
await doc.locator('[data-seg]', { hasText: '退款金额以财务确认日为准' }).first().click()
await page.waitForTimeout(300)
const quote = await S(page, () => window.__studio.getState().lineage)
check('换一段(引文):路径换成检索节点 → 报告', quote?.producers?.join(',') === 'manual' && quote?.consumers?.join(',') === 'write', JSON.stringify(quote))
// 面板里的节点名:点一下选中并对准画布上的节点
const chip = page.locator('[data-assistant-panel] [data-evidence-panel] button[data-ev-node="manual"]').first()
check('面板里的节点名是按钮,写画布上的节点名', (await chip.innerText().catch(() => '')).includes('查手册'))
await chip.click().catch(() => {})
await page.waitForTimeout(250)
const sel = await S(page, () => ({ selected: window.__studio.getState().selectedId, focus: window.__studio.getState().focusRequest?.id }))
check('点节点名:选中并对准画布上的那个节点', sel.selected === 'manual' && sel.focus === 'manual', JSON.stringify(sel))
await S(page, () => window.__studio.getState().select(null))
await page.waitForTimeout(250)
if (SHOTS) {
await doc.locator('[data-seg]', { hasText: /^orders$/ }).first().click()
await page.waitForTimeout(300)
for (const theme of ['dark', 'light']) {
await page.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme)
await page.waitForTimeout(250)
await page.screenshot({ path: `${SHOTS}/studio-evidence-path-${theme}.png` })
}
await page.evaluate(() => document.documentElement.setAttribute('data-theme', 'dark'))
}
// 收起面板:只收回自己画的那条路径
const back = page.locator('[data-assistant-panel] [data-evidence-panel] button', { hasText: '回到正文' })
await back.click().catch(() => {})
await page.waitForTimeout(250)
check('收起面板:画布上的证据路径一起收掉', await page.locator('style[data-lineage]').count() === 0
&& await S(page, () => window.__studio.getState().lineage) === null)
// 「清除」把刚才那次运行摘下来:打开画布时这张图还没跑过(取到的是空的),现在最近一次就是刚才那次——
// 章要跟着换,不能拿打开画布时的「没跑过」当最近一次
latestRun = EV_RUN
const refetched = await clearAndRefetch(page)
check('「清除」摘下运行:重新取这张图的最近一次运行', refetched)
check('……章取刚才那次(打开画布时还没跑过,不能照旧不画)', (await stamp(page).innerText().catch(() => '没有章')) === `引用 ${cited} · 无证据 ${none}`,
await stamp(page).innerText().catch(() => '没有章'))
check('没有运行时报错(证据路径)', errors.length === 0, errors.slice(0, 2).join(' | '))
await page.context().close()
// 没挂运行、这张图最近一次运行有报告核对:章从那次运行的证据图取
let latestNow = fxe.run_id
const latest = await open({ path: '/studio/st-ev', before: routes(() => latestNow) })
await stamp(latest.page).waitFor({ timeout: 4000 }).catch(() => {})
check('没挂运行:章取这张图最近一次运行的核对统计(和挂着运行时一样,结论句策略也算上)',
(await stamp(latest.page).innerText().catch(() => '')) === `引用 ${cited} · 无证据 ${none}`,
await stamp(latest.page).innerText().catch(() => '没有章'))
if (SHOTS) {
// 取景到报告节点(可读的缩放),截卡片连同下沿的章,四周留一圈
await S(latest.page, () => window.__studio.getState().focusNode('write'))
await latest.page.waitForTimeout(600)
for (const theme of ['dark', 'light']) {
await latest.page.evaluate((t) => document.documentElement.setAttribute('data-theme', t), theme)
await latest.page.waitForTimeout(250)
const box = await latest.page.locator('.react-flow__node[data-id="write"]').boundingBox()
if (box) {
await latest.page.screenshot({ path: `${SHOTS}/studio-report-stamp-${theme}.png`,
clip: { x: box.x - 40, y: box.y - 40, width: box.width + 80, height: box.height + 80 } }).catch(() => {})
}
}
}
// 在画布上又跑了一次(另一份报告),跑完「清除」:最近一次换成了这次,章不能还是打开画布时取的那份
await S(latest.page, ({ id, rc, output }) => {
const s = window.__studio
const t = Date.now() / 1000 - 3
s.setState({ run: { id, workflow_id: 'st-ev', workflow_name: '__studio_check_ev__', status: 'queued', input: {},
output: {}, error: null, usage: {}, run_class: 'exploratory', version: null } })
const list = [['run.started', null, {}], ['node.started', 'write', { label: '写周报', node_type: 'report' }],
['report.checked', 'write', rc], ['node.finished', 'write', { duration_ms: 900 }],
['run.finished', null, { output, usage: {}, timing: { wall_ms: 1000, active_ms: 1000, wait_ms: 0 } }]]
list.forEach((e, i) => s.getState().applyEvent({ seq: i + 1, type: e[0], node_id: e[1], data: e[2], ts: t + i * 0.2 }))
}, { id: fxe.dup.run_id, rc: fxe.dup.report_checked, output: fxe.dup.output })
await latest.page.waitForTimeout(300)
check(`挂上新跑的一次:章跟着它(${dupText})`, (await stamp(latest.page).innerText().catch(() => '')) === dupText,
await stamp(latest.page).innerText().catch(() => '没有章'))
latestNow = fxe.dup.run_id
const again = await clearAndRefetch(latest.page)
check('「清除」之后重新取最近一次运行(打开画布时取的那份作废)', again)
check(`……章换成新跑的那次(${dupText}),不是打开画布时的「引用 ${cited} · 无证据 ${none}」`,
(await stamp(latest.page).innerText().catch(() => '没有章')) === dupText, await stamp(latest.page).innerText().catch(() => '没有章'))
check('没有运行时报错(最近一次运行的章)', latest.errors.length === 0, latest.errors.slice(0, 2).join(' | '))
await latest.ctx.close()
// 取「最近一次运行」的请求还没回来,就在画布上跑完、清除了:先发的那次(那时还没跑过)晚回来,不能把后来
// 取到的那份盖掉。页面加载等网络静下来才算完,慢请求只能在加载后从 store 发(和报告卡发的是同一个 loadLatest)
let raceRun = null
let raceHits = 0
const race = await open({ path: '/studio/st-ev', before: async (pg) => {
await routes(() => raceRun)(pg)
await pg.route((u) => { const x = new URL(u); return x.pathname === '/api/runs' && x.searchParams.get('workflow_id') === 'st-ev' },
async (r) => {
if (raceHits++ !== 1) return r.fallback()
const body = raceRun ? [{ id: raceRun, workflow_id: 'st-ev', status: 'succeeded', run_class: 'exploratory' }] : []
await new Promise((res) => setTimeout(res, 1800))
return json(r, body).catch(() => {})
})
} })
await race.page.evaluate(async () => {
const { useEvidence } = await window.__appImport('/src/store/evidence.ts')
useEvidence.getState().dropLatest('st-ev')
void useEvidence.getState().loadLatest('st-ev')
})
await race.page.waitForTimeout(150)
await S(race.page, ({ id, rc, output }) => {
const s = window.__studio
const t = Date.now() / 1000 - 2
s.setState({ run: { id, workflow_id: 'st-ev', workflow_name: '__studio_check_ev__', status: 'queued', input: {},
output: {}, error: null, usage: {}, run_class: 'exploratory', version: null } })
const list = [['run.started', null, {}], ['report.checked', 'write', rc],
['run.finished', null, { output, usage: {}, timing: { wall_ms: 1000, active_ms: 1000, wait_ms: 0 } }]]
list.forEach((e, i) => s.getState().applyEvent({ seq: i + 1, type: e[0], node_id: e[1], data: e[2], ts: t + i * 0.2 }))
}, { id: fxe.dup.run_id, rc: fxe.dup.report_checked, output: fxe.dup.output })
raceRun = fxe.dup.run_id
await race.page.waitForTimeout(150)
await S(race.page, () => window.__studio.getState().clearRun())
await race.page.waitForTimeout(2600)
check(`先发的请求晚回来(那时还没跑过):章照旧是清除后取到的那次(${dupText}),没被盖成不画`,
raceHits >= 3 && (await stamp(race.page).innerText().catch(() => '没有章')) === dupText,
`${raceHits} 次请求,${await stamp(race.page).innerText().catch(() => '没有章')}`)
check('没有运行时报错(请求先后颠倒)', race.errors.length === 0, race.errors.slice(0, 2).join(' | '))
await race.ctx.close()
})
check('整个检查没有弹出原生 confirm / prompt', dialogs === 0, `${dialogs} 次`)
check('检查没有写库(兜底拦下的写请求)', writes.length === 0, writes.slice(0, 4).join('、'))
+22 -3
View File
@@ -289,9 +289,28 @@ await section('lib/evidence:证据状态的四通道元数据、报告节点
const eq = (name, got, want) => out.push([name, got === want, got === want ? '' : `得到 ${JSON.stringify(got)},应为 ${JSON.stringify(want)}`])
const ok = (name, cond, detail = '') => out.push([name, !!cond, cond ? '' : detail])
const codes = ev.EVIDENCE_STATES
// 方案 6.1 的八种:确定性、概率性四种、无证据、连接性、旧运行候选
ok('八种状态一种不少', ['deterministic', 'supported', 'partial', 'unsupported', 'unjudged', 'none', 'connective', 'candidate']
.every((c) => codes.includes(c)) && codes.length === 8, codes.join(','))
// 方案 6.1 的八种(确定性、概率性四种、无证据、连接性、旧运行候选),加三期无证据的两个变体:
// 可疑实体(可能是编造的名字)、核对不了(表结构快照不全)
ok('十种状态一种不少', ['deterministic', 'supported', 'partial', 'unsupported', 'unjudged', 'none', 'connective', 'candidate',
'suspect', 'unverified'].every((c) => codes.includes(c)) && codes.length === 10, codes.join(','))
eq('可疑实体:点状线(同无证据)、?!、可能是编造的名字', `${ev.EVIDENCE_STATE.suspect.line}/${ev.EVIDENCE_STATE.suspect.glyph}/${ev.EVIDENCE_STATE.suspect.label}`,
'dotted/?!/可能是编造的名字')
ok('可疑实体醒目(进 n / N),核对不了只是标注', ev.EVIDENCE_STATE.suspect.alert && !ev.EVIDENCE_STATE.unverified.alert)
eq('片段状态:issue 为 unknown_entity 的是可疑实体', ev.segmentState({ kind: 'entity', state: 'none', issue: 'unknown_entity' }), 'suspect')
eq('片段状态:[[t:编造]] 的 cite 带 unknown 也是可疑实体', ev.segmentState({ kind: 'entity', state: 'none', cite: { unknown: true } }), 'suspect')
eq('片段状态:unverified_entity 是核对不了', ev.segmentState({ kind: 'entity', state: 'none', issue: 'unverified_entity' }), 'unverified')
eq('片段状态:原话对不上的引文仍是无证据', ev.segmentState({ kind: 'quote', state: 'none', issue: 'unresolved_ref' }), 'none')
ok('实体、引文另有字形和说法,线型颜色同「有出处」', ev.EVIDENCE_KIND_STYLE.entity.line === 'solid' && ev.EVIDENCE_KIND_STYLE.quote.line === 'solid'
&& ev.EVIDENCE_KIND_STYLE.quote.glyph === '“' && ev.EVIDENCE_KIND_STYLE.entity.color === ev.EVIDENCE_STATE.deterministic.color
&& /^var\(--st-[\w-]+\)$/.test(ev.EVIDENCE_KIND_STYLE.quote.color))
eq('章的两个数:引用 = 有出处的数字、值、实体、引文;无证据含可疑实体,require_citation 时加没挂依据的结论句',
JSON.stringify(ev.stampCounts({ numbers: 4, numbers_cited: 3, values: 1, entities: 5, quotes: 1, uncited_numbers: 1, unresolved: 1,
unknown_entities: 2, uncited_claims: 1 }, 'require_citation')), JSON.stringify({ cited: 10, none: 5 }))
eq('……结论句策略是 off 时不算没挂依据的结论句', ev.stampCounts({ numbers: 4, numbers_cited: 3, uncited_numbers: 1, unresolved: 0,
uncited_claims: 3 }, 'off')?.none, 1)
eq('可疑实体的提示:目录键、{name, table} 都写成去掉前缀的名字,最多 3 个',
ev.closestNames(['c:orders.amount', { alias: 't:refunds' }, { name: 'region', table: 'orders', kind: 'column' }, 'c:x']).join(','),
'orders.amount,refunds,orders.region')
const lines = new Set(['solid', 'dotted', 'badge', 'none'])
const bad = codes.filter((c) => {
const m = ev.EVIDENCE_STATE[c]