M2 复核第 3 轮:报告自查只查引用、别的表里同名的列不算;裁判摘录退回只按数据源设的遮罩列;收遮罩值只收确定取自遮罩列的值;遮罩分析和收到的值按快照工件记在进程里

第 2 轮的按值自查和面板那套宁可多遮的认法带进了误判,正式运行因此失败,代价远大于收益,这一轮回到规格原意:

- 报告自查去掉按值在正文里找遮罩值(engine/evidence.py 的 masked_values、report.py 的收值):两个字的姓名会撞上「销售
  高峰」这类常用词,同名列的值会撞上门店名;照抄成正文的号码本来就会被「数字没有出处」拦下。时间线日志 report_masked_cell
  不再带 written,decode.ts、terms.ts、证据审计的 CSV 回到「引用了遮罩列」一种说法,check-decode 同步
- 自查认引用时从 _mask_sets 的 sure 起,再按 SQL 血缘(sqlglot.lineage,按快照里冻结的表结构)核一遍值取自哪张表的哪一列
  (api/evidence.py 的 _cited_masks / _owned_columns):遮罩列不带表名,同名的列在几张表里都有、其中有的表另有别的遮罩列时
  (customers 有 name 和 phone,stores 只有 name),只认那几张表,门店名不算客户姓名;值根本不取自遮罩列的(region AS
  phone)也不算。认不出出处的照算,没探查过表结构的照列名认。违规说明点名它取自的那一列遮罩列
- 交给裁判的摘录退回 M2 之前的做法,只按数据源设的列名遮(删掉 _judge_masks):并进面板那套宁可多遮的列后,整份遮的快照、
  只数个数的列从摘录里消失,被引用的金额、人数裁判看不到。按需裁判和节点里当场裁判都改回
- 证据面板、推演收「这次运行查到过的遮罩值」只收 sure 的列(_snapshot_secrets,同 masking 收值的口径):SQL 认不准、整份
  遮的快照只遮显示,里面的区域名、金额不再收成遮罩值
- 按快照工件 id 在进程里记下遮罩分析和收到的值(_SnapshotMemo,键里带数据源现在设的遮罩、在不在、类型,设置改了不拿旧的;
  只记读得出来的快照,存取加锁,分析结果冻结、交出去的值另抄一份):之后点开片段、推演都不再把全部快照读盘、复验、分析,
  只读这一步要显示的那一份。30 份 500 行的快照,第一次约 0.7 秒,之后不到 1 毫秒

测试:新增 test_report_masked_owners.py(高峰、门店名写成文字照常出具,照抄号码由数字核对拦下,引用门店名不算、引用客户名算,
同一条查询里两张表的同名列、只设 name、* 展开、WITH、UNION、没有表结构、表值函数)、test_evidence_mask_memo.py(整份遮
的快照不收值、只读一次只分析一次、设置改了重新收、推演共用、再点开片段只读本步快照);裁判摘录改测被引用的金额和计数
都在摘录里;按值自查的用例改成不按值认的回归
This commit is contained in:
ThearchyHelios
2026-10-12 12:33:32 +08:00
parent ce4be9cc8f
commit 65d6a1e63c
13 changed files with 1029 additions and 367 deletions
+332 -69
View File
@@ -20,7 +20,8 @@ artifacts 表可以事后插行,run.output 是可以改写的一列,封存
查询步骤只给被引用的行和前后各 2 行,完整快照仍走 /api/artifacts/{id}。数据源
options.mask_columns 里的列换成「已遮罩」,SQL 里给它们起了别名、套了函数得来的结果列也换
(_mask_sets,和可验证报告同一个函数,合并查询另按合并 SQL 认);用到这些列的指标,指标步骤的值、
证据图的标签也不给(_metric_masks,和推演同一个判断);SQL、合并警告里写着的遮罩值展示时换掉
证据图的标签也不给(_metric_masks,和推演同一个判断);SQL、合并警告里写着的遮罩值(这次运行里确定取自遮罩列的值)
展示时换掉。一份快照的遮罩分析、收到的值按快照工件记在进程里(_MEMO),之后的请求不再读盘、不再分析
——在有身份体系之前,这只减少暴露,不是安全边界:知道工件 id 的人照样能取到整份快照。
期 4 的核查状态同样只读封存范围内的东西:指标步骤带双盲复算的结果(_dual_check_of,读双盲复算节点的产出和台账
@@ -41,8 +42,9 @@ import io
import logging
import re
import sqlite3
import threading
from collections import OrderedDict
from collections.abc import Callable, Iterable, Iterator
from collections.abc import Callable, Iterator
from dataclasses import asdict, dataclass, field, replace
from pathlib import Path
from typing import Any, get_args
@@ -1368,7 +1370,8 @@ class _Masks:
"""一次请求里查过的数据源遮罩:数据源名 → (要遮的列(小写), 数据源现在还在不在)。顺带记下数据源的类型(kind),
按 SQL 认改了名字的遮罩列时照它的方言解析(_mask_sets);数据源不在了的按通用读法。
给了 sealed(证据面板的片段、证据图)时另能交回这次运行查到过的遮罩值(run_secrets),一次请求只收一次。"""
给了 sealed(证据面板的片段、证据图)时另能交回这次运行查到过的遮罩值(run_secrets),一次请求只收一次。一份快照的
遮罩分析、收到的值另按快照工件记在进程里(_MEMO),之后的请求不再读、不再分析。"""
def __init__(self, sealed: _Sealed | None = None) -> None:
self._memo: dict[str, tuple[set[str], bool]] = {}
@@ -1391,7 +1394,7 @@ class _Masks:
读不出来的为 None。"""
key = (artifact, str(source or ""))
if artifact is None or key not in self._sets:
found = await _mask_sets(snap, source, self)
found = await _mask_sets(snap, source, self, artifact=artifact)
if artifact is None:
return found
self._sets[key] = found
@@ -1415,27 +1418,49 @@ class _Masks:
await self.of(name)
return self._kinds.get(name)
async def memo_key(self, artifact: str | None, source: Any) -> tuple[Any, ...] | None:
"""一份快照在进程内记录(_MEMO)里的键:(工件, 数据源名, 数据源现在设的遮罩, 数据源在不在, 类型)。工件按内容
寻址、不会变,一份快照的遮罩只取决于它的内容和后面几样;遮罩设置改了键就跟着变,不会拿旧的结果。合并查询、
没有工件或数据源名的没有键(不记)。"""
name = str(source or "")
if not artifact or not name or name == MERGE_SOURCE:
return None
current, found = await self.of(name)
return str(artifact), name, frozenset(current), found, self._kinds.get(name)
async def value_keys(self, queries: dict[str, Any]) -> list[tuple[str, Any, tuple[Any, ...]]]:
"""封存范围内要收遮罩值的那几份查询快照:(工件, 数据源名, 进程内记录的键)。数据源现在设了遮罩、或者已经不在了的
才收(masked_source);合并查询不收:它的值都来自它的输入,输入另有快照。"""
out: list[tuple[str, Any, tuple[Any, ...]]] = []
for artifact, info in queries.items():
source = (info or {}).get("source")
if await self.masked_source(source) and (key := await self.memo_key(artifact, source)) is not None:
out.append((str(artifact), source, key))
return out
async def run_secrets(self) -> Any:
"""这次运行查到过的遮罩值(masking._Secrets):封存范围内、数据源现在设了遮罩(或者已经不在了)的每份查询快照里,
取自遮罩列的那几列(_mask_sets:列名对上的、SQL 里改了名字的都算,只装着分档标签的格子不收)的值。认不出出处、
值却就是遮罩值的地方据此也不给:指标和输入的值(数据整形算出来再赋给变量的一格)、SQL 里的字面量(按号码筛的
WHERE phone IN (…))、合并警告里举的例子——和可验证报告认「这次运行里查到过的遮罩值」(bundle/build.py 的
_Store.seen)同一个口径。合并查询的快照不收:它的值都来自它的输入,输入另有快照。
确定取自遮罩列的那几列的值(_snapshot_secrets:_mask_sets 的 sure,SQL 认不准、整份遮的快照只遮显示,不收)。
认不出出处、值却就是遮罩值的地方据此也不给:指标和输入的值(数据整形算出来再赋给变量的一格)、SQL 里的字面量
(按号码筛的 WHERE phone IN (…))、合并警告里举的例子。
要把这些快照读一遍,只在用到时收、一次请求只收一次;没给 sealed、运行里没有一份这样的快照时为 None。"""
只在用到时收、一次请求只收一次;每份快照收到的值按快照工件记在进程里(_MEMO),之后的请求(推演也一样)不再读盘、
复验哈希、做 SQL 分析,只读还没记下的那几份。没给 sealed、一个值都没收到时为 None。"""
if self._collected:
return self._secrets
self._collected = True
if self._sealed is None:
return None
found: list[tuple[dict[str, Any], _MaskSets]] = []
for artifact, info in self._sealed.queries.items():
if not await self.masked_source(info.get("source")):
continue
snap, ok = self.fetch(artifact)
if ok is True and isinstance(snap, dict) and snap.get("source") != MERGE_SOURCE:
found.append((snap, await self.sets(artifact, info.get("source"), snap)))
self._secrets = _collect_secrets(found)
parts: list[tuple[tuple[Any, ...], Any]] = []
for artifact, source, key in await self.value_keys(self._sealed.queries):
known, got = _MEMO.values(key)
if not known:
snap, ok = self.fetch(artifact)
if ok is not True or not isinstance(snap, dict) or snap.get("source") == MERGE_SOURCE:
continue
got = _MEMO.keep_values(key, _snapshot_secrets(snap, await self.sets(artifact, source, snap)))
parts.append((key, got))
self._secrets = _MEMO.pooled(parts)
return self._secrets
async def masked_source(self, source: Any) -> bool:
@@ -1447,18 +1472,139 @@ class _Masks:
return bool(cols) or not found
def _collect_secrets(found: Iterable[tuple[dict[str, Any], _MaskSets]]) -> Any:
"""几份查询快照里取自遮罩列的值(masking._Secrets):每份按它的遮罩(_MaskSets)收要遮的那几列,只装着分档标签的
格子不收。一个值都没收到的为 None。证据面板(_Masks.run_secrets)、推演(api/whatif.py 的 _Hidden.secrets)都按它收。"""
def _snapshot_secrets(snap: dict[str, Any], sets: _MaskSets) -> Any:
"""一份查询快照里确定取自遮罩列的值(masking._Secrets,一个都没有为 None):只收 sets.sure 那几列(列名就是遮罩列的、
SQL 里认得出取自遮罩列的),只装着分档标签的格子不收,和核查 agent 在工具边界收值同一个口径(masking:整份遮的快照只收
确定的那几列)。SQL 认不准、整份遮的快照只遮显示:以前连它的区域名、金额一起收,之后每条 SQL 里的「'华东'」都显示成
「已遮罩」。证据面板(_Masks.run_secrets)、推演(api/whatif.py 的 _Hidden.secrets)都按它收。"""
from app.engine.masking import _Secrets
secrets = _Secrets()
for snap, sets in found:
secrets.add(snap, sets.hidden, labels=sets.labels, base=sets.base)
if not secrets.strong:
secrets.add(snap, set(sets.sure), labels=dict(sets.labels), base=set(sets.base))
return secrets if secrets.weak else None
#: 进程内记录里「没有这一项」(记下的值本身可以是 None:这份快照一个遮罩值都没有)
_ABSENT = object()
class _SnapshotMemo:
"""按查询快照工件记在进程里的遮罩分析(_MaskSets)和收到的遮罩值(masking._Secrets),跨请求共用。证据面板每点开一个
片段、推演每算一次,都要知道这次运行查到过的遮罩值(_Masks.run_secrets、api/whatif.py 的 _Hidden.secrets);以前缓存
只在一次请求里有效,每次都把设了遮罩的数据源上的快照读盘、复验哈希、逐份做 SQL 分析(一份几十毫秒)。
键见 _Masks.memo_key。只记读得出来的快照(读不出来、与哈希不一致的下次照旧去读:文件可能修好了);合并查询不记(它的
遮罩还取决于各个输入,输入另按自己的键记)。只记分析结果和收到的值,不记快照本身(几 MB 的行):分析结果按份数封顶,
收到的值按条数合计封顶,并起来的值按运行封顶,都是最近用到的留着。
推演在工作线程里取、证据面板在事件循环里取,所以存取都加锁;记下的东西调用方只读不改:分析结果存成冻结的集合,
并起来的值每次交出去一份新的(_shared:集合另抄一份、共用建好的索引),各自的正则缓存互不相扰。"""
SETS_MAX = 1024
VALUES_MAX = 200_000
POOLS_MAX = 8
def __init__(self) -> None:
self._lock = threading.Lock()
self._sets: OrderedDict[tuple[Any, ...], _MaskSets] = OrderedDict()
self._values: OrderedDict[tuple[Any, ...], Any] = OrderedDict()
self._size = 0
self._pools: OrderedDict[tuple[Any, ...], Any] = OrderedDict()
def clear(self) -> None:
with self._lock:
self._sets.clear()
self._values.clear()
self._pools.clear()
self._size = 0
def sets(self, key: tuple[Any, ...]) -> _MaskSets | None:
with self._lock:
found = self._sets.get(key)
if found is not None:
self._sets.move_to_end(key)
return found
def keep_sets(self, key: tuple[Any, ...], sets: _MaskSets) -> _MaskSets:
frozen = _MaskSets(frozenset(sets.hidden), frozenset(sets.sure), frozenset(sets.base), # type: ignore[arg-type]
{k: frozenset(v) for k, v in sets.labels.items()}, list(sets.gone)) # type: ignore[misc]
with self._lock:
self._sets[key] = frozen
self._sets.move_to_end(key)
while len(self._sets) > self.SETS_MAX:
self._sets.popitem(last=False)
return frozen
def values(self, key: tuple[Any, ...]) -> tuple[bool, Any]:
"""(记没记过, 这份快照收到的值)。"""
with self._lock:
found = self._values.get(key, _ABSENT)
if found is _ABSENT:
return False, None
self._values.move_to_end(key)
return True, found
def keep_values(self, key: tuple[Any, ...], secrets: Any) -> Any:
with self._lock:
old = self._values.pop(key, _ABSENT)
if old is not _ABSENT:
self._size -= _value_count(old)
self._values[key] = secrets
self._size += _value_count(secrets)
while self._size > self.VALUES_MAX and len(self._values) > 1:
_, dropped = self._values.popitem(last=False)
self._size -= _value_count(dropped)
return secrets
def pooled(self, parts: list[tuple[tuple[Any, ...], Any]]) -> Any:
"""几份快照收到的值并成一份、建好索引(上万个值建一次要几十毫秒),按这几份的键记下;交出去的是 _shared 的一份。
一个值都没有为 None。"""
if not parts:
return None
key = tuple(k for k, _ in parts)
with self._lock:
found = self._pools.get(key, _ABSENT)
if found is not _ABSENT:
self._pools.move_to_end(key)
if found is _ABSENT:
found = _pool([one for _, one in parts])
with self._lock:
self._pools[key] = found
while len(self._pools) > self.POOLS_MAX:
self._pools.popitem(last=False)
return _shared(found)
def _value_count(secrets: Any) -> int:
return len(secrets.weak) if secrets is not None else 0
def _pool(parts: list[Any]) -> Any:
"""几份 masking._Secrets 并成一份(合起来最多 SECRETS_MAX 种,同 _Secrets.merge),建好索引;一个值都没有为 None。"""
from app.engine.masking import _Secrets
out = _Secrets()
for one in parts:
out.merge(one)
if not out.strong:
return None
secrets.prepare()
return secrets
out.prepare()
return out
def _shared(secrets: Any) -> Any:
"""记在进程里的一份遮罩值交给一个请求用:值的集合另抄一份(调用方往里并值、免遮不碍别人),建好的索引共用(只读),
正则缓存各起一份(_Secrets.pattern 会改它)。"""
if secrets is None:
return None
from app.engine.masking import _Secrets
return _Secrets(strong=set(secrets.strong), weak=set(secrets.weak), shown=set(secrets.shown),
protected=secrets.protected, _index=secrets._index)
#: 证据面板、推演、报告自查共用的进程内记录(见 _SnapshotMemo)
_MEMO = _SnapshotMemo()
@dataclass
@@ -1466,11 +1612,14 @@ class _MaskSets:
"""一份查询快照的遮罩(_mask_sets):
- hidden:要遮的列(小写)。证据面板、推演、勾行按它遮,宁可多遮:SQL 认不准时整份的列都算
- sure:其中能确定取自遮罩列的(报告自查按它判违规):列名就是遮罩列的,SQL 里认得出取自遮罩列的(别名、函数、
子查询、WITH)。SQL 认不准、整份算进来的那几列不在里面,只装着分档标签的列(CASE 写死的字)也不在
- base:遮罩列本身(查询当时记下的加数据源现在设的,小写),违规说明里点名
- sure:其中能确定取自遮罩列的:列名就是遮罩列的,SQL 里认得出取自遮罩列的(别名、函数、子查询、WITH)。SQL 认不准、
整份算进来的那几列不在里面,只装着分档标签的列(CASE 写死的字)也不在。收「这次运行查到过的遮罩值」只收这几列
(_snapshot_secrets);报告自查从这几列起,再按血缘去掉别的表里同名的列(_cited_masks)
- base:遮罩列本身(查询当时记下的加数据源现在设的,小写)
- labels:只装着分档标签的列 → 标签的写法(masking._snapshot_labels),收遮罩值时不收标签
- gone:已经不在的数据源"""
- gone:已经不在的数据源
记进进程内记录(_MEMO)的是冻结的集合:几个请求共用一份,调用方只读不改。"""
hidden: set[str]
sure: set[str]
@@ -1572,7 +1721,8 @@ def _counted(snap: dict[str, Any], kind: str | None) -> dict[str, Any]:
return snap
async def _mask_sets(snap: dict[str, Any] | None, source: Any, masks: _Masks, *, depth: int = 0) -> _MaskSets:
async def _mask_sets(snap: dict[str, Any] | None, source: Any, masks: _Masks, *, depth: int = 0,
artifact: str | None = None) -> _MaskSets:
"""一份查询快照的遮罩(_MaskSets):证据面板的查询步骤、指标的输入、推演、报告自查(engine/nodes/report.py)都按这一个
函数认。
@@ -1580,7 +1730,9 @@ async def _mask_sets(snap: dict[str, Any] | None, source: Any, masks: _Masks, *,
遮着的列亮出来;数据源不在了的,它的名字进 gone
- 列名对上的之外,按 SQL 认出改了名字的遮罩列:要遮的按 _derived(和可验证报告同一个函数),确定的按 _sure
- 合并查询不是数据源:按它合并的那几个数据源、逐格来历和合并 SQL 认(_merge_mask_sets)
- snap 为 None(快照读不出来):只能按 source 现在设的遮罩算,没有 SQL 可认"""
- snap 为 None(快照读不出来):只能按 source 现在设的遮罩算,没有 SQL 可认
artifact 是 snap 的工件 id:给了的,分析结果按快照记在进程里(_MEMO,键见 _Masks.memo_key),之后的请求不再分析。"""
if snap is not None and snap.get("source") == MERGE_SOURCE:
return await _merge_mask_sets(snap, masks, depth=depth)
name = source or (snap or {}).get("source")
@@ -1589,16 +1741,154 @@ async def _mask_sets(snap: dict[str, Any] | None, source: Any, masks: _Masks, *,
gone = [] if found or not name else [str(name)]
if snap is None or not base:
return _MaskSets(set(base), set(base), base, gone=gone)
key = await masks.memo_key(artifact, name) if artifact else None
if key is not None and (kept := _MEMO.sets(key)) is not None:
return kept
kind = await masks.kind(name)
sure, labels = _sure(snap, base, kind)
return _MaskSets(_derived(snap, base, kind) | sure, sure, base, labels, gone)
out = _MaskSets(_derived(snap, base, kind) | sure, sure, base, labels, gone)
return _MEMO.keep_sets(key, out) if key is not None else out
async def _snapshot_masks(snap: dict[str, Any] | None, source: Any, masks: _Masks, *,
depth: int = 0) -> tuple[set[str], list[str]]:
"""一份查询快照要遮的列(小写)和已经不在的数据源(_mask_sets 的 hidden、gone)。"""
sets = await _mask_sets(snap, source, masks, depth=depth)
return sets.hidden, sets.gone
async def _cited_masks(snap: dict[str, Any], source: Any, masks: _Masks,
artifact: str | None = None) -> dict[str, list[str]]:
"""报告自查(engine/nodes/report.py 的 _MaskedCells)认一份查询快照里引用了就记 masked_cell 的列:{结果列(小写):
它取自的遮罩列},违规说明点名后者。
从 _mask_sets 的 sure 起(列名就是遮罩列的、SQL 里认得出取自遮罩列的;SQL 认不准、整份算进来的和只装着分档标签的
不在里面),普通查询再按血缘核一遍值取自哪张表的哪一列,只认取自遮罩表里那一列的(_owned_columns:stores.name 和设了
遮罩的 customers.name 同名不算)。合并查询的值来自它的各个输入、不是数据源里的表,照 sure 算。"""
sets = await _mask_sets(snap, source, masks, artifact=artifact)
if not sets.sure:
return {}
if snap.get("source") == MERGE_SOURCE:
return {c: sorted(sets.base) for c in sorted(sets.sure)}
return _owned_columns(snap, set(sets.sure), set(sets.base), await masks.kind(source or snap.get("source")))
def _owned_columns(snap: dict[str, Any], sure: set[str], base: set[str], kind: str | None) -> dict[str, list[str]]:
"""sure 里值确实取自遮罩表里那一列的结果列 → 它取自的遮罩列(按血缘里出现的先后)。报告自查按它判违规。
遮罩按列名设在整个数据源上(data/engine.py 的 masked_columns,不带表名):设了 name,stores.name 和 customers.name 按
列名都算。证据面板、可验证报告、核查 agent 照这样遮(宁可多遮);自查多算一列就是判失败,所以按 SQL 的血缘
(sqlglot.lineage,按查询当时冻结的表结构展开 *、认出不带表名的列出自哪张表)核一遍每一列的值取自哪张表的哪一列:
- 遮罩列在几张表里都有、其中有的表里另有别的遮罩列(customers 有 name 和 phone,stores 只有 name):只认那几张表
(_mask_owners)。别的情形(只在一张表里有、哪张都没有别的遮罩列)照列名认
- 血缘里有认不出出处的(找不到这一列、表值函数、不带表名又有歧义的列、* 展开不了):排除不了,照算
- 认得出、可一处都不取自遮罩表里的遮罩列(门店名、SELECT region AS phone):不算
- 快照里没有冻结的表结构(数据源没探查过):认不出同名的列还在哪几张表里,照列名认;值根本不取自遮罩列的照样不算
只排除、不新增:sure 里没有的列血缘认出来也不算。分析出了意外,照 sure 算。纯计算,在调用方的线程里算。"""
plain = {c: sorted(base) for c in sorted(sure)}
sql = snap.get("sql")
if not sure or not isinstance(sql, str) or not sql.strip():
return plain
try:
from app.data.sql_semantics import _parse, _read
from app.engine.masking import _frozen_columns
read = _read(kind)
root, _ = _parse(sql, read)
if root is None:
return plain
frozen = _frozen_columns(snap["schema_artifact"]) if isinstance(snap.get("schema_artifact"), str) else None
# 表结构里一张表按好几个名字记着(带不带库名),这里只要不带库名的那一份
tables = {name: cols for name, cols in (frozen or {}).items() if "." not in name}
owners = _mask_owners(tables, base)
schema = {name: {c: "TEXT" for c in cols} for name, cols in tables.items()} or None
nodes = _lineages(root, schema, read, [str(c) for c in snap.get("columns") or [] if str(c).lower() in sure])
out: dict[str, list[str]] = {}
for column in sorted(sure):
found = _owned_masks(nodes.get(column), base, owners, set(tables))
if found:
out[column] = found
return out
except Exception: # noqa: BLE001 - 核对不了出自哪张表:照 sure 算
logger.warning("报告自查:按血缘核对遮罩列出自哪张表时出错,照列名认", exc_info=True)
return plain
def _mask_owners(tables: dict[str, list[str]], base: set[str]) -> dict[str, set[str]]:
"""遮罩列 → 它的遮罩表(小写的表名),只记排除得掉一些表的:遮罩列在几张表里都有,其中有的表里另有别的遮罩列,
只认那几张(设了 name, phone,customers 有这两列,stores 只有 name:name 指的是客户姓名)。只在一张表里有、几张表
里都有可哪张都没有别的遮罩列的不记:照列名认,哪张表的都算。"""
out: dict[str, set[str]] = {}
for mask in base:
having = {t for t, cols in tables.items() if mask in cols}
paired = {t for t in having if any(c in base and c != mask for c in tables[t])}
if paired and paired != having:
out[mask] = paired
return out
def _lineages(root: Any, schema: dict[str, Any] | None, read: str | None, columns: list[str]) -> dict[str, Any]:
"""结果列(小写)→ 它的血缘(sqlglot.lineage 的节点)。先一次认全部的列(只解析、补全一遍);认不了的逐列再认,还是
认不出的不在里面(_owned_masks 当认不出出处)。"""
from sqlglot.lineage import lineage
try:
found = lineage(None, root.copy(), schema=schema, dialect=read)
every = {str(k).lower(): v for k, v in found.items()} if isinstance(found, dict) else {}
except Exception: # noqa: BLE001 - 有一列认不了就整条认不了:下面逐列认
every = {}
out: dict[str, Any] = {}
for column in columns:
if (node := every.get(column.lower())) is None:
try:
node = lineage(column, root.copy(), schema=schema, dialect=read)
except Exception: # noqa: BLE001 - 认不出这一列(没起别名的表达式……)
continue
out[column.lower()] = node
return out
def _owned_masks(node: Any, base: set[str], owners: dict[str, set[str]], known: set[str]) -> list[str]:
"""一个结果列的值取自哪几列遮罩列(按血缘里出现的先后),别的表里同名的列不算(owners,_mask_owners)。血缘认不出来
(node 为 None)、里面有认不出出处的(_leaf_of 交回 None),有取自遮罩列的就点名它们、没有就照全部遮罩列算:排除不了。
known 是冻结的表结构里有的表:不在里面的表认不出有没有别的遮罩列,照算。"""
if node is None:
return sorted(base)
found: list[str] = []
unplaced = False
for leaf in node.walk():
if leaf.downstream:
continue
place = _leaf_of(leaf)
if place is None:
unplaced = True
continue
if place is _NO_COLUMN:
continue
table, column = place
if column not in base or (column in owners and table in known and table not in owners[column]):
continue
if column not in found:
found.append(column)
return found or (sorted(base) if unplaced else [])
#: 血缘里的一个末端没有取任何列(写死的值、COUNT(*))
_NO_COLUMN = object()
def _leaf_of(leaf: Any) -> Any:
"""血缘的一个末端出自哪张表的哪一列:(表名, 列名),都是小写;没有取任何列的(写死的值、COUNT(*))交回 _NO_COLUMN;
认不出出处的(sqlglot 找不到来源、表值函数、整张表的 *、派生来源里的 *)交回 None。"""
from sqlglot import exp
source = leaf.source
if isinstance(source, exp.Table) and isinstance(source.this, exp.Identifier):
try:
column = exp.to_column(leaf.name)
except Exception: # noqa: BLE001 - 名字读不出来:认不出
return None
if not isinstance(column, exp.Column) or isinstance(column.this, exp.Star) or not column.name:
return None
return source.name.lower(), column.name.lower()
if isinstance(source, (exp.Table, exp.Placeholder)) or leaf.expression is source:
return None
return _NO_COLUMN
async def _query_step(artifact: str, cells: list[tuple[Any, Any]], doc: dict[str, Any], sealed: _Sealed,
@@ -1920,7 +2210,7 @@ async def _merge_input_sets(item: dict[str, Any], masks: _Masks, depth: int, rec
artifact = str(item.get("artifact") or "")
content, ok = masks.fetch(artifact) if artifact and depth < MERGE_DEPTH else (None, None)
if ok is True and isinstance(content, dict):
return await _mask_sets(content, source, masks, depth=depth + 1)
return await _mask_sets(content, source, masks, depth=depth + 1, artifact=artifact)
current, found = await masks.of(source) if source and source != MERGE_SOURCE else (set(), True)
if not (current or recorded or not found or source == MERGE_SOURCE or not source):
return _MaskSets(set(), set(), set())
@@ -2991,33 +3281,6 @@ async def judge_units(run_id: str, payload: Any = Body(None), report: str | None
return await _judge_now(sealed, chosen, units)
async def _judge_masks(catalog: dict[str, Any], masked: dict[str, list[str]],
loader: Callable[[str], Any] | None = None) -> dict[str, list[str]]:
"""交给裁判的摘录要遮的列:{数据源名: 列名},裁判按数据源名合起来认(judge._hidden)。数据源现在设的(judge.source_masks)
之外,目录里每份查询快照按证据面板同一个函数(_mask_sets)认出的、改了名字的遮罩列(c.phone AS tel 的 tel)也并进它的
数据源名下(合并查询并进「合并查询」名下),多遮无妨。以前摘录只按列名去掉 phone,tel 那一列的号码照样交给裁判,裁判
理由可能原样带进证据面板。报告撰写节点当场裁判(report.py)用同一个函数。loader 同 judge.prepare(缺省读工件库)。"""
from app.core.artifact_store import load as default_load
read = loader or default_load
out = {str(k): [str(c) for c in v] for k, v in masked.items()}
masks = _Masks()
for entry in catalog.values():
if not isinstance(entry, dict) or entry.get("kind") != "query" or not entry.get("artifact"):
continue
try:
snap = read(str(entry["artifact"]))
except Exception: # noqa: BLE001 - 读不出来、不在封存范围里:摘录本来就不给它
continue
if not isinstance(snap, dict):
continue
source = str(entry.get("source") or snap.get("source") or "")
have = {c.lower() for c in out.get(source, [])}
if extra := sorted((await _mask_sets(snap, source, masks)).hidden - have):
out.setdefault(source, []).extend(extra)
return out
async def _judge_now(sealed: _Sealed, report: _Report, units: list[str]) -> dict[str, Any]:
from app.engine import judge as judging
@@ -3040,7 +3303,7 @@ async def _judge_now(sealed: _Sealed, report: _Report, units: list[str]) -> dict
if pending:
# 带期 4 标记的文档才让裁判取得到哈希链上的清单(摘录的来历行要用);老文档的 loader 和以前完全相同
loader = _sealed_loader(sealed, chain=doc.get("provenance") == DOC_PROVENANCE)
masked = await _judge_masks(catalog, await judging.source_masks(catalog, loader=loader), loader)
masked = await judging.source_masks(catalog, loader=loader)
request = judging.prepare(doc, catalog, units=pending, loader=loader, masked=masked)
skipped = dict(request.skipped)
if request.cands:
@@ -3156,8 +3419,8 @@ GROUPS = (("cited", "有出处"), ("none", "无证据"), ("suspicious", "可疑
#: 文档本身对不上的违规:不管落在哪个片段上都单独列一行,不能被那个片段「有出处」的行盖住
_INTEGRITY = frozenset({"bad_schema", "segment_mismatch", "structural_text", "render_mismatch", "eid_mismatch",
"state_mismatch"})
#: 落在已经列出的片段上也单列一行的违规:文档对不上的,和写进了遮罩列的值的(masked_cell:引用的那一格落在的正是一个
#: 「有出处」的单元格片段,经指标带出来的落在指标片段上,不单列的话审计表、导出的 CSV 里只有一行普通的「有出处」)
#: 落在已经列出的片段上也单列一行的违规:文档对不上的,和引用了遮罩列的格子的(masked_cell:引用的那一格落在的正是一个
#: 「有出处」的单元格片段,不单列的话审计表、导出的 CSV 里只有一行普通的「有出处」)
_LISTED_ALWAYS = _INTEGRITY | {"masked_cell"}
UNCITED_CLAIM = "这句结论没有附依据(既没有引用,也没有注明依据)"
UNCITED_CLAIM_COUNTED = "这句结论没有附依据。本报告要求结论句附依据,出具时计入缺口"
@@ -3197,7 +3460,7 @@ _CSV_LABELS: dict[str, dict[Any, str]] = {
"issue": {"uncited_number": "没有出处的数字", "unresolved_ref": "引用无法解析", "unknown_entity": "疑似不存在的名称",
"unverified_entity": "无法核实", "uncited_claim": "结论句未附依据", "no_candidate": "未找到候选",
"contradicted_claim": "证据相矛盾", "insufficient_claim": "证据不足", "unsupported_claim": "证据不支持",
"masked_cell": "写进了遮罩列的值",
"masked_cell": "引用了遮罩列",
**{code: "文档不一致" for code in _INTEGRITY}},
"evidence_kind": {"metric": "口径卡指标", "cell": "查询单元格", "input": "运行输入", "table": "表", "column": "字段",
"quote": "知识库原话", "query": "查询结果"},
+24 -17
View File
@@ -31,7 +31,7 @@ from app.api.evidence import (
MISSING_QUERY,
REPORT_NOT_FOUND,
TAMPERED_QUERY,
_collect_secrets,
_MEMO,
_Masks,
_MaskSets,
_reports,
@@ -39,6 +39,7 @@ from app.api.evidence import (
_sealed,
_sealed_cards,
_sealed_loader,
_snapshot_secrets,
)
from app.core.artifact_store import load
from app.engine import recompute
@@ -323,8 +324,9 @@ class _Hidden(Mapping[str, frozenset[str]]):
设的遮罩要访问库,交回事件循环去查(_Masks 是异步的),线程在这里等它。以前每次请求在事件循环里把这次运行的
全部快照读一遍、复验一遍哈希,不管卡上用没用到。
secrets 同样在工作线程里收:这次运行查到过的遮罩值(和证据面板的 _Masks.run_secrets 同一个口径),重算里认不出出处
的输入、整份结果当输入的指标要按值认时才收,一次请求只收一次。
secrets 同样在工作线程里收:这次运行查到过的遮罩值(和证据面板的 _Masks.run_secrets 同一个口径、同一份进程内记录
_MEMO),重算里认不出出处的输入、整份结果当输入的指标要按值认时才收,一次请求只收一次;证据面板、之前的推演收过的
快照不再读。分析结果同样按快照工件记在进程里。
"""
def __init__(self, sealed: _Sealed, loader: Any, loop: asyncio.AbstractEventLoop) -> None:
@@ -348,23 +350,26 @@ class _Hidden(Mapping[str, frozenset[str]]):
snap = None
snap = snap if isinstance(snap, dict) else None
if key not in self._memo:
found = _mask_sets_of(snap, self._sealed.queries[key], self._masks)
found = _mask_sets_of(snap, self._sealed.queries[key], self._masks, key)
self._memo[key] = self._wait(found)
return snap, self._memo[key]
def secrets(self) -> Any:
"""这次运行查到过的遮罩值(masking._Secrets,没有为 None):数据源现在设了遮罩、或者已经不在了的查询快照里取自
遮罩列的值,合并查询不收(api/evidence.py 的 _collect_secrets)。"""
"""这次运行查到过的遮罩值(masking._Secrets,没有为 None):数据源现在设了遮罩、或者已经不在了的查询快照里确定
取自遮罩列的值,合并查询不收(api/evidence.py 的 _snapshot_secrets)。每份快照收到的值先查进程内记录(_MEMO),
记过的不再读。"""
if not self._collected:
self._collected = True
found = []
for artifact, info in self._sealed.queries.items():
if not self._wait(self._masks.masked_source(info.get("source"))):
continue
snap, sets = self._sets(artifact)
if snap is not None and snap.get("source") != MERGE_SOURCE:
found.append((snap, sets))
self._secrets = _collect_secrets(found)
parts = []
for artifact, _source, key in self._wait(self._masks.value_keys(self._sealed.queries)):
known, got = _MEMO.values(key)
if not known:
snap, sets = self._sets(artifact)
if snap is None or snap.get("source") == MERGE_SOURCE:
continue
got = _MEMO.keep_values(key, _snapshot_secrets(snap, sets))
parts.append((key, got))
self._secrets = _MEMO.pooled(parts)
return self._secrets
def _wait(self, coro: Any) -> Any:
@@ -385,10 +390,12 @@ class _Hidden(Mapping[str, frozenset[str]]):
return len(self._sealed.queries)
async def _mask_sets_of(snap: dict[str, Any] | None, info: dict[str, Any], masks: _Masks) -> _MaskSets:
async def _mask_sets_of(snap: dict[str, Any] | None, info: dict[str, Any], masks: _Masks,
artifact: str | None = None) -> _MaskSets:
"""一份查询快照的遮罩,和证据面板的查询步骤同一个函数(api/evidence.py 的 _mask_sets):查询当时记下的加数据源现在
设的,SQL 里给遮罩列起了别名、套了函数的结果列也算。snap 为 None(读不出来)时只能按台账记的数据源现在设的遮罩算。"""
return await masks.sets(None, info.get("source"), snap)
设的,SQL 里给遮罩列起了别名、套了函数的结果列也算。snap 为 None(读不出来)时只能按台账记的数据源现在设的遮罩算。
artifact 是 snap 的工件 id:给了的,分析结果按快照记在进程里,之后的推演、证据面板不再分析。"""
return await masks.sets(artifact, info.get("source"), snap)
async def _mask_of(snap: dict[str, Any] | None, info: dict[str, Any], masks: _Masks) -> set[str]:
+6 -67
View File
@@ -3131,9 +3131,9 @@ def absence_claims(doc: dict[str, Any]) -> list[dict[str, Any]]:
def _masked_cell_message(ref: str, column: str, sources: list[str]) -> _Reason:
"""违规 masked_cell 的说法:列名就是遮罩列的说「在数据源中设置了遮罩」,按 SQL 认出来的点名取自哪几列遮罩列(sources,
这份快照的数据源设的、查询当时记下的)——遮罩按列名设在整个数据源上,别的表里同名的列也算,读的人要看得出是哪一列
的遮罩管到了这里。给写作者的那句说清这一列的值不能以任何写法写进报告:只说「删掉引用」,写作者会把值照抄成文字。"""
"""违规 masked_cell 的说法:列名就是遮罩列的说「在数据源中设置了遮罩」,按 SQL 认出来的点名它取自的那几列遮罩列(sources,
按血缘认出来的;认不出时是这份快照的全部遮罩列)——遮罩按列名设在整个数据源上,读的人要看得出是哪一项遮罩设置管到了
这里。给写作者的那句说清这一列的值不能以任何写法写进报告:只说「删掉引用」,写作者会把值照抄成文字。"""
own = column.lower() in {s.lower() for s in sources} or not sources
where = (f"列「{column}」在数据源中设置了遮罩" if own
else f"列「{column}」按 SQL 认出取自设置了遮罩的列「{'、'.join(sources[:3])}」")
@@ -3148,9 +3148,10 @@ def masked_cells(doc: dict[str, Any], catalog: dict[str, Any], masked: dict[str,
"""正文直接引用了取自遮罩列的格子([[v:Q2.r0.tel]],整表展开出来的格也算),每格一条违规 masked_cell。
masked 是 {查询快照工件: 取自遮罩列的结果列},结果列可以是列名的集合,也可以是 {列名: 它取自的遮罩列}(说法里点名)。
由报告节点按证据面板同一个函数算好交进来(api/evidence.py 的 _mask_sets 的 sure:列名就是遮罩列的、SQL 里给遮罩列
起了别名套了函数的;SQL 认不准、整份算进来的不在里面):遮罩设在数据源上,这里只认文档和目录,查不了库。写作目录
由报告节点算好交进来(api/evidence.py 的 _cited_masks:列名就是遮罩列的、SQL 里给遮罩列起了别名套了函数的,按血缘
去掉别的表里同名的列;SQL 认不准、整份算进来的不在里面):遮罩设在数据源上,这里只认文档和目录,查不了库。写作目录
不遮罩(写作者要看得到数据),所以拦在自查:和没有出处的数字一样交回写作者重写,仍不改的按「违规处置」办(report.py)。
只认引用,不按值在正文里找遮罩值(两个字的姓名会撞上常用词):照抄成正文的号码由数字核对拦下。
违规不带那一格的值(不写 text、context):它会进 report.checked、重写的日志、出具缺口,这些地方不能再露一次。带着
ref、列名、span 和所在片段:证据面板能定位,写作者知道要去掉哪一处。"""
@@ -3180,68 +3181,6 @@ def masked_cells(doc: dict[str, Any], catalog: dict[str, Any], masked: dict[str,
return found
def _masked_value_message(ref: str | None) -> _Reason:
"""违规 masked_cell(按值认出来的,by_value)的说法。不点名列:按值认只知道这个值在写作目录的遮罩列里出现过。经引用
带出来的(指标的值就是那一格)点名那处引用;写在正文里的,交回写作者时另附那一句(context,值换成「已遮罩」)。"""
if ref:
return _Reason(
f"引用 [[{ref}]] 的值与设置了遮罩的列中的值相同,值不应写进报告",
f"[[{ref}]] 的值就是证据目录里取自遮罩列的值(手机号、姓名这类),这个值不能以任何形式写进报告,引用、数字、"
"照抄的文字都不行:删掉这处引用和相应的内容",
)
return _Reason(
"正文写着设置了遮罩的列中的值,值不应写进报告",
"正文里直接写着证据目录里取自遮罩列的值(手机号、姓名这类),这些值不能以任何形式写进报告,照抄的文字、数字都"
"不行:删掉这处文字,或者改成不含这个值的说法",
)
def masked_values(doc: dict[str, Any], secrets: Any,
skip: set[str] | frozenset[str] = frozenset()) -> list[dict[str, Any]]:
"""正文里直接写着的遮罩值,每处一条违规 masked_cell(带 by_value):写作者把号码、姓名照抄成文字、写成裸数字,或者
经指标、输入的引用带了出来([[m:phone_raw]] 的值就是号码)。masked_cells 只认 [[v:]] 引用的格子,这几种它认不出;
数字核对管得到裸数字、管不到文字,「数字核对」关掉时数字也不管。
secrets 是写作目录里取自遮罩列的值(masking._Secrets),由报告节点按证据面板同一个函数算好交进来(report.py 的
_MaskedCells:只收确定取自遮罩列的列,运行输入里写着的免);None 时一条都不记。skip 是已经按引用记过 masked_cell
的片段:同一处不记两条。按整句认:换了写法的长号码(199-0000-0702)组装时被拆成好几个数字片段,一个片段里认不全,
认出来的那一段落在哪几个片段上,就记在第一个上。
违规不带值:context 是那一句,值换成「已遮罩」——交回写作者时据此找到那一处,进事件、日志也不再露一次。"""
if secrets is None:
return []
skipped = {str(s) for s in skip}
found: list[dict[str, Any]] = []
for _, unit in iter_units(doc):
placed: list[tuple[int, int, dict[str, Any]]] = []
sentence = ""
for seg in unit.get("segments") or []:
text = seg.get("text") if isinstance(seg.get("text"), str) else ""
placed.append((len(sentence), len(sentence) + len(text), seg))
sentence += text
hits = secrets.spans(sentence, loose=True) if sentence else []
named: set[str] = set()
context: str | None = None
for start, end in hits:
owners = [(a, seg) for a, b, seg in placed if a < end and start < b]
if not owners or any(str(seg.get("id")) in skipped for _, seg in owners):
continue
first_at, first = owners[0]
if str(first.get("id")) in named:
continue
named.add(str(first.get("id")))
context = context or secrets.scrub(sentence, loose=True).strip()[:120]
ref = next((str(seg["ref"]) for _, seg in owners if seg.get("ref")), None)
last_at, last = owners[-1]
head, tail = first.get("span"), last.get("span")
where = None
if all(isinstance(s, list) and len(s) == 2 for s in (head, tail)):
where = [head[0] + max(start - first_at, 0), tail[0] + min(end - last_at, len(last.get("text") or ""))]
found.append(_violation("masked_cell", _masked_value_message(ref), ref=ref, by_value=True, context=context,
span=where, segment=first.get("id"), unit=unit.get("id")))
return found
#: 引用查询结果的写法。只跟着目录里的查询出现(catalog_prompt):写作规则本身保持一期那份,升级前
#: 发起的运行跑到报告节点时提示一字不差;只有口径卡的目录也不该教写作者去引用不存在的 Q1
CELL_RULES = (
+26 -84
View File
@@ -32,7 +32,6 @@
from __future__ import annotations
import asyncio
import json
import re
import time
from collections import Counter
@@ -321,36 +320,30 @@ class _RenderedStream:
class _MaskedCells:
"""报告自查:报告里写进了取自遮罩列的值,记一条违规 masked_cell,和没有出处的数字一样交回写作者重写。写作目录本身
不遮(写作者要看得到数据),只拦写进报告的。两种认法:
"""报告自查:写作者引用了取自遮罩列的格子([[v:Q2.r0.tel]]),记一条违规 masked_cell,和没有出处的数字一样交回写作者
重写(evidence.masked_cells)。写作目录本身不遮(写作者要看得到数据),只拦引用。
- 引用(evidence.masked_cells):[[v:Q2.r0.tel]] 引用的那一列取自遮罩列。哪几列算:和证据面板同一个函数(api/evidence.py
的 _mask_sets)认出来的「确定取自遮罩列」的那几列(sure)——列名就是遮罩列的,SQL 里给遮罩列起了别名、套了函数、经
子查询或 WITH 转一道的,合并查询按逐格来历和合并 SQL 认。证据面板为了展示宁可多遮(SQL 认不准时整份的列都遮),自查
不跟着多算:多算一列就是一次判失败,写作者也改不掉
- 按值(evidence.masked_values):正文里直接写着这些列里的值——照抄成文字、写成数字,或者经指标带了出来([[m:phone_raw]]
的值就是号码)。值从数据源现在设了遮罩的那几份查询快照里收,同样只收 sure 的列、不收只装着分档标签的格子;合并查询
的不收(它的值来自它的输入,输入是目录里另外几份快照)。protected 是运行输入(发起运行的人自己填的),里面写着的值
不算,和推演、证据面板一样
哪几列算按 api/evidence.py 的 _cited_masks 认:列名就是遮罩列的,SQL 里给遮罩列起了别名、套了函数、经子查询或 WITH
转一道的(_mask_sets 的 sure,和证据面板同一个函数),再按血缘去掉别的表里同名的列(设了 name, phone,门店名不算客户
姓名);合并查询按逐格来历和合并 SQL 认。证据面板为了展示宁可多遮(SQL 认不准时整份的列都遮),自查不跟着多算:多算
一列就是一次判失败,写作者也改不掉。
组装好一稿之后才认:引用只读正文引用到的那几份快照,按值只读数据源现在设了遮罩的那几份(读盘复验哈希放进线程),都只读
一次,重写的稿子不再读。没设遮罩的运行一份都不读。以前写作之前就把目录里每份查询快照都读一遍。"""
只查引用,不按值在正文里找遮罩值:两个字的姓名会撞上「销售高峰」这类常用词,同名列的值会撞上门店名,正式运行因此判
失败(复核第 3 轮去掉了按值认);正文里照抄的号码本来就会被「数字没有出处」拦下。
def __init__(self, catalog: dict[str, Any], protected: list[str] | tuple[str, ...] = ()) -> None:
组装好一稿之后才认:只读正文引用到的那几份快照(读盘复验哈希放进线程),每份只读一次,重写的稿子不再读;没引用格子的
一份都不读。"""
def __init__(self, catalog: dict[str, Any]) -> None:
from app.api.evidence import _Masks
self._catalog = catalog
self._masks = _Masks()
self._protected = [str(t) for t in protected if t]
self._memo: dict[str, dict[str, list[str]]] = {}
#: 按值认之前已经读过的快照(为了认引用):收值时不再读第二遍,收完就放掉
self._loaded: dict[str, tuple[dict[str, Any], Any]] = {}
self._secrets: Any = None
self._collected = False
async def __call__(self, doc: dict[str, Any]) -> dict[str, Any]:
"""把这一稿写进了遮罩列的值的地方记进 doc 的违规(接在最后,和 compose_doc 给了 masked 时一样),交回同一份文档。"""
from app.engine.evidence import iter_segments, masked_cells, masked_values
"""把这一稿引用了遮罩列的格子记进 doc 的违规(接在最后,和 compose_doc 给了 masked 时一样),交回同一份文档。"""
from app.engine.evidence import iter_segments, masked_cells
wanted: dict[str, Any] = {}
for seg in iter_segments(doc):
@@ -363,71 +356,26 @@ class _MaskedCells:
for artifact, source in wanted.items():
await self._read(artifact, source)
cited = masked_cells(doc, self._catalog, {a: m for a, m in self._memo.items() if m})
written = masked_values(doc, await self._values(), {str(v.get("segment")) for v in cited})
doc["violations"] = [*doc["violations"], *cited, *written]
doc["violations"] = [*doc["violations"], *cited]
return doc
async def _read(self, artifact: str, source: Any) -> tuple[dict[str, Any], Any] | None:
"""读一份查询快照、认它的遮罩(_mask_sets),确定取自遮罩列的列记进 _memo。读不出来的为 None(引用它的格子本来就
解析不了)。"""
from app.api.evidence import _mask_sets
async def _read(self, artifact: str, source: Any) -> None:
"""读一份查询快照,引用了就算违规的列(_cited_masks:{结果列: 它取自的遮罩列})记进 _memo。读不出来的记空:引用它的
格子本来就解析不了。"""
from app.api.evidence import _cited_masks
from app.core.artifact_store import load
try:
snap = await asyncio.to_thread(load, artifact)
except ValueError:
snap = None
if not isinstance(snap, dict):
self._memo[artifact] = {}
return None
sets = await _mask_sets(snap, source, self._masks)
self._memo[artifact] = {c: sorted(sets.base) for c in sets.sure}
if not self._collected:
self._loaded[artifact] = (snap, sets)
return snap, sets
async def _values(self) -> Any:
"""写作目录里取自遮罩列的值(masking._Secrets),一个都没有为 None。只收一次。"""
if self._collected:
return self._secrets
self._collected = True
from app.engine.masking import _Secrets
from app.engine.merge_query import MERGE_SOURCE
secrets = _Secrets()
seen: set[str] = set()
for entry in self._catalog.values():
artifact = entry.get("artifact") if isinstance(entry, dict) and entry.get("kind") == "query" else None
source = entry.get("source") if isinstance(entry, dict) else None
if not isinstance(artifact, str) or not artifact or artifact in seen or not source \
or source == MERGE_SOURCE or not (await self._masks.of(source))[0]:
continue
seen.add(artifact)
# 认引用时读过的不再读;读过、读不出来的(_memo 里有、_loaded 里没有)不再试
found = self._loaded.get(artifact)
if found is None and artifact not in self._memo:
found = await self._read(artifact, source)
if found is not None:
snap, sets = found
secrets.add(snap, sets.sure, labels=sets.labels, base=sets.base)
self._loaded.clear()
secrets.protect(self._protected)
if secrets.strong:
secrets.prepare()
self._secrets = secrets
return self._secrets
self._memo[artifact] = await _cited_masks(snap, source, self._masks, artifact) if isinstance(snap, dict) else {}
def _masked_said(hits: list[dict[str, Any]]) -> str:
"""报告写进了遮罩列的值的那几处在缺口、日志里怎么说:「引用了 2 处取自遮罩列的格子(「v:Q2.r0.tel」、…)」「写进了 1 处
设置了遮罩的列中的值」,两样都有时连起来。只点名引用,不写值。"""
cited = [v for v in hits if not v.get("by_value")]
written = [v for v in hits if v.get("by_value")]
parts = [f"引用了 {len(cited)} 处取自遮罩列的格子({_named(cited)})"] if cited else []
if written:
named = _named(written)
parts.append(f"写进了 {len(written)} 处设置了遮罩的列中的值" + (f"({named})" if named else ""))
return ",".join(parts)
"""报告引用了遮罩列的那几处在缺口、日志里怎么说:「引用了 2 处取自遮罩列的格子(「v:Q2.r0.tel」、…)」。只点名引用,
不写值。"""
return f"引用了 {len(hits)} 处取自遮罩列的格子({_named(hits)})"
async def _run_class(run_id: str) -> str | None:
@@ -647,9 +595,8 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
spec = ctx.run.spec
catalog = report_catalog(state, spec, ctx.node)
# 报告写进了取自遮罩列的值(引用了那一格、照抄成正文):自查记一条违规,和没有出处的数字一样交回写作者重写
# (_MaskedCells)。运行输入里写着的值是发起运行的人自己填的,不算
masked = _MaskedCells(catalog, protected=[json.dumps(state.get("input") or {}, ensure_ascii=False, default=str)])
# 报告引用了取自遮罩列的格子:自查记一条违规,和没有出处的数字一样交回写作者重写(_MaskedCells)
masked = _MaskedCells(catalog)
allow = report_allowance(spec, ctx.node)
cells = True
# 结论句策略只对升级后发起的运行生效:升级前的运行(图里也不会有 claims)事件、产出一个字不加
@@ -754,9 +701,7 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
tried = f"已要求模型重写 {repairs} 次仍未改掉" if repairs else "未要求重写"
refs = list(dict.fromkeys(str(v["ref"]) for v in hits if v.get("ref")))
gaps.append(f"报告{_masked_said(hits)},{tried}:数据源设置了遮罩的列不应写进报告")
# written:其中按值认出来的几处(正文里直接写着、经指标带出来的),时间线据此换说法
ctx.emit(EventType.LOG, level="warn", code="report_masked_cell", count=len(hits), refs=refs,
written=sum(1 for v in hits if v.get("by_value")),
columns=list(dict.fromkeys(str(v["column"]) for v in hits if v.get("column"))),
message=f"{gaps[0]}。报告照常产出,这几处已标注,已记为缺口。请在报告撰写节点的指令中去掉对这些列的引用")
doc_artifact = await _store(doc, ctx)
@@ -857,10 +802,7 @@ async def _judge_inline(ctx: NodeContext, doc: dict[str, Any], catalog: dict[str
settings = await judging.judge_settings(session)
judge_spec = await judging.judge_model_spec(session, judge_cfg, settings=settings)
budget = judging.report_budget(settings, judge_cfg)
# 交给裁判的摘录要遮的列:数据源设的之外,SQL 里改了名字的遮罩列也算(和按需裁判同一个函数)
from app.api.evidence import _judge_masks
masks = await _judge_masks(catalog, await judging.source_masks(catalog))
masks = await judging.source_masks(catalog)
@task
async def judge_step(request: judging.JudgeRequest, key: str, limits: judging.Budget, known: dict[str, Any],
+238
View File
@@ -0,0 +1,238 @@
"""证据面板、推演收「这次运行查到过的遮罩值」:只收确定取自遮罩列的值;按快照工件在进程里记下,不再每次都读一遍。
复核发现(第 3 轮):
- 收值按的是面板展示用的 hidden(宁可多遮):SQL 认不准、整份遮的快照,hidden 是全部列,里面的区域名、金额都被收成
遮罩值。之后任何查询步骤里展示的 SQL「WHERE region = '华东' AND amount > 12345.5」都显示成「已遮罩」,整份结果当输入
的指标值等于这些数时也写「已遮罩」。现在只收确定取自遮罩列的那几列(_MaskSets.sure,同 masking 收值的口径),整份遮
的快照只遮显示
- 每点开一个片段,都把这次运行所有设了遮罩的数据源上的快照读盘、复验哈希,再逐份做 SQL 分析(一份几十毫秒),缓存
只在一次请求里有效。现在按快照工件 id 在进程里记下分析和收到的值(工件按内容寻址、不会变;键里带着数据源现在设的
遮罩和类型,设置改了不会拿旧的),之后的请求不再读、不再分析;推演同一份记录
快照、号码都是虚构的。
"""
from __future__ import annotations
import asyncio
import json
import sqlite3
from types import SimpleNamespace
import pytest
from httpx import ASGITransport, AsyncClient
from langchain_core.messages import AIMessage
from sqlalchemy import select
from app.core import artifact_store
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import DataSource, Run
from app.engine.evidence import iter_segments
from app.engine.runner import run_manager
from app.main import app
CUSTOMERS = [(1, "客户甲", "19900000901"), (2, "客户乙", "19900000902"), (3, "客户丙", "19900000903")]
ORDERS = [(1, 1, 120.5), (2, 2, 300.0), (3, 2, 80.0), (4, 3, 210.25)]
PHONES = [c[2] for c in CUSTOMERS]
#: 派生来源里有没起别名的 UPPER(c.phone):认遮罩列的那一套认不准,整份的列都遮(只遮显示)
SQL_WHOLE = ("SELECT region, amount FROM (SELECT 'east' AS region, UPPER(c.phone), o.amount FROM customers c "
"JOIN orders o ON o.customer_id = c.id) t")
SQL_TEL = "SELECT c.name AS customer, c.phone AS tel FROM customers c ORDER BY c.id"
@pytest.fixture
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
@pytest.fixture
async def client(engine_up):
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
yield c
async def set_source(path, options=None) -> str:
async with SessionLocal() as session:
row = (await session.execute(select(DataSource).where(DataSource.name == "mall"))).scalar_one_or_none()
if row is None:
row = DataSource(name="mall", kind="sqlite", database=str(path), readonly=True, enabled=True)
session.add(row)
row.database, row.kind, row.enabled, row.readonly = str(path), "sqlite", True, True
row.options = dict(options or {})
await session.commit()
return row.id
@pytest.fixture
async def mall(tmp_path, engine_up):
from app.api.evidence import _MEMO
path = tmp_path / "mall.db"
db = sqlite3.connect(path)
db.executescript("CREATE TABLE customers (id INTEGER PRIMARY KEY, name TEXT, phone TEXT);"
"CREATE TABLE orders (id INTEGER PRIMARY KEY, customer_id INTEGER, amount REAL);")
db.executemany("INSERT INTO customers VALUES (?, ?, ?)", CUSTOMERS)
db.executemany("INSERT INTO orders VALUES (?, ?, ?)", ORDERS)
db.commit()
db.close()
source_id = await set_source(path, {"mask_columns": ["phone"]})
await engines.invalidate(source_id)
# 同一个测试进程里别的测试记下的结果(内容相同的快照工件 id 一样)不算进这里的读盘次数
_MEMO.clear()
yield path
await engines.invalidate(source_id)
async def put(sql: str, columns: list[str], rows: list[list]) -> str:
return await artifact_store.put_json({"sql": sql, "columns": columns, "rows": rows, "source": "mall"}, kind="query")
def sealed_of(*artifacts: str) -> SimpleNamespace:
return SimpleNamespace(queries={a: {"source": "mall"} for a in artifacts})
# --------------------------------------------------------------------------
# 只收确定取自遮罩列的值
# --------------------------------------------------------------------------
async def test_values_of_a_snapshot_masked_whole_are_not_collected(engine_up, mall):
"""整份遮的快照只遮显示:里面的区域名、金额不收成遮罩值,别的查询步骤里写着它们的 SQL 照原样显示;改了名字的号码列
(c.phone AS tel)照收。"""
from app.api.evidence import _Masks, _Shown
whole = await put(SQL_WHOLE, ["region", "amount"], [["华东", 12345.5], ["华南", 6789.25]])
tel = await put(SQL_TEL, ["customer", "tel"], [[c[1], c[2]] for c in CUSTOMERS])
assert await _Masks(sealed_of(whole)).run_secrets() is None
secrets = await _Masks(sealed_of(whole, tel)).run_secrets()
sql = f"SELECT SUM(amount) FROM orders WHERE region = '华东' AND amount > 12345.5 AND phone <> '{PHONES[0]}'"
shown = _Shown(secrets)
assert shown.text(sql) == sql.replace(PHONES[0], "已遮罩") and shown.changed, shown.text(sql)
# --------------------------------------------------------------------------
# 按快照工件在进程里记下
# --------------------------------------------------------------------------
def spies(monkeypatch) -> tuple[list[str], list[str]]:
"""(读过的工件, 做过的 SQL 分析):证据面板经 artifact_store.load 读快照(查询步骤直接读,收值经 _fetch),按 SQL 认
遮罩列经可验证报告和 masking 的两个函数。"""
from app.api import evidence as api
from app.bundle import build
from app.engine import masking
fetched: list[str] = []
analysed: list[str] = []
real_load, real_bundle, real_masking = api.load, build._derived_columns, masking._derived_hidden
monkeypatch.setattr(api, "load", lambda artifact: fetched.append(artifact) or real_load(artifact))
monkeypatch.setattr(build, "_derived_columns",
lambda *a, **k: analysed.append("bundle") or real_bundle(*a, **k))
monkeypatch.setattr(masking, "_derived_hidden", lambda *a, **k: analysed.append("masking") or real_masking(*a, **k))
return fetched, analysed
async def test_the_values_are_read_and_analysed_once_per_process(engine_up, mall, monkeypatch):
"""第一次收:设了遮罩的快照各读一次、分析一次。之后的请求(新的 _Masks)一份都不再读、不再分析,收到的值一样。"""
from app.api.evidence import _Masks
tel = await put(SQL_TEL, ["customer", "tel"], [[c[1], c[2]] for c in CUSTOMERS])
raw = await put("SELECT id, phone FROM customers ORDER BY id", ["id", "phone"], [[c[0], c[2]] for c in CUSTOMERS])
fetched, analysed = spies(monkeypatch)
first = await _Masks(sealed_of(tel, raw)).run_secrets()
assert sorted(fetched) == sorted([tel, raw]) and analysed, (fetched, analysed)
assert set(PHONES) <= first.strong
fetched.clear()
analysed.clear()
second = await _Masks(sealed_of(tel, raw)).run_secrets()
assert fetched == [] and analysed == [] and second.strong == first.strong, (fetched, analysed)
async def test_a_changed_mask_setting_is_not_served_from_the_memo(engine_up, mall, monkeypatch):
"""数据源的遮罩设置改了:键里带着现在设的遮罩,按新的设置重新收;不再设遮罩的数据源一份都不读。"""
from app.api.evidence import _Masks, _Shown
tel = await put(SQL_TEL, ["customer", "tel"], [[c[1], c[2]] for c in CUSTOMERS])
assert "客户甲" not in (await _Masks(sealed_of(tel)).run_secrets()).strong
await set_source(mall, {"mask_columns": ["phone", "name"]})
widened = await _Masks(sealed_of(tel)).run_secrets()
assert {"客户甲", PHONES[0]} <= widened.strong
assert _Shown(widened).text("WHERE c.name = '客户甲'") == "WHERE c.name = '已遮罩'"
await set_source(mall, {})
fetched, _ = spies(monkeypatch)
assert await _Masks(sealed_of(tel)).run_secrets() is None and fetched == []
async def test_the_whatif_collects_from_the_same_memo(engine_up, mall):
"""推演在工作线程里收(_Hidden.secrets,经推演自己的 loader 读):第一次读一份,之后的推演、证据面板都不再读。"""
from app.api.evidence import _Masks
from app.api.whatif import _Hidden
tel = await put(SQL_TEL, ["customer", "tel"], [[c[1], c[2]] for c in CUSTOMERS])
reads: list[str] = []
def loader(artifact: str):
reads.append(artifact)
return artifact_store.load(artifact)
loop = asyncio.get_running_loop()
first = await asyncio.to_thread(_Hidden(sealed_of(tel), loader, loop).secrets)
assert reads == [tel] and set(PHONES) <= first.strong
reads.clear()
second = await asyncio.to_thread(_Hidden(sealed_of(tel), loader, loop).secrets)
assert reads == [] and second.strong == first.strong
assert (await _Masks(sealed_of(tel)).run_secrets()).strong == first.strong
# --------------------------------------------------------------------------
# 片段接口:之后点开只读这一步要显示的快照
# --------------------------------------------------------------------------
def node(nid, ntype, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
SQL_BY_PHONE = ("SELECT ROUND(SUM(o.amount), 2) AS spend FROM orders o JOIN customers c ON c.id = o.customer_id "
f"WHERE c.phone IN ('{PHONES[0]}', '{PHONES[1]}')")
async def test_a_segment_opened_again_reads_only_its_own_snapshot(client, monkeypatch, mall):
"""点开一个片段:第一次要把这次运行设了遮罩的快照读一遍(收遮罩值,SQL 里写着号码时展示要换掉);再点开(新的请求)
只读这一步要显示的那一份、不再做 SQL 分析,号码照样换掉。"""
from app.providers import mock_model
monkeypatch.setattr(mock_model.MockChatModel, "_decide",
lambda self, messages: AIMessage(content="两位重点客户共消费 [[v:Q2.r0.spend]]。"))
nodes = [node("start", "input"),
node("q_cust", "tool", tool="db_query__mall", args={"sql": "SELECT id, phone AS tel FROM customers ORDER BY id"}),
node("q_one", "tool", tool="db_query__mall", args={"sql": SQL_BY_PHONE}),
node("write", "report", instructions="写一句", on_violation="flag"),
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}])]
spec = {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
run_id = (await run_manager.start(graph=spec, input_payload={})).id
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run_id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
break
assert row.status == "succeeded", row.error
[report] = (await client.get(f"/api/runs/{run_id}/evidence")).json()["reports"]
doc = artifact_store.load(report["doc_artifact"])
seg = next(s for s in iter_segments(doc) if s.get("ref") == "v:Q2.r0.spend")
catalog = doc["catalog"]
cust, one = catalog["Q1"]["artifact"], catalog["Q2"]["artifact"]
fetched, analysed = spies(monkeypatch)
for attempt in range(2):
fetched.clear()
analysed.clear()
body = (await client.get(f"/api/runs/{run_id}/evidence/segments/{seg['id']}")).json()
[step] = [s for s in body["chain"] if s["step"] == "query"]
assert "IN ('已遮罩', '已遮罩')" in step["sql"] and PHONES[0] not in json.dumps(body, ensure_ascii=False)
assert one in fetched, fetched
assert (cust in fetched) is (attempt == 0), (attempt, fetched)
assert bool(analysed) is (attempt == 0), (attempt, analysed)
@@ -1,4 +1,4 @@
"""证据面板认遮罩列和可验证报告是同一个函数;查询步骤里写着的遮罩值、交给裁判的摘录同样不给。
"""证据面板认遮罩列和可验证报告是同一个函数;查询步骤里写着的遮罩值同样不给;交给裁判的摘录照数据源设的遮罩列。
复核发现:
- 证据面板只按数据源的一种方言调 masking._derived_hidden,可验证报告另有几道(语法树认不准的写法、不点列名也读得出表里
@@ -6,8 +6,9 @@
照给、可验证报告整份遮。现在面板调可验证报告的那个函数(bundle/build.py 的 _derived_columns),再按数据源的方言认一遍
- 查询步骤原样给出 SQL 里写着的遮罩值(按号码筛的 WHERE c.phone IN (…)),合并 SQL、SQL 检查的片段也一样;可验证报告
按「这次运行查到过的遮罩值」认出来、整份只给哈希。现在展示时换成「已遮罩」,底下写明只是展示时换的
- 按需裁判交给裁判的摘录只按列名去掉 phone,c.phone AS tel 那一列的号码照样交给裁判。现在按证据面板同一个函数认出的列
也并进去
- 第 2 轮把面板那套宁可多遮的列按数据源名并给裁判(裁判按数据源名合起来遮,对同一数据源的每一份快照都生效):整份遮的
快照的全部列、只数个数的列一并从摘录里消失,被引用的金额、人数裁判看不到,只能判证据不足。第 3 轮退回 M2 之前的做法,
只按数据源设的列名遮;c.phone AS tel 这种改了名字的号码列仍会进摘录,是裁判摘录已知的缺口,另行处理(judge.py)
真跑的部分用虚构的 SQLite 商城库(号码是编的),报告由 MockChatModel 照剧本写。
"""
@@ -15,6 +16,7 @@ from __future__ import annotations
import asyncio
import json
import re
import sqlite3
import pytest
@@ -26,7 +28,7 @@ from app.core import artifact_store
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import DataSource, Run
from app.engine.evidence import iter_segments
from app.engine.evidence import iter_segments, iter_units
from app.engine.runner import run_manager
from app.main import app
@@ -196,26 +198,94 @@ def test_sql_check_excerpts_and_merge_parts_are_shown_with_values_masked():
# --------------------------------------------------------------------------
# 交给裁判的摘录
# 交给裁判的摘录:照数据源设的遮罩列(复核第 3 轮退回 M2 之前的做法)
# --------------------------------------------------------------------------
#: 派生来源里有没起别名的 UPPER(c.phone):证据面板整份遮(宁可多遮),可这份快照的列和号码无关
SQL_WHOLE = ("SELECT name, amount FROM (SELECT c.name, UPPER(c.phone), o.amount FROM customers c "
"JOIN orders o ON o.customer_id = c.id) t ORDER BY amount DESC")
#: 金额、只数个数的留号人数:值里没有号码,面板照可验证报告的认法把只数个数的那一列也遮
SQL_TOTAL = ("SELECT ROUND(SUM(o.amount), 2) AS amount, COUNT(*) FILTER (WHERE c.phone IS NOT NULL) AS with_phone "
"FROM orders o JOIN customers c ON c.id = o.customer_id")
JUDGED = "全部订单金额 [[v:Q2.r0.amount]],其中 [[v:Q2.r0.with_phone]] 笔的客户留了手机号,最大一笔 [[v:Q1.r0.amount]]。"
async def test_the_judge_excerpt_drops_a_column_renamed_from_a_masked_one(engine_up, mall):
"""c.phone AS tel:交给裁判的遮罩列表把 tel 并进数据源名下,摘录里不给 tel 那一列,号码一个都不交出去。"""
from app.api.evidence import _judge_masks
from app.engine import judge as judging
art = await artifact_store.put_json({"columns": ["customer", "tel", "spend"],
"rows": [["客户乙", PHONES[1], 380.0], ["客户甲", PHONES[0], 220.0]],
"source": "mall", "mask_columns": ["phone"],
"sql": "SELECT c.name AS customer, c.phone AS tel, 1 AS spend FROM customers c"},
kind="query")
entry = {"alias": "Q1", "kind": "query", "artifact": art, "columns": ["customer", "tel", "spend"], "rows": 2,
"node_id": "top", "source": "mall", "tool": "db_query__mall"}
masked = await _judge_masks({"Q1": entry}, {"mall": ["phone"]})
assert {c.lower() for c in masked["mall"]} >= {"phone", "tel"}, masked
excerpt = judging._query_excerpt("Q1", entry, {0}, ["tel", "spend"], True, judging._Fetch(None), masked)
assert excerpt and leaks(excerpt) == [] and "380" in excerpt, excerpt
# 只按列名(以前的办法):号码照样在摘录里
before = judging._query_excerpt("Q1", entry, {0}, ["tel", "spend"], True, judging._Fetch(None), {"mall": ["phone"]})
assert leaks(before), "对照:只按列名遮时号码在摘录里"
def judge_models(monkeypatch) -> list[str]:
"""写作者回 JUDGED;裁判一律判有依据。返回每次交给裁判的请求原文。"""
from app.engine import judge as judge_mod
from app.engine.judge import JUDGE_ROLE
from app.providers.mock_model import MockChatModel
seen: list[str] = []
def decide(model, messages):
if JUDGE_ROLE in str(messages[0].content):
request = str(messages[-1].content)
seen.append(request)
items = [{"unit": uid, "verdict": "supported", "rationale": "虚构的理由", "used": []}
for uid in re.findall(r"^\[(u\d+)\]", request, re.M)]
return AIMessage(content=json.dumps({"items": items}, ensure_ascii=False))
return AIMessage(content=JUDGED)
monkeypatch.setattr(MockChatModel, "_decide", decide)
async def judge_model_of(session, spec):
return MockChatModel(model_name="claude-sonnet-5"), "claude-sonnet-5"
monkeypatch.setattr(judge_mod, "get_chat_model", judge_model_of)
return seen
async def judged_run(run_class: str, **report) -> Run:
from sqlalchemy import delete
from app.db.models import Setting
from app.engine.judge import SPEND_KEY
async with SessionLocal() as session:
await session.execute(delete(Setting).where(Setting.key.in_([SPEND_KEY, "judge"])))
await session.commit()
nodes = [node("start", "input"),
node("q_top", "tool", tool="db_query__mall", args={"sql": SQL_WHOLE}),
node("q_total", "tool", tool="db_query__mall", args={"sql": SQL_TOTAL}),
node("write", "report", instructions="写一句", **report),
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}])]
spec = {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
run_id = (await run_manager.start(graph=spec, input_payload={}, run_class=run_class)).id
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run_id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
assert row.status == "succeeded", row.error
return row
raise AssertionError(f"运行 {run_id} 没跑完:{row.status}")
def excerpt_of(request: str) -> str:
"""裁判请求里 Q2 那一段摘录。"""
return request.split("【Q2】", 1)[1].split("【", 1)[0]
@pytest.mark.parametrize("where", ["on_demand", "inline"])
async def test_the_judge_sees_the_cited_columns_of_every_snapshot(client, monkeypatch, mall, where):
"""交给裁判的遮罩列只按数据源设的列名(judge.source_masks),不并证据面板那套宁可多遮的列:Q1 整份遮、Q2 只数个数
的列,面板都不显示,可它们和号码无关,裁判要看得到被引用的金额、人数才判得了。按需裁判(探索运行)、节点里当场裁判
(正式运行)同一个办法。号码列照旧不给(摘录里一个号码都没有)。"""
seen = judge_models(monkeypatch)
if where == "on_demand":
row = await judged_run("exploratory", claims="judge")
[report] = (await client.get(f"/api/runs/{row.id}/evidence")).json()["reports"]
doc = artifact_store.load(report["doc_artifact"])
uid = next(u["id"] for _, u in iter_units(doc)
if any(seg.get("ref") == "v:Q2.r0.amount" for seg in u.get("segments") or []))
res = await client.post(f"/api/runs/{row.id}/evidence/judge", json={"units": [uid]})
assert res.status_code == 200, res.text
else:
await judged_run("formal", claims="judge")
[request] = seen
q2 = excerpt_of(request)
assert "amount" in q2 and "with_phone" in q2 and "710.75" in q2, q2
[q1_columns] = [line for line in request.split("【Q1】", 1)[1].split("【", 1)[0].splitlines() if line.startswith("列:")]
assert "amount" in q1_columns and "name" in q1_columns, q1_columns
assert leaks(request) == []
+1 -1
View File
@@ -5,7 +5,7 @@
所以拦在自查:
- 组装文档时(compose_doc 的 masked:{查询快照工件: 取自遮罩列的列}),引用了这些列的格子记一条 masked_cell,列名对上
的、按 SQL 认出改了名字的都算(报告节点按证据面板同一个函数 api/evidence.py 的 _snapshot_masks 算)
的、按 SQL 认出改了名字的都算(报告节点按 api/evidence.py 的 _cited_masks 算,和证据面板同一个 _mask_sets 起步)
- 违规里不带那一格的值(不写 text、context):违规会进 report.checked、重写的日志、出具缺口,这些地方不能再露一次号码
- 报告节点把它交回写作者重写(不是只标注的那几类);重写后还在:「违规处置」为失败的判失败,为标注的照常产出,
时间线上另起一行说清(report_masked_cell),节点产出里写一条出具缺口,出口降档
+261
View File
@@ -0,0 +1,261 @@
"""报告自查只查引用,而且只认真取自遮罩表里那一列的格子(复核第 3 轮)。
复核发现:
- 第 2 轮加的按值自查把两个字的遮罩值当子串去认:客户里有一位叫「高峰」,正文写「本周销售高峰」就判 masked_cell,正式
运行(默认违规即失败)因此失败,报错还叫人去改写作指令
- 按值自查连同名列的值一起收:mask_columns 设了 name(本意是客户姓名),正文提到门店名(stores.name)同样判失败
- 引用同理:遮罩按列名设在整个数据源上(不带表名),s.name AS store 按列名认也算取自遮罩列「name」
现在自查只认写作者用 [[v:Qn.rX.列]] 引用的格子(正文里照抄的号码本来就会被「数字没有出处」拦下),并且按 SQL 的血缘核一遍
这一列的值取自哪张表的哪一列:遮罩列在几张表里都有、其中有的表里另有别的遮罩列时(customers 有 name 和 phone,stores 只有
name),只算那几张表。证据面板、可验证报告照列名多遮,不受影响。
真跑的部分用虚构的 SQLite 零售库(号码、姓名、门店名都是编的),数据源探查过表结构(查询快照里冻结着表结构),写作者是
MockChatModel 照剧本写。
"""
from __future__ import annotations
import asyncio
import json
import sqlite3
import pytest
from langchain_core.messages import AIMessage
from sqlalchemy import select
from app.core import artifact_store
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import DataSource, Run, RunEvent
from app.engine.evidence import SCHEMA_SNAPSHOT, compose_doc
from app.engine.runner import run_manager
SOURCE = "m2shop"
STORES = [(1, "杭州西湖店", "华东"), (2, "广州天河店", "华南")]
CUSTOMERS = [(1, "陈思远", "19900000801"), (2, "高峰", "19900000802"), (3, "林若溪", "19900000803")]
ORDERS = [(1, 1, 1, 120.5), (2, 2, 2, 300.0), (3, 1, 2, 80.0), (4, 2, 3, 210.25)]
PHONES = [c[2] for c in CUSTOMERS]
SQL_STORE = ("SELECT s.name AS store, ROUND(SUM(o.amount), 2) AS sales FROM stores s JOIN orders o ON o.store_id = s.id "
"GROUP BY s.name ORDER BY sales DESC")
SQL_CUSTOMER = ("SELECT c.name AS customer, ROUND(SUM(o.amount), 2) AS spend FROM customers c "
"JOIN orders o ON o.customer_id = c.id GROUP BY c.name ORDER BY spend DESC")
@pytest.fixture
async def engine_up():
await run_manager.setup()
yield
await run_manager.shutdown()
async def set_source(path, options: dict, *, probe: bool = True) -> str:
"""数据源 m2shop:probe 时探查表结构(工具查询时把它冻结进快照),不然清空。名字只这个文件用,探查的结构不会
冻结进别的测试的快照。"""
from app.data.introspect import introspect
async with SessionLocal() as session:
row = (await session.execute(select(DataSource).where(DataSource.name == SOURCE))).scalar_one_or_none()
if row is None:
row = DataSource(name=SOURCE, kind="sqlite", database=str(path), readonly=True, enabled=True)
session.add(row)
row.database, row.kind, row.enabled, row.readonly = str(path), "sqlite", True, True
row.options = dict(options)
row.schema_cache = (await introspect(row)) if probe else None
await session.commit()
return row.id
@pytest.fixture
async def shop(tmp_path, engine_up):
path = tmp_path / "shop.db"
db = sqlite3.connect(path)
db.executescript("CREATE TABLE stores (id INTEGER PRIMARY KEY, name TEXT, region TEXT);"
"CREATE TABLE customers (id INTEGER PRIMARY KEY, name TEXT, phone TEXT);"
"CREATE TABLE orders (id INTEGER PRIMARY KEY, store_id INTEGER, customer_id INTEGER, amount REAL);")
db.executemany("INSERT INTO stores VALUES (?, ?, ?)", STORES)
db.executemany("INSERT INTO customers VALUES (?, ?, ?)", CUSTOMERS)
db.executemany("INSERT INTO orders VALUES (?, ?, ?, ?)", ORDERS)
db.commit()
db.close()
source_id = await set_source(path, {"mask_columns": ["name", "phone"]})
await engines.invalidate(source_id)
yield path
await set_source(path, {}, probe=False)
await engines.invalidate(source_id)
def node(nid, ntype, **config):
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
def graph(sql: str, **report) -> dict:
nodes = [node("start", "input"), node("top", "tool", tool=f"db_query__{SOURCE}", args={"sql": sql}),
node("write", "report", instructions="写一句", **report),
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}])]
return {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
async def finish(monkeypatch, spec: dict, text: str) -> Run:
from app.providers import mock_model
monkeypatch.setattr(mock_model.MockChatModel, "_decide", lambda self, messages: AIMessage(content=text))
run_id = (await run_manager.start(graph=spec, input_payload={})).id
for _ in range(400):
await asyncio.sleep(0.05)
async with SessionLocal() as session:
row = await session.get(Run, run_id)
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
return row
raise AssertionError(f"运行 {run_id} 没跑完:{row.status}")
async def checked_of(run_id: str) -> dict:
async with SessionLocal() as session:
q = select(RunEvent).where(RunEvent.run_id == run_id, RunEvent.type == "report.checked")
return (await session.execute(q)).scalars().one().data
def masked_of(violations: list[dict]) -> list[dict]:
return [v for v in violations if v["code"] == "masked_cell"]
def leaks(value) -> list[str]:
raw = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False, default=str)
return [p for p in PHONES if p in raw or p in raw.replace(",", "")]
# --------------------------------------------------------------------------
# 正文里写着的字不按值认
# --------------------------------------------------------------------------
@pytest.mark.parametrize("sql, text", [
# 客户里有一位叫「高峰」:常用词「销售高峰」不是在写这位客户
(SQL_CUSTOMER, "本周销售高峰出现在周五,最高一位客户消费 [[v:Q1.r0.spend]]。"),
# 设了 name 本意是客户姓名:正文用文字写门店名照常
(SQL_STORE, "销售额最高的门店是广州天河店,销售额 [[v:Q1.r0.sales]]。"),
])
async def test_words_that_happen_to_equal_masked_values_do_not_fail_a_formal_report(monkeypatch, shop, sql, text):
"""违规即失败(正式运行的默认值):正文里的字和遮罩列中的值相同,不判 masked_cell,运行照常成功。"""
row = await finish(monkeypatch, graph(sql, on_violation="fail"), text)
assert row.status == "succeeded", row.error
checked = await checked_of(row.id)
assert masked_of(checked["violations"]) == [] and checked["ok"] is True, checked["violations"]
async def test_a_phone_copied_into_the_text_is_stopped_by_the_number_check(monkeypatch, shop):
"""照抄成正文的号码不按值认,「数字没有出处」照样拦下:交回重写、仍不改就判失败;违规里没有 masked_cell。"""
row = await finish(monkeypatch, graph(SQL_CUSTOMER, on_violation="fail"),
f"消费最高的客户电话 {PHONES[1]},共消费 [[v:Q1.r0.spend]]。")
assert row.status == "failed", row.error
checked = await checked_of(row.id)
assert {v["code"] for v in checked["violations"]} == {"uncited_number"}, checked["violations"]
assert "遮罩" not in (row.error or "")
# --------------------------------------------------------------------------
# 引用:只认真取自遮罩表里那一列的
# --------------------------------------------------------------------------
async def test_citing_a_store_name_is_not_flagged_though_name_is_masked(monkeypatch, shop):
"""s.name AS store:门店表只有 name、没有别的遮罩列,customers 里 name 和 phone 都在——设了 name 管的是客户姓名。"""
row = await finish(monkeypatch, graph(SQL_STORE, on_violation="fail"),
"销售额最高的门店是 [[v:Q1.r0.store]],销售额 [[v:Q1.r0.sales]]。")
assert row.status == "succeeded", row.error
assert masked_of((await checked_of(row.id))["violations"]) == []
async def test_citing_a_customer_name_is_flagged_and_names_the_masked_column(monkeypatch, shop):
"""c.name AS customer:取自客户表的 name,照旧拦下;说明只点名它取自的那一列遮罩列,不把 phone 也列进来。"""
row = await finish(monkeypatch, graph(SQL_CUSTOMER, on_violation="flag"),
"消费最高的客户是 [[v:Q1.r0.customer]],共消费 [[v:Q1.r0.spend]]。")
assert row.status == "succeeded", row.error
[v] = masked_of((await checked_of(row.id))["violations"])
assert v["ref"] == "v:Q1.r0.customer" and "取自设置了遮罩的列「name」" in v["message"], v["message"]
# --------------------------------------------------------------------------
# 认法:_cited_masks(报告节点经 _MaskedCells 调用)
# --------------------------------------------------------------------------
#: 冻结的表结构(introspect 的形状):stores 只有 name,customers 有 name 和 phone
TABLES = {"stores": ["id", "name", "region"], "customers": ["id", "name", "phone"],
"orders": ["id", "store_id", "customer_id", "amount"]}
async def schema_of(tables: dict[str, list[str]]) -> str:
return await artifact_store.put_json(
{"source": SOURCE, "tables": {t: {"columns": [{"name": c, "type": "TEXT"} for c in cols]}
for t, cols in tables.items()}}, kind=SCHEMA_SNAPSHOT)
async def cited(sql: str, columns: list[str], *, masks: list[str], schema: bool = True,
tables: dict[str, list[str]] = TABLES) -> dict[str, list[str]]:
"""一份快照按报告自查认出的列:{结果列: 它取自的遮罩列}。"""
from app.api.evidence import _cited_masks, _Masks
await set_source(":memory:", {"mask_columns": masks}, probe=False)
snap = {"sql": sql, "columns": columns, "rows": [], "source": SOURCE,
**({"schema_artifact": await schema_of(tables)} if schema else {})}
art = await artifact_store.put_json(snap, kind="query")
return await _cited_masks(snap, SOURCE, _Masks(), art)
MIXED = ("SELECT s.name AS store, c.name AS customer, c.phone AS tel, SUM(o.amount) AS spend FROM orders o "
"JOIN stores s ON s.id = o.store_id JOIN customers c ON c.id = o.customer_id GROUP BY s.name, c.name, c.phone")
@pytest.mark.parametrize("label, sql, columns, masks, expected", [
("两张表的同名列在一条查询里:只认客户表的", MIXED, ["store", "customer", "tel", "spend"], ["name", "phone"],
{"customer": ["name"], "tel": ["phone"]}),
("只设了 name:几张表都有 name、哪张都没有别的遮罩列,照列名认", MIXED, ["store", "customer", "tel", "spend"],
["name"], {"store": ["name"], "customer": ["name"]}),
("直接选出门店表的 name(列名对上遮罩列)", "SELECT name, region FROM stores", ["name", "region"],
["name", "phone"], {}),
("* 展开门店表", "SELECT * FROM stores", ["id", "name", "region"], ["name", "phone"], {}),
("* 展开客户表", "SELECT * FROM customers", ["id", "name", "phone"], ["name", "phone"],
{"name": ["name"], "phone": ["phone"]}),
("经 WITH 转两道", "WITH a AS (SELECT c.phone AS tel FROM customers c), b AS (SELECT tel AS contact FROM a) "
"SELECT contact FROM b", ["contact"], ["phone"], {"contact": ["phone"]}),
("列名和遮罩列一样,值却取自别的列", "SELECT region AS phone FROM stores", ["phone"], ["phone"], {}),
("UNION:有一支取自客户表就算", "SELECT name FROM stores UNION ALL SELECT name FROM customers", ["name"],
["name", "phone"], {"name": ["name"]}),
])
async def test_cited_masks_follow_the_table_the_value_comes_from(engine_up, label, sql, columns, masks, expected):
assert await cited(sql, columns, masks=masks) == expected, label
async def test_without_a_frozen_schema_the_mask_is_read_by_column_name(engine_up):
"""数据源没探查过表结构(快照里没有冻结的表结构):认不出同名的列还在哪几张表里,照列名认,门店名也算;值根本不取自
遮罩列的(region AS phone)照样不算。"""
assert await cited(MIXED, ["store", "customer", "tel", "spend"], masks=["name", "phone"], schema=False) == \
{"store": ["name"], "customer": ["name"], "tel": ["phone"]}
assert await cited("SELECT region AS phone FROM stores", ["phone"], masks=["phone"], schema=False) == {}
@pytest.mark.parametrize("sql, columns", [
# 表值函数:拆的是号码,血缘认不出出处,照遮罩那一套认
("SELECT x.value AS p FROM customers c, json_each(c.phone) x", ["p"]),
# 没起别名的表达式:血缘按列名找不到这一列
("SELECT id, UPPER(phone) FROM customers", ["id", "UPPER(phone)"]),
])
async def test_columns_the_lineage_cannot_place_stay_flagged(engine_up, sql, columns):
assert set(await cited(sql, columns, masks=["name", "phone"])) == {columns[-1].lower()}
async def test_the_report_node_flags_only_the_customer_column(engine_up):
"""报告节点的自查(_MaskedCells)按同一个函数认:同一条查询里引用门店名不算、引用客户名算。"""
from app.engine.nodes.report import _MaskedCells
await set_source(":memory:", {"mask_columns": ["name", "phone"]}, probe=False)
art = await artifact_store.put_json({"sql": MIXED, "columns": ["store", "customer", "tel", "spend"],
"rows": [["广州天河店", "高峰", PHONES[1], 300.0]], "source": SOURCE,
"schema_artifact": await schema_of(TABLES)}, kind="query")
catalog = {"Q1": {"alias": "Q1", "kind": "query", "artifact": art, "columns": ["store", "customer", "tel", "spend"],
"rows": 1, "node_id": "top", "source": SOURCE}}
doc = compose_doc("门店 [[v:Q1.r0.store]],客户 [[v:Q1.r0.customer]],消费 [[v:Q1.r0.spend]]。", catalog,
node_id="write", run_id="r1")
[v] = masked_of((await _MaskedCells(catalog)(doc))["violations"])
assert v["ref"] == "v:Q1.r0.customer" and "「name」" in v["message"] and leaks(v) == [], v
@@ -236,8 +236,8 @@ async def test_the_rewrite_request_says_the_value_must_not_be_written_in_any_for
async def test_only_the_cited_snapshots_are_read_and_each_only_once(engine_up, mall, monkeypatch):
"""组装好一稿之后才读:引用到的那几份、数据源现在设了遮罩的那几份(按值认要收里面的值),每份只读一次,重写的稿子
不再读;没设遮罩的数据源上的一份都不读(test_report_masked_values)。"""
"""组装好一稿之后才读:只读正文引用到的那几份,每份只读一次,重写的稿子不再读;同一个数据源上没被引用的快照不读
(第 2 轮按值认时要把设了遮罩的都读一遍,第 3 轮去掉了按值认)。"""
from app.core import artifact_store as store
from app.engine.nodes.report import _MaskedCells
@@ -259,10 +259,11 @@ async def test_only_the_cited_snapshots_are_read_and_each_only_once(engine_up, m
monkeypatch.setattr(store, "load", counting)
check = _MaskedCells(catalog)
first = await check(compose_doc("电话 [[v:Q1.r0.tel]]。", catalog, node_id="write", run_id="r1"))
assert masked_of(first["violations"]) == ["v:Q1.r0.tel"] and sorted(read) == sorted([cited, other]), read
assert masked_of(first["violations"]) == ["v:Q1.r0.tel"] and read == [cited], read
second = await check(compose_doc("客户 [[v:Q1.r0.customer]],电话 [[v:Q1.r0.tel]]。", catalog, node_id="write",
run_id="r1"))
assert masked_of(second["violations"]) == ["v:Q1.r0.tel"] and len(read) == 2, "同一份快照不再读第二遍"
assert masked_of(second["violations"]) == ["v:Q1.r0.tel"] and read == [cited], "同一份快照不再读第二遍"
assert other not in read
# --------------------------------------------------------------------------
@@ -280,4 +281,4 @@ async def test_the_audit_lists_a_masked_cell_on_its_own_row(client, monkeypatch,
# 这一行自己不写值(所在的句子是报告原文,标注之后照常产出,读者本来就看得到)
assert leaks([flagged[0][k] for k in ("text", "ref", "note")]) == []
csv = (await client.get(f"/api/runs/{row.id}/evidence/audit", params={"format": "csv"})).text
assert "写进了遮罩列的值" in csv
assert "引用了遮罩列" in csv
+24 -71
View File
@@ -1,4 +1,4 @@
"""报告自查(第 2 轮):只数个数的列不算取自遮罩列;正文里直接写着的遮罩值按值认出来。
"""报告自查:只数个数的列不算取自遮罩列;正文里写着的值不按值认。
复核发现:
- 只数个数、只决定算不算进来的写法——COUNT(*) FILTER (WHERE phone IS NULL)、SUM(phone IS NULL)、AVG(phone IS NOT NULL)、
@@ -6,16 +6,15 @@
条件、汇总里的真假条件当成值的来历,自查跟着判 masked_cell,正式运行(默认违规即失败)因此判失败。等价的
SUM(CASE WHEN phone IS NULL THEN 1 ELSE 0 END) 本来就不算。现在自查认之前先把这几种写法改写成认得出的样子
(api/evidence.py 的 _counted),证据面板、推演照可验证报告的认法多遮(展示上多遮无妨)
- 自查只认 [[v:]] 引用:写作者照着「删掉引用」把号码、姓名照抄成正文,或者经指标带出来([[m:phone_raw]] 的值就是号码),
一条违规都没有。现在按值认一遍(engine/evidence.py 的 masked_values):写作目录里取自遮罩列的值(只认确定取自遮罩列的
列,整份算进来的不算)出现在正文里,记一条 masked_cell,不带值;运行输入里写着的值是发起运行的人自己填的,不算
- 第 2 轮另按值认过正文里写着的遮罩值(照抄的号码、姓名,经指标带出来的值),第 3 轮去掉了:两个字的姓名会撞上「销售
高峰」这类常用词,同名列的值会撞上门店名,正式运行因此判失败,代价远大于收益;照抄的号码本来就会被「数字没有出处」
拦下(test_report_masked_owners)。自查只查 [[v:]] 引用,这里留着不按值认的回归
真跑的部分用虚构的 SQLite 商城库(号码、姓名是编的),写作者是 MockChatModel 照剧本写。
"""
from __future__ import annotations
import asyncio
import json
import sqlite3
import pytest
@@ -26,7 +25,7 @@ from app.core import artifact_store
from app.data.engine import engines
from app.db.base import SessionLocal
from app.db.models import DataSource, Run, RunEvent
from app.engine.evidence import compose_doc, describe_violations
from app.engine.evidence import compose_doc
from app.engine.runner import run_manager
CUSTOMERS = [(1, "陈思远", "19900000701"), (2, "林若溪", "19900000702"), (3, "周以宁", "19900000703")]
@@ -70,12 +69,6 @@ async def mall(tmp_path, engine_up):
await engines.invalidate(source_id)
def leaks(value) -> list[str]:
"""写着的号码、姓名,按千分位写的(19,900,000,701)也算。"""
raw = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False, default=str)
return [p for p in [*PHONES, *NAMES] if p in raw or p in raw.replace(",", "")]
def masked_of(violations: list[dict]) -> list[dict]:
return [v for v in violations if v["code"] == "masked_cell"]
@@ -182,7 +175,7 @@ async def test_a_formal_report_citing_a_count_of_missing_phones_succeeds(monkeyp
# --------------------------------------------------------------------------
# 正文里直接写着的遮罩值
# 正文里写着的值不按值认
# --------------------------------------------------------------------------
@@ -198,77 +191,49 @@ TOP_ROWS = [[NAMES[1], PHONES[1], 380.0], [NAMES[0], PHONES[0], 220.0]]
@pytest.mark.parametrize("text", [
f"消费最高的客户电话是 {PHONES[1]}。",
# 换了写法的号码也认
"消费最高的客户电话是 199-0000-0702。",
f"消费最高的客户电话是 +86 {PHONES[1][:3]} {PHONES[1][3:7]} {PHONES[1][7:]}。",
])
async def test_a_masked_value_copied_into_the_text_is_flagged(engine_up, mall, text):
"""写作者把号码照抄成正文(没写成引用):记一条 masked_cell(按值认出来的),不带值;交回写作者的那句说清在哪一句、
值不能以任何形式写进报告,句子里的值换成「已遮罩」。"""
async def test_a_value_copied_into_the_text_is_not_a_masked_cell(engine_up, mall, text):
"""写作者把号码照抄成正文(没写成引用):自查不按值认,不记 masked_cell;裸数字由数字核对记 uncited_number。"""
from app.engine.nodes.report import _MaskedCells
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
doc = await _MaskedCells(catalog)(compose_doc(text, catalog, node_id="write", run_id="r1"))
[v] = masked_of(doc["violations"])
assert v["by_value"] is True and v["segment"] and v["unit"] and "ref" not in v, v
assert "设置了遮罩的列中的值" in v["message"] and "不能以任何形式写进报告" in v["for_model"]
assert "已遮罩" in v["context"] and leaks(v) == [] and leaks(describe_violations([v])) == []
assert masked_of(doc["violations"]) == [] and any(v["code"] == "uncited_number" for v in doc["violations"])
async def test_a_text_masked_column_copied_as_text_is_flagged(engine_up, mall):
"""文字类的遮罩列(姓名):数字核对管不到它,按值认才拦得住。"""
async def test_a_name_copied_as_text_is_not_flagged(engine_up, mall):
"""文字类的遮罩列(姓名)照抄成正文:同样不按值认(两个字的姓名会撞上常用词)。"""
from app.engine.nodes.report import _MaskedCells
await set_source(mall, {"mask_columns": ["name"]})
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
doc = await _MaskedCells(catalog)(compose_doc(f"消费最高的是{NAMES[1]},花了 [[v:Q1.r0.spend]]。", catalog,
node_id="write", run_id="r1"))
[v] = masked_of(doc["violations"])
assert v["by_value"] is True and leaks(v) == [], v
assert masked_of(doc["violations"]) == [], doc["violations"]
async def test_a_cited_masked_cell_is_not_counted_twice(engine_up, mall):
"""[[v:Q1.r0.tel]] 已经按引用记过:同一处不再按值另记一条。"""
async def test_a_cited_masked_cell_is_flagged_once(engine_up, mall):
"""[[v:Q1.r0.tel]]:按引用记一条,没有按值另记的那一类(by_value)。"""
from app.engine.nodes.report import _MaskedCells
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
doc = await _MaskedCells(catalog)(compose_doc("电话 [[v:Q1.r0.tel]]。", catalog, node_id="write", run_id="r1"))
[v] = masked_of(doc["violations"])
assert v["ref"] == "v:Q1.r0.tel" and not v.get("by_value"), v
assert v["ref"] == "v:Q1.r0.tel" and "by_value" not in v, v
async def test_values_only_conservatively_masked_or_given_as_run_input_are_not_flagged(engine_up, mall):
"""不算违规的:SQL 认不准、整份算进来的列(姓名没设遮罩,只是面板多遮);运行输入里写着的号码(发起运行的人自己填的);
数据源没设遮罩。"""
from app.engine.nodes.report import _MaskedCells
unsure = ("SELECT name, amount FROM (SELECT c.name, UPPER(c.phone), o.amount FROM customers c "
"JOIN orders o ON o.customer_id = c.id) t ORDER BY amount DESC")
catalog = await masked_catalog(unsure, ["name", "amount"], [[NAMES[1], 300.0]])
doc = await _MaskedCells(catalog)(compose_doc(f"金额最大的一笔来自{NAMES[1]}。", catalog, node_id="write", run_id="r1"))
assert masked_of(doc["violations"]) == []
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
asked = _MaskedCells(catalog, protected=[json.dumps({"phone": PHONES[1]})])
doc = await asked(compose_doc(f"号码 {PHONES[1]} 的客户消费最高。", catalog, node_id="write", run_id="r1"))
assert masked_of(doc["violations"]) == []
await set_source(mall, {})
doc = await _MaskedCells(catalog)(compose_doc(f"电话 {PHONES[1]}。", catalog, node_id="write", run_id="r1"))
assert masked_of(doc["violations"]) == []
async def test_snapshots_of_unmasked_sources_are_not_read_for_values(engine_up, mall, monkeypatch):
"""按值认要读快照:数据源现在没设遮罩的一份都不读,没引用到格子的也不为引用去读。"""
async def test_no_snapshot_is_read_without_a_cited_cell(engine_up, mall, monkeypatch):
"""正文没引用格子:数据源设了遮罩也一份快照都不读(以前按值认要把设了遮罩的快照都读一遍)。"""
from app.core import artifact_store as store
from app.engine.nodes.report import _MaskedCells
await set_source(mall, {})
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
read: list[str] = []
real = store.load
monkeypatch.setattr(store, "load", lambda a: read.append(a) or real(a))
await _MaskedCells(catalog)(compose_doc("一共三位客户。", catalog, node_id="write", run_id="r1"))
await _MaskedCells(catalog)(compose_doc(f"一共三位客户,电话 {PHONES[0]}。", catalog, node_id="write", run_id="r1"))
assert read == [], read
@@ -287,24 +252,12 @@ def card_graph(**report) -> dict:
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}]))
async def test_a_metric_whose_value_is_a_masked_value_is_flagged(monkeypatch, mall):
"""[[m:phone_raw]] 的值就是号码:按值认出来,点名那处引用;取客户编号的 [[m:id3]] 照常。标注照常产出时,时间线那一行
说清有几处是正文里写着的值,缺口、日志都不写值。"""
row = await finish(monkeypatch, card_graph(on_violation="flag"), "第三位客户编号 [[m:id3]],首位客户电话 [[m:phone_raw]]。")
async def test_a_metric_reference_is_not_checked_by_value(monkeypatch, mall):
"""自查只查 [[v:]] 引用:[[m:phone_raw]] 的值就是号码,自查不按值认(指标的值在证据面板、推演里另有判断,写「已遮罩」,
见 test_masked_metric_values)。违规即失败的运行照常成功,时间线上没有「引用了遮罩列」那一行。"""
row = await finish(monkeypatch, card_graph(on_violation="fail"), "第三位客户编号 [[m:id3]],首位客户电话 [[m:phone_raw]]。")
assert row.status == "succeeded", row.error
rows = await events(row.id)
checked = next(e.data for e in rows if e.type == "report.checked")
[v] = masked_of(checked["violations"])
assert v["ref"] == "m:phone_raw" and v["by_value"] is True and "[[m:phone_raw]]" in v["message"], v
[log] = [e.data for e in rows if e.type == "log" and e.data.get("code") == "report_masked_cell"]
assert log["count"] == 1 and log["written"] == 1 and log["refs"] == ["m:phone_raw"], log
finished = next(e.data for e in rows if e.type == "node.finished" and e.node_id == "write")
[gap] = artifact_store.load(finished["artifact"])["issuance_gaps"]
assert "设置了遮罩的列中的值" in gap and leaks([checked["violations"], log, gap]) == [], (gap, log)
async def test_a_formal_report_copying_a_phone_fails_with_advice_about_the_instructions(monkeypatch, mall):
row = await finish(monkeypatch, card_graph(on_violation="fail", numbers="off"),
f"首位客户电话 {PHONES[0]},第三位客户编号 [[m:id3]]。")
assert row.status == "failed" and "设置了遮罩的列中的值" in row.error, row.error
assert "指令中去掉" in row.error and "指令遵循能力" not in row.error and leaks(row.error) == [], row.error
assert masked_of(checked["violations"]) == [], checked["violations"]
assert not [e for e in rows if e.type == "log" and e.data.get("code") == "report_masked_cell"]
+5 -9
View File
@@ -248,18 +248,14 @@ export const ABSENCE_CLAIM_TEXT = {
}
/**
* 报告写进了取自遮罩列的值(报告自查的 masked_cell):直接引用了那一格([[v:Q2.r0.tel]],c.phone AS tel 这种改了名字的
* 也算),或者按值认出正文里直接写着、经指标带出来的(违规带 by_value),重写后仍在、「违规处置」为标注时照常产出:
* 时间线上那一行(日志 report_masked_cell,written 是其中按值认出来的几处)和证据面板违规清单里的叫法。
* 报告直接引用了取自遮罩列的格子(报告自查的 masked_cell:[[v:Q2.r0.tel]],c.phone AS tel 这种改了名字的也算,别的表里
* 同名的列不算),重写后仍在、「违规处置」为标注时照常产出:时间线上那一行(日志 report_masked_cell)和证据面板违规清单
* 里的叫法。自查只查引用,不按值在正文里找遮罩值。
* 只说记为缺口,不说出具:探索运行、没设出具契约的成果不出具,报告接在侧枝上时缺口也进不了出具,降不降档由出具那一行说
*/
export const MASKED_CELL_TEXT = {
label: '写进了遮罩列的值',
title: (n?: number, written = 0) => {
if (!written) return `报告引用了${n ? ` ${formatNumber(n)} 处` : ''}取自遮罩列的格子,已标注`
if (n && n > written) return `报告引用、写进了 ${formatNumber(n)} 处取自遮罩列的值,已标注`
return `报告写进了 ${formatNumber(written)} 处设置了遮罩的列中的值,已标注`
},
label: '引用了遮罩列',
title: (n?: number) => `报告引用了${n ? ` ${formatNumber(n)} 处` : ''}取自遮罩列的格子,已标注`,
next: '数据源设置了遮罩的列不应写进报告,这几处已记为缺口。请在报告撰写节点的指令中去掉对这些列的引用后重新运行',
}
+3 -4
View File
@@ -874,13 +874,12 @@ function explainLog(code: string | undefined, message: string, d: Record<string,
}
}
case 'report_masked_cell': {
// 报告写进了取自遮罩列的值,重写后仍在、照常产出(report.py):读结构化字段 count、written(其中按值认出、正文里
// 直接写着的几处)、refs,副标题只点名引用,不写那几格的值(后端也没给)
// 报告引用了取自遮罩列的格子,重写后仍在、照常产出(report.py):读结构化字段 count、refs,副标题只点名引用,
// 不写那几格的值(后端也没给)
const n = typeof d.count === 'number' && d.count > 0 ? d.count : undefined
const written = typeof d.written === 'number' && d.written > 0 ? d.written : 0
const refs = strs(d.refs)
return {
title: MASKED_CELL_TEXT.title(n, written),
title: MASKED_CELL_TEXT.title(n),
sub: refs.length ? refs.map((r) => `[[${r}]]`).join('、') : undefined,
next: MASKED_CELL_TEXT.next,
}
+10 -17
View File
@@ -1377,14 +1377,14 @@ await section('报告核对 report.checked、口径卡的台账(可点击证
&& !!rcAbsent?.detail?.includes('缺失断言') && !!rcAbsent?.detail?.includes('依据不足'), `${rcAbsent?.status} ${rcAbsent?.detail}`)
})
await section('报告写进了取自遮罩列的值(report_masked_cell):标题说几处、已标注,副标题点名引用,不写那几格的值', async () => {
// 契约:data = {level: warn, code: report_masked_cell, message, count, refs, written, columns},重写后仍在、「违规处置」
// 为标注时才发。written 是其中按值认出来的几处(正文里直接写着的、经指标带出来的,违规带 by_value)
await section('报告引用了取自遮罩列的格子(report_masked_cell):标题说几处、已标注,副标题点名引用,不写那几格的值', async () => {
// 契约:data = {level: warn, code: report_masked_cell, message, count, refs, columns},重写后仍在、「违规处置」
// 为标注时才发。自查只查 [[v:]] 引用,不按值认正文里写着的值(复核第 3 轮去掉了按值认,日志也不再带 written)
const E = (seq, type, node_id, data = {}) => ({ seq, type, node_id, ts: 1790000000 + seq, data })
const reportPy = readFileSync(`${root}backend/app/engine/nodes/report.py`, 'utf8')
check('后端发写进了遮罩列的值的警告,带着 count、refs、written、columns', reportPy.includes('code="report_masked_cell"')
&& reportPy.includes('count=len(hits), refs=refs') && reportPy.includes('written=sum(')
&& reportPy.includes('columns=list(dict.fromkeys('),
check('后端发引用了遮罩列的警告,带着 count、refs、columns,不再带按值认的 written', reportPy.includes('code="report_masked_cell"')
&& reportPy.includes('count=len(hits), refs=refs') && reportPy.includes('columns=list(dict.fromkeys(')
&& !reportPy.includes('written=sum('),
'改了 report.py 的字段就同步改 decode.ts 的 explainLog 和这里的夹具')
const MASKED = '报告引用了 2 处取自遮罩列的格子(「v:Q2.r0.tel」、「v:Q2.r1.tel」),已要求模型重写 1 次仍未改掉:'
+ '数据源设置了遮罩的列不应写进报告。报告照常产出,这几处已标注,已记为缺口。请在报告撰写节点的指令中去掉对这些列的引用'
@@ -1413,17 +1413,10 @@ await section('报告写进了取自遮罩列的值(report_masked_cell):
.find((s) => s.code === 'report_masked_cell')
check('……缺字段时标题照样说人话,不出 undefined', bare?.title === '报告引用了取自遮罩列的格子,已标注'
&& bare?.sub === undefined, `${bare?.title} / ${bare?.sub}`)
// 按值认出来的:正文里直接写着号码、姓名,或者经指标带了出来([[m:phone_raw]])。标题不说「引用了格子」
const titleOf = (data) => flatten(mod.decodeRun([E(1, 'log', 'w', { level: 'warn', code: 'report_masked_cell',
message: '报告写进了遮罩列的值', ...data })])).find((s) => s.code === 'report_masked_cell')
const onlyWritten = titleOf({ count: 1, written: 1, refs: [] })
check('……全是正文里直接写着的:标题说写进了几处遮罩列中的值,没有引用可点名就不写副标题',
onlyWritten?.title === '报告写进了 1 处设置了遮罩的列中的值,已标注' && onlyWritten?.sub === undefined,
`${onlyWritten?.title} / ${onlyWritten?.sub}`)
const viaMetric = titleOf({ count: 1, written: 1, refs: ['m:phone_raw'] })
check('……经指标带出来的:副标题点名那处引用', viaMetric?.sub === '[[m:phone_raw]]', viaMetric?.sub)
const both = titleOf({ count: 3, written: 1, refs: ['v:Q2.r0.tel', 'v:Q2.r1.tel'] })
check('……两样都有:标题说引用、写进了几处', both?.title === '报告引用、写进了 3 处取自遮罩列的值,已标注', both?.title)
// 第 2 轮的日志带过 written(按值认出的几处):现在的后端不再发,万一读到也只按引用说,不改标题
const legacy = flatten(mod.decodeRun([E(1, 'log', 'w', { level: 'warn', code: 'report_masked_cell',
message: '报告引用了遮罩列', count: 1, written: 1, refs: ['v:Q2.r0.tel'] })])).find((s) => s.code === 'report_masked_cell')
check('……带着 written 的日志照引用说', legacy?.title === '报告引用了 1 处取自遮罩列的格子,已标注', legacy?.title)
})
await section('整句面板挂的依据:按证据图的标签写,不读文档目录里带值的标签(citeLabel)', async () => {