mirror of
https://github.com/ThearchyHelios/AgentLab.git
synced 2026-11-17 23:21:58 +00:00
M2 复核第 3 轮:报告自查只查引用、别的表里同名的列不算;裁判摘录退回只按数据源设的遮罩列;收遮罩值只收确定取自遮罩列的值;遮罩分析和收到的值按快照工件记在进程里
第 2 轮的按值自查和面板那套宁可多遮的认法带进了误判,正式运行因此失败,代价远大于收益,这一轮回到规格原意: - 报告自查去掉按值在正文里找遮罩值(engine/evidence.py 的 masked_values、report.py 的收值):两个字的姓名会撞上「销售 高峰」这类常用词,同名列的值会撞上门店名;照抄成正文的号码本来就会被「数字没有出处」拦下。时间线日志 report_masked_cell 不再带 written,decode.ts、terms.ts、证据审计的 CSV 回到「引用了遮罩列」一种说法,check-decode 同步 - 自查认引用时从 _mask_sets 的 sure 起,再按 SQL 血缘(sqlglot.lineage,按快照里冻结的表结构)核一遍值取自哪张表的哪一列 (api/evidence.py 的 _cited_masks / _owned_columns):遮罩列不带表名,同名的列在几张表里都有、其中有的表另有别的遮罩列时 (customers 有 name 和 phone,stores 只有 name),只认那几张表,门店名不算客户姓名;值根本不取自遮罩列的(region AS phone)也不算。认不出出处的照算,没探查过表结构的照列名认。违规说明点名它取自的那一列遮罩列 - 交给裁判的摘录退回 M2 之前的做法,只按数据源设的列名遮(删掉 _judge_masks):并进面板那套宁可多遮的列后,整份遮的快照、 只数个数的列从摘录里消失,被引用的金额、人数裁判看不到。按需裁判和节点里当场裁判都改回 - 证据面板、推演收「这次运行查到过的遮罩值」只收 sure 的列(_snapshot_secrets,同 masking 收值的口径):SQL 认不准、整份 遮的快照只遮显示,里面的区域名、金额不再收成遮罩值 - 按快照工件 id 在进程里记下遮罩分析和收到的值(_SnapshotMemo,键里带数据源现在设的遮罩、在不在、类型,设置改了不拿旧的; 只记读得出来的快照,存取加锁,分析结果冻结、交出去的值另抄一份):之后点开片段、推演都不再把全部快照读盘、复验、分析, 只读这一步要显示的那一份。30 份 500 行的快照,第一次约 0.7 秒,之后不到 1 毫秒 测试:新增 test_report_masked_owners.py(高峰、门店名写成文字照常出具,照抄号码由数字核对拦下,引用门店名不算、引用客户名算, 同一条查询里两张表的同名列、只设 name、* 展开、WITH、UNION、没有表结构、表值函数)、test_evidence_mask_memo.py(整份遮 的快照不收值、只读一次只分析一次、设置改了重新收、推演共用、再点开片段只读本步快照);裁判摘录改测被引用的金额和计数 都在摘录里;按值自查的用例改成不按值认的回归
This commit is contained in:
+332
-69
@@ -20,7 +20,8 @@ artifacts 表可以事后插行,run.output 是可以改写的一列,封存
|
||||
查询步骤只给被引用的行和前后各 2 行,完整快照仍走 /api/artifacts/{id}。数据源
|
||||
options.mask_columns 里的列换成「已遮罩」,SQL 里给它们起了别名、套了函数得来的结果列也换
|
||||
(_mask_sets,和可验证报告同一个函数,合并查询另按合并 SQL 认);用到这些列的指标,指标步骤的值、
|
||||
证据图的标签也不给(_metric_masks,和推演同一个判断);SQL、合并警告里写着的遮罩值展示时换掉
|
||||
证据图的标签也不给(_metric_masks,和推演同一个判断);SQL、合并警告里写着的遮罩值(这次运行里确定取自遮罩列的值)
|
||||
展示时换掉。一份快照的遮罩分析、收到的值按快照工件记在进程里(_MEMO),之后的请求不再读盘、不再分析
|
||||
——在有身份体系之前,这只减少暴露,不是安全边界:知道工件 id 的人照样能取到整份快照。
|
||||
|
||||
期 4 的核查状态同样只读封存范围内的东西:指标步骤带双盲复算的结果(_dual_check_of,读双盲复算节点的产出和台账
|
||||
@@ -41,8 +42,9 @@ import io
|
||||
import logging
|
||||
import re
|
||||
import sqlite3
|
||||
import threading
|
||||
from collections import OrderedDict
|
||||
from collections.abc import Callable, Iterable, Iterator
|
||||
from collections.abc import Callable, Iterator
|
||||
from dataclasses import asdict, dataclass, field, replace
|
||||
from pathlib import Path
|
||||
from typing import Any, get_args
|
||||
@@ -1368,7 +1370,8 @@ class _Masks:
|
||||
"""一次请求里查过的数据源遮罩:数据源名 → (要遮的列(小写), 数据源现在还在不在)。顺带记下数据源的类型(kind),
|
||||
按 SQL 认改了名字的遮罩列时照它的方言解析(_mask_sets);数据源不在了的按通用读法。
|
||||
|
||||
给了 sealed(证据面板的片段、证据图)时另能交回这次运行查到过的遮罩值(run_secrets),一次请求只收一次。"""
|
||||
给了 sealed(证据面板的片段、证据图)时另能交回这次运行查到过的遮罩值(run_secrets),一次请求只收一次。一份快照的
|
||||
遮罩分析、收到的值另按快照工件记在进程里(_MEMO),之后的请求不再读、不再分析。"""
|
||||
|
||||
def __init__(self, sealed: _Sealed | None = None) -> None:
|
||||
self._memo: dict[str, tuple[set[str], bool]] = {}
|
||||
@@ -1391,7 +1394,7 @@ class _Masks:
|
||||
读不出来的为 None。"""
|
||||
key = (artifact, str(source or ""))
|
||||
if artifact is None or key not in self._sets:
|
||||
found = await _mask_sets(snap, source, self)
|
||||
found = await _mask_sets(snap, source, self, artifact=artifact)
|
||||
if artifact is None:
|
||||
return found
|
||||
self._sets[key] = found
|
||||
@@ -1415,27 +1418,49 @@ class _Masks:
|
||||
await self.of(name)
|
||||
return self._kinds.get(name)
|
||||
|
||||
async def memo_key(self, artifact: str | None, source: Any) -> tuple[Any, ...] | None:
|
||||
"""一份快照在进程内记录(_MEMO)里的键:(工件, 数据源名, 数据源现在设的遮罩, 数据源在不在, 类型)。工件按内容
|
||||
寻址、不会变,一份快照的遮罩只取决于它的内容和后面几样;遮罩设置改了键就跟着变,不会拿旧的结果。合并查询、
|
||||
没有工件或数据源名的没有键(不记)。"""
|
||||
name = str(source or "")
|
||||
if not artifact or not name or name == MERGE_SOURCE:
|
||||
return None
|
||||
current, found = await self.of(name)
|
||||
return str(artifact), name, frozenset(current), found, self._kinds.get(name)
|
||||
|
||||
async def value_keys(self, queries: dict[str, Any]) -> list[tuple[str, Any, tuple[Any, ...]]]:
|
||||
"""封存范围内要收遮罩值的那几份查询快照:(工件, 数据源名, 进程内记录的键)。数据源现在设了遮罩、或者已经不在了的
|
||||
才收(masked_source);合并查询不收:它的值都来自它的输入,输入另有快照。"""
|
||||
out: list[tuple[str, Any, tuple[Any, ...]]] = []
|
||||
for artifact, info in queries.items():
|
||||
source = (info or {}).get("source")
|
||||
if await self.masked_source(source) and (key := await self.memo_key(artifact, source)) is not None:
|
||||
out.append((str(artifact), source, key))
|
||||
return out
|
||||
|
||||
async def run_secrets(self) -> Any:
|
||||
"""这次运行查到过的遮罩值(masking._Secrets):封存范围内、数据源现在设了遮罩(或者已经不在了)的每份查询快照里,
|
||||
取自遮罩列的那几列(_mask_sets:列名对上的、SQL 里改了名字的都算,只装着分档标签的格子不收)的值。认不出出处、
|
||||
值却就是遮罩值的地方据此也不给:指标和输入的值(数据整形算出来再赋给变量的一格)、SQL 里的字面量(按号码筛的
|
||||
WHERE phone IN (…))、合并警告里举的例子——和可验证报告认「这次运行里查到过的遮罩值」(bundle/build.py 的
|
||||
_Store.seen)同一个口径。合并查询的快照不收:它的值都来自它的输入,输入另有快照。
|
||||
确定取自遮罩列的那几列的值(_snapshot_secrets:_mask_sets 的 sure,SQL 认不准、整份遮的快照只遮显示,不收)。
|
||||
认不出出处、值却就是遮罩值的地方据此也不给:指标和输入的值(数据整形算出来再赋给变量的一格)、SQL 里的字面量
|
||||
(按号码筛的 WHERE phone IN (…))、合并警告里举的例子。
|
||||
|
||||
要把这些快照读一遍,只在用到时收、一次请求只收一次;没给 sealed、运行里没有一份这样的快照时为 None。"""
|
||||
只在用到时收、一次请求只收一次;每份快照收到的值按快照工件记在进程里(_MEMO),之后的请求(推演也一样)不再读盘、
|
||||
复验哈希、做 SQL 分析,只读还没记下的那几份。没给 sealed、一个值都没收到时为 None。"""
|
||||
if self._collected:
|
||||
return self._secrets
|
||||
self._collected = True
|
||||
if self._sealed is None:
|
||||
return None
|
||||
found: list[tuple[dict[str, Any], _MaskSets]] = []
|
||||
for artifact, info in self._sealed.queries.items():
|
||||
if not await self.masked_source(info.get("source")):
|
||||
continue
|
||||
snap, ok = self.fetch(artifact)
|
||||
if ok is True and isinstance(snap, dict) and snap.get("source") != MERGE_SOURCE:
|
||||
found.append((snap, await self.sets(artifact, info.get("source"), snap)))
|
||||
self._secrets = _collect_secrets(found)
|
||||
parts: list[tuple[tuple[Any, ...], Any]] = []
|
||||
for artifact, source, key in await self.value_keys(self._sealed.queries):
|
||||
known, got = _MEMO.values(key)
|
||||
if not known:
|
||||
snap, ok = self.fetch(artifact)
|
||||
if ok is not True or not isinstance(snap, dict) or snap.get("source") == MERGE_SOURCE:
|
||||
continue
|
||||
got = _MEMO.keep_values(key, _snapshot_secrets(snap, await self.sets(artifact, source, snap)))
|
||||
parts.append((key, got))
|
||||
self._secrets = _MEMO.pooled(parts)
|
||||
return self._secrets
|
||||
|
||||
async def masked_source(self, source: Any) -> bool:
|
||||
@@ -1447,18 +1472,139 @@ class _Masks:
|
||||
return bool(cols) or not found
|
||||
|
||||
|
||||
def _collect_secrets(found: Iterable[tuple[dict[str, Any], _MaskSets]]) -> Any:
|
||||
"""几份查询快照里取自遮罩列的值(masking._Secrets):每份按它的遮罩(_MaskSets)收要遮的那几列,只装着分档标签的
|
||||
格子不收。一个值都没收到的为 None。证据面板(_Masks.run_secrets)、推演(api/whatif.py 的 _Hidden.secrets)都按它收。"""
|
||||
def _snapshot_secrets(snap: dict[str, Any], sets: _MaskSets) -> Any:
|
||||
"""一份查询快照里确定取自遮罩列的值(masking._Secrets,一个都没有为 None):只收 sets.sure 那几列(列名就是遮罩列的、
|
||||
SQL 里认得出取自遮罩列的),只装着分档标签的格子不收,和核查 agent 在工具边界收值同一个口径(masking:整份遮的快照只收
|
||||
确定的那几列)。SQL 认不准、整份遮的快照只遮显示:以前连它的区域名、金额一起收,之后每条 SQL 里的「'华东'」都显示成
|
||||
「已遮罩」。证据面板(_Masks.run_secrets)、推演(api/whatif.py 的 _Hidden.secrets)都按它收。"""
|
||||
from app.engine.masking import _Secrets
|
||||
|
||||
secrets = _Secrets()
|
||||
for snap, sets in found:
|
||||
secrets.add(snap, sets.hidden, labels=sets.labels, base=sets.base)
|
||||
if not secrets.strong:
|
||||
secrets.add(snap, set(sets.sure), labels=dict(sets.labels), base=set(sets.base))
|
||||
return secrets if secrets.weak else None
|
||||
|
||||
|
||||
#: 进程内记录里「没有这一项」(记下的值本身可以是 None:这份快照一个遮罩值都没有)
|
||||
_ABSENT = object()
|
||||
|
||||
|
||||
class _SnapshotMemo:
|
||||
"""按查询快照工件记在进程里的遮罩分析(_MaskSets)和收到的遮罩值(masking._Secrets),跨请求共用。证据面板每点开一个
|
||||
片段、推演每算一次,都要知道这次运行查到过的遮罩值(_Masks.run_secrets、api/whatif.py 的 _Hidden.secrets);以前缓存
|
||||
只在一次请求里有效,每次都把设了遮罩的数据源上的快照读盘、复验哈希、逐份做 SQL 分析(一份几十毫秒)。
|
||||
|
||||
键见 _Masks.memo_key。只记读得出来的快照(读不出来、与哈希不一致的下次照旧去读:文件可能修好了);合并查询不记(它的
|
||||
遮罩还取决于各个输入,输入另按自己的键记)。只记分析结果和收到的值,不记快照本身(几 MB 的行):分析结果按份数封顶,
|
||||
收到的值按条数合计封顶,并起来的值按运行封顶,都是最近用到的留着。
|
||||
|
||||
推演在工作线程里取、证据面板在事件循环里取,所以存取都加锁;记下的东西调用方只读不改:分析结果存成冻结的集合,
|
||||
并起来的值每次交出去一份新的(_shared:集合另抄一份、共用建好的索引),各自的正则缓存互不相扰。"""
|
||||
|
||||
SETS_MAX = 1024
|
||||
VALUES_MAX = 200_000
|
||||
POOLS_MAX = 8
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._lock = threading.Lock()
|
||||
self._sets: OrderedDict[tuple[Any, ...], _MaskSets] = OrderedDict()
|
||||
self._values: OrderedDict[tuple[Any, ...], Any] = OrderedDict()
|
||||
self._size = 0
|
||||
self._pools: OrderedDict[tuple[Any, ...], Any] = OrderedDict()
|
||||
|
||||
def clear(self) -> None:
|
||||
with self._lock:
|
||||
self._sets.clear()
|
||||
self._values.clear()
|
||||
self._pools.clear()
|
||||
self._size = 0
|
||||
|
||||
def sets(self, key: tuple[Any, ...]) -> _MaskSets | None:
|
||||
with self._lock:
|
||||
found = self._sets.get(key)
|
||||
if found is not None:
|
||||
self._sets.move_to_end(key)
|
||||
return found
|
||||
|
||||
def keep_sets(self, key: tuple[Any, ...], sets: _MaskSets) -> _MaskSets:
|
||||
frozen = _MaskSets(frozenset(sets.hidden), frozenset(sets.sure), frozenset(sets.base), # type: ignore[arg-type]
|
||||
{k: frozenset(v) for k, v in sets.labels.items()}, list(sets.gone)) # type: ignore[misc]
|
||||
with self._lock:
|
||||
self._sets[key] = frozen
|
||||
self._sets.move_to_end(key)
|
||||
while len(self._sets) > self.SETS_MAX:
|
||||
self._sets.popitem(last=False)
|
||||
return frozen
|
||||
|
||||
def values(self, key: tuple[Any, ...]) -> tuple[bool, Any]:
|
||||
"""(记没记过, 这份快照收到的值)。"""
|
||||
with self._lock:
|
||||
found = self._values.get(key, _ABSENT)
|
||||
if found is _ABSENT:
|
||||
return False, None
|
||||
self._values.move_to_end(key)
|
||||
return True, found
|
||||
|
||||
def keep_values(self, key: tuple[Any, ...], secrets: Any) -> Any:
|
||||
with self._lock:
|
||||
old = self._values.pop(key, _ABSENT)
|
||||
if old is not _ABSENT:
|
||||
self._size -= _value_count(old)
|
||||
self._values[key] = secrets
|
||||
self._size += _value_count(secrets)
|
||||
while self._size > self.VALUES_MAX and len(self._values) > 1:
|
||||
_, dropped = self._values.popitem(last=False)
|
||||
self._size -= _value_count(dropped)
|
||||
return secrets
|
||||
|
||||
def pooled(self, parts: list[tuple[tuple[Any, ...], Any]]) -> Any:
|
||||
"""几份快照收到的值并成一份、建好索引(上万个值建一次要几十毫秒),按这几份的键记下;交出去的是 _shared 的一份。
|
||||
一个值都没有为 None。"""
|
||||
if not parts:
|
||||
return None
|
||||
key = tuple(k for k, _ in parts)
|
||||
with self._lock:
|
||||
found = self._pools.get(key, _ABSENT)
|
||||
if found is not _ABSENT:
|
||||
self._pools.move_to_end(key)
|
||||
if found is _ABSENT:
|
||||
found = _pool([one for _, one in parts])
|
||||
with self._lock:
|
||||
self._pools[key] = found
|
||||
while len(self._pools) > self.POOLS_MAX:
|
||||
self._pools.popitem(last=False)
|
||||
return _shared(found)
|
||||
|
||||
|
||||
def _value_count(secrets: Any) -> int:
|
||||
return len(secrets.weak) if secrets is not None else 0
|
||||
|
||||
|
||||
def _pool(parts: list[Any]) -> Any:
|
||||
"""几份 masking._Secrets 并成一份(合起来最多 SECRETS_MAX 种,同 _Secrets.merge),建好索引;一个值都没有为 None。"""
|
||||
from app.engine.masking import _Secrets
|
||||
|
||||
out = _Secrets()
|
||||
for one in parts:
|
||||
out.merge(one)
|
||||
if not out.strong:
|
||||
return None
|
||||
secrets.prepare()
|
||||
return secrets
|
||||
out.prepare()
|
||||
return out
|
||||
|
||||
|
||||
def _shared(secrets: Any) -> Any:
|
||||
"""记在进程里的一份遮罩值交给一个请求用:值的集合另抄一份(调用方往里并值、免遮不碍别人),建好的索引共用(只读),
|
||||
正则缓存各起一份(_Secrets.pattern 会改它)。"""
|
||||
if secrets is None:
|
||||
return None
|
||||
from app.engine.masking import _Secrets
|
||||
|
||||
return _Secrets(strong=set(secrets.strong), weak=set(secrets.weak), shown=set(secrets.shown),
|
||||
protected=secrets.protected, _index=secrets._index)
|
||||
|
||||
|
||||
#: 证据面板、推演、报告自查共用的进程内记录(见 _SnapshotMemo)
|
||||
_MEMO = _SnapshotMemo()
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -1466,11 +1612,14 @@ class _MaskSets:
|
||||
"""一份查询快照的遮罩(_mask_sets):
|
||||
|
||||
- hidden:要遮的列(小写)。证据面板、推演、勾行按它遮,宁可多遮:SQL 认不准时整份的列都算
|
||||
- sure:其中能确定取自遮罩列的(报告自查按它判违规):列名就是遮罩列的,SQL 里认得出取自遮罩列的(别名、函数、
|
||||
子查询、WITH)。SQL 认不准、整份算进来的那几列不在里面,只装着分档标签的列(CASE 写死的字)也不在
|
||||
- base:遮罩列本身(查询当时记下的加数据源现在设的,小写),违规说明里点名
|
||||
- sure:其中能确定取自遮罩列的:列名就是遮罩列的,SQL 里认得出取自遮罩列的(别名、函数、子查询、WITH)。SQL 认不准、
|
||||
整份算进来的那几列不在里面,只装着分档标签的列(CASE 写死的字)也不在。收「这次运行查到过的遮罩值」只收这几列
|
||||
(_snapshot_secrets);报告自查从这几列起,再按血缘去掉别的表里同名的列(_cited_masks)
|
||||
- base:遮罩列本身(查询当时记下的加数据源现在设的,小写)
|
||||
- labels:只装着分档标签的列 → 标签的写法(masking._snapshot_labels),收遮罩值时不收标签
|
||||
- gone:已经不在的数据源"""
|
||||
- gone:已经不在的数据源
|
||||
|
||||
记进进程内记录(_MEMO)的是冻结的集合:几个请求共用一份,调用方只读不改。"""
|
||||
|
||||
hidden: set[str]
|
||||
sure: set[str]
|
||||
@@ -1572,7 +1721,8 @@ def _counted(snap: dict[str, Any], kind: str | None) -> dict[str, Any]:
|
||||
return snap
|
||||
|
||||
|
||||
async def _mask_sets(snap: dict[str, Any] | None, source: Any, masks: _Masks, *, depth: int = 0) -> _MaskSets:
|
||||
async def _mask_sets(snap: dict[str, Any] | None, source: Any, masks: _Masks, *, depth: int = 0,
|
||||
artifact: str | None = None) -> _MaskSets:
|
||||
"""一份查询快照的遮罩(_MaskSets):证据面板的查询步骤、指标的输入、推演、报告自查(engine/nodes/report.py)都按这一个
|
||||
函数认。
|
||||
|
||||
@@ -1580,7 +1730,9 @@ async def _mask_sets(snap: dict[str, Any] | None, source: Any, masks: _Masks, *,
|
||||
遮着的列亮出来;数据源不在了的,它的名字进 gone
|
||||
- 列名对上的之外,按 SQL 认出改了名字的遮罩列:要遮的按 _derived(和可验证报告同一个函数),确定的按 _sure
|
||||
- 合并查询不是数据源:按它合并的那几个数据源、逐格来历和合并 SQL 认(_merge_mask_sets)
|
||||
- snap 为 None(快照读不出来):只能按 source 现在设的遮罩算,没有 SQL 可认"""
|
||||
- snap 为 None(快照读不出来):只能按 source 现在设的遮罩算,没有 SQL 可认
|
||||
|
||||
artifact 是 snap 的工件 id:给了的,分析结果按快照记在进程里(_MEMO,键见 _Masks.memo_key),之后的请求不再分析。"""
|
||||
if snap is not None and snap.get("source") == MERGE_SOURCE:
|
||||
return await _merge_mask_sets(snap, masks, depth=depth)
|
||||
name = source or (snap or {}).get("source")
|
||||
@@ -1589,16 +1741,154 @@ async def _mask_sets(snap: dict[str, Any] | None, source: Any, masks: _Masks, *,
|
||||
gone = [] if found or not name else [str(name)]
|
||||
if snap is None or not base:
|
||||
return _MaskSets(set(base), set(base), base, gone=gone)
|
||||
key = await masks.memo_key(artifact, name) if artifact else None
|
||||
if key is not None and (kept := _MEMO.sets(key)) is not None:
|
||||
return kept
|
||||
kind = await masks.kind(name)
|
||||
sure, labels = _sure(snap, base, kind)
|
||||
return _MaskSets(_derived(snap, base, kind) | sure, sure, base, labels, gone)
|
||||
out = _MaskSets(_derived(snap, base, kind) | sure, sure, base, labels, gone)
|
||||
return _MEMO.keep_sets(key, out) if key is not None else out
|
||||
|
||||
|
||||
async def _snapshot_masks(snap: dict[str, Any] | None, source: Any, masks: _Masks, *,
|
||||
depth: int = 0) -> tuple[set[str], list[str]]:
|
||||
"""一份查询快照要遮的列(小写)和已经不在的数据源(_mask_sets 的 hidden、gone)。"""
|
||||
sets = await _mask_sets(snap, source, masks, depth=depth)
|
||||
return sets.hidden, sets.gone
|
||||
async def _cited_masks(snap: dict[str, Any], source: Any, masks: _Masks,
|
||||
artifact: str | None = None) -> dict[str, list[str]]:
|
||||
"""报告自查(engine/nodes/report.py 的 _MaskedCells)认一份查询快照里引用了就记 masked_cell 的列:{结果列(小写):
|
||||
它取自的遮罩列},违规说明点名后者。
|
||||
|
||||
从 _mask_sets 的 sure 起(列名就是遮罩列的、SQL 里认得出取自遮罩列的;SQL 认不准、整份算进来的和只装着分档标签的
|
||||
不在里面),普通查询再按血缘核一遍值取自哪张表的哪一列,只认取自遮罩表里那一列的(_owned_columns:stores.name 和设了
|
||||
遮罩的 customers.name 同名不算)。合并查询的值来自它的各个输入、不是数据源里的表,照 sure 算。"""
|
||||
sets = await _mask_sets(snap, source, masks, artifact=artifact)
|
||||
if not sets.sure:
|
||||
return {}
|
||||
if snap.get("source") == MERGE_SOURCE:
|
||||
return {c: sorted(sets.base) for c in sorted(sets.sure)}
|
||||
return _owned_columns(snap, set(sets.sure), set(sets.base), await masks.kind(source or snap.get("source")))
|
||||
|
||||
|
||||
def _owned_columns(snap: dict[str, Any], sure: set[str], base: set[str], kind: str | None) -> dict[str, list[str]]:
|
||||
"""sure 里值确实取自遮罩表里那一列的结果列 → 它取自的遮罩列(按血缘里出现的先后)。报告自查按它判违规。
|
||||
|
||||
遮罩按列名设在整个数据源上(data/engine.py 的 masked_columns,不带表名):设了 name,stores.name 和 customers.name 按
|
||||
列名都算。证据面板、可验证报告、核查 agent 照这样遮(宁可多遮);自查多算一列就是判失败,所以按 SQL 的血缘
|
||||
(sqlglot.lineage,按查询当时冻结的表结构展开 *、认出不带表名的列出自哪张表)核一遍每一列的值取自哪张表的哪一列:
|
||||
|
||||
- 遮罩列在几张表里都有、其中有的表里另有别的遮罩列(customers 有 name 和 phone,stores 只有 name):只认那几张表
|
||||
(_mask_owners)。别的情形(只在一张表里有、哪张都没有别的遮罩列)照列名认
|
||||
- 血缘里有认不出出处的(找不到这一列、表值函数、不带表名又有歧义的列、* 展开不了):排除不了,照算
|
||||
- 认得出、可一处都不取自遮罩表里的遮罩列(门店名、SELECT region AS phone):不算
|
||||
- 快照里没有冻结的表结构(数据源没探查过):认不出同名的列还在哪几张表里,照列名认;值根本不取自遮罩列的照样不算
|
||||
|
||||
只排除、不新增:sure 里没有的列血缘认出来也不算。分析出了意外,照 sure 算。纯计算,在调用方的线程里算。"""
|
||||
plain = {c: sorted(base) for c in sorted(sure)}
|
||||
sql = snap.get("sql")
|
||||
if not sure or not isinstance(sql, str) or not sql.strip():
|
||||
return plain
|
||||
try:
|
||||
from app.data.sql_semantics import _parse, _read
|
||||
from app.engine.masking import _frozen_columns
|
||||
|
||||
read = _read(kind)
|
||||
root, _ = _parse(sql, read)
|
||||
if root is None:
|
||||
return plain
|
||||
frozen = _frozen_columns(snap["schema_artifact"]) if isinstance(snap.get("schema_artifact"), str) else None
|
||||
# 表结构里一张表按好几个名字记着(带不带库名),这里只要不带库名的那一份
|
||||
tables = {name: cols for name, cols in (frozen or {}).items() if "." not in name}
|
||||
owners = _mask_owners(tables, base)
|
||||
schema = {name: {c: "TEXT" for c in cols} for name, cols in tables.items()} or None
|
||||
nodes = _lineages(root, schema, read, [str(c) for c in snap.get("columns") or [] if str(c).lower() in sure])
|
||||
out: dict[str, list[str]] = {}
|
||||
for column in sorted(sure):
|
||||
found = _owned_masks(nodes.get(column), base, owners, set(tables))
|
||||
if found:
|
||||
out[column] = found
|
||||
return out
|
||||
except Exception: # noqa: BLE001 - 核对不了出自哪张表:照 sure 算
|
||||
logger.warning("报告自查:按血缘核对遮罩列出自哪张表时出错,照列名认", exc_info=True)
|
||||
return plain
|
||||
|
||||
|
||||
def _mask_owners(tables: dict[str, list[str]], base: set[str]) -> dict[str, set[str]]:
|
||||
"""遮罩列 → 它的遮罩表(小写的表名),只记排除得掉一些表的:遮罩列在几张表里都有,其中有的表里另有别的遮罩列,
|
||||
只认那几张(设了 name, phone,customers 有这两列,stores 只有 name:name 指的是客户姓名)。只在一张表里有、几张表
|
||||
里都有可哪张都没有别的遮罩列的不记:照列名认,哪张表的都算。"""
|
||||
out: dict[str, set[str]] = {}
|
||||
for mask in base:
|
||||
having = {t for t, cols in tables.items() if mask in cols}
|
||||
paired = {t for t in having if any(c in base and c != mask for c in tables[t])}
|
||||
if paired and paired != having:
|
||||
out[mask] = paired
|
||||
return out
|
||||
|
||||
|
||||
def _lineages(root: Any, schema: dict[str, Any] | None, read: str | None, columns: list[str]) -> dict[str, Any]:
|
||||
"""结果列(小写)→ 它的血缘(sqlglot.lineage 的节点)。先一次认全部的列(只解析、补全一遍);认不了的逐列再认,还是
|
||||
认不出的不在里面(_owned_masks 当认不出出处)。"""
|
||||
from sqlglot.lineage import lineage
|
||||
|
||||
try:
|
||||
found = lineage(None, root.copy(), schema=schema, dialect=read)
|
||||
every = {str(k).lower(): v for k, v in found.items()} if isinstance(found, dict) else {}
|
||||
except Exception: # noqa: BLE001 - 有一列认不了就整条认不了:下面逐列认
|
||||
every = {}
|
||||
out: dict[str, Any] = {}
|
||||
for column in columns:
|
||||
if (node := every.get(column.lower())) is None:
|
||||
try:
|
||||
node = lineage(column, root.copy(), schema=schema, dialect=read)
|
||||
except Exception: # noqa: BLE001 - 认不出这一列(没起别名的表达式……)
|
||||
continue
|
||||
out[column.lower()] = node
|
||||
return out
|
||||
|
||||
|
||||
def _owned_masks(node: Any, base: set[str], owners: dict[str, set[str]], known: set[str]) -> list[str]:
|
||||
"""一个结果列的值取自哪几列遮罩列(按血缘里出现的先后),别的表里同名的列不算(owners,_mask_owners)。血缘认不出来
|
||||
(node 为 None)、里面有认不出出处的(_leaf_of 交回 None),有取自遮罩列的就点名它们、没有就照全部遮罩列算:排除不了。
|
||||
known 是冻结的表结构里有的表:不在里面的表认不出有没有别的遮罩列,照算。"""
|
||||
if node is None:
|
||||
return sorted(base)
|
||||
found: list[str] = []
|
||||
unplaced = False
|
||||
for leaf in node.walk():
|
||||
if leaf.downstream:
|
||||
continue
|
||||
place = _leaf_of(leaf)
|
||||
if place is None:
|
||||
unplaced = True
|
||||
continue
|
||||
if place is _NO_COLUMN:
|
||||
continue
|
||||
table, column = place
|
||||
if column not in base or (column in owners and table in known and table not in owners[column]):
|
||||
continue
|
||||
if column not in found:
|
||||
found.append(column)
|
||||
return found or (sorted(base) if unplaced else [])
|
||||
|
||||
|
||||
#: 血缘里的一个末端没有取任何列(写死的值、COUNT(*))
|
||||
_NO_COLUMN = object()
|
||||
|
||||
|
||||
def _leaf_of(leaf: Any) -> Any:
|
||||
"""血缘的一个末端出自哪张表的哪一列:(表名, 列名),都是小写;没有取任何列的(写死的值、COUNT(*))交回 _NO_COLUMN;
|
||||
认不出出处的(sqlglot 找不到来源、表值函数、整张表的 *、派生来源里的 *)交回 None。"""
|
||||
from sqlglot import exp
|
||||
|
||||
source = leaf.source
|
||||
if isinstance(source, exp.Table) and isinstance(source.this, exp.Identifier):
|
||||
try:
|
||||
column = exp.to_column(leaf.name)
|
||||
except Exception: # noqa: BLE001 - 名字读不出来:认不出
|
||||
return None
|
||||
if not isinstance(column, exp.Column) or isinstance(column.this, exp.Star) or not column.name:
|
||||
return None
|
||||
return source.name.lower(), column.name.lower()
|
||||
if isinstance(source, (exp.Table, exp.Placeholder)) or leaf.expression is source:
|
||||
return None
|
||||
return _NO_COLUMN
|
||||
|
||||
|
||||
async def _query_step(artifact: str, cells: list[tuple[Any, Any]], doc: dict[str, Any], sealed: _Sealed,
|
||||
@@ -1920,7 +2210,7 @@ async def _merge_input_sets(item: dict[str, Any], masks: _Masks, depth: int, rec
|
||||
artifact = str(item.get("artifact") or "")
|
||||
content, ok = masks.fetch(artifact) if artifact and depth < MERGE_DEPTH else (None, None)
|
||||
if ok is True and isinstance(content, dict):
|
||||
return await _mask_sets(content, source, masks, depth=depth + 1)
|
||||
return await _mask_sets(content, source, masks, depth=depth + 1, artifact=artifact)
|
||||
current, found = await masks.of(source) if source and source != MERGE_SOURCE else (set(), True)
|
||||
if not (current or recorded or not found or source == MERGE_SOURCE or not source):
|
||||
return _MaskSets(set(), set(), set())
|
||||
@@ -2991,33 +3281,6 @@ async def judge_units(run_id: str, payload: Any = Body(None), report: str | None
|
||||
return await _judge_now(sealed, chosen, units)
|
||||
|
||||
|
||||
async def _judge_masks(catalog: dict[str, Any], masked: dict[str, list[str]],
|
||||
loader: Callable[[str], Any] | None = None) -> dict[str, list[str]]:
|
||||
"""交给裁判的摘录要遮的列:{数据源名: 列名},裁判按数据源名合起来认(judge._hidden)。数据源现在设的(judge.source_masks)
|
||||
之外,目录里每份查询快照按证据面板同一个函数(_mask_sets)认出的、改了名字的遮罩列(c.phone AS tel 的 tel)也并进它的
|
||||
数据源名下(合并查询并进「合并查询」名下),多遮无妨。以前摘录只按列名去掉 phone,tel 那一列的号码照样交给裁判,裁判
|
||||
理由可能原样带进证据面板。报告撰写节点当场裁判(report.py)用同一个函数。loader 同 judge.prepare(缺省读工件库)。"""
|
||||
from app.core.artifact_store import load as default_load
|
||||
|
||||
read = loader or default_load
|
||||
out = {str(k): [str(c) for c in v] for k, v in masked.items()}
|
||||
masks = _Masks()
|
||||
for entry in catalog.values():
|
||||
if not isinstance(entry, dict) or entry.get("kind") != "query" or not entry.get("artifact"):
|
||||
continue
|
||||
try:
|
||||
snap = read(str(entry["artifact"]))
|
||||
except Exception: # noqa: BLE001 - 读不出来、不在封存范围里:摘录本来就不给它
|
||||
continue
|
||||
if not isinstance(snap, dict):
|
||||
continue
|
||||
source = str(entry.get("source") or snap.get("source") or "")
|
||||
have = {c.lower() for c in out.get(source, [])}
|
||||
if extra := sorted((await _mask_sets(snap, source, masks)).hidden - have):
|
||||
out.setdefault(source, []).extend(extra)
|
||||
return out
|
||||
|
||||
|
||||
async def _judge_now(sealed: _Sealed, report: _Report, units: list[str]) -> dict[str, Any]:
|
||||
from app.engine import judge as judging
|
||||
|
||||
@@ -3040,7 +3303,7 @@ async def _judge_now(sealed: _Sealed, report: _Report, units: list[str]) -> dict
|
||||
if pending:
|
||||
# 带期 4 标记的文档才让裁判取得到哈希链上的清单(摘录的来历行要用);老文档的 loader 和以前完全相同
|
||||
loader = _sealed_loader(sealed, chain=doc.get("provenance") == DOC_PROVENANCE)
|
||||
masked = await _judge_masks(catalog, await judging.source_masks(catalog, loader=loader), loader)
|
||||
masked = await judging.source_masks(catalog, loader=loader)
|
||||
request = judging.prepare(doc, catalog, units=pending, loader=loader, masked=masked)
|
||||
skipped = dict(request.skipped)
|
||||
if request.cands:
|
||||
@@ -3156,8 +3419,8 @@ GROUPS = (("cited", "有出处"), ("none", "无证据"), ("suspicious", "可疑
|
||||
#: 文档本身对不上的违规:不管落在哪个片段上都单独列一行,不能被那个片段「有出处」的行盖住
|
||||
_INTEGRITY = frozenset({"bad_schema", "segment_mismatch", "structural_text", "render_mismatch", "eid_mismatch",
|
||||
"state_mismatch"})
|
||||
#: 落在已经列出的片段上也单列一行的违规:文档对不上的,和写进了遮罩列的值的(masked_cell:引用的那一格落在的正是一个
|
||||
#: 「有出处」的单元格片段,经指标带出来的落在指标片段上,不单列的话审计表、导出的 CSV 里只有一行普通的「有出处」)
|
||||
#: 落在已经列出的片段上也单列一行的违规:文档对不上的,和引用了遮罩列的格子的(masked_cell:引用的那一格落在的正是一个
|
||||
#: 「有出处」的单元格片段,不单列的话审计表、导出的 CSV 里只有一行普通的「有出处」)
|
||||
_LISTED_ALWAYS = _INTEGRITY | {"masked_cell"}
|
||||
UNCITED_CLAIM = "这句结论没有附依据(既没有引用,也没有注明依据)"
|
||||
UNCITED_CLAIM_COUNTED = "这句结论没有附依据。本报告要求结论句附依据,出具时计入缺口"
|
||||
@@ -3197,7 +3460,7 @@ _CSV_LABELS: dict[str, dict[Any, str]] = {
|
||||
"issue": {"uncited_number": "没有出处的数字", "unresolved_ref": "引用无法解析", "unknown_entity": "疑似不存在的名称",
|
||||
"unverified_entity": "无法核实", "uncited_claim": "结论句未附依据", "no_candidate": "未找到候选",
|
||||
"contradicted_claim": "证据相矛盾", "insufficient_claim": "证据不足", "unsupported_claim": "证据不支持",
|
||||
"masked_cell": "写进了遮罩列的值",
|
||||
"masked_cell": "引用了遮罩列",
|
||||
**{code: "文档不一致" for code in _INTEGRITY}},
|
||||
"evidence_kind": {"metric": "口径卡指标", "cell": "查询单元格", "input": "运行输入", "table": "表", "column": "字段",
|
||||
"quote": "知识库原话", "query": "查询结果"},
|
||||
|
||||
+24
-17
@@ -31,7 +31,7 @@ from app.api.evidence import (
|
||||
MISSING_QUERY,
|
||||
REPORT_NOT_FOUND,
|
||||
TAMPERED_QUERY,
|
||||
_collect_secrets,
|
||||
_MEMO,
|
||||
_Masks,
|
||||
_MaskSets,
|
||||
_reports,
|
||||
@@ -39,6 +39,7 @@ from app.api.evidence import (
|
||||
_sealed,
|
||||
_sealed_cards,
|
||||
_sealed_loader,
|
||||
_snapshot_secrets,
|
||||
)
|
||||
from app.core.artifact_store import load
|
||||
from app.engine import recompute
|
||||
@@ -323,8 +324,9 @@ class _Hidden(Mapping[str, frozenset[str]]):
|
||||
设的遮罩要访问库,交回事件循环去查(_Masks 是异步的),线程在这里等它。以前每次请求在事件循环里把这次运行的
|
||||
全部快照读一遍、复验一遍哈希,不管卡上用没用到。
|
||||
|
||||
secrets 同样在工作线程里收:这次运行查到过的遮罩值(和证据面板的 _Masks.run_secrets 同一个口径),重算里认不出出处
|
||||
的输入、整份结果当输入的指标要按值认时才收,一次请求只收一次。
|
||||
secrets 同样在工作线程里收:这次运行查到过的遮罩值(和证据面板的 _Masks.run_secrets 同一个口径、同一份进程内记录
|
||||
_MEMO),重算里认不出出处的输入、整份结果当输入的指标要按值认时才收,一次请求只收一次;证据面板、之前的推演收过的
|
||||
快照不再读。分析结果同样按快照工件记在进程里。
|
||||
"""
|
||||
|
||||
def __init__(self, sealed: _Sealed, loader: Any, loop: asyncio.AbstractEventLoop) -> None:
|
||||
@@ -348,23 +350,26 @@ class _Hidden(Mapping[str, frozenset[str]]):
|
||||
snap = None
|
||||
snap = snap if isinstance(snap, dict) else None
|
||||
if key not in self._memo:
|
||||
found = _mask_sets_of(snap, self._sealed.queries[key], self._masks)
|
||||
found = _mask_sets_of(snap, self._sealed.queries[key], self._masks, key)
|
||||
self._memo[key] = self._wait(found)
|
||||
return snap, self._memo[key]
|
||||
|
||||
def secrets(self) -> Any:
|
||||
"""这次运行查到过的遮罩值(masking._Secrets,没有为 None):数据源现在设了遮罩、或者已经不在了的查询快照里取自
|
||||
遮罩列的值,合并查询不收(api/evidence.py 的 _collect_secrets)。"""
|
||||
"""这次运行查到过的遮罩值(masking._Secrets,没有为 None):数据源现在设了遮罩、或者已经不在了的查询快照里确定
|
||||
取自遮罩列的值,合并查询不收(api/evidence.py 的 _snapshot_secrets)。每份快照收到的值先查进程内记录(_MEMO),
|
||||
记过的不再读。"""
|
||||
if not self._collected:
|
||||
self._collected = True
|
||||
found = []
|
||||
for artifact, info in self._sealed.queries.items():
|
||||
if not self._wait(self._masks.masked_source(info.get("source"))):
|
||||
continue
|
||||
snap, sets = self._sets(artifact)
|
||||
if snap is not None and snap.get("source") != MERGE_SOURCE:
|
||||
found.append((snap, sets))
|
||||
self._secrets = _collect_secrets(found)
|
||||
parts = []
|
||||
for artifact, _source, key in self._wait(self._masks.value_keys(self._sealed.queries)):
|
||||
known, got = _MEMO.values(key)
|
||||
if not known:
|
||||
snap, sets = self._sets(artifact)
|
||||
if snap is None or snap.get("source") == MERGE_SOURCE:
|
||||
continue
|
||||
got = _MEMO.keep_values(key, _snapshot_secrets(snap, sets))
|
||||
parts.append((key, got))
|
||||
self._secrets = _MEMO.pooled(parts)
|
||||
return self._secrets
|
||||
|
||||
def _wait(self, coro: Any) -> Any:
|
||||
@@ -385,10 +390,12 @@ class _Hidden(Mapping[str, frozenset[str]]):
|
||||
return len(self._sealed.queries)
|
||||
|
||||
|
||||
async def _mask_sets_of(snap: dict[str, Any] | None, info: dict[str, Any], masks: _Masks) -> _MaskSets:
|
||||
async def _mask_sets_of(snap: dict[str, Any] | None, info: dict[str, Any], masks: _Masks,
|
||||
artifact: str | None = None) -> _MaskSets:
|
||||
"""一份查询快照的遮罩,和证据面板的查询步骤同一个函数(api/evidence.py 的 _mask_sets):查询当时记下的加数据源现在
|
||||
设的,SQL 里给遮罩列起了别名、套了函数的结果列也算。snap 为 None(读不出来)时只能按台账记的数据源现在设的遮罩算。"""
|
||||
return await masks.sets(None, info.get("source"), snap)
|
||||
设的,SQL 里给遮罩列起了别名、套了函数的结果列也算。snap 为 None(读不出来)时只能按台账记的数据源现在设的遮罩算。
|
||||
artifact 是 snap 的工件 id:给了的,分析结果按快照记在进程里,之后的推演、证据面板不再分析。"""
|
||||
return await masks.sets(artifact, info.get("source"), snap)
|
||||
|
||||
|
||||
async def _mask_of(snap: dict[str, Any] | None, info: dict[str, Any], masks: _Masks) -> set[str]:
|
||||
|
||||
@@ -3131,9 +3131,9 @@ def absence_claims(doc: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
|
||||
|
||||
def _masked_cell_message(ref: str, column: str, sources: list[str]) -> _Reason:
|
||||
"""违规 masked_cell 的说法:列名就是遮罩列的说「在数据源中设置了遮罩」,按 SQL 认出来的点名取自哪几列遮罩列(sources,
|
||||
这份快照的数据源设的、查询当时记下的)——遮罩按列名设在整个数据源上,别的表里同名的列也算,读的人要看得出是哪一列
|
||||
的遮罩管到了这里。给写作者的那句说清这一列的值不能以任何写法写进报告:只说「删掉引用」,写作者会把值照抄成文字。"""
|
||||
"""违规 masked_cell 的说法:列名就是遮罩列的说「在数据源中设置了遮罩」,按 SQL 认出来的点名它取自的那几列遮罩列(sources,
|
||||
按血缘认出来的;认不出时是这份快照的全部遮罩列)——遮罩按列名设在整个数据源上,读的人要看得出是哪一项遮罩设置管到了
|
||||
这里。给写作者的那句说清这一列的值不能以任何写法写进报告:只说「删掉引用」,写作者会把值照抄成文字。"""
|
||||
own = column.lower() in {s.lower() for s in sources} or not sources
|
||||
where = (f"列「{column}」在数据源中设置了遮罩" if own
|
||||
else f"列「{column}」按 SQL 认出取自设置了遮罩的列「{'、'.join(sources[:3])}」")
|
||||
@@ -3148,9 +3148,10 @@ def masked_cells(doc: dict[str, Any], catalog: dict[str, Any], masked: dict[str,
|
||||
"""正文直接引用了取自遮罩列的格子([[v:Q2.r0.tel]],整表展开出来的格也算),每格一条违规 masked_cell。
|
||||
|
||||
masked 是 {查询快照工件: 取自遮罩列的结果列},结果列可以是列名的集合,也可以是 {列名: 它取自的遮罩列}(说法里点名)。
|
||||
由报告节点按证据面板同一个函数算好交进来(api/evidence.py 的 _mask_sets 的 sure:列名就是遮罩列的、SQL 里给遮罩列
|
||||
起了别名套了函数的;SQL 认不准、整份算进来的不在里面):遮罩设在数据源上,这里只认文档和目录,查不了库。写作目录
|
||||
由报告节点算好交进来(api/evidence.py 的 _cited_masks:列名就是遮罩列的、SQL 里给遮罩列起了别名套了函数的,按血缘
|
||||
去掉别的表里同名的列;SQL 认不准、整份算进来的不在里面):遮罩设在数据源上,这里只认文档和目录,查不了库。写作目录
|
||||
不遮罩(写作者要看得到数据),所以拦在自查:和没有出处的数字一样交回写作者重写,仍不改的按「违规处置」办(report.py)。
|
||||
只认引用,不按值在正文里找遮罩值(两个字的姓名会撞上常用词):照抄成正文的号码由数字核对拦下。
|
||||
|
||||
违规不带那一格的值(不写 text、context):它会进 report.checked、重写的日志、出具缺口,这些地方不能再露一次。带着
|
||||
ref、列名、span 和所在片段:证据面板能定位,写作者知道要去掉哪一处。"""
|
||||
@@ -3180,68 +3181,6 @@ def masked_cells(doc: dict[str, Any], catalog: dict[str, Any], masked: dict[str,
|
||||
return found
|
||||
|
||||
|
||||
def _masked_value_message(ref: str | None) -> _Reason:
|
||||
"""违规 masked_cell(按值认出来的,by_value)的说法。不点名列:按值认只知道这个值在写作目录的遮罩列里出现过。经引用
|
||||
带出来的(指标的值就是那一格)点名那处引用;写在正文里的,交回写作者时另附那一句(context,值换成「已遮罩」)。"""
|
||||
if ref:
|
||||
return _Reason(
|
||||
f"引用 [[{ref}]] 的值与设置了遮罩的列中的值相同,值不应写进报告",
|
||||
f"[[{ref}]] 的值就是证据目录里取自遮罩列的值(手机号、姓名这类),这个值不能以任何形式写进报告,引用、数字、"
|
||||
"照抄的文字都不行:删掉这处引用和相应的内容",
|
||||
)
|
||||
return _Reason(
|
||||
"正文写着设置了遮罩的列中的值,值不应写进报告",
|
||||
"正文里直接写着证据目录里取自遮罩列的值(手机号、姓名这类),这些值不能以任何形式写进报告,照抄的文字、数字都"
|
||||
"不行:删掉这处文字,或者改成不含这个值的说法",
|
||||
)
|
||||
|
||||
|
||||
def masked_values(doc: dict[str, Any], secrets: Any,
|
||||
skip: set[str] | frozenset[str] = frozenset()) -> list[dict[str, Any]]:
|
||||
"""正文里直接写着的遮罩值,每处一条违规 masked_cell(带 by_value):写作者把号码、姓名照抄成文字、写成裸数字,或者
|
||||
经指标、输入的引用带了出来([[m:phone_raw]] 的值就是号码)。masked_cells 只认 [[v:]] 引用的格子,这几种它认不出;
|
||||
数字核对管得到裸数字、管不到文字,「数字核对」关掉时数字也不管。
|
||||
|
||||
secrets 是写作目录里取自遮罩列的值(masking._Secrets),由报告节点按证据面板同一个函数算好交进来(report.py 的
|
||||
_MaskedCells:只收确定取自遮罩列的列,运行输入里写着的免);None 时一条都不记。skip 是已经按引用记过 masked_cell
|
||||
的片段:同一处不记两条。按整句认:换了写法的长号码(199-0000-0702)组装时被拆成好几个数字片段,一个片段里认不全,
|
||||
认出来的那一段落在哪几个片段上,就记在第一个上。
|
||||
|
||||
违规不带值:context 是那一句,值换成「已遮罩」——交回写作者时据此找到那一处,进事件、日志也不再露一次。"""
|
||||
if secrets is None:
|
||||
return []
|
||||
skipped = {str(s) for s in skip}
|
||||
found: list[dict[str, Any]] = []
|
||||
for _, unit in iter_units(doc):
|
||||
placed: list[tuple[int, int, dict[str, Any]]] = []
|
||||
sentence = ""
|
||||
for seg in unit.get("segments") or []:
|
||||
text = seg.get("text") if isinstance(seg.get("text"), str) else ""
|
||||
placed.append((len(sentence), len(sentence) + len(text), seg))
|
||||
sentence += text
|
||||
hits = secrets.spans(sentence, loose=True) if sentence else []
|
||||
named: set[str] = set()
|
||||
context: str | None = None
|
||||
for start, end in hits:
|
||||
owners = [(a, seg) for a, b, seg in placed if a < end and start < b]
|
||||
if not owners or any(str(seg.get("id")) in skipped for _, seg in owners):
|
||||
continue
|
||||
first_at, first = owners[0]
|
||||
if str(first.get("id")) in named:
|
||||
continue
|
||||
named.add(str(first.get("id")))
|
||||
context = context or secrets.scrub(sentence, loose=True).strip()[:120]
|
||||
ref = next((str(seg["ref"]) for _, seg in owners if seg.get("ref")), None)
|
||||
last_at, last = owners[-1]
|
||||
head, tail = first.get("span"), last.get("span")
|
||||
where = None
|
||||
if all(isinstance(s, list) and len(s) == 2 for s in (head, tail)):
|
||||
where = [head[0] + max(start - first_at, 0), tail[0] + min(end - last_at, len(last.get("text") or ""))]
|
||||
found.append(_violation("masked_cell", _masked_value_message(ref), ref=ref, by_value=True, context=context,
|
||||
span=where, segment=first.get("id"), unit=unit.get("id")))
|
||||
return found
|
||||
|
||||
|
||||
#: 引用查询结果的写法。只跟着目录里的查询出现(catalog_prompt):写作规则本身保持一期那份,升级前
|
||||
#: 发起的运行跑到报告节点时提示一字不差;只有口径卡的目录也不该教写作者去引用不存在的 Q1
|
||||
CELL_RULES = (
|
||||
|
||||
@@ -32,7 +32,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
from collections import Counter
|
||||
@@ -321,36 +320,30 @@ class _RenderedStream:
|
||||
|
||||
|
||||
class _MaskedCells:
|
||||
"""报告自查:报告里写进了取自遮罩列的值,记一条违规 masked_cell,和没有出处的数字一样交回写作者重写。写作目录本身
|
||||
不遮(写作者要看得到数据),只拦写进报告的。两种认法:
|
||||
"""报告自查:写作者引用了取自遮罩列的格子([[v:Q2.r0.tel]]),记一条违规 masked_cell,和没有出处的数字一样交回写作者
|
||||
重写(evidence.masked_cells)。写作目录本身不遮(写作者要看得到数据),只拦引用。
|
||||
|
||||
- 引用(evidence.masked_cells):[[v:Q2.r0.tel]] 引用的那一列取自遮罩列。哪几列算:和证据面板同一个函数(api/evidence.py
|
||||
的 _mask_sets)认出来的「确定取自遮罩列」的那几列(sure)——列名就是遮罩列的,SQL 里给遮罩列起了别名、套了函数、经
|
||||
子查询或 WITH 转一道的,合并查询按逐格来历和合并 SQL 认。证据面板为了展示宁可多遮(SQL 认不准时整份的列都遮),自查
|
||||
不跟着多算:多算一列就是一次判失败,写作者也改不掉
|
||||
- 按值(evidence.masked_values):正文里直接写着这些列里的值——照抄成文字、写成数字,或者经指标带了出来([[m:phone_raw]]
|
||||
的值就是号码)。值从数据源现在设了遮罩的那几份查询快照里收,同样只收 sure 的列、不收只装着分档标签的格子;合并查询
|
||||
的不收(它的值来自它的输入,输入是目录里另外几份快照)。protected 是运行输入(发起运行的人自己填的),里面写着的值
|
||||
不算,和推演、证据面板一样
|
||||
哪几列算按 api/evidence.py 的 _cited_masks 认:列名就是遮罩列的,SQL 里给遮罩列起了别名、套了函数、经子查询或 WITH
|
||||
转一道的(_mask_sets 的 sure,和证据面板同一个函数),再按血缘去掉别的表里同名的列(设了 name, phone,门店名不算客户
|
||||
姓名);合并查询按逐格来历和合并 SQL 认。证据面板为了展示宁可多遮(SQL 认不准时整份的列都遮),自查不跟着多算:多算
|
||||
一列就是一次判失败,写作者也改不掉。
|
||||
|
||||
组装好一稿之后才认:引用只读正文引用到的那几份快照,按值只读数据源现在设了遮罩的那几份(读盘复验哈希放进线程),都只读
|
||||
一次,重写的稿子不再读。没设遮罩的运行一份都不读。以前写作之前就把目录里每份查询快照都读一遍。"""
|
||||
只查引用,不按值在正文里找遮罩值:两个字的姓名会撞上「销售高峰」这类常用词,同名列的值会撞上门店名,正式运行因此判
|
||||
失败(复核第 3 轮去掉了按值认);正文里照抄的号码本来就会被「数字没有出处」拦下。
|
||||
|
||||
def __init__(self, catalog: dict[str, Any], protected: list[str] | tuple[str, ...] = ()) -> None:
|
||||
组装好一稿之后才认:只读正文引用到的那几份快照(读盘复验哈希放进线程),每份只读一次,重写的稿子不再读;没引用格子的
|
||||
一份都不读。"""
|
||||
|
||||
def __init__(self, catalog: dict[str, Any]) -> None:
|
||||
from app.api.evidence import _Masks
|
||||
|
||||
self._catalog = catalog
|
||||
self._masks = _Masks()
|
||||
self._protected = [str(t) for t in protected if t]
|
||||
self._memo: dict[str, dict[str, list[str]]] = {}
|
||||
#: 按值认之前已经读过的快照(为了认引用):收值时不再读第二遍,收完就放掉
|
||||
self._loaded: dict[str, tuple[dict[str, Any], Any]] = {}
|
||||
self._secrets: Any = None
|
||||
self._collected = False
|
||||
|
||||
async def __call__(self, doc: dict[str, Any]) -> dict[str, Any]:
|
||||
"""把这一稿写进了遮罩列的值的地方记进 doc 的违规(接在最后,和 compose_doc 给了 masked 时一样),交回同一份文档。"""
|
||||
from app.engine.evidence import iter_segments, masked_cells, masked_values
|
||||
"""把这一稿引用了遮罩列的格子记进 doc 的违规(接在最后,和 compose_doc 给了 masked 时一样),交回同一份文档。"""
|
||||
from app.engine.evidence import iter_segments, masked_cells
|
||||
|
||||
wanted: dict[str, Any] = {}
|
||||
for seg in iter_segments(doc):
|
||||
@@ -363,71 +356,26 @@ class _MaskedCells:
|
||||
for artifact, source in wanted.items():
|
||||
await self._read(artifact, source)
|
||||
cited = masked_cells(doc, self._catalog, {a: m for a, m in self._memo.items() if m})
|
||||
written = masked_values(doc, await self._values(), {str(v.get("segment")) for v in cited})
|
||||
doc["violations"] = [*doc["violations"], *cited, *written]
|
||||
doc["violations"] = [*doc["violations"], *cited]
|
||||
return doc
|
||||
|
||||
async def _read(self, artifact: str, source: Any) -> tuple[dict[str, Any], Any] | None:
|
||||
"""读一份查询快照、认它的遮罩(_mask_sets),确定取自遮罩列的列记进 _memo。读不出来的为 None(引用它的格子本来就
|
||||
解析不了)。"""
|
||||
from app.api.evidence import _mask_sets
|
||||
async def _read(self, artifact: str, source: Any) -> None:
|
||||
"""读一份查询快照,引用了就算违规的列(_cited_masks:{结果列: 它取自的遮罩列})记进 _memo。读不出来的记空:引用它的
|
||||
格子本来就解析不了。"""
|
||||
from app.api.evidence import _cited_masks
|
||||
from app.core.artifact_store import load
|
||||
|
||||
try:
|
||||
snap = await asyncio.to_thread(load, artifact)
|
||||
except ValueError:
|
||||
snap = None
|
||||
if not isinstance(snap, dict):
|
||||
self._memo[artifact] = {}
|
||||
return None
|
||||
sets = await _mask_sets(snap, source, self._masks)
|
||||
self._memo[artifact] = {c: sorted(sets.base) for c in sets.sure}
|
||||
if not self._collected:
|
||||
self._loaded[artifact] = (snap, sets)
|
||||
return snap, sets
|
||||
|
||||
async def _values(self) -> Any:
|
||||
"""写作目录里取自遮罩列的值(masking._Secrets),一个都没有为 None。只收一次。"""
|
||||
if self._collected:
|
||||
return self._secrets
|
||||
self._collected = True
|
||||
from app.engine.masking import _Secrets
|
||||
from app.engine.merge_query import MERGE_SOURCE
|
||||
|
||||
secrets = _Secrets()
|
||||
seen: set[str] = set()
|
||||
for entry in self._catalog.values():
|
||||
artifact = entry.get("artifact") if isinstance(entry, dict) and entry.get("kind") == "query" else None
|
||||
source = entry.get("source") if isinstance(entry, dict) else None
|
||||
if not isinstance(artifact, str) or not artifact or artifact in seen or not source \
|
||||
or source == MERGE_SOURCE or not (await self._masks.of(source))[0]:
|
||||
continue
|
||||
seen.add(artifact)
|
||||
# 认引用时读过的不再读;读过、读不出来的(_memo 里有、_loaded 里没有)不再试
|
||||
found = self._loaded.get(artifact)
|
||||
if found is None and artifact not in self._memo:
|
||||
found = await self._read(artifact, source)
|
||||
if found is not None:
|
||||
snap, sets = found
|
||||
secrets.add(snap, sets.sure, labels=sets.labels, base=sets.base)
|
||||
self._loaded.clear()
|
||||
secrets.protect(self._protected)
|
||||
if secrets.strong:
|
||||
secrets.prepare()
|
||||
self._secrets = secrets
|
||||
return self._secrets
|
||||
self._memo[artifact] = await _cited_masks(snap, source, self._masks, artifact) if isinstance(snap, dict) else {}
|
||||
|
||||
|
||||
def _masked_said(hits: list[dict[str, Any]]) -> str:
|
||||
"""报告写进了遮罩列的值的那几处在缺口、日志里怎么说:「引用了 2 处取自遮罩列的格子(「v:Q2.r0.tel」、…)」「写进了 1 处
|
||||
设置了遮罩的列中的值」,两样都有时连起来。只点名引用,不写值。"""
|
||||
cited = [v for v in hits if not v.get("by_value")]
|
||||
written = [v for v in hits if v.get("by_value")]
|
||||
parts = [f"引用了 {len(cited)} 处取自遮罩列的格子({_named(cited)})"] if cited else []
|
||||
if written:
|
||||
named = _named(written)
|
||||
parts.append(f"写进了 {len(written)} 处设置了遮罩的列中的值" + (f"({named})" if named else ""))
|
||||
return ",".join(parts)
|
||||
"""报告引用了遮罩列的那几处在缺口、日志里怎么说:「引用了 2 处取自遮罩列的格子(「v:Q2.r0.tel」、…)」。只点名引用,
|
||||
不写值。"""
|
||||
return f"引用了 {len(hits)} 处取自遮罩列的格子({_named(hits)})"
|
||||
|
||||
|
||||
async def _run_class(run_id: str) -> str | None:
|
||||
@@ -647,9 +595,8 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
|
||||
|
||||
spec = ctx.run.spec
|
||||
catalog = report_catalog(state, spec, ctx.node)
|
||||
# 报告写进了取自遮罩列的值(引用了那一格、照抄成正文):自查记一条违规,和没有出处的数字一样交回写作者重写
|
||||
# (_MaskedCells)。运行输入里写着的值是发起运行的人自己填的,不算
|
||||
masked = _MaskedCells(catalog, protected=[json.dumps(state.get("input") or {}, ensure_ascii=False, default=str)])
|
||||
# 报告引用了取自遮罩列的格子:自查记一条违规,和没有出处的数字一样交回写作者重写(_MaskedCells)
|
||||
masked = _MaskedCells(catalog)
|
||||
allow = report_allowance(spec, ctx.node)
|
||||
cells = True
|
||||
# 结论句策略只对升级后发起的运行生效:升级前的运行(图里也不会有 claims)事件、产出一个字不加
|
||||
@@ -754,9 +701,7 @@ async def run_report(state: GraphState, ctx: NodeContext) -> dict[str, Any]:
|
||||
tried = f"已要求模型重写 {repairs} 次仍未改掉" if repairs else "未要求重写"
|
||||
refs = list(dict.fromkeys(str(v["ref"]) for v in hits if v.get("ref")))
|
||||
gaps.append(f"报告{_masked_said(hits)},{tried}:数据源设置了遮罩的列不应写进报告")
|
||||
# written:其中按值认出来的几处(正文里直接写着、经指标带出来的),时间线据此换说法
|
||||
ctx.emit(EventType.LOG, level="warn", code="report_masked_cell", count=len(hits), refs=refs,
|
||||
written=sum(1 for v in hits if v.get("by_value")),
|
||||
columns=list(dict.fromkeys(str(v["column"]) for v in hits if v.get("column"))),
|
||||
message=f"{gaps[0]}。报告照常产出,这几处已标注,已记为缺口。请在报告撰写节点的指令中去掉对这些列的引用")
|
||||
doc_artifact = await _store(doc, ctx)
|
||||
@@ -857,10 +802,7 @@ async def _judge_inline(ctx: NodeContext, doc: dict[str, Any], catalog: dict[str
|
||||
settings = await judging.judge_settings(session)
|
||||
judge_spec = await judging.judge_model_spec(session, judge_cfg, settings=settings)
|
||||
budget = judging.report_budget(settings, judge_cfg)
|
||||
# 交给裁判的摘录要遮的列:数据源设的之外,SQL 里改了名字的遮罩列也算(和按需裁判同一个函数)
|
||||
from app.api.evidence import _judge_masks
|
||||
|
||||
masks = await _judge_masks(catalog, await judging.source_masks(catalog))
|
||||
masks = await judging.source_masks(catalog)
|
||||
|
||||
@task
|
||||
async def judge_step(request: judging.JudgeRequest, key: str, limits: judging.Budget, known: dict[str, Any],
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
"""证据面板、推演收「这次运行查到过的遮罩值」:只收确定取自遮罩列的值;按快照工件在进程里记下,不再每次都读一遍。
|
||||
|
||||
复核发现(第 3 轮):
|
||||
- 收值按的是面板展示用的 hidden(宁可多遮):SQL 认不准、整份遮的快照,hidden 是全部列,里面的区域名、金额都被收成
|
||||
遮罩值。之后任何查询步骤里展示的 SQL「WHERE region = '华东' AND amount > 12345.5」都显示成「已遮罩」,整份结果当输入
|
||||
的指标值等于这些数时也写「已遮罩」。现在只收确定取自遮罩列的那几列(_MaskSets.sure,同 masking 收值的口径),整份遮
|
||||
的快照只遮显示
|
||||
- 每点开一个片段,都把这次运行所有设了遮罩的数据源上的快照读盘、复验哈希,再逐份做 SQL 分析(一份几十毫秒),缓存
|
||||
只在一次请求里有效。现在按快照工件 id 在进程里记下分析和收到的值(工件按内容寻址、不会变;键里带着数据源现在设的
|
||||
遮罩和类型,设置改了不会拿旧的),之后的请求不再读、不再分析;推演同一份记录
|
||||
|
||||
快照、号码都是虚构的。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from langchain_core.messages import AIMessage
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.core import artifact_store
|
||||
from app.data.engine import engines
|
||||
from app.db.base import SessionLocal
|
||||
from app.db.models import DataSource, Run
|
||||
from app.engine.evidence import iter_segments
|
||||
from app.engine.runner import run_manager
|
||||
from app.main import app
|
||||
|
||||
CUSTOMERS = [(1, "客户甲", "19900000901"), (2, "客户乙", "19900000902"), (3, "客户丙", "19900000903")]
|
||||
ORDERS = [(1, 1, 120.5), (2, 2, 300.0), (3, 2, 80.0), (4, 3, 210.25)]
|
||||
PHONES = [c[2] for c in CUSTOMERS]
|
||||
|
||||
#: 派生来源里有没起别名的 UPPER(c.phone):认遮罩列的那一套认不准,整份的列都遮(只遮显示)
|
||||
SQL_WHOLE = ("SELECT region, amount FROM (SELECT 'east' AS region, UPPER(c.phone), o.amount FROM customers c "
|
||||
"JOIN orders o ON o.customer_id = c.id) t")
|
||||
SQL_TEL = "SELECT c.name AS customer, c.phone AS tel FROM customers c ORDER BY c.id"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def engine_up():
|
||||
await run_manager.setup()
|
||||
yield
|
||||
await run_manager.shutdown()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def client(engine_up):
|
||||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
|
||||
yield c
|
||||
|
||||
|
||||
async def set_source(path, options=None) -> str:
|
||||
async with SessionLocal() as session:
|
||||
row = (await session.execute(select(DataSource).where(DataSource.name == "mall"))).scalar_one_or_none()
|
||||
if row is None:
|
||||
row = DataSource(name="mall", kind="sqlite", database=str(path), readonly=True, enabled=True)
|
||||
session.add(row)
|
||||
row.database, row.kind, row.enabled, row.readonly = str(path), "sqlite", True, True
|
||||
row.options = dict(options or {})
|
||||
await session.commit()
|
||||
return row.id
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def mall(tmp_path, engine_up):
|
||||
from app.api.evidence import _MEMO
|
||||
|
||||
path = tmp_path / "mall.db"
|
||||
db = sqlite3.connect(path)
|
||||
db.executescript("CREATE TABLE customers (id INTEGER PRIMARY KEY, name TEXT, phone TEXT);"
|
||||
"CREATE TABLE orders (id INTEGER PRIMARY KEY, customer_id INTEGER, amount REAL);")
|
||||
db.executemany("INSERT INTO customers VALUES (?, ?, ?)", CUSTOMERS)
|
||||
db.executemany("INSERT INTO orders VALUES (?, ?, ?)", ORDERS)
|
||||
db.commit()
|
||||
db.close()
|
||||
source_id = await set_source(path, {"mask_columns": ["phone"]})
|
||||
await engines.invalidate(source_id)
|
||||
# 同一个测试进程里别的测试记下的结果(内容相同的快照工件 id 一样)不算进这里的读盘次数
|
||||
_MEMO.clear()
|
||||
yield path
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
async def put(sql: str, columns: list[str], rows: list[list]) -> str:
|
||||
return await artifact_store.put_json({"sql": sql, "columns": columns, "rows": rows, "source": "mall"}, kind="query")
|
||||
|
||||
|
||||
def sealed_of(*artifacts: str) -> SimpleNamespace:
|
||||
return SimpleNamespace(queries={a: {"source": "mall"} for a in artifacts})
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 只收确定取自遮罩列的值
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_values_of_a_snapshot_masked_whole_are_not_collected(engine_up, mall):
|
||||
"""整份遮的快照只遮显示:里面的区域名、金额不收成遮罩值,别的查询步骤里写着它们的 SQL 照原样显示;改了名字的号码列
|
||||
(c.phone AS tel)照收。"""
|
||||
from app.api.evidence import _Masks, _Shown
|
||||
|
||||
whole = await put(SQL_WHOLE, ["region", "amount"], [["华东", 12345.5], ["华南", 6789.25]])
|
||||
tel = await put(SQL_TEL, ["customer", "tel"], [[c[1], c[2]] for c in CUSTOMERS])
|
||||
assert await _Masks(sealed_of(whole)).run_secrets() is None
|
||||
secrets = await _Masks(sealed_of(whole, tel)).run_secrets()
|
||||
sql = f"SELECT SUM(amount) FROM orders WHERE region = '华东' AND amount > 12345.5 AND phone <> '{PHONES[0]}'"
|
||||
shown = _Shown(secrets)
|
||||
assert shown.text(sql) == sql.replace(PHONES[0], "已遮罩") and shown.changed, shown.text(sql)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 按快照工件在进程里记下
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def spies(monkeypatch) -> tuple[list[str], list[str]]:
|
||||
"""(读过的工件, 做过的 SQL 分析):证据面板经 artifact_store.load 读快照(查询步骤直接读,收值经 _fetch),按 SQL 认
|
||||
遮罩列经可验证报告和 masking 的两个函数。"""
|
||||
from app.api import evidence as api
|
||||
from app.bundle import build
|
||||
from app.engine import masking
|
||||
|
||||
fetched: list[str] = []
|
||||
analysed: list[str] = []
|
||||
real_load, real_bundle, real_masking = api.load, build._derived_columns, masking._derived_hidden
|
||||
monkeypatch.setattr(api, "load", lambda artifact: fetched.append(artifact) or real_load(artifact))
|
||||
monkeypatch.setattr(build, "_derived_columns",
|
||||
lambda *a, **k: analysed.append("bundle") or real_bundle(*a, **k))
|
||||
monkeypatch.setattr(masking, "_derived_hidden", lambda *a, **k: analysed.append("masking") or real_masking(*a, **k))
|
||||
return fetched, analysed
|
||||
|
||||
|
||||
async def test_the_values_are_read_and_analysed_once_per_process(engine_up, mall, monkeypatch):
|
||||
"""第一次收:设了遮罩的快照各读一次、分析一次。之后的请求(新的 _Masks)一份都不再读、不再分析,收到的值一样。"""
|
||||
from app.api.evidence import _Masks
|
||||
|
||||
tel = await put(SQL_TEL, ["customer", "tel"], [[c[1], c[2]] for c in CUSTOMERS])
|
||||
raw = await put("SELECT id, phone FROM customers ORDER BY id", ["id", "phone"], [[c[0], c[2]] for c in CUSTOMERS])
|
||||
fetched, analysed = spies(monkeypatch)
|
||||
first = await _Masks(sealed_of(tel, raw)).run_secrets()
|
||||
assert sorted(fetched) == sorted([tel, raw]) and analysed, (fetched, analysed)
|
||||
assert set(PHONES) <= first.strong
|
||||
fetched.clear()
|
||||
analysed.clear()
|
||||
second = await _Masks(sealed_of(tel, raw)).run_secrets()
|
||||
assert fetched == [] and analysed == [] and second.strong == first.strong, (fetched, analysed)
|
||||
|
||||
|
||||
async def test_a_changed_mask_setting_is_not_served_from_the_memo(engine_up, mall, monkeypatch):
|
||||
"""数据源的遮罩设置改了:键里带着现在设的遮罩,按新的设置重新收;不再设遮罩的数据源一份都不读。"""
|
||||
from app.api.evidence import _Masks, _Shown
|
||||
|
||||
tel = await put(SQL_TEL, ["customer", "tel"], [[c[1], c[2]] for c in CUSTOMERS])
|
||||
assert "客户甲" not in (await _Masks(sealed_of(tel)).run_secrets()).strong
|
||||
await set_source(mall, {"mask_columns": ["phone", "name"]})
|
||||
widened = await _Masks(sealed_of(tel)).run_secrets()
|
||||
assert {"客户甲", PHONES[0]} <= widened.strong
|
||||
assert _Shown(widened).text("WHERE c.name = '客户甲'") == "WHERE c.name = '已遮罩'"
|
||||
await set_source(mall, {})
|
||||
fetched, _ = spies(monkeypatch)
|
||||
assert await _Masks(sealed_of(tel)).run_secrets() is None and fetched == []
|
||||
|
||||
|
||||
async def test_the_whatif_collects_from_the_same_memo(engine_up, mall):
|
||||
"""推演在工作线程里收(_Hidden.secrets,经推演自己的 loader 读):第一次读一份,之后的推演、证据面板都不再读。"""
|
||||
from app.api.evidence import _Masks
|
||||
from app.api.whatif import _Hidden
|
||||
|
||||
tel = await put(SQL_TEL, ["customer", "tel"], [[c[1], c[2]] for c in CUSTOMERS])
|
||||
reads: list[str] = []
|
||||
|
||||
def loader(artifact: str):
|
||||
reads.append(artifact)
|
||||
return artifact_store.load(artifact)
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
first = await asyncio.to_thread(_Hidden(sealed_of(tel), loader, loop).secrets)
|
||||
assert reads == [tel] and set(PHONES) <= first.strong
|
||||
reads.clear()
|
||||
second = await asyncio.to_thread(_Hidden(sealed_of(tel), loader, loop).secrets)
|
||||
assert reads == [] and second.strong == first.strong
|
||||
assert (await _Masks(sealed_of(tel)).run_secrets()).strong == first.strong
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 片段接口:之后点开只读这一步要显示的快照
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def node(nid, ntype, **config):
|
||||
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
|
||||
|
||||
|
||||
SQL_BY_PHONE = ("SELECT ROUND(SUM(o.amount), 2) AS spend FROM orders o JOIN customers c ON c.id = o.customer_id "
|
||||
f"WHERE c.phone IN ('{PHONES[0]}', '{PHONES[1]}')")
|
||||
|
||||
|
||||
async def test_a_segment_opened_again_reads_only_its_own_snapshot(client, monkeypatch, mall):
|
||||
"""点开一个片段:第一次要把这次运行设了遮罩的快照读一遍(收遮罩值,SQL 里写着号码时展示要换掉);再点开(新的请求)
|
||||
只读这一步要显示的那一份、不再做 SQL 分析,号码照样换掉。"""
|
||||
from app.providers import mock_model
|
||||
|
||||
monkeypatch.setattr(mock_model.MockChatModel, "_decide",
|
||||
lambda self, messages: AIMessage(content="两位重点客户共消费 [[v:Q2.r0.spend]]。"))
|
||||
nodes = [node("start", "input"),
|
||||
node("q_cust", "tool", tool="db_query__mall", args={"sql": "SELECT id, phone AS tel FROM customers ORDER BY id"}),
|
||||
node("q_one", "tool", tool="db_query__mall", args={"sql": SQL_BY_PHONE}),
|
||||
node("write", "report", instructions="写一句", on_violation="flag"),
|
||||
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}])]
|
||||
spec = {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
|
||||
run_id = (await run_manager.start(graph=spec, input_payload={})).id
|
||||
for _ in range(400):
|
||||
await asyncio.sleep(0.05)
|
||||
async with SessionLocal() as session:
|
||||
row = await session.get(Run, run_id)
|
||||
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
|
||||
break
|
||||
assert row.status == "succeeded", row.error
|
||||
[report] = (await client.get(f"/api/runs/{run_id}/evidence")).json()["reports"]
|
||||
doc = artifact_store.load(report["doc_artifact"])
|
||||
seg = next(s for s in iter_segments(doc) if s.get("ref") == "v:Q2.r0.spend")
|
||||
catalog = doc["catalog"]
|
||||
cust, one = catalog["Q1"]["artifact"], catalog["Q2"]["artifact"]
|
||||
fetched, analysed = spies(monkeypatch)
|
||||
for attempt in range(2):
|
||||
fetched.clear()
|
||||
analysed.clear()
|
||||
body = (await client.get(f"/api/runs/{run_id}/evidence/segments/{seg['id']}")).json()
|
||||
[step] = [s for s in body["chain"] if s["step"] == "query"]
|
||||
assert "IN ('已遮罩', '已遮罩')" in step["sql"] and PHONES[0] not in json.dumps(body, ensure_ascii=False)
|
||||
assert one in fetched, fetched
|
||||
assert (cust in fetched) is (attempt == 0), (attempt, fetched)
|
||||
assert bool(analysed) is (attempt == 0), (attempt, analysed)
|
||||
@@ -1,4 +1,4 @@
|
||||
"""证据面板认遮罩列和可验证报告是同一个函数;查询步骤里写着的遮罩值、交给裁判的摘录同样不给。
|
||||
"""证据面板认遮罩列和可验证报告是同一个函数;查询步骤里写着的遮罩值同样不给;交给裁判的摘录照数据源设的遮罩列。
|
||||
|
||||
复核发现:
|
||||
- 证据面板只按数据源的一种方言调 masking._derived_hidden,可验证报告另有几道(语法树认不准的写法、不点列名也读得出表里
|
||||
@@ -6,8 +6,9 @@
|
||||
照给、可验证报告整份遮。现在面板调可验证报告的那个函数(bundle/build.py 的 _derived_columns),再按数据源的方言认一遍
|
||||
- 查询步骤原样给出 SQL 里写着的遮罩值(按号码筛的 WHERE c.phone IN (…)),合并 SQL、SQL 检查的片段也一样;可验证报告
|
||||
按「这次运行查到过的遮罩值」认出来、整份只给哈希。现在展示时换成「已遮罩」,底下写明只是展示时换的
|
||||
- 按需裁判交给裁判的摘录只按列名去掉 phone,c.phone AS tel 那一列的号码照样交给裁判。现在按证据面板同一个函数认出的列
|
||||
也并进去
|
||||
- 第 2 轮把面板那套宁可多遮的列按数据源名并给裁判(裁判按数据源名合起来遮,对同一数据源的每一份快照都生效):整份遮的
|
||||
快照的全部列、只数个数的列一并从摘录里消失,被引用的金额、人数裁判看不到,只能判证据不足。第 3 轮退回 M2 之前的做法,
|
||||
只按数据源设的列名遮;c.phone AS tel 这种改了名字的号码列仍会进摘录,是裁判摘录已知的缺口,另行处理(judge.py)
|
||||
|
||||
真跑的部分用虚构的 SQLite 商城库(号码是编的),报告由 MockChatModel 照剧本写。
|
||||
"""
|
||||
@@ -15,6 +16,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
@@ -26,7 +28,7 @@ from app.core import artifact_store
|
||||
from app.data.engine import engines
|
||||
from app.db.base import SessionLocal
|
||||
from app.db.models import DataSource, Run
|
||||
from app.engine.evidence import iter_segments
|
||||
from app.engine.evidence import iter_segments, iter_units
|
||||
from app.engine.runner import run_manager
|
||||
from app.main import app
|
||||
|
||||
@@ -196,26 +198,94 @@ def test_sql_check_excerpts_and_merge_parts_are_shown_with_values_masked():
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 交给裁判的摘录
|
||||
# 交给裁判的摘录:照数据源设的遮罩列(复核第 3 轮退回 M2 之前的做法)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
#: 派生来源里有没起别名的 UPPER(c.phone):证据面板整份遮(宁可多遮),可这份快照的列和号码无关
|
||||
SQL_WHOLE = ("SELECT name, amount FROM (SELECT c.name, UPPER(c.phone), o.amount FROM customers c "
|
||||
"JOIN orders o ON o.customer_id = c.id) t ORDER BY amount DESC")
|
||||
#: 金额、只数个数的留号人数:值里没有号码,面板照可验证报告的认法把只数个数的那一列也遮
|
||||
SQL_TOTAL = ("SELECT ROUND(SUM(o.amount), 2) AS amount, COUNT(*) FILTER (WHERE c.phone IS NOT NULL) AS with_phone "
|
||||
"FROM orders o JOIN customers c ON c.id = o.customer_id")
|
||||
JUDGED = "全部订单金额 [[v:Q2.r0.amount]],其中 [[v:Q2.r0.with_phone]] 笔的客户留了手机号,最大一笔 [[v:Q1.r0.amount]]。"
|
||||
|
||||
async def test_the_judge_excerpt_drops_a_column_renamed_from_a_masked_one(engine_up, mall):
|
||||
"""c.phone AS tel:交给裁判的遮罩列表把 tel 并进数据源名下,摘录里不给 tel 那一列,号码一个都不交出去。"""
|
||||
from app.api.evidence import _judge_masks
|
||||
from app.engine import judge as judging
|
||||
|
||||
art = await artifact_store.put_json({"columns": ["customer", "tel", "spend"],
|
||||
"rows": [["客户乙", PHONES[1], 380.0], ["客户甲", PHONES[0], 220.0]],
|
||||
"source": "mall", "mask_columns": ["phone"],
|
||||
"sql": "SELECT c.name AS customer, c.phone AS tel, 1 AS spend FROM customers c"},
|
||||
kind="query")
|
||||
entry = {"alias": "Q1", "kind": "query", "artifact": art, "columns": ["customer", "tel", "spend"], "rows": 2,
|
||||
"node_id": "top", "source": "mall", "tool": "db_query__mall"}
|
||||
masked = await _judge_masks({"Q1": entry}, {"mall": ["phone"]})
|
||||
assert {c.lower() for c in masked["mall"]} >= {"phone", "tel"}, masked
|
||||
excerpt = judging._query_excerpt("Q1", entry, {0}, ["tel", "spend"], True, judging._Fetch(None), masked)
|
||||
assert excerpt and leaks(excerpt) == [] and "380" in excerpt, excerpt
|
||||
# 只按列名(以前的办法):号码照样在摘录里
|
||||
before = judging._query_excerpt("Q1", entry, {0}, ["tel", "spend"], True, judging._Fetch(None), {"mall": ["phone"]})
|
||||
assert leaks(before), "对照:只按列名遮时号码在摘录里"
|
||||
def judge_models(monkeypatch) -> list[str]:
|
||||
"""写作者回 JUDGED;裁判一律判有依据。返回每次交给裁判的请求原文。"""
|
||||
from app.engine import judge as judge_mod
|
||||
from app.engine.judge import JUDGE_ROLE
|
||||
from app.providers.mock_model import MockChatModel
|
||||
|
||||
seen: list[str] = []
|
||||
|
||||
def decide(model, messages):
|
||||
if JUDGE_ROLE in str(messages[0].content):
|
||||
request = str(messages[-1].content)
|
||||
seen.append(request)
|
||||
items = [{"unit": uid, "verdict": "supported", "rationale": "虚构的理由", "used": []}
|
||||
for uid in re.findall(r"^\[(u\d+)\]", request, re.M)]
|
||||
return AIMessage(content=json.dumps({"items": items}, ensure_ascii=False))
|
||||
return AIMessage(content=JUDGED)
|
||||
|
||||
monkeypatch.setattr(MockChatModel, "_decide", decide)
|
||||
|
||||
async def judge_model_of(session, spec):
|
||||
return MockChatModel(model_name="claude-sonnet-5"), "claude-sonnet-5"
|
||||
|
||||
monkeypatch.setattr(judge_mod, "get_chat_model", judge_model_of)
|
||||
return seen
|
||||
|
||||
|
||||
async def judged_run(run_class: str, **report) -> Run:
|
||||
from sqlalchemy import delete
|
||||
|
||||
from app.db.models import Setting
|
||||
from app.engine.judge import SPEND_KEY
|
||||
|
||||
async with SessionLocal() as session:
|
||||
await session.execute(delete(Setting).where(Setting.key.in_([SPEND_KEY, "judge"])))
|
||||
await session.commit()
|
||||
nodes = [node("start", "input"),
|
||||
node("q_top", "tool", tool="db_query__mall", args={"sql": SQL_WHOLE}),
|
||||
node("q_total", "tool", tool="db_query__mall", args={"sql": SQL_TOTAL}),
|
||||
node("write", "report", instructions="写一句", **report),
|
||||
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}])]
|
||||
spec = {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
|
||||
run_id = (await run_manager.start(graph=spec, input_payload={}, run_class=run_class)).id
|
||||
for _ in range(400):
|
||||
await asyncio.sleep(0.05)
|
||||
async with SessionLocal() as session:
|
||||
row = await session.get(Run, run_id)
|
||||
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
|
||||
assert row.status == "succeeded", row.error
|
||||
return row
|
||||
raise AssertionError(f"运行 {run_id} 没跑完:{row.status}")
|
||||
|
||||
|
||||
def excerpt_of(request: str) -> str:
|
||||
"""裁判请求里 Q2 那一段摘录。"""
|
||||
return request.split("【Q2】", 1)[1].split("【", 1)[0]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("where", ["on_demand", "inline"])
|
||||
async def test_the_judge_sees_the_cited_columns_of_every_snapshot(client, monkeypatch, mall, where):
|
||||
"""交给裁判的遮罩列只按数据源设的列名(judge.source_masks),不并证据面板那套宁可多遮的列:Q1 整份遮、Q2 只数个数
|
||||
的列,面板都不显示,可它们和号码无关,裁判要看得到被引用的金额、人数才判得了。按需裁判(探索运行)、节点里当场裁判
|
||||
(正式运行)同一个办法。号码列照旧不给(摘录里一个号码都没有)。"""
|
||||
seen = judge_models(monkeypatch)
|
||||
if where == "on_demand":
|
||||
row = await judged_run("exploratory", claims="judge")
|
||||
[report] = (await client.get(f"/api/runs/{row.id}/evidence")).json()["reports"]
|
||||
doc = artifact_store.load(report["doc_artifact"])
|
||||
uid = next(u["id"] for _, u in iter_units(doc)
|
||||
if any(seg.get("ref") == "v:Q2.r0.amount" for seg in u.get("segments") or []))
|
||||
res = await client.post(f"/api/runs/{row.id}/evidence/judge", json={"units": [uid]})
|
||||
assert res.status_code == 200, res.text
|
||||
else:
|
||||
await judged_run("formal", claims="judge")
|
||||
[request] = seen
|
||||
q2 = excerpt_of(request)
|
||||
assert "amount" in q2 and "with_phone" in q2 and "710.75" in q2, q2
|
||||
[q1_columns] = [line for line in request.split("【Q1】", 1)[1].split("【", 1)[0].splitlines() if line.startswith("列:")]
|
||||
assert "amount" in q1_columns and "name" in q1_columns, q1_columns
|
||||
assert leaks(request) == []
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
所以拦在自查:
|
||||
|
||||
- 组装文档时(compose_doc 的 masked:{查询快照工件: 取自遮罩列的列}),引用了这些列的格子记一条 masked_cell,列名对上
|
||||
的、按 SQL 认出改了名字的都算(报告节点按证据面板同一个函数 api/evidence.py 的 _snapshot_masks 算)
|
||||
的、按 SQL 认出改了名字的都算(报告节点按 api/evidence.py 的 _cited_masks 算,和证据面板同一个 _mask_sets 起步)
|
||||
- 违规里不带那一格的值(不写 text、context):违规会进 report.checked、重写的日志、出具缺口,这些地方不能再露一次号码
|
||||
- 报告节点把它交回写作者重写(不是只标注的那几类);重写后还在:「违规处置」为失败的判失败,为标注的照常产出,
|
||||
时间线上另起一行说清(report_masked_cell),节点产出里写一条出具缺口,出口降档
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
"""报告自查只查引用,而且只认真取自遮罩表里那一列的格子(复核第 3 轮)。
|
||||
|
||||
复核发现:
|
||||
- 第 2 轮加的按值自查把两个字的遮罩值当子串去认:客户里有一位叫「高峰」,正文写「本周销售高峰」就判 masked_cell,正式
|
||||
运行(默认违规即失败)因此失败,报错还叫人去改写作指令
|
||||
- 按值自查连同名列的值一起收:mask_columns 设了 name(本意是客户姓名),正文提到门店名(stores.name)同样判失败
|
||||
- 引用同理:遮罩按列名设在整个数据源上(不带表名),s.name AS store 按列名认也算取自遮罩列「name」
|
||||
|
||||
现在自查只认写作者用 [[v:Qn.rX.列]] 引用的格子(正文里照抄的号码本来就会被「数字没有出处」拦下),并且按 SQL 的血缘核一遍
|
||||
这一列的值取自哪张表的哪一列:遮罩列在几张表里都有、其中有的表里另有别的遮罩列时(customers 有 name 和 phone,stores 只有
|
||||
name),只算那几张表。证据面板、可验证报告照列名多遮,不受影响。
|
||||
|
||||
真跑的部分用虚构的 SQLite 零售库(号码、姓名、门店名都是编的),数据源探查过表结构(查询快照里冻结着表结构),写作者是
|
||||
MockChatModel 照剧本写。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
from langchain_core.messages import AIMessage
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.core import artifact_store
|
||||
from app.data.engine import engines
|
||||
from app.db.base import SessionLocal
|
||||
from app.db.models import DataSource, Run, RunEvent
|
||||
from app.engine.evidence import SCHEMA_SNAPSHOT, compose_doc
|
||||
from app.engine.runner import run_manager
|
||||
|
||||
SOURCE = "m2shop"
|
||||
STORES = [(1, "杭州西湖店", "华东"), (2, "广州天河店", "华南")]
|
||||
CUSTOMERS = [(1, "陈思远", "19900000801"), (2, "高峰", "19900000802"), (3, "林若溪", "19900000803")]
|
||||
ORDERS = [(1, 1, 1, 120.5), (2, 2, 2, 300.0), (3, 1, 2, 80.0), (4, 2, 3, 210.25)]
|
||||
PHONES = [c[2] for c in CUSTOMERS]
|
||||
|
||||
SQL_STORE = ("SELECT s.name AS store, ROUND(SUM(o.amount), 2) AS sales FROM stores s JOIN orders o ON o.store_id = s.id "
|
||||
"GROUP BY s.name ORDER BY sales DESC")
|
||||
SQL_CUSTOMER = ("SELECT c.name AS customer, ROUND(SUM(o.amount), 2) AS spend FROM customers c "
|
||||
"JOIN orders o ON o.customer_id = c.id GROUP BY c.name ORDER BY spend DESC")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def engine_up():
|
||||
await run_manager.setup()
|
||||
yield
|
||||
await run_manager.shutdown()
|
||||
|
||||
|
||||
async def set_source(path, options: dict, *, probe: bool = True) -> str:
|
||||
"""数据源 m2shop:probe 时探查表结构(工具查询时把它冻结进快照),不然清空。名字只这个文件用,探查的结构不会
|
||||
冻结进别的测试的快照。"""
|
||||
from app.data.introspect import introspect
|
||||
|
||||
async with SessionLocal() as session:
|
||||
row = (await session.execute(select(DataSource).where(DataSource.name == SOURCE))).scalar_one_or_none()
|
||||
if row is None:
|
||||
row = DataSource(name=SOURCE, kind="sqlite", database=str(path), readonly=True, enabled=True)
|
||||
session.add(row)
|
||||
row.database, row.kind, row.enabled, row.readonly = str(path), "sqlite", True, True
|
||||
row.options = dict(options)
|
||||
row.schema_cache = (await introspect(row)) if probe else None
|
||||
await session.commit()
|
||||
return row.id
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def shop(tmp_path, engine_up):
|
||||
path = tmp_path / "shop.db"
|
||||
db = sqlite3.connect(path)
|
||||
db.executescript("CREATE TABLE stores (id INTEGER PRIMARY KEY, name TEXT, region TEXT);"
|
||||
"CREATE TABLE customers (id INTEGER PRIMARY KEY, name TEXT, phone TEXT);"
|
||||
"CREATE TABLE orders (id INTEGER PRIMARY KEY, store_id INTEGER, customer_id INTEGER, amount REAL);")
|
||||
db.executemany("INSERT INTO stores VALUES (?, ?, ?)", STORES)
|
||||
db.executemany("INSERT INTO customers VALUES (?, ?, ?)", CUSTOMERS)
|
||||
db.executemany("INSERT INTO orders VALUES (?, ?, ?, ?)", ORDERS)
|
||||
db.commit()
|
||||
db.close()
|
||||
source_id = await set_source(path, {"mask_columns": ["name", "phone"]})
|
||||
await engines.invalidate(source_id)
|
||||
yield path
|
||||
await set_source(path, {}, probe=False)
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
def node(nid, ntype, **config):
|
||||
return {"id": nid, "type": ntype, "position": {"x": 0, "y": 0}, "data": {"label": nid, "config": config}}
|
||||
|
||||
|
||||
def graph(sql: str, **report) -> dict:
|
||||
nodes = [node("start", "input"), node("top", "tool", tool=f"db_query__{SOURCE}", args={"sql": sql}),
|
||||
node("write", "report", instructions="写一句", **report),
|
||||
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}])]
|
||||
return {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
|
||||
|
||||
|
||||
async def finish(monkeypatch, spec: dict, text: str) -> Run:
|
||||
from app.providers import mock_model
|
||||
|
||||
monkeypatch.setattr(mock_model.MockChatModel, "_decide", lambda self, messages: AIMessage(content=text))
|
||||
run_id = (await run_manager.start(graph=spec, input_payload={})).id
|
||||
for _ in range(400):
|
||||
await asyncio.sleep(0.05)
|
||||
async with SessionLocal() as session:
|
||||
row = await session.get(Run, run_id)
|
||||
if row.status in ("succeeded", "failed") and row.manifest_seq is not None:
|
||||
return row
|
||||
raise AssertionError(f"运行 {run_id} 没跑完:{row.status}")
|
||||
|
||||
|
||||
async def checked_of(run_id: str) -> dict:
|
||||
async with SessionLocal() as session:
|
||||
q = select(RunEvent).where(RunEvent.run_id == run_id, RunEvent.type == "report.checked")
|
||||
return (await session.execute(q)).scalars().one().data
|
||||
|
||||
|
||||
def masked_of(violations: list[dict]) -> list[dict]:
|
||||
return [v for v in violations if v["code"] == "masked_cell"]
|
||||
|
||||
|
||||
def leaks(value) -> list[str]:
|
||||
raw = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False, default=str)
|
||||
return [p for p in PHONES if p in raw or p in raw.replace(",", "")]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 正文里写着的字不按值认
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize("sql, text", [
|
||||
# 客户里有一位叫「高峰」:常用词「销售高峰」不是在写这位客户
|
||||
(SQL_CUSTOMER, "本周销售高峰出现在周五,最高一位客户消费 [[v:Q1.r0.spend]]。"),
|
||||
# 设了 name 本意是客户姓名:正文用文字写门店名照常
|
||||
(SQL_STORE, "销售额最高的门店是广州天河店,销售额 [[v:Q1.r0.sales]]。"),
|
||||
])
|
||||
async def test_words_that_happen_to_equal_masked_values_do_not_fail_a_formal_report(monkeypatch, shop, sql, text):
|
||||
"""违规即失败(正式运行的默认值):正文里的字和遮罩列中的值相同,不判 masked_cell,运行照常成功。"""
|
||||
row = await finish(monkeypatch, graph(sql, on_violation="fail"), text)
|
||||
assert row.status == "succeeded", row.error
|
||||
checked = await checked_of(row.id)
|
||||
assert masked_of(checked["violations"]) == [] and checked["ok"] is True, checked["violations"]
|
||||
|
||||
|
||||
async def test_a_phone_copied_into_the_text_is_stopped_by_the_number_check(monkeypatch, shop):
|
||||
"""照抄成正文的号码不按值认,「数字没有出处」照样拦下:交回重写、仍不改就判失败;违规里没有 masked_cell。"""
|
||||
row = await finish(monkeypatch, graph(SQL_CUSTOMER, on_violation="fail"),
|
||||
f"消费最高的客户电话 {PHONES[1]},共消费 [[v:Q1.r0.spend]]。")
|
||||
assert row.status == "failed", row.error
|
||||
checked = await checked_of(row.id)
|
||||
assert {v["code"] for v in checked["violations"]} == {"uncited_number"}, checked["violations"]
|
||||
assert "遮罩" not in (row.error or "")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 引用:只认真取自遮罩表里那一列的
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_citing_a_store_name_is_not_flagged_though_name_is_masked(monkeypatch, shop):
|
||||
"""s.name AS store:门店表只有 name、没有别的遮罩列,customers 里 name 和 phone 都在——设了 name 管的是客户姓名。"""
|
||||
row = await finish(monkeypatch, graph(SQL_STORE, on_violation="fail"),
|
||||
"销售额最高的门店是 [[v:Q1.r0.store]],销售额 [[v:Q1.r0.sales]]。")
|
||||
assert row.status == "succeeded", row.error
|
||||
assert masked_of((await checked_of(row.id))["violations"]) == []
|
||||
|
||||
|
||||
async def test_citing_a_customer_name_is_flagged_and_names_the_masked_column(monkeypatch, shop):
|
||||
"""c.name AS customer:取自客户表的 name,照旧拦下;说明只点名它取自的那一列遮罩列,不把 phone 也列进来。"""
|
||||
row = await finish(monkeypatch, graph(SQL_CUSTOMER, on_violation="flag"),
|
||||
"消费最高的客户是 [[v:Q1.r0.customer]],共消费 [[v:Q1.r0.spend]]。")
|
||||
assert row.status == "succeeded", row.error
|
||||
[v] = masked_of((await checked_of(row.id))["violations"])
|
||||
assert v["ref"] == "v:Q1.r0.customer" and "取自设置了遮罩的列「name」" in v["message"], v["message"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 认法:_cited_masks(报告节点经 _MaskedCells 调用)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
#: 冻结的表结构(introspect 的形状):stores 只有 name,customers 有 name 和 phone
|
||||
TABLES = {"stores": ["id", "name", "region"], "customers": ["id", "name", "phone"],
|
||||
"orders": ["id", "store_id", "customer_id", "amount"]}
|
||||
|
||||
|
||||
async def schema_of(tables: dict[str, list[str]]) -> str:
|
||||
return await artifact_store.put_json(
|
||||
{"source": SOURCE, "tables": {t: {"columns": [{"name": c, "type": "TEXT"} for c in cols]}
|
||||
for t, cols in tables.items()}}, kind=SCHEMA_SNAPSHOT)
|
||||
|
||||
|
||||
async def cited(sql: str, columns: list[str], *, masks: list[str], schema: bool = True,
|
||||
tables: dict[str, list[str]] = TABLES) -> dict[str, list[str]]:
|
||||
"""一份快照按报告自查认出的列:{结果列: 它取自的遮罩列}。"""
|
||||
from app.api.evidence import _cited_masks, _Masks
|
||||
|
||||
await set_source(":memory:", {"mask_columns": masks}, probe=False)
|
||||
snap = {"sql": sql, "columns": columns, "rows": [], "source": SOURCE,
|
||||
**({"schema_artifact": await schema_of(tables)} if schema else {})}
|
||||
art = await artifact_store.put_json(snap, kind="query")
|
||||
return await _cited_masks(snap, SOURCE, _Masks(), art)
|
||||
|
||||
|
||||
MIXED = ("SELECT s.name AS store, c.name AS customer, c.phone AS tel, SUM(o.amount) AS spend FROM orders o "
|
||||
"JOIN stores s ON s.id = o.store_id JOIN customers c ON c.id = o.customer_id GROUP BY s.name, c.name, c.phone")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("label, sql, columns, masks, expected", [
|
||||
("两张表的同名列在一条查询里:只认客户表的", MIXED, ["store", "customer", "tel", "spend"], ["name", "phone"],
|
||||
{"customer": ["name"], "tel": ["phone"]}),
|
||||
("只设了 name:几张表都有 name、哪张都没有别的遮罩列,照列名认", MIXED, ["store", "customer", "tel", "spend"],
|
||||
["name"], {"store": ["name"], "customer": ["name"]}),
|
||||
("直接选出门店表的 name(列名对上遮罩列)", "SELECT name, region FROM stores", ["name", "region"],
|
||||
["name", "phone"], {}),
|
||||
("* 展开门店表", "SELECT * FROM stores", ["id", "name", "region"], ["name", "phone"], {}),
|
||||
("* 展开客户表", "SELECT * FROM customers", ["id", "name", "phone"], ["name", "phone"],
|
||||
{"name": ["name"], "phone": ["phone"]}),
|
||||
("经 WITH 转两道", "WITH a AS (SELECT c.phone AS tel FROM customers c), b AS (SELECT tel AS contact FROM a) "
|
||||
"SELECT contact FROM b", ["contact"], ["phone"], {"contact": ["phone"]}),
|
||||
("列名和遮罩列一样,值却取自别的列", "SELECT region AS phone FROM stores", ["phone"], ["phone"], {}),
|
||||
("UNION:有一支取自客户表就算", "SELECT name FROM stores UNION ALL SELECT name FROM customers", ["name"],
|
||||
["name", "phone"], {"name": ["name"]}),
|
||||
])
|
||||
async def test_cited_masks_follow_the_table_the_value_comes_from(engine_up, label, sql, columns, masks, expected):
|
||||
assert await cited(sql, columns, masks=masks) == expected, label
|
||||
|
||||
|
||||
async def test_without_a_frozen_schema_the_mask_is_read_by_column_name(engine_up):
|
||||
"""数据源没探查过表结构(快照里没有冻结的表结构):认不出同名的列还在哪几张表里,照列名认,门店名也算;值根本不取自
|
||||
遮罩列的(region AS phone)照样不算。"""
|
||||
assert await cited(MIXED, ["store", "customer", "tel", "spend"], masks=["name", "phone"], schema=False) == \
|
||||
{"store": ["name"], "customer": ["name"], "tel": ["phone"]}
|
||||
assert await cited("SELECT region AS phone FROM stores", ["phone"], masks=["phone"], schema=False) == {}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("sql, columns", [
|
||||
# 表值函数:拆的是号码,血缘认不出出处,照遮罩那一套认
|
||||
("SELECT x.value AS p FROM customers c, json_each(c.phone) x", ["p"]),
|
||||
# 没起别名的表达式:血缘按列名找不到这一列
|
||||
("SELECT id, UPPER(phone) FROM customers", ["id", "UPPER(phone)"]),
|
||||
])
|
||||
async def test_columns_the_lineage_cannot_place_stay_flagged(engine_up, sql, columns):
|
||||
assert set(await cited(sql, columns, masks=["name", "phone"])) == {columns[-1].lower()}
|
||||
|
||||
|
||||
async def test_the_report_node_flags_only_the_customer_column(engine_up):
|
||||
"""报告节点的自查(_MaskedCells)按同一个函数认:同一条查询里引用门店名不算、引用客户名算。"""
|
||||
from app.engine.nodes.report import _MaskedCells
|
||||
|
||||
await set_source(":memory:", {"mask_columns": ["name", "phone"]}, probe=False)
|
||||
art = await artifact_store.put_json({"sql": MIXED, "columns": ["store", "customer", "tel", "spend"],
|
||||
"rows": [["广州天河店", "高峰", PHONES[1], 300.0]], "source": SOURCE,
|
||||
"schema_artifact": await schema_of(TABLES)}, kind="query")
|
||||
catalog = {"Q1": {"alias": "Q1", "kind": "query", "artifact": art, "columns": ["store", "customer", "tel", "spend"],
|
||||
"rows": 1, "node_id": "top", "source": SOURCE}}
|
||||
doc = compose_doc("门店 [[v:Q1.r0.store]],客户 [[v:Q1.r0.customer]],消费 [[v:Q1.r0.spend]]。", catalog,
|
||||
node_id="write", run_id="r1")
|
||||
[v] = masked_of((await _MaskedCells(catalog)(doc))["violations"])
|
||||
assert v["ref"] == "v:Q1.r0.customer" and "「name」" in v["message"] and leaks(v) == [], v
|
||||
@@ -236,8 +236,8 @@ async def test_the_rewrite_request_says_the_value_must_not_be_written_in_any_for
|
||||
|
||||
|
||||
async def test_only_the_cited_snapshots_are_read_and_each_only_once(engine_up, mall, monkeypatch):
|
||||
"""组装好一稿之后才读:引用到的那几份、数据源现在设了遮罩的那几份(按值认要收里面的值),每份只读一次,重写的稿子
|
||||
不再读;没设遮罩的数据源上的一份都不读(test_report_masked_values)。"""
|
||||
"""组装好一稿之后才读:只读正文引用到的那几份,每份只读一次,重写的稿子不再读;同一个数据源上没被引用的快照不读
|
||||
(第 2 轮按值认时要把设了遮罩的都读一遍,第 3 轮去掉了按值认)。"""
|
||||
from app.core import artifact_store as store
|
||||
from app.engine.nodes.report import _MaskedCells
|
||||
|
||||
@@ -259,10 +259,11 @@ async def test_only_the_cited_snapshots_are_read_and_each_only_once(engine_up, m
|
||||
monkeypatch.setattr(store, "load", counting)
|
||||
check = _MaskedCells(catalog)
|
||||
first = await check(compose_doc("电话 [[v:Q1.r0.tel]]。", catalog, node_id="write", run_id="r1"))
|
||||
assert masked_of(first["violations"]) == ["v:Q1.r0.tel"] and sorted(read) == sorted([cited, other]), read
|
||||
assert masked_of(first["violations"]) == ["v:Q1.r0.tel"] and read == [cited], read
|
||||
second = await check(compose_doc("客户 [[v:Q1.r0.customer]],电话 [[v:Q1.r0.tel]]。", catalog, node_id="write",
|
||||
run_id="r1"))
|
||||
assert masked_of(second["violations"]) == ["v:Q1.r0.tel"] and len(read) == 2, "同一份快照不再读第二遍"
|
||||
assert masked_of(second["violations"]) == ["v:Q1.r0.tel"] and read == [cited], "同一份快照不再读第二遍"
|
||||
assert other not in read
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
@@ -280,4 +281,4 @@ async def test_the_audit_lists_a_masked_cell_on_its_own_row(client, monkeypatch,
|
||||
# 这一行自己不写值(所在的句子是报告原文,标注之后照常产出,读者本来就看得到)
|
||||
assert leaks([flagged[0][k] for k in ("text", "ref", "note")]) == []
|
||||
csv = (await client.get(f"/api/runs/{row.id}/evidence/audit", params={"format": "csv"})).text
|
||||
assert "写进了遮罩列的值" in csv
|
||||
assert "引用了遮罩列" in csv
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""报告自查(第 2 轮):只数个数的列不算取自遮罩列;正文里直接写着的遮罩值按值认出来。
|
||||
"""报告自查:只数个数的列不算取自遮罩列;正文里写着的值不按值认。
|
||||
|
||||
复核发现:
|
||||
- 只数个数、只决定算不算进来的写法——COUNT(*) FILTER (WHERE phone IS NULL)、SUM(phone IS NULL)、AVG(phone IS NOT NULL)、
|
||||
@@ -6,16 +6,15 @@
|
||||
条件、汇总里的真假条件当成值的来历,自查跟着判 masked_cell,正式运行(默认违规即失败)因此判失败。等价的
|
||||
SUM(CASE WHEN phone IS NULL THEN 1 ELSE 0 END) 本来就不算。现在自查认之前先把这几种写法改写成认得出的样子
|
||||
(api/evidence.py 的 _counted),证据面板、推演照可验证报告的认法多遮(展示上多遮无妨)
|
||||
- 自查只认 [[v:]] 引用:写作者照着「删掉引用」把号码、姓名照抄成正文,或者经指标带出来([[m:phone_raw]] 的值就是号码),
|
||||
一条违规都没有。现在按值认一遍(engine/evidence.py 的 masked_values):写作目录里取自遮罩列的值(只认确定取自遮罩列的
|
||||
列,整份算进来的不算)出现在正文里,记一条 masked_cell,不带值;运行输入里写着的值是发起运行的人自己填的,不算
|
||||
- 第 2 轮另按值认过正文里写着的遮罩值(照抄的号码、姓名,经指标带出来的值),第 3 轮去掉了:两个字的姓名会撞上「销售
|
||||
高峰」这类常用词,同名列的值会撞上门店名,正式运行因此判失败,代价远大于收益;照抄的号码本来就会被「数字没有出处」
|
||||
拦下(test_report_masked_owners)。自查只查 [[v:]] 引用,这里留着不按值认的回归
|
||||
|
||||
真跑的部分用虚构的 SQLite 商城库(号码、姓名是编的),写作者是 MockChatModel 照剧本写。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
@@ -26,7 +25,7 @@ from app.core import artifact_store
|
||||
from app.data.engine import engines
|
||||
from app.db.base import SessionLocal
|
||||
from app.db.models import DataSource, Run, RunEvent
|
||||
from app.engine.evidence import compose_doc, describe_violations
|
||||
from app.engine.evidence import compose_doc
|
||||
from app.engine.runner import run_manager
|
||||
|
||||
CUSTOMERS = [(1, "陈思远", "19900000701"), (2, "林若溪", "19900000702"), (3, "周以宁", "19900000703")]
|
||||
@@ -70,12 +69,6 @@ async def mall(tmp_path, engine_up):
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
def leaks(value) -> list[str]:
|
||||
"""写着的号码、姓名,按千分位写的(19,900,000,701)也算。"""
|
||||
raw = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False, default=str)
|
||||
return [p for p in [*PHONES, *NAMES] if p in raw or p in raw.replace(",", "")]
|
||||
|
||||
|
||||
def masked_of(violations: list[dict]) -> list[dict]:
|
||||
return [v for v in violations if v["code"] == "masked_cell"]
|
||||
|
||||
@@ -182,7 +175,7 @@ async def test_a_formal_report_citing_a_count_of_missing_phones_succeeds(monkeyp
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 正文里直接写着的遮罩值
|
||||
# 正文里写着的值不按值认
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
@@ -198,77 +191,49 @@ TOP_ROWS = [[NAMES[1], PHONES[1], 380.0], [NAMES[0], PHONES[0], 220.0]]
|
||||
|
||||
@pytest.mark.parametrize("text", [
|
||||
f"消费最高的客户电话是 {PHONES[1]}。",
|
||||
# 换了写法的号码也认
|
||||
"消费最高的客户电话是 199-0000-0702。",
|
||||
f"消费最高的客户电话是 +86 {PHONES[1][:3]} {PHONES[1][3:7]} {PHONES[1][7:]}。",
|
||||
])
|
||||
async def test_a_masked_value_copied_into_the_text_is_flagged(engine_up, mall, text):
|
||||
"""写作者把号码照抄成正文(没写成引用):记一条 masked_cell(按值认出来的),不带值;交回写作者的那句说清在哪一句、
|
||||
值不能以任何形式写进报告,句子里的值换成「已遮罩」。"""
|
||||
async def test_a_value_copied_into_the_text_is_not_a_masked_cell(engine_up, mall, text):
|
||||
"""写作者把号码照抄成正文(没写成引用):自查不按值认,不记 masked_cell;裸数字由数字核对记 uncited_number。"""
|
||||
from app.engine.nodes.report import _MaskedCells
|
||||
|
||||
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
|
||||
doc = await _MaskedCells(catalog)(compose_doc(text, catalog, node_id="write", run_id="r1"))
|
||||
[v] = masked_of(doc["violations"])
|
||||
assert v["by_value"] is True and v["segment"] and v["unit"] and "ref" not in v, v
|
||||
assert "设置了遮罩的列中的值" in v["message"] and "不能以任何形式写进报告" in v["for_model"]
|
||||
assert "已遮罩" in v["context"] and leaks(v) == [] and leaks(describe_violations([v])) == []
|
||||
assert masked_of(doc["violations"]) == [] and any(v["code"] == "uncited_number" for v in doc["violations"])
|
||||
|
||||
|
||||
async def test_a_text_masked_column_copied_as_text_is_flagged(engine_up, mall):
|
||||
"""文字类的遮罩列(姓名):数字核对管不到它,按值认才拦得住。"""
|
||||
async def test_a_name_copied_as_text_is_not_flagged(engine_up, mall):
|
||||
"""文字类的遮罩列(姓名)照抄成正文:同样不按值认(两个字的姓名会撞上常用词)。"""
|
||||
from app.engine.nodes.report import _MaskedCells
|
||||
|
||||
await set_source(mall, {"mask_columns": ["name"]})
|
||||
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
|
||||
doc = await _MaskedCells(catalog)(compose_doc(f"消费最高的是{NAMES[1]},花了 [[v:Q1.r0.spend]]。", catalog,
|
||||
node_id="write", run_id="r1"))
|
||||
[v] = masked_of(doc["violations"])
|
||||
assert v["by_value"] is True and leaks(v) == [], v
|
||||
assert masked_of(doc["violations"]) == [], doc["violations"]
|
||||
|
||||
|
||||
async def test_a_cited_masked_cell_is_not_counted_twice(engine_up, mall):
|
||||
"""[[v:Q1.r0.tel]] 已经按引用记过:同一处不再按值另记一条。"""
|
||||
async def test_a_cited_masked_cell_is_flagged_once(engine_up, mall):
|
||||
"""[[v:Q1.r0.tel]]:按引用记一条,没有按值另记的那一类(by_value)。"""
|
||||
from app.engine.nodes.report import _MaskedCells
|
||||
|
||||
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
|
||||
doc = await _MaskedCells(catalog)(compose_doc("电话 [[v:Q1.r0.tel]]。", catalog, node_id="write", run_id="r1"))
|
||||
[v] = masked_of(doc["violations"])
|
||||
assert v["ref"] == "v:Q1.r0.tel" and not v.get("by_value"), v
|
||||
assert v["ref"] == "v:Q1.r0.tel" and "by_value" not in v, v
|
||||
|
||||
|
||||
async def test_values_only_conservatively_masked_or_given_as_run_input_are_not_flagged(engine_up, mall):
|
||||
"""不算违规的:SQL 认不准、整份算进来的列(姓名没设遮罩,只是面板多遮);运行输入里写着的号码(发起运行的人自己填的);
|
||||
数据源没设遮罩。"""
|
||||
from app.engine.nodes.report import _MaskedCells
|
||||
|
||||
unsure = ("SELECT name, amount FROM (SELECT c.name, UPPER(c.phone), o.amount FROM customers c "
|
||||
"JOIN orders o ON o.customer_id = c.id) t ORDER BY amount DESC")
|
||||
catalog = await masked_catalog(unsure, ["name", "amount"], [[NAMES[1], 300.0]])
|
||||
doc = await _MaskedCells(catalog)(compose_doc(f"金额最大的一笔来自{NAMES[1]}。", catalog, node_id="write", run_id="r1"))
|
||||
assert masked_of(doc["violations"]) == []
|
||||
|
||||
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
|
||||
asked = _MaskedCells(catalog, protected=[json.dumps({"phone": PHONES[1]})])
|
||||
doc = await asked(compose_doc(f"号码 {PHONES[1]} 的客户消费最高。", catalog, node_id="write", run_id="r1"))
|
||||
assert masked_of(doc["violations"]) == []
|
||||
|
||||
await set_source(mall, {})
|
||||
doc = await _MaskedCells(catalog)(compose_doc(f"电话 {PHONES[1]}。", catalog, node_id="write", run_id="r1"))
|
||||
assert masked_of(doc["violations"]) == []
|
||||
|
||||
|
||||
async def test_snapshots_of_unmasked_sources_are_not_read_for_values(engine_up, mall, monkeypatch):
|
||||
"""按值认要读快照:数据源现在没设遮罩的一份都不读,没引用到格子的也不为引用去读。"""
|
||||
async def test_no_snapshot_is_read_without_a_cited_cell(engine_up, mall, monkeypatch):
|
||||
"""正文没引用格子:数据源设了遮罩也一份快照都不读(以前按值认要把设了遮罩的快照都读一遍)。"""
|
||||
from app.core import artifact_store as store
|
||||
from app.engine.nodes.report import _MaskedCells
|
||||
|
||||
await set_source(mall, {})
|
||||
catalog = await masked_catalog(SQL_TOP, ["customer", "tel", "spend"], TOP_ROWS)
|
||||
read: list[str] = []
|
||||
real = store.load
|
||||
monkeypatch.setattr(store, "load", lambda a: read.append(a) or real(a))
|
||||
await _MaskedCells(catalog)(compose_doc("一共三位客户。", catalog, node_id="write", run_id="r1"))
|
||||
await _MaskedCells(catalog)(compose_doc(f"一共三位客户,电话 {PHONES[0]}。", catalog, node_id="write", run_id="r1"))
|
||||
assert read == [], read
|
||||
|
||||
|
||||
@@ -287,24 +252,12 @@ def card_graph(**report) -> dict:
|
||||
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}]))
|
||||
|
||||
|
||||
async def test_a_metric_whose_value_is_a_masked_value_is_flagged(monkeypatch, mall):
|
||||
"""[[m:phone_raw]] 的值就是号码:按值认出来,点名那处引用;取客户编号的 [[m:id3]] 照常。标注照常产出时,时间线那一行
|
||||
说清有几处是正文里写着的值,缺口、日志都不写值。"""
|
||||
row = await finish(monkeypatch, card_graph(on_violation="flag"), "第三位客户编号 [[m:id3]],首位客户电话 [[m:phone_raw]]。")
|
||||
async def test_a_metric_reference_is_not_checked_by_value(monkeypatch, mall):
|
||||
"""自查只查 [[v:]] 引用:[[m:phone_raw]] 的值就是号码,自查不按值认(指标的值在证据面板、推演里另有判断,写「已遮罩」,
|
||||
见 test_masked_metric_values)。违规即失败的运行照常成功,时间线上没有「引用了遮罩列」那一行。"""
|
||||
row = await finish(monkeypatch, card_graph(on_violation="fail"), "第三位客户编号 [[m:id3]],首位客户电话 [[m:phone_raw]]。")
|
||||
assert row.status == "succeeded", row.error
|
||||
rows = await events(row.id)
|
||||
checked = next(e.data for e in rows if e.type == "report.checked")
|
||||
[v] = masked_of(checked["violations"])
|
||||
assert v["ref"] == "m:phone_raw" and v["by_value"] is True and "[[m:phone_raw]]" in v["message"], v
|
||||
[log] = [e.data for e in rows if e.type == "log" and e.data.get("code") == "report_masked_cell"]
|
||||
assert log["count"] == 1 and log["written"] == 1 and log["refs"] == ["m:phone_raw"], log
|
||||
finished = next(e.data for e in rows if e.type == "node.finished" and e.node_id == "write")
|
||||
[gap] = artifact_store.load(finished["artifact"])["issuance_gaps"]
|
||||
assert "设置了遮罩的列中的值" in gap and leaks([checked["violations"], log, gap]) == [], (gap, log)
|
||||
|
||||
|
||||
async def test_a_formal_report_copying_a_phone_fails_with_advice_about_the_instructions(monkeypatch, mall):
|
||||
row = await finish(monkeypatch, card_graph(on_violation="fail", numbers="off"),
|
||||
f"首位客户电话 {PHONES[0]},第三位客户编号 [[m:id3]]。")
|
||||
assert row.status == "failed" and "设置了遮罩的列中的值" in row.error, row.error
|
||||
assert "指令中去掉" in row.error and "指令遵循能力" not in row.error and leaks(row.error) == [], row.error
|
||||
assert masked_of(checked["violations"]) == [], checked["violations"]
|
||||
assert not [e for e in rows if e.type == "log" and e.data.get("code") == "report_masked_cell"]
|
||||
|
||||
@@ -248,18 +248,14 @@ export const ABSENCE_CLAIM_TEXT = {
|
||||
}
|
||||
|
||||
/**
|
||||
* 报告写进了取自遮罩列的值(报告自查的 masked_cell):直接引用了那一格([[v:Q2.r0.tel]],c.phone AS tel 这种改了名字的
|
||||
* 也算),或者按值认出正文里直接写着、经指标带出来的(违规带 by_value),重写后仍在、「违规处置」为标注时照常产出:
|
||||
* 时间线上那一行(日志 report_masked_cell,written 是其中按值认出来的几处)和证据面板违规清单里的叫法。
|
||||
* 报告直接引用了取自遮罩列的格子(报告自查的 masked_cell:[[v:Q2.r0.tel]],c.phone AS tel 这种改了名字的也算,别的表里
|
||||
* 同名的列不算),重写后仍在、「违规处置」为标注时照常产出:时间线上那一行(日志 report_masked_cell)和证据面板违规清单
|
||||
* 里的叫法。自查只查引用,不按值在正文里找遮罩值。
|
||||
* 只说记为缺口,不说出具:探索运行、没设出具契约的成果不出具,报告接在侧枝上时缺口也进不了出具,降不降档由出具那一行说
|
||||
*/
|
||||
export const MASKED_CELL_TEXT = {
|
||||
label: '写进了遮罩列的值',
|
||||
title: (n?: number, written = 0) => {
|
||||
if (!written) return `报告引用了${n ? ` ${formatNumber(n)} 处` : ''}取自遮罩列的格子,已标注`
|
||||
if (n && n > written) return `报告引用、写进了 ${formatNumber(n)} 处取自遮罩列的值,已标注`
|
||||
return `报告写进了 ${formatNumber(written)} 处设置了遮罩的列中的值,已标注`
|
||||
},
|
||||
label: '引用了遮罩列',
|
||||
title: (n?: number) => `报告引用了${n ? ` ${formatNumber(n)} 处` : ''}取自遮罩列的格子,已标注`,
|
||||
next: '数据源设置了遮罩的列不应写进报告,这几处已记为缺口。请在报告撰写节点的指令中去掉对这些列的引用后重新运行',
|
||||
}
|
||||
|
||||
|
||||
@@ -874,13 +874,12 @@ function explainLog(code: string | undefined, message: string, d: Record<string,
|
||||
}
|
||||
}
|
||||
case 'report_masked_cell': {
|
||||
// 报告写进了取自遮罩列的值,重写后仍在、照常产出(report.py):读结构化字段 count、written(其中按值认出、正文里
|
||||
// 直接写着的几处)、refs,副标题只点名引用,不写那几格的值(后端也没给)
|
||||
// 报告引用了取自遮罩列的格子,重写后仍在、照常产出(report.py):读结构化字段 count、refs,副标题只点名引用,
|
||||
// 不写那几格的值(后端也没给)
|
||||
const n = typeof d.count === 'number' && d.count > 0 ? d.count : undefined
|
||||
const written = typeof d.written === 'number' && d.written > 0 ? d.written : 0
|
||||
const refs = strs(d.refs)
|
||||
return {
|
||||
title: MASKED_CELL_TEXT.title(n, written),
|
||||
title: MASKED_CELL_TEXT.title(n),
|
||||
sub: refs.length ? refs.map((r) => `[[${r}]]`).join('、') : undefined,
|
||||
next: MASKED_CELL_TEXT.next,
|
||||
}
|
||||
|
||||
+10
-17
@@ -1377,14 +1377,14 @@ await section('报告核对 report.checked、口径卡的台账(可点击证
|
||||
&& !!rcAbsent?.detail?.includes('缺失断言') && !!rcAbsent?.detail?.includes('依据不足'), `${rcAbsent?.status} ${rcAbsent?.detail}`)
|
||||
})
|
||||
|
||||
await section('报告写进了取自遮罩列的值(report_masked_cell):标题说几处、已标注,副标题点名引用,不写那几格的值', async () => {
|
||||
// 契约:data = {level: warn, code: report_masked_cell, message, count, refs, written, columns},重写后仍在、「违规处置」
|
||||
// 为标注时才发。written 是其中按值认出来的几处(正文里直接写着的、经指标带出来的,违规带 by_value)
|
||||
await section('报告引用了取自遮罩列的格子(report_masked_cell):标题说几处、已标注,副标题点名引用,不写那几格的值', async () => {
|
||||
// 契约:data = {level: warn, code: report_masked_cell, message, count, refs, columns},重写后仍在、「违规处置」
|
||||
// 为标注时才发。自查只查 [[v:]] 引用,不按值认正文里写着的值(复核第 3 轮去掉了按值认,日志也不再带 written)
|
||||
const E = (seq, type, node_id, data = {}) => ({ seq, type, node_id, ts: 1790000000 + seq, data })
|
||||
const reportPy = readFileSync(`${root}backend/app/engine/nodes/report.py`, 'utf8')
|
||||
check('后端发写进了遮罩列的值的警告,带着 count、refs、written、columns', reportPy.includes('code="report_masked_cell"')
|
||||
&& reportPy.includes('count=len(hits), refs=refs') && reportPy.includes('written=sum(')
|
||||
&& reportPy.includes('columns=list(dict.fromkeys('),
|
||||
check('后端发引用了遮罩列的警告,带着 count、refs、columns,不再带按值认的 written', reportPy.includes('code="report_masked_cell"')
|
||||
&& reportPy.includes('count=len(hits), refs=refs') && reportPy.includes('columns=list(dict.fromkeys(')
|
||||
&& !reportPy.includes('written=sum('),
|
||||
'改了 report.py 的字段就同步改 decode.ts 的 explainLog 和这里的夹具')
|
||||
const MASKED = '报告引用了 2 处取自遮罩列的格子(「v:Q2.r0.tel」、「v:Q2.r1.tel」),已要求模型重写 1 次仍未改掉:'
|
||||
+ '数据源设置了遮罩的列不应写进报告。报告照常产出,这几处已标注,已记为缺口。请在报告撰写节点的指令中去掉对这些列的引用'
|
||||
@@ -1413,17 +1413,10 @@ await section('报告写进了取自遮罩列的值(report_masked_cell):
|
||||
.find((s) => s.code === 'report_masked_cell')
|
||||
check('……缺字段时标题照样说人话,不出 undefined', bare?.title === '报告引用了取自遮罩列的格子,已标注'
|
||||
&& bare?.sub === undefined, `${bare?.title} / ${bare?.sub}`)
|
||||
// 按值认出来的:正文里直接写着号码、姓名,或者经指标带了出来([[m:phone_raw]])。标题不说「引用了格子」
|
||||
const titleOf = (data) => flatten(mod.decodeRun([E(1, 'log', 'w', { level: 'warn', code: 'report_masked_cell',
|
||||
message: '报告写进了遮罩列的值', ...data })])).find((s) => s.code === 'report_masked_cell')
|
||||
const onlyWritten = titleOf({ count: 1, written: 1, refs: [] })
|
||||
check('……全是正文里直接写着的:标题说写进了几处遮罩列中的值,没有引用可点名就不写副标题',
|
||||
onlyWritten?.title === '报告写进了 1 处设置了遮罩的列中的值,已标注' && onlyWritten?.sub === undefined,
|
||||
`${onlyWritten?.title} / ${onlyWritten?.sub}`)
|
||||
const viaMetric = titleOf({ count: 1, written: 1, refs: ['m:phone_raw'] })
|
||||
check('……经指标带出来的:副标题点名那处引用', viaMetric?.sub === '[[m:phone_raw]]', viaMetric?.sub)
|
||||
const both = titleOf({ count: 3, written: 1, refs: ['v:Q2.r0.tel', 'v:Q2.r1.tel'] })
|
||||
check('……两样都有:标题说引用、写进了几处', both?.title === '报告引用、写进了 3 处取自遮罩列的值,已标注', both?.title)
|
||||
// 第 2 轮的日志带过 written(按值认出的几处):现在的后端不再发,万一读到也只按引用说,不改标题
|
||||
const legacy = flatten(mod.decodeRun([E(1, 'log', 'w', { level: 'warn', code: 'report_masked_cell',
|
||||
message: '报告引用了遮罩列', count: 1, written: 1, refs: ['v:Q2.r0.tel'] })])).find((s) => s.code === 'report_masked_cell')
|
||||
check('……带着 written 的日志照引用说', legacy?.title === '报告引用了 1 处取自遮罩列的格子,已标注', legacy?.title)
|
||||
})
|
||||
|
||||
await section('整句面板挂的依据:按证据图的标签写,不读文档目录里带值的标签(citeLabel)', async () => {
|
||||
|
||||
Reference in New Issue
Block a user