K3 复核修正(第 1 轮):后面的查询按此前遮掉的值遮、质询工件照回答时记下的位置重放;表值函数拆开遮罩列也认;汇总遮罩列的指标不派盲算;反方理由按值遮

复核发现质询工件接口相对基线回退:agent 自己的查询快照在工具边界遮好之后,事后取质询工件收不到它另一次查询里的号码,
没设遮罩的备注、联系方式列原样给出。另有几处模型照样看得到遮罩值、或者遮多了的地方。

- 工具边界(tools/datasource、engine/masking.mask_query_result):此前已经认出的遮罩值(这个 agent 前面的查询遮掉的、
  调用方交进来的)也用来按值遮后面的结果,数据源没设遮罩的也一样。审批质询、反方审稿在派 agent 之前把报告快照里的
  遮罩值交进来(inquiry.snapshot_secrets,run_verify_agent 的 masked_values),模型从第一条查询起就看不到。已经摆在
  读者眼前的值(提问人写的、报告指标、被审的结论原文)跟着并进来的值一起免遮(_Secrets.protected);这份结果自己写
  的分档标签不按前面收的值换。
- 先查备注、后查出号码的:回答时只靠内存里的值才换掉的地方按快照记成位置(不含值,masking.view_edits),写进质询
  事件的 masked_cells,质询工件接口先照着重放(replay_edits)再按原来的办法遮。工具边界按值换掉的整格照样画成遮罩列。
  agent 查到的值先收、报告快照的后收:上限收满时不丢 agent 自己查到的。
- _derived_hidden 认 FROM、JOIN、LATERAL 里拆开遮罩列的表值函数(json_each、json_tree、unnest、jsonb_each_text、
  regexp_split_to_table、JSON_TABLE、LATERAL VALUES):按出列名认,不认得的函数又没写列名表的整份遮。
- 汇总里的 CASE 只在条件里用到遮罩列的不算取自遮罩列(同写在 WHERE 里)。双盲复算按同一套认法认出指标取自遮罩列
  (SUM(amount)、amount 设了遮罩)时,派盲算之前就记为无法复算并说清原因,不白花一次盲算(nodes/dual_check 的
  _query_input,改动在文件所有权之外,只加这一处判断)。
- 反方审稿:理由正文和理由文档按值换掉遮罩值(换了写法的也认),这句结论原文里写着的不换;重新组装时引用到的格子按
  遮好的快照渲染。
- 换了写法的号码另认不换行空格、窄空格、细空格、制表符、减号,最多四个分隔符;SQL 的字符串、带引号的名字、注释里
  也认(不带引号的数照旧是算式),字符串里换掉的号码同一条 SQL 里写成阈值的也不留;列名同样认。
- 查询失败时报错原话里的值换掉(mask_error_text:引号里、不带引号的长数,SQL 里写着的照留)。
- 快照的 mask_columns 只记数据源设的列名,改了名字的结果列只在 masked_at_query:证据面板并各份快照的这一项从表的
  字段清单里去列,混进别名会误藏同源别的表里同名的真实字段。
- * 展开数据源里的表时按冻结的表结构认它带不带同名列,没有的照常认分档标签(J2 收紧之后的多遮)。
- 格子引用指到按值换掉的格子时照实说;上万个值的合并、免遮放进线程。

测试:质询 agent 先查号码后查备注、先查备注后查号码、另一张表的号码列两种先后,质询工件、通用工件接口、事件都没有
原值;报告快照里的值在第一条查询就遮;表值函数(SQLite、Postgres、MySQL 的写法)、汇总里 CASE 的条件、* 连着的表按
表结构认标签、新的分隔符、SQL 字符串和注释和列名、报错原话、mask_columns 不混别名、遮罩列汇总的指标不派盲算、反方
理由按值遮且免遮结论原文、上限收满时 agent 的值先收。逐处去掉实现(24 处)对应的用例都会变红。
This commit is contained in:
ThearchyHelios
2026-10-06 19:11:14 +08:00
parent 450758185d
commit 099473cd6d
13 changed files with 1387 additions and 193 deletions
+126 -81
View File
@@ -13,9 +13,12 @@
自己查询的格子,或者报告里已有的指标、运行输入;回答文档由 compose_doc 组装,裸数字被标出来(提问人自己写的数除外)。
封存之后的追问查的是数据源现在的数据,任务里给报告生成的时间,回答末尾注明;触顶、查询失败、引用解析不了也在末尾注明
3. 遮罩(_plan):agent 的查询结果在核查 agent 的工具边界就遮好了(engine/masking.py 的 mask_query_result):取自
遮罩列的格子交给模型、落进快照之前就是「已遮罩」,模型看不到遮罩值。这里是第二道防线:查询时遮掉的值由 verify_agent
交回(masked_values,只在内存里),连同报告自己快照里的遮罩值,照下面的办法按值遮;事后取质询工件时内存里的值已经
不在,SQL 检查结果照事件里记下的 SQL 换(_recorded_swaps)。
遮罩列的格子交给模型、落进快照之前就是「已遮罩」,模型看不到遮罩值。报告自己快照里的遮罩值在派 agent 之前就交给
查询工具(snapshot_secrets → run_verify_agent 的 masked_values),agent 前面的查询遮掉的值也一样:后面查到的备注、
别的表里写着同一个号码的,交给模型、落库之前就按值换掉。这里是第二道防线:查询时遮掉的值由 verify_agent 交回
(masked_values,只在内存里),连同报告自己快照里的遮罩值,照下面的办法按值遮。事后取质询工件时内存里的值已经
不在:回答时只靠它们才换掉的地方(先查备注、后查出号码的那份备注快照)记成位置写进事件(REPLAY_KEY,不含值,
masking.view_edits),取工件时照着再换一遍;SQL 检查结果照事件里记下的 SQL 换(_recorded_swaps)。
哪些列要遮照证据面板的规则(查询当时记下的,加上数据源现在设的),另按 SQL 认出改了名字的遮罩列
(起别名、套函数、经子查询转一道、和 * 写在同一支里,_derived_hidden;认不出落在哪一列的,整份快照遮掉;按遮罩列
分档、当分组键或套在汇总里数个数的不算,_band_only);快照按列遮,再按值遮(别的列里写着遮罩值的格子,_masked_view;质询
@@ -24,8 +27,9 @@
认用在哪(_literal_uses),数字照原样留着的只有范围比较的边界(没把某一个值夹住的,_pinned)和 LIMIT 的行数,而且
同一个数在这条 SQL 别处没被换掉,封存的是实际执行的语句;在找某一个值的(等值、IN、简单 CASE、夹住某一个值的
范围……,值换个写法、写成几个数的算式也认)和别处够长的遮罩值都换。已经摆在读者眼前的值不换:问题和选中原文里的、
报告列出的指标和运行输入的值;SQL 里照原样留着的阈值只在回答正文里不换,格子照旧换。回答正文、质询工件的文字格里,
换了写法的长号码(199-1474-2785、(199)1474-2785、+86 199 1474 2785、全角数字)按纯数字写法认(masking 的
报告列出的指标和运行输入的值(agent 后面的查询遮掉的值照样免遮,_Secrets.protected);SQL 里照原样留着的阈值只在
回答正文里不换,格子照旧换。回答正文、质询工件的文字格、SQL 的字符串、带引号的名字和注释里,换了写法的长号码
(199-1474-2785、(199)1474-2785、+86 199 1474 2785、全角数字、不换行空格或减号隔开的)按纯数字写法认(masking 的
_Secrets.loose_spans)。组装之后还认得出遮罩值的(换了写法的也算),这份回答不交出去
4. 写入(_record):一条 approval.inquiry 事件,node_id 为空,用量写在数据的 usage 里(计进运行用量,见 runner 的
_usage_of_events)。质询的查询快照只放在 inquiry_artifacts / queries 里,绝不放在 query_artifact 下:证据接口按那个键
@@ -33,7 +37,8 @@
append_events 不写,回答照样交回给提问的人,说明没有写进运行记录(这样的回答在进程里留一阵,_HELD)。触顶、失败
照样记一条
5. 读取:列表读 approval.inquiry 事件(封存前后的都算,按先后);质询工件只认质询引用过的(查询快照、回答文档、回答里
引用的报告口径卡),遮罩照回答时的办法,快照里 SQL 检查结果的片段、说明照整条 SQL 的换法换(_masked_checks)
引用的报告口径卡),先照事件里记下的位置重放回答时换掉的地方,再照回答时的办法遮,快照里 SQL 检查结果的片段、说明
照整条 SQL 的换法换(_masked_checks)
不发 tool.start、llm.start 这类事件(run_verify_agent 不传 emit):界面从接口的返回取回答;runner 的 _stale_replay
只看 tool.start,质询不影响恢复后的重放。
@@ -45,17 +50,24 @@
正好等于某一笔金额)也遮掉。这是产品取舍,不按上下文去猜
- 遮罩数值列的 SUM / AVG / MIN / MAX 算派生自遮罩列,整列遮(COUNT 不算,见 _value_columns),CASE 里有一支取的是
遮罩列的也算:MIN / MAX 就是某一行的原值,只有一行的分组 SUM / AVG 也是,按列认不出哪些汇总值安全,宁可多遮。
汇总里套着结果全是写死常量的 CASE(SUM(CASE WHEN amount <= 20000 THEN 1 ELSE 0 END))只是按条件数个数,同 COUNT
核查 agent 的查询在工具边界照这个办法遮,所以这样的指标双盲复算不了(原 SQL 里认得出的,派盲算之前就记为无法
复算),反方审稿核对这类汇总数的结论也拿不到数。汇总里套着结果全是写死常量的 CASE(SUM(CASE WHEN amount <= 20000
THEN 1 ELSE 0 END))只是按条件数个数,同 COUNT;汇总里的 CASE 只在条件里用到遮罩列的(SUM(CASE WHEN buyer_phone
IS NOT NULL THEN amount ELSE 0 END))同写在 WHERE 里,不算
- FROM、JOIN、LATERAL 里的表值函数(json_each、json_tree、unnest、jsonb_each_text、JSON_TABLE、LATERAL VALUES)拆开
遮罩列的,按出列名认它出的列(常见函数的出列名、写的列名表、别名、函数名,_function_columns),同名的别的列一并遮
(多遮);不认得的函数又没写列名表的,整份快照都遮
- 按遮罩列分档的标签(CASE、IIF 的结果全是写死的常量):标签文字是 agent 写在 SQL 里的,不当遮罩值,SQL、回答正文里
照原样;标签列当分组键(每一档一行)照常显示,不分组的逐行标签列整列遮(格子里看不出某一笔落在哪一档),可 agent
在正文里写出「这一笔是大单」遮不住,按 id 分组再按档数个数也看得出(同按阈值查出某几行)。只有只装着标签的列不收:
结果的这一列在 UNION 的每一支里都是这样的 CASE / IIF,或者按作用域认得出取自子查询、CTE 里只装着标签的一列;有一支
取自数据源里的表(遮罩列原样选出、经子查询按原名传出,哪怕起了标签的名字)就照收。认不准的也照收(多遮),比如
按这一列 USING、NATURAL 合成的同名列,递归的 CTE,UNION 里落在某一支 * 里的列,写死的字符串、和列一起算的值;
派生来源里同名的几列(SQLite 都留着,外层按名字取排在最前面的那一列):标签前面有 * 带得出这一列(没写表名的 *
展开了数据源里的表就算带得出),或者有没起别名、套了括号或 COLLATE 的同名列((vip_level)、vip_level COLLATE
NOCASE),都不算只装着标签(_block_label_of、_star_label)。标签恰好和别的列里的遮罩值相同时,正文里的档名、SQL
里写成字符串的标签也换(多遮)
派生来源里同名的几列(SQLite 都留着,外层按名字取排在最前面的那一列):标签前面有 * 带得出这一列(* 展开了数据源里
的表、冻结的表结构里查不到这张表或者它有同名的列,就算带得出),或者有没起别名、套了括号或 COLLATE 的同名列
((vip_level)、vip_level COLLATE NOCASE),都不算只装着标签(_block_label_of、_star_label)。* 展开的表在表结构里
查得到、又没有同名的列时,照常按派生来源认标签(_table_lacks)。标签恰好和别的列里的遮罩值相同时,正文里的档名、
SQL 里写成字符串的标签也换(多遮)
- 「夹住某一个值」(_pinned):范围比较在整条 SQL 里配对,只有 CASE 的不同分支、SELECT 里不筛行的不同项(选出列、
GROUP BY、ORDER BY)、UNION 的不同支各算各的,两头误差都有:把找值拆进这几处(CASE WHEN age > 25 THEN … ELSE CASE
WHEN age >= 25 …,两个选出列 SUM(age >= 25)、SUM(age > 25) 再自己相减)认不出,边界照原样留着,正文里复述也不遮;
@@ -77,14 +89,18 @@
- 质询工件里 SQL 检查结果的片段是检查器按语法树重新写出来的,按整条 SQL 里整个换掉了的字面量照值换:同一个值在
片段里别处出现也换(多遮,算式里换掉的 0、1 这类也是);截断处半截的值去掉(停在字符串、带引号的名字、注释里的,
从开头的引号、注释起整个去掉)。说明里码值表列出的码值和别的字一样按值换,恰好是遮罩值的码值也换(多遮)
- 运行详情的「工件」页签、通用的工件接口按工件原样给:质询的查询快照落库时就遮好了(工具边界先遮),从那里打开也
看不到取自遮罩列的值;但工具边界只认这份结果里的遮罩值,别的查询、报告快照里遮罩列的值写在这份结果没设遮罩的列里时
(没设遮罩的联系方式列、备注),从那里打开照原样(质询工件接口按回答时的办法换)。报告自己的查询快照(写作时的查询)
仍原样存着,从那里打开不经这里的遮罩
- 遮罩值最多收 SECRETS_MAX 种写法,再多的按值认不出(按列遮照旧)
- 换了写法的号码只认数字之间夹着一到三个空格、连字符、点、括号(含全角),带 +86 / 0086 / 86 前缀、全角数字:写成
汉字(一三九……)、中间夹着别的字、拆成几处写的认不出;和别的数字只隔着一个连字符或点连着的(13900000001-2、
19914.74)不算号码;SQL 里不这样认(199-1474-2785 在 SQL 里是一个算式)
- 运行详情的「工件」页签、通用的工件接口按工件原样给:质询的查询快照落库时就遮好了(工具边界先遮),取自遮罩列的值、
报告快照里的遮罩值、agent 前面的查询里遮掉的值都看不到。只有一种照原样:先查了写着号码的备注(那时还不知道这个号码
是遮罩值),后来才查出号码——备注快照落库时原样,从通用接口打开照原样,质询工件接口照事件里记下的位置换。报告自己
的查询快照(写作时的查询)仍原样存着,从那里打开不经这里的遮罩
- 遮罩值最多收 SECRETS_MAX 种写法,再多的按值认不出(按列遮照旧)。agent 自己查到的值先收,报告快照里的后收
- 换了写法的号码只认数字之间夹着一到四个分隔符(空格、制表符、全角空格、不换行空格、窄空格这类空白,连字符、减号、
点、括号,含全角),带 +86 / 0086 / 86 前缀、全角数字:写成汉字(一三九……)、中间夹着别的字、拆成几处写的、跨行
写的认不出;和别的数字只隔着一个连字符或点连着的(13900000001-2、19914.74)不算号码;SQL 里不带引号的数不这样认
(199-1474-2785 在 SQL 里是三个数相减),字符串、带引号的名字、注释里照样认
- 质询 agent 的任务里的证据摘录照 judge.prepare 按列名遮:报告取数把遮罩列改了名字选出来(buyer_phone AS 手机号)时,
摘录里有原值,agent 看得到;它写进回答、SQL 的,按值认得出的照样换掉(报告快照里的值都收了),封存的是换过的。摘录
按 SQL 认改了名字的遮罩列要改 judge.py,另行处理
"""
from __future__ import annotations
@@ -113,7 +129,7 @@ from app.data.provenance_types import DOC_PROVENANCE
from app.db.base import SessionLocal
from app.db.models import Approval, Artifact, DataSource, Run, RunEvent
from app.engine import judge as judging
from app.engine.evidence import MARKER_RE, compose_doc, iter_segments, iter_units
from app.engine.evidence import compose_doc, iter_segments, iter_units
from app.engine.issuance import extract_numbers
# 认遮罩列、遮罩值的共用部分在 engine/masking.py。名字照旧从这里导出:nodes/dual_check、测试按 inquiry 的名字引用
from app.engine.masking import ( # noqa: F401
@@ -122,8 +138,8 @@ from app.engine.masking import ( # noqa: F401
_branch_results, _branches, _canonical, _constant, _constants, _decimal, _derived_block, _derived_hidden,
_digits_of, _fixed, _forms, _from_sources, _is_star, _item_label, _label_columns, _leaves, _literal_types,
_masked_view, _masked_word, _mentions, _merging, _numeric, _output_name, _pattern, _places, _plain_text,
_point_text, _query_shaped, _result_label, _snapshot_labels, _source_label, _source_names, _star_label, _strong,
_tainted, _value_columns, _worked_out,
_point_text, _query_shaped, _result_label, _scrubbed_source, _snapshot_labels, _source_label, _source_names,
_star_label, _strong, _tainted, _value_columns, _worked_out, replay_edits, view_edits,
)
from app.engine.merge_query import MERGE_SOURCE
from app.engine.nodes.challenge import mark_original, with_note
@@ -156,6 +172,9 @@ LIMITS = ("max_queries", "timeout_s", "max_cost_usd", "daily_max_usd")
USAGE_KEYS = ("input_tokens", "output_tokens", "cost_usd", "calls", "unpriced_calls")
#: 查询清单每条给的几项:没有行,查询结果只在快照里(取的时候按遮罩遮好)
QUERY_KEYS = ("alias", "artifact", "sql", "row_count", "truncated", "source")
#: 质询事件数据里记「回答时只靠 agent 查询时遮掉的值才换掉的位置」的键:{快照工件: masking.view_edits 的形状}。
#: 不含值;事后取质询工件时内存里的值已经不在,照着位置再换一遍(inquiry_artifact)
REPLAY_KEY = "masked_cells"
#: 回答文档存成工件时的种类;工件引用的 meta 里记下问题的前这么多字,运行的工件列表据此认得出是哪个问题的回答
ANSWER_KIND = "inquiry_answer"
QUESTION_BRIEF = 40
@@ -676,33 +695,6 @@ async def _hidden_columns(snapshot: dict[str, Any], masks: Any) -> tuple[set[str
cols, found = await masks.of(snapshot.get("source"))
return cols | _recorded_masks(snapshot), ([] if found or not snapshot.get("source") else [str(snapshot["source"])])
#: 单元格引用的前一截(Q1.r0.):后面才是列名
_CELL_HEAD = re.compile(r"^\s*[QK]\d+\.r\d+\.")
def _scrubbed_source(text: str, secrets: _Secrets) -> str:
"""回答原文里的遮罩值换成「已遮罩」:标记之外按正文的办法换(SQL 里照原样留着的阈值不换);标记里的按段换(| 隔开
的几段各自换),单元格引用只换列名那一截——编号、行号换不出东西,整段换的话「r0.」会让列名开头的长号码看着像小数
的一截,反而认不出来。列名照快照、SQL 里的名字那样换(阈值也换),换得一样才对得上。
标记里写着遮罩值的引用换完多半解析不了(列名里写着遮罩值的除外:快照的列名也换过了,正好对上),宁可弄坏一个
引用,也不让值留在存下的原文、片段里,或者从解析不了的占位里露出来。"""
if not text or not secrets.strong:
return text
out: list[str] = []
last = 0
for m in MARKER_RE.finditer(text):
out.append(secrets.scrub(text[last:m.start()], prose=True))
start, end = m.span("body")
parts = text[start:end].split("|")
head = _CELL_HEAD.match(parts[0]) if m.group("kind").strip() == "v" else None
first = parts[0][:head.end()] + secrets.scrub(parts[0][head.end():]) if head \
else secrets.scrub(parts[0], prose=True)
parts = [first, *(secrets.scrub(p, prose=True) for p in parts[1:])]
out.append(text[m.start():start] + "|".join(parts) + text[end:m.end()])
last = m.end()
out.append(secrets.scrub(text[last:], prose=True))
return "".join(out)
#: 按词认在找某一个值的比较(SQL 解析不了时的退路,见 _context):col = 'x'、col <> 'x'、col LIKE 'x'、SQLite 的
#: col IS 'x'(IN 列表另外认)
@@ -1404,9 +1396,12 @@ def _swap_literal(raw: str, text: str, kind: str, use: _Use, secrets: _Secrets,
手机号)。数换个写法照样认(25.0、025、2.5e1 按 25 认,_canonical);夹住某一个值的范围按夹住的值认(age > 24
AND age < 26 的两头都按 25 认),算式里的数按算出来的值认,同一个数在别处换掉了的阈值、行数也按等值认(见
_literals);
- 字符串在哪都按值换里面的遮罩值(范围边界也一样:写成字符串的长号码做边界,多半是在找那一个人);
- 字符串在哪都按值换里面的遮罩值(范围边界也一样:写成字符串的长号码做边界,多半是在找那一个人),换了写法的号码
也认(LIKE '%199-1474-2785%':字符串里的 199-1474-2785 不是算式);换掉的号码按纯数字写法记,同一个数写成阈值的
据此不留;
- 别的数:范围比较的边界、LIMIT / OFFSET 的行数照原样(agent 自己定的阈值,换了封存的就不是实际执行的语句);
别处(选出列、函数参数、算式)认不准是不是在找某一个值,够长的遮罩值整个换(算出来的值是也换)。"""
别处(选出列、函数参数、算式)认不准是不是在找某一个值,够长的遮罩值整个换(算出来的值是也换)。不带引号的数
不按换了写法认:199-1474-2785 在 SQL 里是三个数相减。"""
how, columns, points = use
word = _masked_word()
canonical = _canonical(text)
@@ -1418,11 +1413,9 @@ def _swap_literal(raw: str, text: str, kind: str, use: _Use, secrets: _Secrets,
found: list[str] = []
def scrub(inner: str) -> str:
pattern = secrets.pattern(inner)
if pattern is None:
return inner
found.extend(m.group() for m in pattern.finditer(inner))
return pattern.sub(word, inner)
out, swapped = secrets.swap(inner, loose=True)
found.extend(swapped)
return out
return _requote(raw, scrub), _number_forms(found)
if how not in _KEPT and any(secrets.hit(v) for v in looked):
@@ -1453,7 +1446,9 @@ def _scrub_sql(sql: str, secrets: _Secrets, compared: set[str], kind: str | None
def _rebuild(sql: str, tokens: list[Any], kinds: list[str], secrets: _Secrets, literal: Callable[[int, str], str]
) -> str:
"""按词重写一段 SQL:字面量交给 literal(第几个词、原文)定换成什么,名字(带不带引号)、注释和别的字按值换。"""
"""按词重写一段 SQL:字面量交给 literal(第几个词、原文)定换成什么,名字(带不带引号)、注释和别的字按值换。
词与词之间只有空白和注释,名字里的数也不是算式:都按文字的办法换(换了写法的号码也认,/* 客户 199-1474-2785 */、
AS "199-1474-2785 的备注")。"""
out: list[str] = []
last = 0
for i, token in enumerate(tokens):
@@ -1461,16 +1456,16 @@ def _rebuild(sql: str, tokens: list[Any], kinds: list[str], secrets: _Secrets, l
if start < last or end > len(sql):
continue
gap = sql[last:start]
out.append(secrets.scrub(gap) if gap.strip() else gap)
out.append(secrets.scrub(gap, loose=True) if gap.strip() else gap)
raw = sql[start:end]
if _is_literal(kinds[i]):
raw = literal(i, raw)
elif kinds[i] in _NAME_TOKENS:
raw = _requote(raw, secrets.scrub)
raw = _requote(raw, lambda inner: secrets.scrub(inner, loose=True))
out.append(raw)
last = end
rest = sql[last:]
out.append(secrets.scrub(rest) if rest.strip() else rest)
out.append(secrets.scrub(rest, loose=True) if rest.strip() else rest)
return "".join(out)
@@ -1515,7 +1510,7 @@ def _scrub_excerpt(text: str, whole: set[str] | None, secrets: _Secrets, kind: s
def literal(i: int, raw: str) -> str:
if tokens[i].text in whole:
return word if kinds[i] == "NUMBER" else _requote(raw, lambda _: word)
return raw if kinds[i] == "NUMBER" else _requote(raw, secrets.scrub)
return raw if kinds[i] == "NUMBER" else _requote(raw, lambda inner: secrets.scrub(inner, loose=True))
out = _rebuild(body, tokens, kinds, secrets, literal)
return out + "…" if cut else out
@@ -1603,7 +1598,8 @@ def _shown_literals(sqls: Iterable[tuple[str, str | None, set[str]]], secrets: _
class _Plan:
"""一次回答(或一次取质询工件)的遮罩:agent 每份查询快照遮好的样子(views)和遮掉的列(masked)、SQL 里指代
遮罩列的名字(tainted)、数据源的类型(kinds,SQL 按它的方言分词)、遮罩列里出现过的值(secrets),以及已经不在的
数据源(gone,按查询当时记下的遮罩处理)。"""
数据源(gone,按查询当时记下的遮罩处理)。replay:回答时只靠 agent 查询时遮掉的值(seen,只在内存里)才换掉的
地方,按快照记成位置(masking.view_edits),写进质询事件,事后取质询工件照着再换一遍;取工件时为空。"""
views: dict[str, dict[str, Any]]
masked: dict[str, list[str]]
@@ -1611,6 +1607,7 @@ class _Plan:
kinds: dict[str, str | None]
secrets: _Secrets
gone: dict[str, list[str]]
replay: dict[str, dict[str, Any]] = field(default_factory=dict)
async def _source_kinds(names: Iterable[str]) -> dict[str, str | None]:
@@ -1659,22 +1656,57 @@ def _settle_plan(own: dict[str, dict[str, Any]], named: dict[str, tuple[set[str]
kind, base = kinds.get(str(snapshot.get("source") or "")), named[artifact][0]
hidden[artifact], tainted[artifact] = _derived_hidden(snapshot, base, kind)
secrets.add(snapshot, hidden[artifact], labels=_snapshot_labels(snapshot, kind), base=base)
for snapshot, base in based:
kind = kinds.get(str(snapshot.get("source") or ""))
secrets.add(snapshot, _derived_hidden(snapshot, base, kind)[0], labels=_snapshot_labels(snapshot, kind),
base=base)
# agent 查询时遮掉的值先收(报告快照里的值收满了上限 SECRETS_MAX,也不丢 agent 自己查到的);不带它们的一份另算,
# 比出事后取质询工件时认不出、要照位置重放的地方
baseline = secrets.copy() if seen is not None and seen.strong else None
secrets.merge(seen)
secrets.protect(given)
for target in (secrets, baseline):
if target is not None:
_add_based(target, based, kinds)
target.protect(given)
secrets.show(_shown_literals([(sql, kinds.get(name), _compared(tainted, artifact, live, name))
for sql, name, artifact in shown], secrets))
secrets.prepare()
views: dict[str, dict[str, Any]] = {}
masked: dict[str, list[str]] = {}
replay: dict[str, dict[str, Any]] = {}
for artifact, snapshot in own.items():
views[artifact], masked[artifact] = _masked_view(snapshot, hidden[artifact], secrets,
embedded=artifact in embedded)
if baseline is not None and (edits := view_edits(snapshot, hidden[artifact], secrets, baseline)):
replay[artifact] = edits
return _Plan(views=views, masked=masked, tainted=tainted, kinds=kinds, secrets=secrets,
gone={artifact: named[artifact][1] for artifact in own})
gone={artifact: named[artifact][1] for artifact in own}, replay=replay)
def _add_based(secrets: _Secrets, based: list[tuple[dict[str, Any], set[str]]], kinds: dict[str, str | None]) -> None:
"""收报告自己的查询快照里遮罩列的值(based:快照和它要遮的列),按 SQL 认出改了名字的遮罩列(_derived_hidden)。"""
for snapshot, base in based:
kind = kinds.get(str(snapshot.get("source") or ""))
secrets.add(snapshot, _derived_hidden(snapshot, base, kind)[0], labels=_snapshot_labels(snapshot, kind),
base=base)
async def snapshot_secrets(snapshots: Iterable[Any], given: Iterable[str | None] = ()) -> _Secrets:
"""几份查询快照(报告自己的)里遮罩列的值,收法同回答时收报告快照(_plan),去掉已经摆在读者眼前的(given)。
审批质询、反方审稿在派核查 agent 之前交给它的查询工具(run_verify_agent 的 masked_values):agent 查到的别的表、
备注里写着同一个号码时,交给模型之前就换掉,模型看不到。"""
from app.api.evidence import _Masks
masks = _Masks()
based = [(snapshot, (await _hidden_columns(snapshot, masks))[0]) for snapshot in snapshots
if _query_shaped(snapshot)]
kinds = await _source_kinds(str(snapshot.get("source") or "") for snapshot, _ in based)
asked = list(given)
def collect() -> _Secrets:
secrets = _Secrets()
_add_based(secrets, based, kinds)
secrets.protect(asked)
secrets.prepare()
return secrets
return await asyncio.to_thread(collect)
def _compared(tainted: dict[str, set[str]], artifact: str, live: dict[str, list[str]], name: str) -> set[str]:
@@ -1760,21 +1792,24 @@ async def _answer(op: _Opening, ask: Ask, target: _Target | None, settings: dict
part = settings.get("inquiry") if isinstance(settings.get("inquiry"), dict) else {}
async with SessionLocal() as session:
spec = await verify_model_spec(session, None, settings=settings)
# 报告自己的查询快照:被问的那句可能直接写着遮罩列的值,agent 会照着复述;里面的遮罩值在派 agent 之前就交给它的
# 查询工具(masked_values),agent 查到写着同一个号码的备注、别的表时,交给模型之前就换掉
given = [ask.question, ask.quote, *_ref_values(refs)]
others = [snapshot for entry in catalog.values() if isinstance(entry, dict) and entry.get("kind") == "query"
and _query_shaped(snapshot := _quiet_load(op.loader, entry.get("artifact")))]
outcome = await run_verify_agent(
spec=spec, system=INQUIRY_SYSTEM, task=_task(op, ask, target, excerpts, refs, live, lost), sources=list(live),
output_schema=INQUIRY_SCHEMA, max_queries=part.get("max_queries"), timeout_s=part.get("timeout_s"),
daily_max_usd=settings.get("daily_max_usd"), run_id=op.run_id, data_versions=op.versions)
daily_max_usd=settings.get("daily_max_usd"), run_id=op.run_id, data_versions=op.versions,
masked_values=await snapshot_secrets(others, given))
# 遮罩:agent 自己的查询快照(查询时就遮好了,遮掉的值在 outcome.masked_values 里)、报告自己的查询快照(被问的
# 那句可能直接写着遮罩列的值,agent 会照着复述)
# 遮罩:agent 自己的查询快照(查询时就遮好了,遮掉的值在 outcome.masked_values 里)、报告自己的查询快照
own: dict[str, dict[str, Any]] = {}
for query in outcome.queries:
snapshot = _quiet_load(artifact_store.load, query.get("artifact"))
if _query_shaped(snapshot):
own[str(query["artifact"])] = snapshot
others = [snapshot for entry in catalog.values() if isinstance(entry, dict) and entry.get("kind") == "query"
and _query_shaped(snapshot := _quiet_load(op.loader, entry.get("artifact")))]
plan = await _plan(own, others, [ask.question, ask.quote, *_ref_values(refs)],
plan = await _plan(own, others, given,
[(str(q.get("sql") or ""), str(q.get("source") or ""), str(q.get("artifact") or ""))
for q in outcome.queries], live, seen=outcome.masked_values)
@@ -1808,6 +1843,8 @@ async def _answer(op: _Opening, ask: Ask, target: _Target | None, settings: dict
"inquiry_artifacts": list(dict.fromkeys(q["artifact"] for q in queries if q.get("artifact"))),
"queries": queries,
"model": outcome.model, "usage": {k: outcome.usage.get(k, 0) for k in USAGE_KEYS},
# 只靠内存里的遮罩值才换掉的位置(不含值),质询工件照着再换一遍;没有就不加这个键,事件和以前一字不差
**({REPLAY_KEY: plan.replay} if plan.replay else {}),
}
@@ -1865,7 +1902,8 @@ def _leaked(doc: dict[str, Any], secrets: _Secrets) -> bool:
def _renamed_entry(entry: Any, secrets: _Secrets) -> Any:
if not (isinstance(entry, dict) and entry.get("kind") == "query" and isinstance(entry.get("columns"), list)):
return entry
names = [secrets.scrub(str(c)) for c in entry["columns"]]
# 同快照的列名一样换(_masked_view):换了写法的号码也认
names = [secrets.scrub(str(c), loose=True) for c in entry["columns"]]
return {**entry, "columns": names} if names != [str(c) for c in entry["columns"]] else entry
@@ -2048,11 +2086,15 @@ async def inquiry_artifact(run_id: str, artifact_id: str) -> dict[str, Any]:
raise InquiryRefused(404, "无法读取这件工件,可能已被删除", code=ARTIFACT_NOT_FOUND)
if not _query_shaped(content):
return {"id": artifact_id, "content": content, "masked": []}
# 遮罩值照回答时收:引用它的那几条质询各自查到的全部快照(查询条件里写着的值,不一定在这一份的结果里),加上报告
# 自己的查询快照;免遮的也照回答时:提问人自己写的、报告里列出的指标和运行输入的值。SQL 里照原样留着的阈值只在
# 回答正文里不遮,格子照旧按值遮,这里不用。文字格里夹着的遮罩值按子串换(embedded):回答那边渲染出来露了就整份
# 不交,这份快照却照样记在事件里、点得开,不换就等于在「不予展示」旁边把原值摆出来
own: dict[str, dict[str, Any]] = {artifact_id: content}
# agent 自己查询里遮掉的值事后已经不在(查询时就遮好了,快照里收不到):回答时只靠它们才换掉的地方记在事件里
# (REPLAY_KEY,只记位置),先照着再换一遍。先查号码、后查备注的,备注查询时就遮好了,用不着这一步
edits = [e for d in referenced if isinstance(d.get(REPLAY_KEY), dict) and (e := d[REPLAY_KEY].get(artifact_id))]
replayed = replay_edits(content, edits) if edits else content
# 再照回答时的办法遮。遮罩值照回答时收:引用它的那几条质询各自查到的全部快照(查询条件里写着的值,不一定在这一份
# 的结果里),加上报告自己的查询快照;免遮的也照回答时:提问人自己写的、报告里列出的指标和运行输入的值。SQL 里照原样
# 留着的阈值只在回答正文里不遮,格子照旧按值遮,这里不用。文字格里夹着的遮罩值按子串换(embedded):回答那边渲染
# 出来露了就整份不交,这份快照却照样记在事件里、点得开,不换就等于在「不予展示」旁边把原值摆出来
own: dict[str, dict[str, Any]] = {artifact_id: replayed}
for data in referenced:
for other in dict.fromkeys(str(a) for a in data.get("inquiry_artifacts") or []):
if other not in own and _query_shaped(snapshot := _quiet_load(artifact_store.load, other)):
@@ -2061,6 +2103,9 @@ async def inquiry_artifact(run_id: str, artifact_id: str) -> dict[str, Any]:
asked = [t for d in referenced for t in (d.get("question"), d.get("quote")) if isinstance(t, str)]
plan = await _plan(own, others, [*asked, *values], embedded=frozenset({artifact_id}))
shown = plan.views[artifact_id]
masked = list(dict.fromkeys([*plan.masked[artifact_id],
*(m for e in edits if isinstance(e, dict) for m in e.get("masked") or []
if isinstance(m, str))]))
recorded = next((q["sql"] for d in referenced for q in d.get("queries") or []
if isinstance(q, dict) and q.get("artifact") == artifact_id and isinstance(q.get("sql"), str)),
None)
@@ -2073,7 +2118,7 @@ async def inquiry_artifact(run_id: str, artifact_id: str) -> dict[str, Any]:
whole, pieces = _recorded_swaps(content.get("sql"), recorded, kind) if recorded is not None else (set(), set())
shown = {**shown, "checks": _masked_checks(shown["checks"], content.get("sql"), plan.secrets, compared, kind,
swapped=whole, pieces=pieces)}
out: dict[str, Any] = {"id": artifact_id, "content": shown, "masked": plan.masked[artifact_id]}
out: dict[str, Any] = {"id": artifact_id, "content": shown, "masked": masked}
if gone := plan.gone.get(artifact_id):
out["mask_note"] = SOURCE_GONE.format(source="、".join(gone))
return out
+465 -69
View File
@@ -6,24 +6,29 @@
质询(engine/inquiry.py)、双盲复算(nodes/dual_check.py,经 inquiry 的同名导出)共用:
- 工具边界先遮(mask_query_result):核查 agent 的查询结果交给模型、落进快照之前,取自遮罩列的格子换成「已遮罩」,
同一份结果里别的文字格夹着的遮罩值也换掉。模型看不到遮罩值,就不会把它写进回答、理由、SQL——事后按值认每一轮
复核都找到新的漏法,根治靠这一步;审批质询回答里的按值遮降为第二道防线
别的格子按值换:这份结果里的遮罩值,加上这个 agent 前面的查询遮掉的、调用方交进来的(报告快照里的)。模型看不到
遮罩值,就不会把它写进回答、理由、SQL——事后按值认每一轮复核都找到新的漏法,根治靠这一步;审批质询回答、反方
审稿理由里的按值遮降为第二道防线。查询失败时报错原话里的值也换(mask_error_text)
- 哪些列(_derived_hidden):从遮罩列本身起,按 SQL 一层层认出值取自遮罩列的结果列(别名、函数、子查询、WITH、UNION 的
另一支、和 * 写在同一支里没起别名的表达式);COUNT 只数个数、窗口的分区键只决定次序、按遮罩列分档又当分组键或套在
汇总里数个数的不算(_value_columns、_band_only)。认不出落在哪一列的、SQL 解析不了又写着遮罩列的,整份快照的列都算
- 只装着分档标签的列(_label_columns):标签是 agent 写在 SQL 里的字,不当遮罩值
另一支、和 * 写在同一支里没起别名的表达式、FROM 里拆开遮罩列的表值函数);COUNT 只数个数、窗口的分区键只决定次序、
按遮罩列分档又当分组键或套在汇总里数个数的、汇总里的 CASE 只在条件里用到的不算(_value_columns、_band_only)。认不出
落在哪一列的、SQL 解析不了又写着遮罩列的,整份快照的列都算。双盲复算按同一套认法在派盲算之前挡下取自遮罩列的指标
(derived_columns)
- 只装着分档标签的列(_label_columns):标签是 agent 写在 SQL 里的字,不当遮罩值;* 展开数据源里的表时按冻结的表结构
认它带不带同名的列(_schema_tables)
- 哪些值(_Secrets):遮罩列里出现过的值,按值认有门槛(SECRET_MIN、SECRET_DIGITS),换的时候先按词、按开头几个字筛出
正文里真有的那几个值,不现编上万个分支的正则;正文、文字格里换了写法的长号码(199-1474-2785、+86 …、全角数字)按
纯数字写法在集合里查(loose_spans)
- 快照遮好的样子(_masked_view)
正文里真有的那几个值,不现编上万个分支的正则;正文、文字格、SQL 的字符串和注释里换了写法的长号码(199-1474-2785、
+86 …、全角数字)按纯数字写法在集合里查(loose_spans);已经摆在读者眼前的值免遮,之后并进来的照样免(protected)
- 快照遮好的样子(_masked_view),回答时才认得出、事后要照位置重放的地方(view_edits、replay_edits)
除 mask_query_result、换了写法的号码之外,这里的函数原来都在 engine/inquiry.py 里,名字不变,inquiry 照旧导出(别处按
inquiry 的名字引用)。
除 mask_query_result、换了写法的号码和这一轮新加的几样之外,这里的函数原来都在 engine/inquiry.py 里,名字不变,
inquiry 照旧导出(别处按 inquiry 的名字引用)。
"""
from __future__ import annotations
import contextlib
import functools
import json
import logging
import re
@@ -32,7 +37,7 @@ from dataclasses import dataclass, field
from decimal import Decimal
from typing import Any
from app.engine.evidence import RenderError, render_cell
from app.engine.evidence import MARKER_RE, RenderError, render_cell
from app.engine.expressions import numeric_text
from app.engine.merge_query import MERGE_SOURCE
@@ -70,7 +75,9 @@ def _masked_word() -> str:
def _value_columns(node: Any) -> set[str]:
"""一个选出列的值取自哪些列(列名小写)。COUNT 里的只数个数,窗口的分区键、排序键只决定次序,结果全是写死常量的
分档(_band_only:当分组键,或者套在汇总里按条件数个数)只决定一行落在哪一档,都带不出值,不算;套在里面的子查询
只看它选出来的列(它的条件、连接里写的列带不出值)。"""
只看它选出来的列(它的条件、连接里写的列带不出值)。汇总里的 CASE / IIF 只看各个结果:条件只决定这一行算不算进来,
同写在 WHERE 里(SUM(CASE WHEN buyer_phone IS NOT NULL THEN amount ELSE 0 END) 的值全部来自金额);不汇总的逐行
CASE 照旧连条件一起算(每一行落在哪一支本身就是这一行的信息)。"""
from sqlglot import exp
out: set[str] = set()
@@ -82,6 +89,9 @@ def _value_columns(node: Any) -> set[str]:
out.add(item.name.lower())
elif isinstance(item, exp.Count) or (isinstance(item, (exp.Case, exp.If)) and _band_only(item)):
continue
elif isinstance(item, (exp.Case, exp.If)) and (results := _branch_results(item)) is not None \
and _aggregated(item):
stack.extend(r for r in results if r is not None)
elif isinstance(item, exp.Window):
stack.append(item.this)
elif isinstance(item, exp.Select):
@@ -91,6 +101,18 @@ def _value_columns(node: Any) -> set[str]:
return out
def _aggregated(node: Any) -> bool:
"""node 套在同一层 SELECT 的汇总函数里(SUM(…)、AVG(…) OVER (…) 也算)。"""
from sqlglot import exp
parent = node.parent
while parent is not None and not isinstance(parent, (exp.Select, exp.Subquery)):
if isinstance(parent, exp.AggFunc):
return True
parent = parent.parent
return False
def _branch_results(node: Any) -> list[Any] | None:
"""CASE 的各个结果(每个 THEN,加上 ELSE;没写 ELSE 记 None,就是 NULL)、IIF / IF 的真假两支;别的写法为 None。
CASE 里的每个 WHEN … THEN 在语法树里也是 If,不单算。"""
@@ -123,11 +145,8 @@ def _band_only(node: Any) -> bool:
results = _branch_results(node)
if results is None or not all(_fixed(r) for r in results):
return False
parent = node.parent
while parent is not None and not isinstance(parent, (exp.Select, exp.Subquery)):
if isinstance(parent, exp.AggFunc):
return True
parent = parent.parent
if _aggregated(node):
return True
item = node
while isinstance(item.parent, (exp.Paren, exp.Alias)) and item.arg_key == "this":
item = item.parent
@@ -145,7 +164,8 @@ def _band_only(node: Any) -> bool:
return False
def _label_columns(sql: Any, kind: str | None, columns: list[str]) -> dict[str, set[str]]:
def _label_columns(sql: Any, kind: str | None, columns: list[str], tables: dict[str, set[str]] | None = None
) -> dict[str, set[str]]:
"""结果里只装着分档标签的列:列名(小写)→ 标签在格子里的写法。标签(CASE、IIF 写死的结果,分档的名字、档位编号)
是 agent 自己写在 SQL 里的字,不是数据源里的值:这几列派生自遮罩列也不收进遮罩值(_Secrets.add),不然 SQL 里的
标签、回答正文里的档名都会被换成「已遮罩」。
@@ -154,7 +174,8 @@ def _label_columns(sql: Any, kind: str | None, columns: list[str]) -> dict[str,
选出(SELECT vip_level)的名字就是它自己,永远算只装着标签,UNION 另一支、子查询里的遮罩列流进标签列也认不出,
等级 1 撞上 THEN 1 就不收。取自数据源里的表的(哪怕和标签同名)、有一支装着别的、认不准的(说不清取自哪个来源、
递归的 CTE、几支的 * 展开在哪说不清)都不算,照收(宁可多遮);同名的几列有一列不算就都不算。解析不了的一个都
不算。"""
不算。tables 是查询当时冻结的表结构里每张表的列名(_schema_tables):没写表名的 * 展开的表查得到、又没有同名的列
时,这张表带不出这一列(_table_lacks),不再因为它不算。"""
from app.data.sql_semantics import _parse, _read
try:
@@ -167,7 +188,8 @@ def _label_columns(sql: Any, kind: str | None, columns: list[str]) -> dict[str,
result = next((top for top, role, _listed in _blocks(root) if role == "result"), None)
if result is None:
return {}
memo: _LabelMemo = {}
memo = _LabelMemo()
memo.tables = tables
found: dict[str, list[set[str] | None]] = {}
for place, column in enumerate(columns):
name = column.lower()
@@ -180,8 +202,42 @@ def _label_columns(sql: Any, kind: str | None, columns: list[str]) -> dict[str,
#: 标签列分析里「这个来源不出这一列」(_block_label、_source_label):和 None(出这一列,但装着别的、认不准)分开
_ABSENT: Any = object()
#: 标签列分析记下的结果:(id(那块查询), 列名) → 标签的写法、None 或 _ABSENT
_LabelMemo = dict[tuple[int, str], Any]
class _LabelMemo(dict): # type: ignore[type-arg]
"""标签列分析记下的结果:(id(那块查询), 列名) → 标签的写法、None 或 _ABSENT。tables 是冻结的表结构里每张表的列名
(小写,_schema_tables),没有为 None。"""
tables: dict[str, set[str]] | None = None
def _schema_tables(schema: Any) -> dict[str, set[str]] | None:
"""冻结的表结构(数据源的 schema_cache、表结构快照的内容:{"tables": {表名: {"columns": [{"name": …}]}}})→
{表名(小写): 列名(小写)},按登记的名字和不带库名的表名各记一份(几个库里同名的表合起来算,宁可多遮)。表结构
里没列出的表不记(截断了的表结构也一样),查不到就当说不清;整份拿不到的为 None。"""
tables = schema.get("tables") if isinstance(schema, dict) else None
if not isinstance(tables, dict):
return None
out: dict[str, set[str]] = {}
for key, meta in tables.items():
columns = meta.get("columns") if isinstance(meta, dict) else None
if not isinstance(columns, list):
continue
names = {str(c["name"]).lower() for c in columns if isinstance(c, dict) and c.get("name") is not None}
for name in {str(key).lower(), str(key).lower().rsplit(".", 1)[-1],
str(meta.get("qualified") or key).lower().rsplit(".", 1)[-1]}:
out.setdefault(name, set()).update(names)
return out
def _table_lacks(source: Any, name: str, memo: _LabelMemo) -> bool:
"""FROM 里的这个来源是数据源里的一张表,冻结的表结构里查得到它、又没有 name 这一列:* 展开它带不出 name。"""
from sqlglot import exp
if not memo.tables or not isinstance(source, exp.Table) or not isinstance(source.this, exp.Identifier):
return False
columns = memo.tables.get(source.name.lower())
return columns is not None and name not in columns
def _result_label(top: Any, place: int, name: str, width: int, memo: _LabelMemo) -> set[str] | None:
@@ -247,7 +303,8 @@ def _star_label(select: Any, name: str, memo: _LabelMemo, *, before: int | None
if not _is_star(item):
continue
qualifier = (item.table.lower() or None) if isinstance(item, exp.Column) else None
if qualifier is None and any(_derived_block(source) is None for _, source in _from_sources(select)):
if qualifier is None and any(_derived_block(source) is None and not _table_lacks(source, name, memo)
for _, source in _from_sources(select)):
return None
texts = _source_label(select, qualifier, name, memo)
if texts is None:
@@ -275,7 +332,8 @@ def _source_label(select: Any, qualifier: str | None, name: str, memo: _LabelMem
for _alias, source in sources:
derived = _derived_block(source)
if derived is None:
opaque = True
# 冻结的表结构里这张表没有这一列:它不出这一列,不算认不准
opaque = opaque or not _table_lacks(source, name, memo)
continue
texts = _block_label(derived[0], derived[1], name, memo)
if texts is None:
@@ -378,6 +436,68 @@ def _derived_block(source: Any) -> tuple[Any, list[str] | None] | None:
return node, ([c.name.lower() for c in alias.columns] if alias is not None and alias.columns else None)
#: 表值函数常见的出列名:SQLite 的 json_each / json_tree(key、value、atom、fullkey、path),Postgres 的 jsonb_each
#: (key、value),Spark 的 explode / posexplode(col、pos),Snowflake 的 FLATTEN(key、value、path、index、this)。
#: 结构性的几列(id、parent、type)带不出值,不算——算上的话,同一条 SQL 里别的 id 列都会被遮
_FUNCTION_COLUMNS = frozenset({"key", "value", "atom", "fullkey", "path", "col", "pos", "index", "this"})
#: 出列名认得全的表值函数:出的是上面那几列,或者一列、名字就是别名(没起别名的是函数名)
_TABLE_FUNCTIONS = frozenset({
"json_each", "json_tree", "jsonb_each", "jsonb_each_text", "json_each_text", "json_array_elements",
"json_array_elements_text", "jsonb_array_elements", "jsonb_array_elements_text", "json_object_keys",
"jsonb_object_keys", "unnest", "explode", "explode_outer", "posexplode", "regexp_split_to_table",
"string_to_table", "string_split", "openjson", "flatten",
})
def _function_sources(select: Any) -> list[Any]:
"""一条 SELECT 的 FROM、JOIN、LATERAL VIEW 里说不清出哪些列、装着什么的来源:不是数据源里的表(含指向 WITH 的名字),
也不是子查询这类派生来源——表值函数(json_each、unnest、JSON_TABLE)、LATERAL 函数、VALUES。"""
from sqlglot import exp
sources = [source for _, source in _from_sources(select)] + list(select.args.get("laterals") or [])
out: list[Any] = []
for source in sources:
if _derived_block(source) is not None:
continue
node = source.this if isinstance(source, exp.Lateral) else source
if isinstance(node, exp.Table) and isinstance(node.this, exp.Identifier):
continue
out.append(source)
return out
def _function_columns(source: Any) -> set[str] | None:
"""表值函数这类来源出的列名(小写):写的列名表、别名(Postgres 里一列的函数出的列就叫别名,没起别名的叫函数名)、
函数名、常见的出列名(_FUNCTION_COLUMNS);JSON_TABLE 按 COLUMNS 里写的,VALUES 按 column1、column2…。不认得的函数
又没写列名表的,说不清出哪几列,为 None。"""
from sqlglot import exp
node = source.this if isinstance(source, exp.Lateral) else source
alias = source.args.get("alias") or (node.args.get("alias") if node is not source else None)
names: set[str] = set(_FUNCTION_COLUMNS)
listed: list[str] = []
if alias is not None:
if alias.name:
names.add(alias.name.lower())
listed = [c.name.lower() for c in alias.args.get("columns") or [] if c.name]
names |= set(listed)
func = node.this if isinstance(node, exp.Table) else node
while isinstance(func, exp.Subquery):
func = func.this
if isinstance(func, exp.JSONTable):
schema = func.args.get("schema")
defined = [str(c.name).lower() for c in (schema.expressions if schema is not None else []) if c.name]
return names | set(defined) if defined or listed else None
if isinstance(func, exp.Values):
width = max((len(row.expressions) for row in func.expressions if isinstance(row, exp.Tuple)), default=1)
return names | {f"column{i}" for i in range(1, width + 1)}
name = (func.name if isinstance(func, exp.Anonymous) else func.sql_name() if isinstance(func, exp.Func)
else "").lower()
if name:
names.add(name)
return names if listed or name in _TABLE_FUNCTIONS else None
def _merging(select: Any, name: str) -> bool:
"""FROM 里按同名的 name 这一列连起来(USING 里写着它,或者 NATURAL):几个来源的这一列合成一列。"""
return any(str(join.args.get("method") or "").upper() == "NATURAL"
@@ -531,7 +651,9 @@ def _tainted(sql: str, kind: str | None, masked: set[str], width: int) -> tuple[
不同层的也一并算上(宁可多遮)。
位置是给没起别名的表达式用的(UPPER(buyer_phone) 的结果列名对不上任何名字):和 * 写在同一支里的,* 前面的从头数、
后面的从末尾倒着数。认不出的——两个 * 之间的项、派生来源里没起别名的表达式(外层的 * 会把它原样带出来,列名
又认不准)——整份快照的列都算(同解析不了)。"""
又认不准)——整份快照的列都算(同解析不了)。
FROM、JOIN、LATERAL 里的表值函数(json_each、unnest、JSON_TABLE、LATERAL VALUES)参数里写着这些名字的,它出的列
都算(_function_columns):遮罩列是 JSON 时,json_each 拆开是最自然的写法。出哪几列说不清的整份快照都算。"""
from sqlglot import exp
from app.data.sql_semantics import _parse, _read
@@ -547,7 +669,9 @@ def _tainted(sql: str, kind: str | None, masked: set[str], width: int) -> tuple[
selects = list(root.find_all(exp.Select))
blocks = _blocks(root)
sources = [(top, listed) for top, role, listed in blocks if role == "source"]
for _ in range(len(selects) + len(sources) + 1):
functions = [source for select_ in selects for source in _function_sources(select_)]
unreadable = False
for _ in range(len(selects) + len(sources) + len(functions) + 1):
before = len(names)
for select_ in selects:
for item in select_.expressions:
@@ -556,9 +680,14 @@ def _tainted(sql: str, kind: str | None, masked: set[str], width: int) -> tuple[
names.add(name)
for top, listed in sources:
names |= _source_names(top, listed, names)[0]
for source in functions:
if names & _value_columns(source):
columns = _function_columns(source)
unreadable = unreadable or columns is None
names |= columns or set()
if len(names) == before:
break
everything = any(_source_names(top, listed, names)[1] for top, listed in sources)
everything = unreadable or any(_source_names(top, listed, names)[1] for top, listed in sources)
positions: set[int] = set()
for top, role, _listed in blocks:
if role != "result":
@@ -598,12 +727,41 @@ def _derived_hidden(snapshot: dict[str, Any], hidden: set[str], kind: str | None
return names | {columns[i].lower() for i in positions if i < len(columns)}, names
def _snapshot_labels(snapshot: dict[str, Any], kind: str | None) -> dict[str, set[str]]:
"""一份快照里只装着分档标签的列(_label_columns)。没有 SQL 的(合并查询)一个都不算。"""
def derived_columns(snapshot: dict[str, Any], hidden: set[str], kind: str | None) -> set[str] | None:
"""按 SQL 认出值取自遮罩列(hidden,小写)的结果列(小写,列名直接对上的也算)。SQL 解析不了、合并查询的快照(不看
SQL)为 None:认不准,宽严由调用方定(_derived_hidden 整份遮;双盲复算照常派盲算,由盲算自己的查询去认)。"""
sql = snapshot.get("sql")
if not hidden or not isinstance(sql, str) or not sql.strip() or snapshot.get("source") == MERGE_SOURCE:
return None
columns = [str(c) for c in snapshot.get("columns") or []]
found = _tainted(sql, kind, set(hidden), len(columns))
if found is None:
return None
names, positions = found
return {c.lower() for i, c in enumerate(columns) if c.lower() in names or i in positions}
def _snapshot_labels(snapshot: dict[str, Any], kind: str | None, tables: dict[str, set[str]] | None = None
) -> dict[str, set[str]]:
"""一份快照里只装着分档标签的列(_label_columns)。没有 SQL 的(合并查询)一个都不算。tables 没给的,按快照记着
的表结构快照(schema_artifact,查询当时冻结的)认。"""
sql = snapshot.get("sql")
if not isinstance(sql, str) or not sql.strip():
return {}
return _label_columns(sql, kind, [str(c) for c in snapshot.get("columns") or []])
if tables is None and isinstance(snapshot.get("schema_artifact"), str):
tables = _frozen_tables(snapshot["schema_artifact"])
return _label_columns(sql, kind, [str(c) for c in snapshot.get("columns") or []], tables)
@functools.lru_cache(maxsize=64)
def _frozen_tables(artifact: str) -> dict[str, set[str]] | None:
"""表结构快照工件里每张表的列名(_schema_tables)。工件按内容寻址、不会变,记下来;取不到、被改过的为 None。"""
from app.core import artifact_store
try:
return _schema_tables(artifact_store.load(artifact))
except Exception: # noqa: BLE001 - 取不到就当不知道表结构(标签照旧当遮罩值,宁可多遮)
return None
# --------------------------------------------------------------------------
@@ -728,13 +886,17 @@ _WORDS = re.compile(r"[0-9A-Za-z_]+")
_DIGITS = re.compile(r"\d[\d,]*(?:\.\d+)?")
#: 中文这类遮罩值按开头几个字分桶找(_Secrets._partition)
_PREFIX = 8
#: 正文里一串允许夹分隔符的数字(换了写法的号码,见 _Secrets.loose_spans):数字(含全角)之间夹着一到三个空格、连字符、
#: 点、括号(含全角),开头可以带 + 和括号。逗号不算分隔符(千分位按 _DIGITS 认),冒号、斜杠也不算(时间、日期),不跨行
_LOOSE_RUN = re.compile(r"[((]?[++]?[((]?[0-90-9]+(?:[ \u3000\-‐‑‒–—-..·・()()]{1,3}[0-90-9]+)*[))]?")
#: 正文里一串允许夹分隔符的数字(换了写法的号码,见 _Secrets.loose_spans):数字(含全角)之间夹着一到四个分隔符,开头
#: 可以带 + 和括号。分隔符是同一行里的各种空白(空格、制表符、全角空格、不换行空格,以及窄空格、细空格这类 U+2000 至
#: U+200B、U+202F、U+205F)、连字符和减号(含全角、U+2010 至 U+2015、U+2212)、点(含全角、间隔号)、括号(含全角)。
#: 逗号不算分隔符(千分位按 _DIGITS 认),冒号、斜杠也不算(时间、日期),不跨行
_LOOSE_SEPARATORS = (" \t\u00a0\u2000-\u200b\u202f\u205f\u3000"
"\\-\u2010-\u2015\u2212\uff0d.\uff0e\u00b7\u30fb()\uff08\uff09")
_LOOSE_RUN = re.compile(rf"[((]?[++]?[((]?[0-90-9]+(?:[{_LOOSE_SEPARATORS}]{{1,4}}[0-90-9]+)*[))]?")
_LOOSE_DIGITS = re.compile(r"[0-90-9]+")
#: 两截数字之间只隔着这样一个字(不带空格):是更长的编号、小数、日期的两截(13900000001-2、13900000001.5),号码到这里
#: 不算完——同 _pattern 里「前后不能是 [.:/-] 加数字」
_JOINERS = frozenset("-‐‑‒–—-..")
#: 不算完——同 _pattern 里「前后不能是 [.:/-] 加数字」。减号(U+2212)、全角连字符同连字符
_JOINERS = frozenset("-\u2010\u2011\u2012\u2013\u2014\u2015\u2212\uff0d.\uff0e")
_POINTS = frozenset("..")
#: 去掉国家码(0086、86)之后至少还要有这么多位才算号码(固话号码最短 7 位):86 开头的短数不拆
_LOCAL_MIN = 7
@@ -857,11 +1019,13 @@ class _Secrets:
留着的阈值、行数(去掉千分位、正负号的数字),只在回答正文里不换(scrub 的 prose)。换的时候先按词、按开头两个
字筛出正文里真有的那几个值,再拿它们现编正则:遮罩值上万个时,现编一个上万分支的正则要一秒上下,每次都卡住整个进程。
正文、文字格里另认换了写法的长号码(199-1474-2785、+86 199 1474 2785、全角数字,见 loose_spans):按纯数字写法
在集合里查,不进正则。"""
在集合里查,不进正则。protected 是 protect 收下的、已经摆在读者眼前的文字:之后并进来的值(核查 agent 后面的
查询遮掉的)照样按它们免遮。"""
strong: set[str] = field(default_factory=set)
weak: set[str] = field(default_factory=set)
shown: set[str] = field(default_factory=set)
protected: tuple[str, ...] = ()
_index: Any = field(default=None, repr=False, compare=False)
_compiled: dict[frozenset[str], re.Pattern[str]] = field(default_factory=dict, repr=False, compare=False)
@@ -899,8 +1063,10 @@ class _Secrets:
self._index = None
def merge(self, other: _Secrets | None) -> None:
"""并进另一份收好的遮罩值(核查 agent 查询时在工具边界遮掉的,见 mask_query_result)。合起来最多 SECRETS_MAX 种。"""
if other is None or not other.weak:
"""并进另一份收好的遮罩值(核查 agent 查询时在工具边界遮掉的,见 mask_query_result)。合起来最多 SECRETS_MAX 种。
两边免遮的文字合起来,并进来的值照样按它们免遮:报告的指标正好等于某一笔金额时,agent 查出了那一笔,别的查询里
正好是这个数的格子也不遮。"""
if other is None:
return
for form in other.weak:
if len(self.weak) >= SECRETS_MAX:
@@ -908,14 +1074,23 @@ class _Secrets:
self.weak.add(form)
if form in other.strong:
self.strong.add(form)
self.protected = tuple(dict.fromkeys((*self.protected, *other.protected)))
self._index, self._compiled = None, {}
if self.protected:
self._spare(" ".join(self.protected))
def protect(self, texts: Iterable[str | None]) -> None:
"""已经摆在读者眼前的值不换:提问人自己写进问题、选中原文里的(「13900000001 这位客户」「排除 300 以下的」),
报告里列出的指标、运行输入的值——换了只会让回答、查询对不上问题和报告。正文、格子、SQL 里都不换。"""
报告里列出的指标、运行输入的值——换了只会让回答、查询对不上问题和报告。正文、格子、SQL 里都不换。文字记下来,
之后并进来的值照样免遮(merge)。"""
given = " ".join(t for t in texts if t)
if not given:
return
self.protected = tuple(dict.fromkeys((*self.protected, given)))
self._spare(given)
def _spare(self, given: str) -> None:
"""去掉 given(几段免遮的文字)里写着的值。"""
numbers = {_digits_of(n) for n in _DIGITS.findall(given)}
# 换了写法写进来的号码(「199-1474-2785 这位客户」)按纯数字写法认
reformatted = {key for _, _, key in self.loose_spans(given)}
@@ -958,15 +1133,40 @@ class _Secrets:
def scrub(self, text: str, *, prose: bool = False, loose: bool | None = None) -> str:
"""text 里的遮罩值换成「已遮罩」。prose:这是回答正文,SQL 里照原样留着的阈值、行数不换(show)。loose:换了写法
的长号码也换(loose_spans),缺省同 prose——只在正文、文字格里这样认,SQL 里的 199-1474-2785 是一个算式。"""
的长号码也换(loose_spans),缺省同 prose——正文、文字格、SQL 的字符串和注释里这样认,SQL 里不带引号的
199-1474-2785 是一个算式。"""
return self.swap(text, prose=prose, loose=loose)[0]
def swap(self, text: str, *, prose: bool = False, loose: bool | None = None) -> tuple[str, list[str]]:
"""scrub,另交回换掉了哪些值:按值认出的照原文的写法,换了写法的号码交它的纯数字写法(SQL 里同一个数写成阈值的
据此不留,见 inquiry._swap_literal)。"""
hits = self._hits(text, prose=prose, loose=loose)
word = _masked_word()
for start, end, _ in reversed(hits):
text = text[:start] + word + text[end:]
return text, [value for _, _, value in hits]
def spans(self, text: str, *, prose: bool = False, loose: bool | None = None) -> list[tuple[int, int]]:
"""scrub 会换掉的那几段(起, 止),按先后、不重叠。质询工件按位置重放回答时换掉的地方(view_edits)。"""
return [(start, end) for start, end, _ in self._hits(text, prose=prose, loose=loose)]
def _hits(self, text: str, *, prose: bool, loose: bool | None) -> list[tuple[int, int, str]]:
"""text 里要换掉的遮罩值:(起, 止, 值),按先后、不重叠。换了写法的号码先认(+86 19914742785 整个换掉,不留下
一截「+86」),按值认的落在它们里面的不再算。"""
if not text or not self.strong:
return text
if prose if loose is None else loose:
# 先换换了写法的:+86 19914742785 整个换掉,不留下一截「+86」
for start, end, _ in reversed(self.loose_spans(text, prose=prose)):
text = text[:start] + _masked_word() + text[end:]
return []
hits = [(start, end, key) for start, end, key in self.loose_spans(text, prose=prose)] \
if (prose if loose is None else loose) else []
pattern = self.pattern(text, prose=prose)
return pattern.sub(_masked_word(), text) if pattern is not None else text
if pattern is not None:
taken = list(hits)
hits += [(m.start(), m.end(), m.group()) for m in pattern.finditer(text)
if not any(m.start() < end and start < m.end() for start, end, _ in taken)]
return sorted(hits)
def copy(self) -> _Secrets:
"""另一份一样的遮罩值(改它不影响这一份):交给线程、按句另加免遮的值时用。"""
return _Secrets(strong=set(self.strong), weak=set(self.weak), shown=set(self.shown), protected=self.protected)
def loose_spans(self, text: str, *, prose: bool = False) -> list[tuple[int, int, str]]:
"""text 里换了写法的长号码(199-1474-2785、(199)1474-2785、+86 199 1474 2785、全角数字……):(起, 止, 纯数字
@@ -1058,6 +1258,34 @@ class _Secrets:
return self._index
#: 单元格引用的前一截(Q1.r0.):后面才是列名
_CELL_HEAD = re.compile(r"^\s*[QK]\d+\.r\d+\.")
def _scrubbed_source(text: str, secrets: _Secrets) -> str:
"""回答原文里的遮罩值换成「已遮罩」:标记之外按正文的办法换(SQL 里照原样留着的阈值不换);标记里的按段换(| 隔开
的几段各自换),单元格引用只换列名那一截——编号、行号换不出东西,整段换的话「r0.」会让列名开头的长号码看着像小数
的一截,反而认不出来。列名照快照、SQL 里的名字那样换(阈值也换,换了写法的号码也认),换得一样才对得上。
标记里写着遮罩值的引用换完多半解析不了(列名里写着遮罩值的除外:快照的列名也换过了,正好对上),宁可弄坏一个
引用,也不让值留在存下的原文、片段里,或者从解析不了的占位里露出来。"""
if not text or not secrets.strong:
return text
out: list[str] = []
last = 0
for m in MARKER_RE.finditer(text):
out.append(secrets.scrub(text[last:m.start()], prose=True))
start, end = m.span("body")
parts = text[start:end].split("|")
head = _CELL_HEAD.match(parts[0]) if m.group("kind").strip() == "v" else None
first = parts[0][:head.end()] + secrets.scrub(parts[0][head.end():], loose=True) if head \
else secrets.scrub(parts[0], prose=True)
parts = [first, *(secrets.scrub(p, prose=True) for p in parts[1:])]
out.append(text[m.start():start] + "|".join(parts) + text[end:m.end()])
last = m.end()
out.append(secrets.scrub(text[last:], prose=True))
return "".join(out)
def _masked_view(snapshot: dict[str, Any], hidden: set[str], secrets: _Secrets, *, embedded: bool = False,
rename: bool = True) -> tuple[dict[str, Any], list[str]]:
"""一份查询快照遮好的样子和遮掉的列:要遮的列(hidden,小写)每一格都换成「已遮罩」、类型记成文本(渲染时不当数);
@@ -1068,22 +1296,30 @@ def _masked_view(snapshot: dict[str, Any], hidden: set[str], secrets: _Secrets,
embedded 只给质询工件用:别的列里一格文字夹着遮罩值(备注里写着手机号,换了写法的也算),把那一截换成「已遮罩」,
这一格别的字照原样,门槛同整格(_Secrets.scrub);这一列不记进遮掉的列——界面把记进去的列整列画成「已遮罩」,这一列在数据源
里也没设遮罩。回答一路不这样换:渲染出来露了就整份不交(_leaked),不然回答被问的那一格看着像查到了别的东西。
rename 为假时列名不换(工具边界先遮时:列名是模型自己写的,换了它引用的列名就对不上)。"""
rename 为假时列名不换(工具边界先遮时:列名是模型自己写的,换了它引用的列名就对不上)。列名按文字的办法换(换了写法的
号码也认,同 SQL 里带引号的名字)。
工具边界遮过的快照(记着 masked_at_query)里,整格就是「已遮罩」的是查询时按值换掉的:这一列照样记进遮掉的列,和回答时
按值换掉的一样画。"""
columns = [str(c) for c in snapshot.get("columns") or []]
index = {i for i, c in enumerate(columns) if c.lower() in hidden}
by_value = bool(secrets.strong)
if not index and not by_value:
premasked = "masked_at_query" in snapshot
if not index and not by_value and not premasked:
return snapshot, []
word = _masked_word()
touched = set(index)
edited = False
position = {c: i for i, c in enumerate(columns)}
names = [secrets.scrub(c) for c in columns] if by_value and rename else columns
names = [secrets.scrub(c, loose=True) for c in columns] if by_value and rename else columns
def cell(i: int | None, value: Any) -> Any:
nonlocal edited
if i in index:
return word
if premasked and value == word:
if i is not None:
touched.add(i)
return value
if by_value and secrets.hit(value):
if i is not None:
touched.add(i)
@@ -1119,6 +1355,122 @@ def _masked_view(snapshot: dict[str, Any], hidden: set[str], secrets: _Secrets,
return view, [names[i] for i in sorted(touched)]
def _cell_text(value: Any) -> str | None:
"""一格按文字换的样子(_masked_view 的 embedded):文字照原样,JSON(对象、数组)按界面画出来的样子;别的为 None。"""
if isinstance(value, str):
return value
if isinstance(value, (dict, list)):
return json.dumps(value, ensure_ascii=False, default=str)
return None
def view_edits(snapshot: dict[str, Any], hidden: set[str], secrets: _Secrets, baseline: _Secrets
) -> dict[str, Any] | None:
"""质询工件按回答时的办法遮(_masked_view 的 embedded)时,secrets 比 baseline 多换掉的地方,相对原快照记成位置,
不含值:{"cells": [[行, 列], …](整格换成「已遮罩」), "spans": [[行, 列, 起, 止], …](文字格里换掉的那一截,起止按
这一格的文字算), "names": [[列, 起, 止], …](列名里换掉的那一截), "masked": [列名, …](因此画成遮罩列的)};没有
多换的为 None。
agent 先查了备注(那时还不知道备注里的号码是遮罩值,原样交回、原样存下),后来又查出了手机号(查询时遮掉,值只在
内存里):回答时按内存里的值(secrets)认得出,事后取质询工件时只剩 baseline 认得出的。这份位置记进质询事件,
事后照着再换一遍(replay_edits)。行是字典的快照不记(核查的查询快照都是行列表)。"""
full, full_masked = _masked_view(snapshot, hidden, secrets, embedded=True)
base, base_masked = _masked_view(snapshot, hidden, baseline, embedded=True)
if full is base:
return None
word = _masked_word()
cells: list[list[int]] = []
spans: list[list[int]] = []
for r, (row, shown, plain) in enumerate(zip(snapshot.get("rows") or [], full["rows"], base["rows"])):
if not (isinstance(row, list) and isinstance(shown, list) and isinstance(plain, list)):
continue
for c, (value, now, before) in enumerate(zip(row, shown, plain)):
if now == before:
continue
if now == word:
cells.append([r, c])
elif (text := _cell_text(value)) is not None:
spans += [[r, c, start, end] for start, end in secrets.spans(text, loose=True)]
columns = [str(c) for c in snapshot.get("columns") or []]
full_names, base_names = full.get("columns", columns), base.get("columns", columns)
names = [[i, start, end] for i, (column, now, before) in enumerate(zip(columns, full_names, base_names))
if now != before for start, end in secrets.spans(column, loose=True)]
# 按位置比:同一列在两边可能叫不同的名字(列名里的值只有一边换了)
already = {i for i, name in enumerate(base_names) if name in base_masked}
masked = [name for i, name in enumerate(full_names) if name in full_masked and i not in already]
if not (cells or spans or names or masked):
return None
return {"cells": cells, "spans": spans, "names": names, "masked": masked}
def replay_edits(snapshot: dict[str, Any], edits: Iterable[Any]) -> dict[str, Any]:
"""照记下的位置(view_edits,可能有好几份:同一份快照被几条质询引用)把回答时换掉的地方再换一遍。位置对不上的
(行、列越界,起止不合理,形状不对)跳过:记录是封存的事件,坏了也不能让取工件出错。没有可换的返回原快照。"""
rows = snapshot.get("rows")
columns = [str(c) for c in snapshot.get("columns") or []]
if not isinstance(rows, list):
return snapshot
def ints(entry: Any, size: int) -> list[int] | None:
ok = isinstance(entry, list) and len(entry) == size and all(
isinstance(x, int) and not isinstance(x, bool) and x >= 0 for x in entry)
return entry if ok else None
whole: set[tuple[int, int]] = set()
parts: dict[tuple[int, int], list[tuple[int, int]]] = {}
heads: dict[int, list[tuple[int, int]]] = {}
for edit in edits:
if not isinstance(edit, dict):
continue
for entry in edit.get("cells") or []:
if (at := ints(entry, 2)) is not None:
whole.add((at[0], at[1]))
for entry in edit.get("spans") or []:
if (at := ints(entry, 4)) is not None and at[2] < at[3]:
parts.setdefault((at[0], at[1]), []).append((at[2], at[3]))
for entry in edit.get("names") or []:
if (at := ints(entry, 3)) is not None and at[1] < at[2]:
heads.setdefault(at[0], []).append((at[1], at[2]))
word = _masked_word()
def swapped(text: str, cuts: list[tuple[int, int]]) -> str:
merged: list[list[int]] = []
for start, end in sorted(cuts):
if end > len(text):
continue
if merged and start <= merged[-1][1]:
merged[-1][1] = max(merged[-1][1], end)
else:
merged.append([start, end])
for start, end in reversed(merged):
text = text[:start] + word + text[end:]
return text
out_rows: list[Any] = []
changed = False
for r, row in enumerate(rows):
if not isinstance(row, list):
out_rows.append(row)
continue
new_row = list(row)
for c, value in enumerate(row):
if (r, c) in whole:
new_row[c] = word
elif (r, c) in parts and (text := _cell_text(value)) is not None:
new_row[c] = swapped(text, parts[(r, c)])
changed = changed or new_row != row
out_rows.append(new_row)
names = [swapped(name, heads[i]) if i in heads else name for i, name in enumerate(columns)]
if not changed and names == columns:
return snapshot
view = {**snapshot, "rows": out_rows}
if names != columns:
view["columns"] = names
if isinstance(snapshot.get("column_types"), dict):
rename = dict(zip(columns, names))
view["column_types"] = {rename.get(str(k), k): v for k, v in snapshot["column_types"].items()}
return view
# --------------------------------------------------------------------------
# 工具边界:查询结果交给核查 agent 之前先遮
# --------------------------------------------------------------------------
@@ -1137,33 +1489,77 @@ class QueryMask:
secrets: _Secrets
def mask_query_result(payload: dict[str, Any], masked: Iterable[str], kind: str | None) -> QueryMask:
def mask_query_result(payload: dict[str, Any], masked: Iterable[str], kind: str | None,
known: _Secrets | None = None, *, schema: Any = None) -> QueryMask:
"""一份查询结果(tools/datasource 的查询工具交回的样子:columns / rows / column_types / sql)在交给核查 agent、落进
快照之前先遮。masked 是数据源设了遮罩的列,kind 是数据源的类型(SQL 按它的方言解析)。
快照之前先遮。masked 是数据源设了遮罩的列,kind 是数据源的类型(SQL 按它的方言解析)。known 是此前已经认出的遮罩值:
这个 agent 前面的查询遮掉的,加上调用方交进来的(审批质询、反方审稿收的报告快照里的值)。schema 是数据源的表结构
(schema_cache,和随快照冻结的是同一份),认分档标签时用(_label_columns 的 tables)。
- 整列遮:列名直接对上遮罩列的,加上按 SQL 认出值取自遮罩列的(_derived_hidden:别名、函数、子查询、WITH、
SELECT *;COUNT、只在条件里用到、分档当分组键的不算);SQL 解析不了、又写着遮罩列的,整份结果的列都遮。这几列的
格子换成「已遮罩」,类型记成文本
- 别的列:整格就是这份结果里某个遮罩值的换掉,文字格里夹着的那一截换掉(换了写法的号码也认,_Secrets.loose_spans);
门槛同按值遮(SECRET_DIGITS、SECRET_MIN),只装着分档标签的列里的标签不算遮罩值
- 列名不换(模型自己写的,换了它引用的列名就对不上);结果里一列都没取自遮罩列的原样交回:数值的汇总、只数个数的
查询不受影响
SELECT *、表值函数;COUNT、只在条件里用到、分档当分组键的不算);SQL 解析不了、又写着遮罩列的,整份结果的列都遮。
这几列的格子换成「已遮罩」,类型记成文本
- 别的列按值换:整格就是遮罩值的换掉,文字格里夹着的那一截换掉(换了写法的号码也认,_Secrets.loose_spans)。遮罩值
是这份结果里的加上 known:先查号码、再查写着号码的备注、没设遮罩的联系方式表(别的库里的也一样)时,后一份同样
遮好。门槛同按值遮(SECRET_DIGITS、SECRET_MIN);只装着分档标签的列里的标签不算遮罩值,这份结果自己写的标签也不
按 known 换(前面的查询认不准、当遮罩值收了的同名标签,不能把这一份的档位列换掉)
- 列名不换(模型自己写的,换了它引用的列名就对不上);结果里一列都没取自遮罩列、也没有可按值换的原样交回:数值的
汇总、只数个数的查询不受影响
分析本身出了意外,整份结果的列都遮:这一步挡在模型前面,分析的缺陷不能把原值放过去(宁可多遮)。"""
交回的 secrets 只有这份结果里的遮罩值(known 调用方自己有)。分析本身出了意外,整份结果的列都遮:这一步挡在模型
前面,分析的缺陷不能把原值放过去(宁可多遮)。"""
base = {str(c).strip().lower() for c in masked if str(c).strip()}
columns = [str(c) for c in payload.get("columns") or []]
if not base or not columns or not _query_shaped(payload):
given = known is not None and bool(known.strong)
if not (base or given) or not columns or not _query_shaped(payload):
return QueryMask(payload, [], _Secrets())
try:
hidden, _ = _derived_hidden(payload, base, kind)
if not any(c.lower() in hidden for c in columns):
return QueryMask(payload, [], _Secrets())
secrets = _Secrets()
secrets.add(payload, hidden, labels=_snapshot_labels(payload, kind), base=base)
secrets.prepare()
view, _ = _masked_view(payload, hidden, secrets, embedded=True, rename=False)
hidden = _derived_hidden(payload, base, kind)[0] if base else set()
own = _Secrets()
labels = _snapshot_labels(payload, kind, _schema_tables(schema))
if any(c.lower() in hidden for c in columns):
own.add(payload, hidden, labels=labels, base=base)
pool = own
if given:
pool = own.copy()
pool.merge(known)
# 这份结果自己写的标签(CASE 写死的档名)不当遮罩值,哪怕前面的查询把同名的字收了
if written := [text for name, texts in labels.items() if name not in base for text in texts]:
pool.protect(written)
if not pool.strong and not any(c.lower() in hidden for c in columns):
return QueryMask(payload, [], own)
pool.prepare()
view, _ = _masked_view(payload, hidden, pool, embedded=True, rename=False)
except Exception: # noqa: BLE001 - 认不准就整份遮掉,原值不交给模型
logger.warning("核查查询的遮罩分析出错,整份结果按遮罩处理", exc_info=True)
hidden, secrets = {c.lower() for c in columns}, _Secrets()
view, _ = _masked_view(payload, hidden, secrets, rename=False)
return QueryMask(view, [c for c in columns if c.lower() in hidden], secrets)
hidden, own = {c.lower() for c in columns}, _Secrets()
view, _ = _masked_view(payload, hidden, own, rename=False)
return QueryMask(view, [c for c in columns if c.lower() in hidden], own)
#: 查询报错里引号括起来的一段(Postgres、DuckDB、MySQL 把转换失败的值写在引号里:invalid input syntax for type
#: bigint: "139-0000-0001")。引号前后紧挨着字母数字的不算(can't、isn't 里的撇号)
_QUOTED_ERROR = re.compile(r"(?<![0-9A-Za-z_])(\"[^\"\n]*\"|'[^'\n]*')(?![0-9A-Za-z_])")
#: 报错里不带引号的长数(DuckDB:Type INT64 with value 13900000001 can't be cast)
_ERROR_NUMBER = re.compile(r"(?<![0-9A-Za-z_.])\d{%d,}(?![0-9A-Za-z_])" % SECRET_DIGITS)
def mask_error_text(message: str, sql: str, known: _Secrets | None = None) -> str:
"""查询失败时交给核查 agent、写进 tool.end 的报错原话。驱动的原话是模型改写 SQL 的线索,可数据库会在报错里带上
数据里的值(类型转换失败、越界),数据源设了遮罩时交出去之前先换:引号里的一段、不带引号的长数,SQL 里没写着的
换成「已遮罩」——SQL 里写着的是模型自己写的表名、列名、字面量,照留,它要靠这些改写;此前查询遮掉的值(known)
按值换。不认得是不是数据的也换(多遮):报错只是线索,说不全也不耽误改写。"""
if not message:
return message
word = _masked_word()
lowered = sql.lower()
def quoted(m: re.Match[str]) -> str:
inner = m.group(1)[1:-1]
if not inner.strip() or inner.lower() in lowered:
return m.group()
return m.group(1)[0] + word + m.group(1)[-1]
out = _QUOTED_ERROR.sub(quoted, message)
out = _ERROR_NUMBER.sub(lambda m: m.group() if m.group() in sql else word, out)
return known.scrub(out, loose=True) if known is not None and known.strong else out
+68 -2
View File
@@ -9,7 +9,11 @@
取自的查询算,合并查询按它合并的那几个源算);这句没挂依据,就在这份报告引用的数据源里查。任务里给这句结论
和它挂的证据摘录(judge.prepare,数据源遮罩的列照旧不给;摘录里原报告的编号都标成「原报告 Qn」,和 agent 自己
的 Q1 分开),要它先想清楚「如果这句不成立,数据里会看到什么」,写只读查询去找,最后交结论(经受住 / 被削弱 /
被推翻 / 无法判断)和理由。理由里的数字只能引用它自己查询的格子,裸数字由 compose_doc 标出来
被推翻 / 无法判断)和理由。理由里的数字只能引用它自己查询的格子,裸数字由 compose_doc 标出来。
遮罩:agent 的查询结果在工具边界就遮好了;这份报告引用的查询快照里的遮罩值(按 SQL 认出改了名字的遮罩列,同审批
质询)反查之前就交给查询工具,agent 查到写着同一个号码的别的表时也看不到。摘录按列名遮,改了名字的遮罩列原值
agent 看得到,会照着写进理由:理由正文和理由文档按值换成「已遮罩」(换了写法的号码也认),这句结论原文里写着的
不换(_masked_rationale)
3. 出处:一条查询都没执行成功的,判什么都记为无法判断;判「被推翻」的,理由里还得引用到自己的查询结果——它要让
正式出具降档,审稿记录里得查得到凭什么。不采信时系统写明原因,反方原本的结论和理由照录在后面(纯文字:解析
不了的引用写成「(引用无效)」,裸数字注明「(无出处)」)。这句该查的数据源都不存在或已停用、核查模型本身用
@@ -44,8 +48,10 @@
"""
from __future__ import annotations
import asyncio
import re
from collections.abc import Callable, Collection, Iterable
from dataclasses import replace
from typing import Any
from langgraph.func import task
@@ -372,6 +378,52 @@ def _claim_of(cand: judging.Candidate, outcome: VerifyOutcome) -> tuple[dict[str
return _claim(cand, verdict, rationale, doc=outcome.rationale, queries=outcome.queries, usage=outcome.usage), None
def _spared(seed: Any, claim: str) -> Any:
"""报告快照里的遮罩值(seed)去掉这句结论原文里写着的,交给这一句的反查。seed 本身不动(下一句另算)。"""
known = seed.copy()
known.protect([claim])
return known
def _masked_rationale(outcome: VerifyOutcome, secrets: Any, *, given: Iterable[str | None]) -> VerifyOutcome:
"""反方的理由里的遮罩值换成「已遮罩」,换了写法的号码也认(199-1474-2785、+86 …)。值是这份报告引用的查询快照里的
(secrets,反查之前就交给了查询工具),加上 agent 查询时在工具边界遮掉的(outcome.masked_values);被审的那句结论
原文里写着的(given)已经摆在读者眼前,不换。反方 agent 的查询结果在工具边界就遮好了,可证据摘录按列名遮,改了名字
的遮罩列(buyer_phone AS 手机号)的原值它看得到,会写进理由。
换的办法同审批质询的回答(_scrubbed_source):原文里的值先换掉,再照原来的目录重新组装理由文档;引用到的格子按
遮好的快照渲染(文字格里夹着的遮罩值换掉那一截)。理由里认不出遮罩值的原样交回。"""
from app.engine.evidence import compose_doc
from app.engine.masking import _masked_view, _scrubbed_source
doc, result = outcome.rationale, outcome.result
pool = secrets.copy()
pool.merge(outcome.masked_values)
pool.protect(given)
if result is None or not isinstance(doc, dict) or not pool.strong:
return outcome
pool.prepare()
source = str(doc.get("source") or "")
if not (pool.spans(str(doc.get("markdown") or ""), prose=True) or pool.spans(source, prose=True)):
return outcome
views: dict[str, Any] = {}
fetch = _fetcher()
for query in outcome.queries:
snapshot = fetch(query.get("artifact"))
if isinstance(snapshot, dict) and isinstance(snapshot.get("rows"), list):
views[str(query["artifact"])] = _masked_view(snapshot, set(), pool, embedded=True)[0]
def load(artifact: str) -> Any:
return views[artifact] if artifact in views else artifact_store.load(artifact)
try:
again = compose_doc(_scrubbed_source(source, pool), doc.get("catalog") or {},
node_id=str(doc.get("node_id") or ""), run_id=doc.get("run_id"), loader=load)
except Exception: # noqa: BLE001 - 重新组装出了意外,值也不能留着
# 正文按值换掉,理由文档不要了(界面照正文显示;判「被推翻」的因此查不到出处,按无法判断记)
said = pool.scrub(str(doc.get("markdown") or ""), prose=True)
return replace(outcome, rationale=None, result={**result, RATIONALE: said})
return replace(outcome, rationale=again, result={**result, RATIONALE: again["markdown"]})
def _cites(doc: dict[str, Any] | None) -> bool:
"""理由文档里有没有一处引用解析到了 agent 自己的查询结果(compose_doc 只把解析成功的引用记进 unit.cites)。"""
return isinstance(doc, dict) and any(unit.get("cites") for _, unit in iter_units(doc))
@@ -566,6 +618,9 @@ class _Review:
# 这句没挂依据时在整份报告引用的数据源里查:写作者看得到的就是这几个库
everywhere = claim_sources(self.catalog, self.catalog, self.fetch)
usable = await _usable_sources(everywhere)
# 报告引用的查询快照里遮罩列的值:反查之前交给查询工具(agent 查到写着同一个号码的别的表时看不到),反查之后
# 据此换掉理由里的遮罩值(_masked_rationale)
seed = await self._seed()
claims: list[dict[str, Any]] = []
usage = _zero_usage()
model: str | None = None
@@ -629,11 +684,14 @@ class _Review:
if allot.tightened:
self.run_seconds = allot.seconds
# 收紧过的:核查 agent 到点自己发的 verify_limit 改说运行时长上限将至(deadline.relabel),不写收紧后的秒数
# 这句结论原文里写着的值已经摆在读者眼前,不当遮罩值。上万个值过一遍要几十毫秒,放进线程
known = await asyncio.to_thread(_spared, seed, cand.text)
outcome = await run_verify_agent(
spec=self.spec, system=CHALLENGE_SYSTEM, task=task_text, sources=live,
output_schema=CHALLENGE_SCHEMA, max_queries=self.max_queries, timeout_s=allot.timeout_s,
daily_max_usd=self.daily_max_usd, emit=deadline.relabel(ctx.emit, allot), run_id=ctx.run.run_id,
node_id=ctx.node.id, data_versions=run_versions(ctx.run))
node_id=ctx.node.id, data_versions=run_versions(ctx.run), masked_values=known)
outcome = await asyncio.to_thread(_masked_rationale, outcome, known, given=[cand.text])
if outcome.error and not outcome.model:
broken = outcome.error # 连模型都没建成(没配好、接入出错):换一句也一样
model = model or outcome.model
@@ -709,6 +767,14 @@ class _Review:
return {"artifact": artifact, "counts": counts, "usage": usage, "unreviewed": missed,
"claims": [{k: c[k] for k in ("unit", "text", "verdict")} for c in claims]}
async def _seed(self) -> Any:
"""这份报告目录里的查询快照里遮罩列的值(engine/masking 的 _Secrets)。收法同审批质询收报告快照
(inquiry.snapshot_secrets:按 SQL 认出改了名字的遮罩列,查询当时记下的遮罩加上数据源现在设的)。"""
from app.engine.inquiry import snapshot_secrets # inquiry 导入了这个模块,用到时再导入
return await snapshot_secrets(self.fetch(entry.get("artifact")) for entry in self.catalog.values()
if isinstance(entry, dict) and entry.get("kind") == "query")
async def _store(self, record: dict[str, Any]) -> str | None:
try:
return await artifact_store.put_json(record, kind="challenge", run_id=self.ctx.run.run_id,
+11 -2
View File
@@ -13,7 +13,9 @@
连取值也不给;遮罩列不给。计算式只有一项来自查询、或者写了口径说明时不给列名;几项都来自查询又没写口径说明的,列名
里看不出日期、口径的取舍时才给,好让它分得清哪一项是哪一项。
**遮罩列上的输入不复算。** cell() 取自遮罩列的输入不能单独复算;别的输入(计算式里写成当时代进去的值)里按位置取自
**遮罩列上的输入不复算。** cell() 取自遮罩列的输入不能单独复算,改了名字、套了函数或汇总的遮罩列(SUM(amount) AS
gmv,amount 设了遮罩)按 SQL 认出来也一样(engine/masking.derived_columns):盲算的查询在工具边界按同一套认法遮掉,
派出去也取不到数,不白花一次盲算。别的输入(计算式里写成当时代进去的值)里按位置取自
查询结果的一格(vars.res.rows[0][2]),顺着取值链认出它取自哪份快照的哪一列(recompute.input_origin,证据面板、推演、
可验证报告同一个认法),按需算那份快照要遮的列(和证据面板同一套,见 _Sources.hidden)——它不一定是 cell() 用到的
那几份。取自遮罩列、遮没遮核对不了的,整个指标按无法复算记:遮罩值不能当字面量写进交给盲算的计算式。伸进查询结果的行、
@@ -75,6 +77,7 @@ from app.engine.expressions import ExpressionError, _cell_call, _chain_root, cel
parse_expression
from app.engine.replay import _in_graph, ask, once
from app.engine.labels import DUAL_CHECK_FIELD_LABEL, DUAL_CHECK_ON_REJECT_LABEL
from app.engine.masking import derived_columns
from app.engine.schema import DUAL_CHECK_ON_REJECT, NodeType, dual_check_card
from app.engine.state import GraphState
@@ -613,9 +616,15 @@ async def _query_input(item: dict[str, Any], sources: _Sources, aliases: dict[st
value = cell_value(item.get("value"))
if not _number(value):
return {}, f"输入 {item['path']} 不是数值"
if column.lower() in await sources.hidden(artifact):
hidden = await sources.hidden(artifact)
if column.lower() in hidden:
return {}, f"输入 {item['path']} 取自设置了遮罩的列,不能用于复算"
found = await sources.source(source)
derived = derived_columns(snap, set(hidden), found.kind if found is not None else None) if hidden else None
if derived and column.lower() in derived:
# 改了名字、套了函数或汇总的遮罩列(SUM(amount) AS gmv):盲算的查询在工具边界按同一套认法遮掉(engine/masking),
# 交回的格子注定取不到,不白派一次盲算。SQL 解析不了的认不准,照常派
return {}, f"输入 {item['path']} 由设置了遮罩的列算出(经改名、函数或汇总),盲算时会被遮罩,不能用于复算"
if found is None or not found.enabled:
return {}, f"数据源「{source}」不存在或已停用"
return {"path": item["path"], "key": f"{artifact}|{row}|{column}", "value": value,
+24 -9
View File
@@ -7,11 +7,12 @@
(tools/datasource.py)。这一路没有审批关卡,不能指望人在半路拦下一条 DELETE。
**先遮再交。** 建查询工具时还打开遮罩开关(tools/datasource 的 mask_results):数据源设了遮罩时,取自遮罩列的格子
(按 SQL 认,起别名、套函数、经子查询或 WITH 转一道、SELECT * 都算)在交给模型、落进查询快照之前就换成「已遮罩」,
同一份结果里别的文字格夹着的遮罩值也换掉(engine/masking.py 的 mask_query_result)。模型看不到遮罩值,就不会把它
写进回答、理由、SQL;以前是事后按值认,每一轮复核都找到新的漏法。快照里的 masked_at_query 记下查询时就遮了的列,
格子引用指到这几列的不采信。遮掉的值只在内存里交回调用方(VerifyOutcome.masked_values,审批质询按值遮的第二道防线
用),不进快照、不交给模型、不进 as_dict。
(按 SQL 认,起别名、套函数、经子查询或 WITH 转一道、SELECT *、表值函数拆开的都算)在交给模型、落进查询快照之前就
换成「已遮罩」,别的格子按值换:这份结果里的遮罩值、前面的查询遮掉的、调用方交进来的(masked_values:审批质询、
反方审稿收的报告快照里的值),文字格里夹着的换那一截(engine/masking.py 的 mask_query_result);查询失败时报错原话
里的值也换。模型看不到遮罩值,就不会把它写进回答、理由、SQL;以前是事后按值认,每一轮复核都找到新的漏法。快照里的
masked_at_query 记下查询时就遮了的列,格子引用指到这几列、指到按值换掉的格子的都不采信。遮掉的值只在内存里交回
调用方(VerifyOutcome.masked_values,审批质询、反方审稿按值遮的第二道防线用),不进快照、不交给模型、不进 as_dict。
**数从快照取。** 每次成功的查询是一份查询快照(工具自己落的 put_json),按这个 agent 自己的顺序编 Q1、Q2…。
最后交结果时,调用方结构里的数值字段一律改成格子引用 {alias, row, column},由代码按引用从快照里取值:模型自己
@@ -66,7 +67,7 @@ from app.engine import spend
from app.engine.evidence import build_catalog, cell_eid, compose_doc, query_entry_fields, schema_ledger_entry
from app.engine.expressions import CellError, column_kind, locate_cell, numeric_text
from app.engine.judge import FALLBACK_MAX_TOKENS, MAX_TOKENS, _json_in, _tokens, _usd
from app.engine.masking import _Secrets
from app.engine.masking import _masked_word, _Secrets
from app.engine.nodes.llm import _DIGEST_BUDGET, _digest, _split_structured, _typed, explain_model_error
from app.engine.state import message_text
from app.engine.toolcalls import ToolTimeout, limit_fields, limit_of, run_bounded
@@ -159,8 +160,8 @@ class VerifyOutcome:
output_tokens, total_tokens, cost_usd, calls, unpriced_calls}。limit:触顶原因(LIMITS 之一),没触顶为 None。
answered:模型是不是自己停下来的(没被上限掐断,也没因为输出到了上限被截断)。被截断的一律没有结果,error 是
TRUNCATED_ERROR。
masked_values:查询时在工具边界遮掉的值(engine/masking 的 _Secrets)。只在内存里交给调用方按值遮(审批质询的回答),
不进 as_dict、不往外交。
masked_values:查询时在工具边界遮掉的值(engine/masking 的 _Secrets),连同调用方交进来的。只在内存里交给调用方按值遮
(审批质询的回答、反方审稿的理由),不进 as_dict、不往外交。
"""
result: dict[str, Any] | None
@@ -428,6 +429,12 @@ class _Cells:
"row": row, "column": name, "artifact": query["artifact"],
"reason": f"{alias} 的「{name}」取自设置了遮罩的列,查询时已遮罩,不采信"})
return None
if raw == _masked_word() and isinstance(content, dict) and "masked_at_query" in content:
# 这一格查询时按值换掉了(和别的查询、报告里的遮罩值相同):同样照实说
self.cells.append({"path": path, "status": "unresolved", "ref": f"{alias}.r{row}.{name}", "alias": alias,
"row": row, "column": name, "artifact": query["artifact"],
"reason": f"{alias} 第 {row} 行「{name}」与设置了遮罩的值相同,查询时已遮罩,不采信"})
return None
truth = _typed(raw, {"type": sorted(types)}, column_kind(content, name))
cited = {"ref": f"{alias}.r{row}.{name}", "alias": alias, "row": row, "column": name,
"artifact": query["artifact"], "eid": cell_eid(query["artifact"], row, name), "value": truth}
@@ -663,7 +670,8 @@ class _State:
queries: list[dict[str, Any]] = field(default_factory=list)
ledger: list[dict[str, Any]] = field(default_factory=list)
model: str | None = None
#: 查询工具在工具边界遮掉的值(经 ToolContext.extra 并进来,只在内存里)
#: 查询工具在工具边界遮掉的值(经 ToolContext.extra 并进来,只在内存里),起头是调用方交进来的。后面的查询按它们
#: 按值遮
seen: _Secrets = field(default_factory=_Secrets)
def outcome(self, **fields: Any) -> VerifyOutcome:
@@ -695,6 +703,7 @@ async def run_verify_agent(
run_id: str = "",
node_id: str = "",
data_versions: dict[str, Any] | Versions | None = Versions.UNSET,
masked_values: Any = None,
) -> VerifyOutcome:
"""让一个只读 agent 查库,按 output_schema 交回有出处的结果。永远不抛。
@@ -709,9 +718,15 @@ async def run_verify_agent(
tool.end(role: verify,查询带 query_artifact);不在节点里(质询接口)就不传,什么都不发。
- run_id / node_id:查询快照、表结构快照记在谁名下。data_versions:上传表格用哪一版(运行固定的那份
data_versions;不在运行里传 Versions.CURRENT;缺省 UNSET 时上传源一律报错,不拿当前版本顶上)。
- masked_values:调用方已经认出的遮罩值(engine/masking 的 _Secrets,审批质询、反方审稿收的报告快照里的值)。
查询工具从第一条查询起就按它们按值遮:agent 查到的备注、别的表里写着同一个号码时,模型看不到。随查询时遮掉的值
一起交回(VerifyOutcome.masked_values)。
"""
state = _State(started=time.monotonic())
try:
if masked_values is not None:
# 上万个值并一遍要几十毫秒,放进线程,别挡住同一进程里别的运行
await asyncio.to_thread(state.seen.merge, masked_values)
return await _verify(state, spec=spec, system=system, task=task, sources=sources,
output_schema=output_schema, max_queries=max_queries, timeout_s=timeout_s,
max_cost_usd=max_cost_usd, daily_max_usd=daily_max_usd, max_steps=max_steps, emit=emit,
+27 -13
View File
@@ -468,6 +468,8 @@ def _make_query_tool(source: Any, ctx: ToolContext, *, fixed: bool = False,
缓存键相同,会把这个源平时用的可写连接也换成只读的。
mask_results:数据源设了遮罩时,取自遮罩列的格子(按 SQL 认,改了名字的也算)在交给模型、落进快照之前就换成
「已遮罩」,见 engine/masking.py 的 mask_query_result;这一次遮掉的值并进 ctx.extra[MASKED_VALUES_KEY](有的话)。
那里已经有的值(这个 agent 前面的查询遮掉的、调用方交进来的)这一次也按值换,数据源没设遮罩也一样:先查号码、
再查写着号码的备注或别的库,后一份同样遮好。查询失败时报错原话里的值也换(masking.mask_error_text)。
"""
# 数据源在 options.query_timeout_s 里配了就用它,否则取缺省。数据库按它停下语句,
# 引擎按它告诉界面上限是多少(metadata["timeout_s"])
@@ -495,6 +497,10 @@ def _make_query_tool(source: Any, ctx: ToolContext, *, fixed: bool = False,
if is_write(statement, dialect=(source.kind or "").lower()):
return f"SQL 被拒绝:{FORCED_READONLY}"
limits = QueryLimits(timeout_seconds=seconds, **({"max_rows": int(limit)} if limit else {}))
mask = masked_columns(source.options)
# 此前认出的遮罩值:交给线程的是一份拷贝,线程里读的时候这边不会再往里并
sink = ctx.extra.get(MASKED_VALUES_KEY) if mask_results else None
known = sink.copy() if sink is not None and sink.strong else None
try:
result = await run_query(source, sql, limits=limits)
except SqlRejected as e:
@@ -511,21 +517,29 @@ def _make_query_tool(source: Any, ctx: ToolContext, *, fixed: bool = False,
# 驱动的原话留着("no such table: x" 正是模型改写 SQL 要的线索),
# 类名和 SQLAlchemy 的包装前缀去掉:这一行也会原样出现在运行面板上
return f"查询失败:{first_line(e) or explain(e)[0]}"
said = first_line(e) or explain(e)[0]
if mask_results and (mask or known is not None):
from app.engine.masking import mask_error_text
# 数据库会把转换失败的值写进报错(invalid input syntax for type bigint: "139-0000-0001")
said = mask_error_text(said, sql, known)
return f"查询失败:{said}"
payload = result.to_payload()
payload["source"] = source.name
mask = masked_columns(source.options)
# 查询时就整列遮掉的结果列;没开遮罩开关、数据源没设遮罩时为 None(快照和原来一字不差)
# 查询时就整列遮掉的结果列;没开遮罩开关、数据源没设遮罩又没有可按值换的时为 None(快照和原来一字不差)
masked_now: list[str] | None = None
if mask_results and mask:
if mask_results and (mask or known is not None):
from app.engine.masking import mask_query_result
# 按 SQL 认列、逐格比对是纯计算,放进线程(同 _sql_checks)
done = await asyncio.to_thread(mask_query_result, payload, mask, source.kind)
payload, masked_now = done.payload, done.columns
if (sink := ctx.extra.get(MASKED_VALUES_KEY)) is not None:
sink.merge(done.secrets)
done = await asyncio.to_thread(mask_query_result, payload, mask, source.kind, known,
schema=source.schema_cache)
if mask or done.payload is not payload:
payload, masked_now = done.payload, done.columns
if sink is not None:
# 上万个值并一遍要几十毫秒(免遮的文字要重新过一遍),也放进线程
await asyncio.to_thread(sink.merge, done.secrets)
# 对照数据目录检查这条 SQL(一对多关联后重复计算、存量跨期求和……)。结果交给模型(Agent 据此改写),
# 也随查询快照存下:证据面板看得到,口径卡据此给指标标「存疑」、出具降档。没查出问题不加这个键,
# 快照和以前一字不差
@@ -541,15 +555,15 @@ def _make_query_tool(source: Any, ctx: ToolContext, *, fixed: bool = False,
# 只进快照、不进交给模型的结果的几样(模型看到多余的字段只会误会:以为数据被遮了,
# 或者把一串十六进制抄进报告):
# - 查询当时数据源设的遮罩列:数据源事后改名、删掉,证据面板仍按当时的遮。查询时就遮了的结果列(改了名字的
# 也算)一并记进去,证据面板、质询工件按这个键把它们画成遮罩列;
# - masked_at_query:开了遮罩开关时,查询时就整列遮掉的结果列(这几列的格子是「已遮罩」,不是数据源里的值);
# - 查询当时数据源设的遮罩列:数据源事后改名、删掉,证据面板仍按当时的遮。只记数据源设的列名——证据面板把
# 同源各份快照记的这一项并起来,从表的字段清单里去掉,混进结果列的别名会把同名的真实字段也去掉;
# - masked_at_query:开了遮罩开关时,查询时就整列遮掉的结果列(这几列的格子是「已遮罩」,不是数据源里的值,
# 改了名字的也在这里)。有这一项的快照,别的格子整格是「已遮罩」的是查询时按值换掉的;
# - data_version:上传表格查的是哪个快照,事后据此认得出这个数出自哪一版数据。
# 手工源、没设遮罩的,快照和原来一字不差
extra: dict[str, Any] = {}
if mask:
own = {m.lower() for m in mask}
extra["mask_columns"] = [*mask, *(c for c in masked_now or [] if c.lower() not in own)]
extra["mask_columns"] = mask
if masked_now is not None:
extra["masked_at_query"] = masked_now
if version := getattr(source, "snapshot_id", None):
+82
View File
@@ -1128,6 +1128,88 @@ async def test_the_challenge_agent_never_sees_masked_values(monkeypatch, tmp_pat
await engines.invalidate(source_id)
async def test_masked_values_written_into_the_rationale_are_masked_in_any_shape(monkeypatch, tmp_path):
"""复核第 1 轮:报告取数把遮罩列改了名字选出来(buyer_phone AS 手机号),摘录按列名遮不住,反方 agent 看得到号码,
把它换个写法写进理由(139-0000-0001、+86 139 0000 0002)。审稿记录里的理由正文、理由文档都换成「已遮罩」:值从这句
挂的报告快照里按 SQL 认出来收(和审批质询收报告快照的办法一样),连同 agent 查询时在工具边界遮掉的值;同时报告
快照里的值在反查之前就交给查询工具,agent 查到写着同一个号码的别的表时也看不到。以前理由原样存下、随运行封存。"""
name, source_id = await _source(tmp_path, options={"mask_columns": ["buyer_phone"]})
try:
sql = "SELECT region AS 区域, buyer_phone AS 手机号, amount AS 金额 FROM sales ORDER BY id"
said = "来自 139-0000-0001 号码的那笔属实,另有 +86 139 0000 0002 的一笔。"
plays = {"华东有一笔": Play(["SELECT COUNT(*) AS 笔数 FROM sales WHERE region = '华东'"], "stands", said)}
Models(monkeypatch, name, writer=("华东有一笔 [[v:Q1.r0.金额]] 的订单。[[see:Q1]]",), plays=plays)
row = await finish(graph(name, sql=sql), run_class="exploratory")
assert row.status == "succeeded", row.error
record = artifact_store.load((await output_of(row.id, "chal"))["challenge_artifact"])
[claim] = [c for c in record["claims"] if "华东有一笔" in c["text"]]
assert claim["verdict"] == "stands"
assert claim["rationale"] == "来自 已遮罩 号码的那笔属实,另有 已遮罩 的一笔。"
assert claim["rationale_doc"]["markdown"] == claim["rationale"]
assert claim["rationale_doc"]["source"] == claim["rationale"], "存下的原文也换掉"
shown = json.dumps(record, ensure_ascii=False)
assert "0000-0001" not in shown and "0000 0002" not in shown
finally:
await engines.invalidate(source_id)
async def test_the_challenge_agent_does_not_see_report_values_in_other_tables(monkeypatch, tmp_path):
"""报告取数选了手机号(遮罩列);反方 agent 去查回访记录,备注里写着同一个号码(备注没设遮罩)。报告快照里的遮罩值
在反查之前就交给了查询工具:交给模型的备注里那一截已经是「已遮罩」,它的查询快照里也是。以前原样交给模型。"""
path = tmp_path / f"{uuid.uuid4().hex[:8]}.db"
db = sqlite3.connect(path)
db.executescript(
"CREATE TABLE sales (id INTEGER PRIMARY KEY, region TEXT, buyer_phone TEXT, amount INTEGER, refunded INTEGER);"
"INSERT INTO sales VALUES (1, '华东', '13900000001', 1200, 0), (2, '华南', '13900000003', 900, 0);"
"CREATE TABLE callbacks (id INTEGER PRIMARY KEY, region TEXT, note TEXT);"
"INSERT INTO callbacks VALUES (1, '华东', '客户 139-0000-0001 来电改址'), (2, '华南', '例行回访');")
db.commit()
db.close()
name = f"shop_{uuid.uuid4().hex[:8]}"
async with SessionLocal() as session:
row = DataSource(name=name, kind="sqlite", database=str(path), readonly=True, enabled=True,
options={"mask_columns": ["buyer_phone"]})
session.add(row)
await session.flush()
row.schema_cache = await introspect.introspect(row)
await session.commit()
source_id = row.id
try:
sql = "SELECT region AS 区域, buyer_phone, amount AS 金额 FROM sales ORDER BY id"
notes = "SELECT region AS 区域, note AS 备注 FROM callbacks ORDER BY id"
plays = {"华东有一笔": Play([notes], "stands", "回访记录里有 [[v:Q1.r0.备注]]。[[see:Q1]]")}
models = Models(monkeypatch, name, writer=("华东有一笔 [[v:Q1.r0.金额]] 的订单。[[see:Q1]]",), plays=plays)
row = await finish(graph(name, sql=sql), run_class="exploratory")
assert row.status == "succeeded", row.error
seen = [str(m.content) for _, key, messages in models.calls if key == "华东有一笔"
for m in messages if isinstance(m, ToolMessage)]
assert any("客户 已遮罩 来电改址" in s for s in seen) and all("0000-0001" not in s for s in seen)
record = artifact_store.load((await output_of(row.id, "chal"))["challenge_artifact"])
[claim] = [c for c in record["claims"] if "华东有一笔" in c["text"]]
assert claim["rationale"] == "回访记录里有 客户 已遮罩 来电改址。"
snapshot = artifact_store.load(claim["queries"][0]["artifact"])
assert snapshot["rows"] == [["华东", "客户 已遮罩 来电改址"], ["华南", "例行回访"]]
finally:
await engines.invalidate(source_id)
def test_values_written_in_the_claim_itself_are_kept_in_the_rationale():
"""被审的那句结论原文里写着的号码已经摆在读者眼前:理由里复述它不换,别的号码照换。"""
from app.engine.evidence import compose_doc
from app.engine.masking import _Secrets
from app.engine.verify_agent import VerifyOutcome
secrets = _Secrets()
secrets.add({"columns": ["p"], "rows": [["13900000001"], ["13900000003"]]}, {"p"})
doc = compose_doc("号码 13900000003 的客户在华南,139-0000-0001 不在。", {}, node_id="chal", run_id="r1")
outcome = VerifyOutcome(result={"verdict": "stands", "rationale": doc["markdown"]}, cells=[], queries=[],
rationale=doc, usage={})
masked = challenge_mod._masked_rationale(outcome, secrets, given=["号码 13900000003 的客户在华南。"])
assert masked.result["rationale"] == "号码 13900000003 的客户在华南,已遮罩 不在。"
assert masked.rationale["markdown"] == masked.result["rationale"] and masked.rationale["node_id"] == "chal"
assert challenge_mod._masked_rationale(outcome, _Secrets(), given=[]) is outcome, "没有遮罩值时原样"
async def test_a_metric_excerpt_marks_the_report_query_numbers_too(monkeypatch, shop):
"""句子挂的是口径卡指标(正式报告的主路径「取数 → 口径卡 → 报告」):摘录正文里「取自查询 Q1」「查询 Q1 的 SQL」
也标成原报告的编号,和反查 agent 自己的 Q1 分开;工具按指标的输入取自的查询找源。"""
+32
View File
@@ -1055,6 +1055,38 @@ async def test_a_renamed_masked_column_in_the_blind_query_is_masked_before_the_b
assert {row[0] for row in shown["rows"]} == {"已遮罩"}, "交给盲算模型的结果里也是「已遮罩」"
async def test_a_metric_summed_over_a_masked_column_is_not_blind_computed(monkeypatch, tmp_path):
"""复核第 1 轮:金额这一列设了遮罩,指标是 SUM(amount)。核查 agent 的查询结果在工具边界遮好,汇总遮罩列的值也遮
(只有一行的汇总就是那一行的原值),盲算交回的格子注定取不到。所以派出盲算之前按 SQL 认出这一列由遮罩列算出,
直接按无法复算记、说清原因,不白花一次盲算的钱。以前只按列名认,gmv 认不出,盲算照样派出去,最后才在格子引用上
被拒,原因说成「盲算交回的格子引用不可用」。"""
name, source_id = await add_source(build(tmp_path / "amount_masked.db"), options={"mask_columns": "amount"})
try:
blind = Blind(monkeypatch, name, {"销售额": (BLIND_TOTAL, "blind_total")})
run = await start(graph(name, RIGHT))
evs = await events(run.id)
out = dual_output(run, evs)
[row] = out["metrics"]
assert row["status"] == "unchecked" and blind.runs == 0, "没派盲算"
assert row["reason"] == ("输入 cell(nodes.fetch, 0, 'gmv') 由设置了遮罩的列算出(经改名、函数或汇总),"
"盲算时会被遮罩,不能用于复算")
assert not [e for e in evs if e.type == "tool.end" and e.node_id == "dual"]
finally:
await drop_source(source_id)
async def test_a_metric_filtered_by_a_masked_column_inside_its_sum_is_still_blind_computed(monkeypatch, masked_shop):
"""遮罩列(区域)只在汇总的 CASE 条件里(SUM(CASE WHEN region = '华东' THEN amount ELSE 0 END)):同写在 WHERE 里,
值全部来自金额,原 SQL 和盲算的查询都不遮,照常比对。"""
name, _ = masked_shop
east = (f"SELECT SUM(CASE WHEN region = '华东' THEN amount ELSE 0 END) AS gmv FROM orders "
f"WHERE status <> 'refunded' AND paid_date {WEEK}")
blind = Blind(monkeypatch, name, {"销售额": (east, "gmv")})
run = await start(graph(name, east))
out = dual_output(run, await events(run.id))
assert blind.runs == 1 and out["metrics"][0]["status"] == "agreed", out["metrics"][0]
async def test_masked_group_columns_are_not_shown_to_the_blind_agent(monkeypatch, masked_shop):
name, _ = masked_shop
first = {**GMV, "id": "first", "name": "首组销售额", "expression": "cell(nodes.fetch, 0, 'gmv')"}
+118 -14
View File
@@ -1474,14 +1474,22 @@ CALLBACKS = SALES + ("CREATE TABLE callbacks (id INTEGER PRIMARY KEY, region TEX
LEAKED = "回答中含有设了遮罩的值,无法遮住,已不予展示,请换个问法重新提问"
async def test_a_masked_value_inside_a_text_cell_is_cut_out_of_the_artifact(monkeypatch, tmp_path, client):
"""备注里写着手机号(备注没设遮罩,手机号设了遮罩、被报告取数选中):agent 引用这一格,回答照旧整份不交;
质询工件里这份快照照样记着、点得开,那一格里的号码换成「已遮罩」,别的字、别的格子照原样,不然「不予展示」
旁边一点就能看到原值。"""
async def test_a_masked_value_inside_a_text_cell_is_cut_out_before_the_agent_sees_it(monkeypatch, tmp_path, client):
"""备注里写着手机号(备注没设遮罩,手机号设了遮罩、被报告取数选中):报告快照里的遮罩值在开始回答之前就交给核查
agent 的查询工具,agent 查备注时那一截就换成「已遮罩」,模型看不到;它引用这一格,回答照常给出,渲染出来也是遮好的。
质询工件、通用工件接口里这份快照同样遮好,别的字、别的格子照原样。以前模型看得到原值,回答整份不交。"""
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=CALLBACKS)
notes = "SELECT region AS 区域, note AS 备注 FROM callbacks ORDER BY id"
plays = {"回访": Play([notes], "华东的回访备注是 [[v:Q1.r0.备注]]。[[see:Q1]]")}
Models(monkeypatch, name, writer="金额最高的一笔 [[v:Q1.r0.金额]]。[[see:Q1]]", plays=plays)
seen: list[str] = []
answering = MockChatModel._decide
def spy(model, messages):
seen.extend(str(m.content) for m in messages if isinstance(m, ToolMessage))
return answering(model, messages)
monkeypatch.setattr(MockChatModel, "_decide", spy)
try:
report_sql = "SELECT buyer_phone, amount AS 金额 FROM sales ORDER BY amount DESC"
paused = await pause(name, graph=fetching(name, report_sql))
@@ -1491,20 +1499,114 @@ async def test_a_masked_value_inside_a_text_cell_is_cut_out_of_the_artifact(monk
assert r.status_code == 200, r.text
assert "1390000000" not in r.text
item = r.json()["inquiry"]
assert (item["answer"]["ok"], item["answer"]["text"], item["answer"]["doc_artifact"]) == (False, "", None)
assert item["answer"]["error"] == LEAKED
assert (item["answer"]["ok"], item["answer"]["text"]) == (True, "华东的回访备注是 客户已遮罩来电改址。")
[query] = item["queries"]
assert query["sql"] == notes
[event] = await events(paused.id, "approval.inquiry")
assert event.data["inquiry_artifacts"] == [query["artifact"]]
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{query['artifact']}")
assert got.status_code == 200, got.text
assert got.json()["content"]["rows"] == [["华东", "客户已遮罩来电改址"], ["华南", "例行回访"]]
assert got.json()["masked"] == [] and "1390000000" not in got.text
assert any("客户已遮罩来电改址" in s for s in seen) and all("1390000000" not in s for s in seen), \
"交给模型的查询结果里就是遮好的"
for url in (f"/api/runs/{paused.id}/inquiries/artifacts/{query['artifact']}",
f"/api/artifacts/{query['artifact']}"):
got = await client.get(url)
assert got.status_code == 200, got.text
assert got.json()["content"]["rows"] == [["华东", "客户已遮罩来电改址"], ["华南", "例行回访"]]
assert "1390000000" not in got.text
assert got.json().get("masked", []) == [], "备注只换了一截,不画成遮罩列"
finally:
await engines.invalidate(source_id)
#: 质询 agent 的两条查询:一条选出手机号(查询时遮掉),一条查回访备注(备注没设遮罩,里面写着手机号)
PHONES_EAST = "SELECT buyer_phone AS 号码, amount AS 金额 FROM sales WHERE region = '华东' ORDER BY amount DESC"
NOTES = "SELECT region AS 区域, note AS 备注 FROM callbacks ORDER BY id"
async def test_a_number_from_the_agents_other_query_is_masked_when_the_number_came_first(monkeypatch, tmp_path, client):
"""复核第 1 轮(对基线回退):报告取数没选手机号,号码只出现在 agent 自己的另一次查询里。先查号码、后查备注:号码
查询时就遮掉、值并进内存,后面那次查询的备注在交给模型、落库之前就按值换掉,回答照常给出;质询工件、通用工件接口
给的都是遮好的。以前事后取质询工件时收不到这个号码,备注原样给出。"""
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=CALLBACKS)
plays = {"回访": Play([PHONES_EAST, NOTES], "华东的回访备注是 [[v:Q2.r0.备注]]。[[see:Q2]]")}
Models(monkeypatch, name, writer="华东销售额 [[v:Q1.r0.销售额]]。[[see:Q1]]", plays=plays)
try:
paused = await pause(name, graph=fetching(name, QUERY))
[approval] = await pending(paused.id)
r = await ask(client, paused.id, report="write", unit=None, question="华东的回访说了什么?",
approval_id=approval.id)
assert r.status_code == 200, r.text
item = r.json()["inquiry"]
assert (item["answer"]["ok"], item["answer"]["text"]) == (True, "华东的回访备注是 客户已遮罩来电改址。")
phones, notes = item["queries"]
for url in (f"/api/runs/{paused.id}/inquiries/artifacts/{notes['artifact']}",
f"/api/artifacts/{notes['artifact']}"):
got = await client.get(url)
assert got.status_code == 200, got.text
assert got.json()["content"]["rows"] == [["华东", "客户已遮罩来电改址"], ["华南", "例行回访"]]
assert "1390000000" not in got.text
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{phones['artifact']}")
assert got.json()["content"]["rows"] == [["已遮罩", 1200], ["已遮罩", 300]] and got.json()["masked"] == ["号码"]
[event] = await events(paused.id, "approval.inquiry")
assert "1390000000" not in json.dumps(event.data, ensure_ascii=False)
finally:
await engines.invalidate(source_id)
async def test_a_number_from_the_agents_other_query_is_masked_when_the_number_came_last(monkeypatch, tmp_path, client):
"""先查备注、后查号码:查备注时还不知道里面写着遮罩值,原样交给模型、原样存下;回答引用这一格,渲染出来才露,整份
不交。回答时按内存里的值换掉的位置记进事件(不含值),事后取质询工件照着再换一遍:质询工件里那一截是「已遮罩」,
和回答时一样。"""
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=CALLBACKS)
plays = {"回访": Play([NOTES, PHONES_EAST], "华东的回访备注是 [[v:Q1.r0.备注]]。[[see:Q1]]")}
Models(monkeypatch, name, writer="华东销售额 [[v:Q1.r0.销售额]]。[[see:Q1]]", plays=plays)
try:
paused = await pause(name, graph=fetching(name, QUERY))
[approval] = await pending(paused.id)
r = await ask(client, paused.id, report="write", unit=None, question="华东的回访说了什么?",
approval_id=approval.id)
assert r.status_code == 200, r.text
assert "1390000000" not in r.text
item = r.json()["inquiry"]
assert (item["answer"]["ok"], item["answer"]["error"]) == (False, LEAKED)
notes, phones = item["queries"]
[event] = await events(paused.id, "approval.inquiry")
assert "1390000000" not in json.dumps(event.data, ensure_ascii=False)
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{notes['artifact']}")
assert got.status_code == 200, got.text
assert got.json()["content"]["rows"] == [["华东", "客户已遮罩来电改址"], ["华南", "例行回访"]]
assert "1390000000" not in got.text
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{phones['artifact']}")
assert got.json()["content"]["rows"] == [["已遮罩", 1200], ["已遮罩", 300]]
finally:
await engines.invalidate(source_id)
async def test_a_number_written_in_another_table_is_masked_whichever_query_came_first(monkeypatch, tmp_path, client):
"""另一张表(联系方式)的号码列没设遮罩,存的是同一批号码:先查手机号的,查询时就按值换;后查的,事后取质询工件
照回答时记下的位置换。两种先后都整列画成遮罩列。"""
phones = "SELECT buyer_phone AS 手机号, amount AS 金额 FROM sales ORDER BY id"
visits = "SELECT phone_num AS 号码, visits AS 次数 FROM contacts ORDER BY id"
for order in ([phones, visits], [visits, phones]):
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=CONTACTS)
at = order.index(visits) + 1
plays = {"常来": Play(order, f"最常来的客户来过 [[v:Q{at}.r0.次数]] 次。[[see:Q{at}]]")}
Models(monkeypatch, name, writer="华东销售额 [[v:Q1.r0.销售额]]。[[see:Q1]]", plays=plays)
try:
paused = await pause(name, graph=fetching(name, QUERY))
[approval] = await pending(paused.id)
r = await ask(client, paused.id, report="write", unit=None, question="常来的客户来过几次?",
approval_id=approval.id)
assert r.status_code == 200, r.text
assert r.json()["inquiry"]["answer"]["text"] == "最常来的客户来过 5 次。"
query = r.json()["inquiry"]["queries"][at - 1]
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{query['artifact']}")
assert got.status_code == 200, got.text
assert got.json()["content"]["rows"] == [["已遮罩", 5], ["已遮罩", 2]]
assert got.json()["masked"] == ["号码"] and "1390000000" not in got.text
finally:
await engines.invalidate(source_id)
async def test_sql_check_excerpts_in_the_artifact_are_masked_like_the_sql(monkeypatch, masked_shop, client):
"""agent 的查询带着 SQL 检查结果(对照数据目录查出的问题,片段摘自这条 SQL):质询工件里片段照查询清单的 SQL
换掉遮罩值,给模型的那句改法照抄了片段,不交出去(同证据面板)。"""
@@ -1727,8 +1829,10 @@ REWRITTEN_CALLBACKS = SALES + ("CREATE TABLE callbacks (id INTEGER PRIMARY KEY,
"(2, '华南', '例行回访');")
async def test_a_reformatted_number_inside_a_text_cell_stops_the_answer(monkeypatch, tmp_path, client):
"""备注里夹着换了写法的号码,agent 引用这一格:渲染出来才露,最后一道关认得出,回答整份不交;质询工件里只换掉那一截。"""
async def test_a_reformatted_number_inside_a_text_cell_is_cut_out_before_the_agent_sees_it(monkeypatch, tmp_path,
client):
"""备注里夹着换了写法的号码(报告取数选了手机号):开始回答之前就认得出,agent 查备注时那一截已经换掉,引用这一格
回答照常给出;质询工件里同样只换掉那一截。"""
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=REWRITTEN_CALLBACKS)
notes = "SELECT region AS 区域, note AS 备注 FROM callbacks ORDER BY id"
plays = {"回访": Play([notes], "华东的回访备注是 [[v:Q1.r0.备注]]。[[see:Q1]]")}
@@ -1740,12 +1844,12 @@ async def test_a_reformatted_number_inside_a_text_cell_stops_the_answer(monkeypa
approval_id=approval.id)
assert r.status_code == 200, r.text
item = r.json()["inquiry"]
assert (item["answer"]["ok"], item["answer"]["error"]) == (False, LEAKED)
assert (item["answer"]["ok"], item["answer"]["text"]) == (True, "华东的回访备注是 客户 已遮罩 来电改址。")
[query] = item["queries"]
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{query['artifact']}")
assert got.status_code == 200, got.text
assert got.json()["content"]["rows"] == [["华东", "客户 已遮罩 来电改址"], ["华南", "例行回访"]]
assert "0000-0001" not in got.text
assert "0000-0001" not in got.text and "0000-0001" not in r.text
finally:
await engines.invalidate(source_id)
+113
View File
@@ -402,6 +402,66 @@ def test_a_label_ahead_of_a_star_or_alone_behind_one_is_still_a_label_column(sql
assert inquiry_mod._label_columns(sql, "sqlite", ["档"]) == {"档": {"0", "1"}}
#: 没写表名的 * 连着数据源里的表和装着标签的派生来源
STAR_JOIN = ("SELECT * FROM sales s JOIN (SELECT id AS bid, CASE WHEN amount > 1000 THEN '大单' ELSE '小单' END AS 档 "
"FROM sales) b ON s.id = b.bid")
STAR_COLUMNS = ["id", "region", "amount", "bid", "档"]
def test_a_star_over_a_table_known_to_lack_the_column_keeps_the_label():
"""复核第 1 轮(收紧之后的多遮):没写表名的 * 展开数据源里的表,以前一律算「说不清带不带这一列」,标签照收成
遮罩值,回答里的「大单」、SQL 里的 '大单' 都换成「已遮罩」。冻结的表结构里查得到这张表、又没有同名的列时,照旧
按派生来源认标签;表里有同名的列、查不到表结构的,照旧不算(宁可多遮)。"""
from app.engine import masking
lacking = {"tables": {"sales": {"columns": [{"name": "id"}, {"name": "region"}, {"name": "amount"}]}}}
holding = {"tables": {"sales": {"columns": [{"name": "id"}, {"name": "档"}, {"name": "amount"}]}}}
assert masking._label_columns(STAR_JOIN, "sqlite", STAR_COLUMNS, masking._schema_tables(lacking)) == {
"档": {"大单", "小单"}}
assert masking._label_columns(STAR_JOIN, "sqlite", STAR_COLUMNS, masking._schema_tables(holding)) == {}
assert masking._label_columns(STAR_JOIN, "sqlite", STAR_COLUMNS) == {}
truncated = {"tables": {"orders": {"columns": [{"name": "id"}]}}, "truncated": True}
assert masking._label_columns(STAR_JOIN, "sqlite", STAR_COLUMNS, masking._schema_tables(truncated)) == {}
# 写了表名的 s.* 展开的是数据源里的表:同样按表结构认它带不带这一列
nested = ("SELECT x.档 FROM (SELECT s.*, b.* FROM sales s JOIN (SELECT id AS bid, CASE WHEN amount > 1000 "
"THEN '大单' ELSE '小单' END AS 档 FROM sales) b ON s.id = b.bid) x")
assert masking._label_columns(nested, "sqlite", ["档"], masking._schema_tables(lacking)) == {"档": {"大单", "小单"}}
assert masking._label_columns(nested, "sqlite", ["档"], masking._schema_tables(holding)) == {}
assert masking._label_columns(nested, "sqlite", ["档"]) == {}
async def test_a_snapshot_reads_labels_with_its_frozen_schema():
"""快照记着查询当时冻结的表结构(schema_artifact):按它认,* 连着的表没有同名的列时标签不当遮罩值,回答正文、
SQL 里的档名照原样;表结构取不到的照旧不算。"""
from app.core import artifact_store
from app.engine import masking
lacking = {"source": "shop", "tables": {"sales": {"columns": [{"name": "id"}, {"name": "region"},
{"name": "amount"}]}}}
schema = await artifact_store.put_json(lacking, kind="schema_snapshot", run_id="r-labels", node_id="n1")
snapshot = {"source": "shop", "sql": STAR_JOIN, "columns": STAR_COLUMNS, "schema_artifact": schema,
"rows": [[1, "华东", 1200, 1, "大单"], [2, "华南", 300, 2, "小单"]]}
assert masking._snapshot_labels(snapshot, "sqlite") == {"档": {"大单", "小单"}}
plan = plan_of(snapshot, {"amount"})
assert plan.secrets.scrub("这一笔属于大单") == "这一笔属于大单"
sql = "SELECT COUNT(*) FROM (SELECT CASE WHEN amount > 1000 THEN '大单' ELSE '小单' END AS 档 FROM sales) WHERE 档 = '大单'"
assert inquiry_mod._scrub_sql(sql, plan.secrets, plan.tainted["a1"], "sqlite") == sql
assert masking._snapshot_labels({**snapshot, "schema_artifact": "missing"}, "sqlite") == {}
def test_the_tool_boundary_reads_labels_with_the_sources_frozen_schema():
"""工具边界按数据源的表结构认标签:* 连着的表没有同名的列,标签不收成遮罩值,同一份结果里别的文字格里的「大单」照常。"""
from app.engine import masking
payload = {"sql": STAR_JOIN, "columns": [*STAR_COLUMNS, "备注"],
"rows": [[1, "华东", 1200, 1, "大单", "大单客户"], [2, "华南", 300, 2, "小单", "例行回访"]]}
schema = {"tables": {"sales": {"columns": [{"name": "id"}, {"name": "region"}, {"name": "amount"},
{"name": "备注"}]}}}
done = masking.mask_query_result(payload, ["amount"], "sqlite", schema=schema)
assert done.columns == ["amount", "档"] and not {"大单", "小单"} & done.secrets.strong
assert [row[-1] for row in done.payload["rows"]] == ["大单客户", "例行回访"]
# --------------------------------------------------------------------------
# 按值认:门槛、边界
# --------------------------------------------------------------------------
@@ -1373,3 +1433,56 @@ def test_hundreds_of_band_edges_are_read_within_a_budget():
assert "已遮罩" in inquiry_mod._scrub_sql(bins(600), every, {"age"}, "sqlite")
elapsed = time.perf_counter() - started
assert elapsed < 5, f"用了 {elapsed:.2f} 秒"
# --------------------------------------------------------------------------
# 回答时才认得出的值:记下位置,事后取质询工件时照着再换一遍
# --------------------------------------------------------------------------
def test_values_known_only_at_answer_time_are_recorded_as_positions_and_replayed():
"""agent 先查了备注(那时还不知道备注里的号码是遮罩值,查询时原样交回、原样存下),后来又查出了手机号(查询时
遮掉,值只在内存里)。回答时按内存里的值换掉的地方记成位置(不含值);事后取质询工件时内存里的值已经不在,照着
位置再换一遍,和回答时看到的一样。"""
from app.engine import masking
note = "客户13900000001来电改址,备用 139-0000-0001"
snapshot = {"columns": ["区域", "备注", "号码", "13900000001 的单"],
"rows": [["华东", note, "13900000001", 1], ["华南", "例行回访", "13800000000", 2]]}
seen = secrets_of({"columns": ["phone"], "rows": [["13900000001"]]}, {"phone"})
edits = masking.view_edits(snapshot, set(), seen, inquiry_mod._Secrets())
first, second = note.index("13900000001"), note.index("139-0000-0001")
assert edits == {"cells": [[0, 2]], "spans": [[0, 1, first, first + 11], [0, 1, second, second + 13]],
"names": [[3, 0, 11]], "masked": ["号码"]}
assert "1390000000" not in json.dumps(edits, ensure_ascii=False), "只记位置,不记值"
replayed = masking.replay_edits(snapshot, [edits])
assert replayed["columns"] == ["区域", "备注", "号码", f"{MASKED} 的单"]
assert replayed["rows"] == [["华东", f"客户{MASKED}来电改址,备用 {MASKED}", MASKED, 1],
["华南", "例行回访", "13800000000", 2]]
# 不靠内存里的值也认得出的(报告快照里就有),事后照样认得出,不记
assert masking.view_edits(snapshot, set(), seen, seen) is None
# 记下的位置对不上(越界、坏形状)的跳过,不出错
broken = {"cells": [[9, 9], "x"], "spans": [[0, 1, 50, 60], [0, 9, 0, 1]], "names": [[7, 0, 1]], "masked": [3]}
assert masking.replay_edits(snapshot, [broken, None, "x"]) == snapshot
def test_the_plan_records_what_only_the_values_seen_by_the_agent_masked():
"""_settle_plan 带上 agent 查询时遮掉的值(seen)时,另记下只靠它们才换掉的位置(replay);不带 seen 的(事后取
质询工件)不记。"""
notes = {"source": "shop", "sql": "SELECT region AS 区域, note AS 备注 FROM callbacks", "columns": ["区域", "备注"],
"rows": [["华东", "客户13900000001来电改址"], ["华南", "例行回访"]]}
seen = secrets_of({"columns": ["phone"], "rows": [["13900000001"]]}, {"phone"})
plan = inquiry_mod._settle_plan({"a1": notes}, {"a1": ({"buyer_phone"}, [])}, [], {"shop": "sqlite"}, [], [], {},
frozenset(), seen)
assert plan.replay == {"a1": {"cells": [], "spans": [[0, 1, 2, 13]], "names": [], "masked": []}}
later = inquiry_mod._settle_plan({"a1": notes}, {"a1": ({"buyer_phone"}, [])}, [], {"shop": "sqlite"}, [], [], {},
frozenset({"a1"}))
assert later.replay == {} and later.views["a1"]["rows"][0][1] == "客户13900000001来电改址"
def test_values_the_agent_saw_survive_a_report_that_fills_the_cap():
"""报告快照里的遮罩值收满了上限(SECRETS_MAX):agent 查询时遮掉的值先收,照样认得出(以前先收报告的,满了就丢)。"""
report = {"columns": ["buyer_phone"], "rows": [[str(13800000000 + i)] for i in range(inquiry_mod.SECRETS_MAX)]}
seen = secrets_of({"columns": ["phone"], "rows": [["19914742785"]]}, {"phone"})
plan = inquiry_mod._settle_plan({}, {}, [(report, {"buyer_phone"})], {}, [], [], {}, frozenset(), seen)
assert plan.secrets.hit("19914742785")
@@ -137,3 +137,73 @@ def test_many_masked_numbers_are_matched_without_stalling():
out = secrets.scrub(text, prose=True)
assert time.perf_counter() - began < 2.0
assert "199-0000" not in out and out.count(MASKED) == text.count("199-0000")
#: 复核第 1 轮:别的空白、减号作分隔符的写法(模型输出里不换行空格、窄空格并不少见)
OTHER_SEPARATORS = [
pytest.param("199 1474 2785", id="不换行空格"),
pytest.param("199 1474 2785", id="窄不换行空格"),
pytest.param("199 1474 2785", id="细空格"),
pytest.param("199 1474 2785", id="发丝空格"),
pytest.param("199\t1474\t2785", id="制表符"),
pytest.param("199−1474−2785", id="减号"),
pytest.param("199 1474 2785", id="四个空格"),
pytest.param("199 - 1474 - 2785", id="连字符两边带空格"),
pytest.param("+86 199 1474 2785", id="+86 加不换行空格"),
]
@pytest.mark.parametrize("written", OTHER_SEPARATORS)
def test_numbers_split_by_other_spaces_and_minus_signs_are_masked_too(written):
secrets = secrets_of([PHONE])
assert secrets.scrub(f"客户 {written} 来电", prose=True) == f"客户 {MASKED} 来电"
assert inquiry_mod._leaked({"markdown": f"来电的是 {written}。", "blocks": []}, secrets)
assert secrets.hit(written), "整格就是这个号码的,格子按值认"
def test_a_line_break_still_ends_a_number():
"""不跨行:两行各写一截的不拼成一个号码。"""
secrets = secrets_of([PHONE])
assert secrets.scrub("编号 199\n1474 2785", prose=True) == "编号 199\n1474 2785"
@pytest.mark.parametrize("sql, sealed", [
pytest.param("SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note LIKE '%199-1474-2785%'",
f"SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note LIKE '%{MASKED}%'", id="LIKE 里的一截"),
pytest.param("SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note = '客户 199 1474 2785 来电'",
f"SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note = '客户 {MASKED} 来电'", id="字符串里夹着"),
pytest.param('SELECT note AS "199-1474-2785 的备注" FROM callbacks',
f'SELECT note AS "{MASKED} 的备注" FROM callbacks', id="带引号的列名"),
pytest.param("SELECT COUNT(*) AS 笔数 FROM callbacks /* 客户 199-1474-2785 */",
f"SELECT COUNT(*) AS 笔数 FROM callbacks /* 客户 {MASKED} */", id="块注释"),
pytest.param("SELECT COUNT(*) AS 笔数 FROM callbacks -- 客户 199 1474 2785\n WHERE id = 1",
f"SELECT COUNT(*) AS 笔数 FROM callbacks -- 客户 {MASKED}\n WHERE id = 1", id="行注释"),
])
def test_a_reformatted_number_in_sql_strings_names_and_comments_is_masked(sql, sealed):
"""SQL 里字符串、带引号的名字、注释里换了写法的号码也认(以前只有整个字面量就是号码的才换)。"""
assert inquiry_mod._scrub_sql(sql, secrets_of([PHONE]), set(), "sqlite") == sealed
def test_a_bare_number_expression_in_sql_is_still_left_alone():
"""不带引号的 199-1474-2785 在 SQL 里是一个算式(三个数相减),照旧不按号码认。"""
sql = "SELECT 199-1474-2785 AS x FROM callbacks"
assert inquiry_mod._scrub_sql(sql, secrets_of([PHONE]), set(), "sqlite") == sql
def test_a_threshold_matching_a_number_swapped_in_a_string_is_not_kept():
"""字符串里按换了写法认出、换掉的号码,同一条 SQL 里写成阈值的同一个数也不留(不然阈值把它露了)。"""
sql = f"SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note LIKE '%199-1474-2785%' AND phone_no >= {PHONE}"
sealed = inquiry_mod._scrub_sql(sql, secrets_of([PHONE]), set(), "sqlite")
assert PHONE not in sealed and "1474" not in sealed
def test_a_reformatted_number_in_a_column_name_is_masked_like_the_sql():
"""agent 把换了写法的号码写进列名(AS "139-0000-0001 的单"):质询工件的列名、回答里引用这一列的标记照 SQL 里带引号
的名字一样换,三处对得上。"""
secrets = secrets_of(["13900000001"])
snapshot = {"columns": ["139-0000-0001 的单"], "rows": [[3]]}
view, _ = _masked_view(snapshot, set(), secrets)
assert view["columns"] == [f"{MASKED} 的单"]
assert inquiry_mod._scrub_sql('SELECT COUNT(*) AS "139-0000-0001 的单" FROM sales', secrets, set(), "sqlite") == (
f'SELECT COUNT(*) AS "{MASKED} 的单" FROM sales')
assert inquiry_mod._scrubbed_source("共 [[v:Q1.r0.139-0000-0001 的单]] 单。", secrets) == f"共 [[v:Q1.r0.{MASKED} 的单]] 单。"
@@ -0,0 +1,109 @@
"""按 SQL 认取自遮罩列的结果列(engine/masking.py 的 _derived_hidden):FROM、JOIN、LATERAL 里的表值函数,汇总里的 CASE。
核查 agent 的查询在工具边界按这套认法整列遮掉(mask_query_result),审批质询、双盲复算也用它:
- 表值函数(SQLite 的 json_each / json_tree,Postgres 的 unnest、jsonb_each_text、regexp_split_to_table、LATERAL
VALUES,MySQL 的 JSON_TABLE)把遮罩列拆开时,它出的列都算取自遮罩列:遮罩列是 JSON 时,用 json_each 拆开是最自然的
写法,以前只追 SELECT 项、子查询和 *,这些写法的原值原样交给模型、落进快照。出列名认得全的(常见的几个函数、写了
列名表的)按名字认,认不全的整份遮
- 汇总里的 CASE / IIF 只在条件里用到遮罩列的(SUM(CASE WHEN buyer_phone IS NOT NULL THEN amount ELSE 0 END)),同
写在 WHERE 里,不算;结果的某一支取的是遮罩列、不汇总的逐行 CASE 照旧算
数据都是虚构的:手机号、区域、金额。
"""
from __future__ import annotations
import pytest
from app.engine import masking
#: 设了遮罩的列:手机号,和装着手机号、姓名的 JSON 列
MASKED = {"buyer_phone", "profile"}
SALES = ["id", "region", "buyer_phone", "amount", "profile"]
#: SQLite 的 json_each / json_tree 出的列
JSON_EACH = ["key", "value", "type", "atom", "id", "parent", "fullkey", "path"]
def hidden_of(sql: str, columns: list[str], kind: str = "sqlite", masked: set[str] = MASKED) -> set[str]:
snapshot = {"source": "shop", "sql": sql, "columns": columns, "rows": []}
return masking._derived_hidden(snapshot, set(masked), kind)[0]
@pytest.mark.parametrize("kind, sql, columns, shown", [
pytest.param("sqlite", "SELECT j.key, j.value FROM sales, json_each(sales.profile) AS j", ["key", "value"],
["key", "value"], id="json_each 拆遮罩的 JSON 列"),
pytest.param("sqlite", "SELECT t.atom AS 值 FROM sales, json_tree(sales.profile) AS t WHERE t.type = 'text'",
["值"], ["值"], id="json_tree"),
pytest.param("sqlite", "SELECT value FROM sales, json_each(json_array(buyer_phone))", ["value"], ["value"],
id="json_each 套 json_array"),
pytest.param("sqlite", "SELECT r.value AS 号码 FROM sales s JOIN json_each(s.profile) r ON 1 = 1", ["号码"],
["号码"], id="JOIN 里的 json_each"),
pytest.param("sqlite", "SELECT value FROM json_each((SELECT json_group_array(buyer_phone) FROM sales))",
["value"], ["value"], id="参数是子查询"),
pytest.param("sqlite", "SELECT c.value AS 号码 FROM (SELECT json_group_array(buyer_phone) AS arr FROM sales) t, "
"json_each(t.arr) c", ["号码"], ["号码"], id="参数取自派生来源里改了名字的遮罩列"),
pytest.param("sqlite", "SELECT * FROM sales, json_each(sales.profile)", [*SALES, *JSON_EACH],
["buyer_phone", "profile", "key", "value", "atom", "fullkey", "path"], id="* 带出 json_each 的列"),
pytest.param("sqlite", "SELECT x FROM (SELECT j.value AS x FROM sales, json_each(sales.profile) j) t", ["x"],
["x"], id="子查询里拆开再改名"),
pytest.param("postgres", "SELECT u FROM sales, unnest(ARRAY[buyer_phone]) AS u", ["u"], ["u"], id="unnest"),
pytest.param("postgres", "SELECT x.p FROM sales s CROSS JOIN LATERAL unnest(ARRAY[s.buyer_phone]) AS x(p)",
["p"], ["p"], id="LATERAL unnest 带列名表"),
pytest.param("postgres", "SELECT x FROM sales, LATERAL unnest(ARRAY[buyer_phone]) AS x", ["x"], ["x"],
id="LATERAL unnest"),
pytest.param("postgres", "SELECT u FROM sales CROSS JOIN LATERAL (VALUES (buyer_phone)) AS v(u)", ["u"], ["u"],
id="LATERAL VALUES"),
pytest.param("postgres", "SELECT d FROM sales, regexp_split_to_table(buyer_phone, '-') AS d", ["d"], ["d"],
id="regexp_split_to_table"),
pytest.param("postgres", "SELECT e.key, e.value FROM sales, jsonb_each_text(profile) AS e", ["key", "value"],
["key", "value"], id="jsonb_each_text"),
pytest.param("postgres", "SELECT x FROM sales, unnest(string_to_array(buyer_phone, ',')) AS x", ["x"], ["x"],
id="unnest 套 string_to_array"),
pytest.param("postgres", "SELECT * FROM sales, unnest(ARRAY[buyer_phone])", [*SALES, "unnest"],
["buyer_phone", "profile", "unnest"], id="没起别名的 unnest 按函数名出列"),
pytest.param("mysql", "SELECT j.p FROM sales s, JSON_TABLE(JSON_ARRAY(s.buyer_phone), '$[*]' "
"COLUMNS (p VARCHAR(20) PATH '$')) j", ["p"], ["p"], id="JSON_TABLE"),
])
def test_a_table_function_over_a_masked_column_hides_what_it_returns(kind, sql, columns, shown):
assert {c.lower() for c in shown} <= hidden_of(sql, columns, kind)
def test_columns_a_table_function_does_not_return_stay_visible():
"""拆的是遮罩列,别的列(区域、金额)照常;拆的不是遮罩列的,一列都不遮。"""
hidden = hidden_of("SELECT s.region, s.amount, j.value AS 值 FROM sales s, json_each(s.profile) j",
["region", "amount", "值"])
assert "值" in hidden and not {"region", "amount"} & hidden
assert hidden_of("SELECT s.region, j.value AS 标签 FROM sales s, json_each(s.tags) j", ["region", "标签"]) == MASKED
def test_a_table_function_that_cannot_be_read_hides_the_whole_snapshot():
"""不认得的函数拆遮罩列、又没写列名表:说不清出哪几列,整份快照的列都遮(同 SQL 解析不了)。"""
columns = ["region", "piece"]
assert hidden_of("SELECT s.region, t.piece FROM sales s, my_split(s.buyer_phone) t", columns, "postgres") == {
*MASKED, "region", "piece"}
@pytest.mark.parametrize("sql", [
"SELECT SUM(CASE WHEN buyer_phone IS NOT NULL THEN amount ELSE 0 END) AS total FROM sales",
"SELECT SUM(CASE buyer_phone WHEN '13900000001' THEN amount END) AS total FROM sales",
"SELECT AVG(IIF(buyer_phone LIKE '139%', amount, NULL)) AS total FROM sales",
"SELECT MAX(CASE WHEN buyer_phone IS NULL THEN 0 ELSE amount END) AS total FROM sales",
"SELECT SUM(CASE WHEN buyer_phone IS NULL THEN 0 ELSE CASE WHEN region = '华东' THEN amount END END) AS total "
"FROM sales",
])
def test_a_masked_column_only_in_the_condition_of_an_aggregated_case_is_like_a_filter(sql):
"""值全部来自金额,手机号只决定这一行算不算进来:同写在 WHERE 里,不算取自遮罩列。"""
assert hidden_of(sql, ["total"]) == MASKED
assert hidden_of(sql.replace("CASE WHEN buyer_phone IS NOT NULL THEN amount ELSE 0 END", "amount")
+ " WHERE buyer_phone IS NOT NULL", ["total"]) == MASKED
@pytest.mark.parametrize("sql", [
# 结果的某一支取的就是遮罩列
"SELECT SUM(CASE WHEN region = '华东' THEN buyer_phone END) AS total FROM sales",
"SELECT MAX(IIF(amount > 1000, buyer_phone, NULL)) AS total FROM sales",
# 不汇总的逐行 CASE:每一行各自落在哪一支
"SELECT CASE WHEN buyer_phone IS NULL THEN 0 ELSE amount END AS total FROM sales",
])
def test_a_case_returning_the_masked_value_or_not_aggregated_is_still_derived(sql):
assert "total" in hidden_of(sql, ["total"])
+142 -3
View File
@@ -34,12 +34,18 @@ from tests.test_verify_agent import SPEC, Script, q
MASKED = "已遮罩"
#: 遮罩列里的值(和备注里换了写法的号码):交给模型的、存下的、接口给的都不能出现
SECRETS = ("1390000000", "张伟", "李娜", "王芳", "刘洋", "139-0000-0003")
SECRETS = ("1390000000", "张伟", "李娜", "王芳", "刘洋", "139-0000-0003", "139 0000 0003")
SALES = ("CREATE TABLE sales (id INTEGER PRIMARY KEY, region TEXT, buyer_phone TEXT, buyer_name TEXT, amount INTEGER,"
" note TEXT);"
"INSERT INTO sales VALUES (1, '华东', '13900000001', '张伟', 1200, '客户13900000001来电改址'),"
" (2, '华东', '13900000002', '李娜', 300, '例行回访'), (3, '华南', '13900000003', '王芳', 900, '回访:联系 139-0000-0003'),"
" (4, '华北', '13900000004', '刘洋', 400, '');")
" (4, '华北', '13900000004', '刘洋', 400, '');"
# 联系方式表的号码列、客户资料的 JSON 列都没设遮罩(json_shop 另设了 profile):同一批号码、姓名写在这里
"CREATE TABLE contacts (id INTEGER PRIMARY KEY, phone_num TEXT, visits INTEGER);"
"INSERT INTO contacts VALUES (1, '13900000001', 5), (2, '139 0000 0003', 2), (3, '13800000000', 1);"
"CREATE TABLE customers (id INTEGER PRIMARY KEY, region TEXT, profile TEXT);"
"INSERT INTO customers VALUES (1, '华东', '{\"phone\": \"13900000001\", \"name\": \"张伟\"}'),"
" (2, '华南', '{\"phone\": \"13900000002\", \"name\": \"李娜\"}');")
VALUE_SCHEMA = {"type": "object", "properties": {"value": {"type": "number", "description": "要核对的数"}},
"required": ["value"]}
@@ -76,6 +82,14 @@ async def masked_shop(tmp_path):
await engines.invalidate(source_id)
@pytest.fixture
async def json_shop(tmp_path):
"""同一个库,客户资料(JSON:手机号、姓名)这一列设了遮罩。返回源名。"""
name, source_id = await _source(tmp_path, {"mask_columns": "profile"})
yield name
await engines.invalidate(source_id)
@pytest.fixture
async def plain_shop(tmp_path):
name, source_id = await _source(tmp_path, None)
@@ -137,7 +151,9 @@ async def test_masked_columns_are_masked_before_the_model_sees_them(monkeypatch,
assert snapshot.get("column_types", {}).get(column, "text") == "text"
assert _cells(snapshot, amount) == [1200, 300, 900, 400]
assert snapshot["masked_at_query"] == masked
assert {c.lower() for c in snapshot["mask_columns"]} == {"buyer_phone", "buyer_name", *(c.lower() for c in masked)}
# mask_columns 照记查询当时数据源设的遮罩列(证据面板按它从表的字段清单里去掉列),改了名字的结果列只记在
# masked_at_query:混进去的话,同源别的表里同名的真实字段(name、region)会从字段清单里消失
assert snapshot["mask_columns"] == ["buyer_phone", "buyer_name"]
# 运行详情「工件」页签走的通用接口:给的就是遮好的
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
@@ -161,6 +177,8 @@ async def test_masked_columns_are_masked_before_the_model_sees_them(monkeypatch,
("SELECT SUM(amount) AS total FROM sales WHERE buyer_phone = '13900000001'", 1200),
("SELECT region, SUM(amount) AS total FROM sales GROUP BY region ORDER BY total DESC", 1500),
("SELECT region, note, amount AS total FROM sales ORDER BY id", 1200),
# 汇总里的 CASE 只在条件里用到手机号:同写在 WHERE 里,值全部来自金额
("SELECT SUM(CASE WHEN buyer_phone LIKE '139%' THEN amount ELSE 0 END) AS total FROM sales", 2800),
])
async def test_queries_that_do_not_read_masked_values_are_untouched(monkeypatch, masked_shop, sql, value):
"""没碰遮罩列的值(汇总、只数个数、只在条件里用到):结果原样,取值照常,快照记下查询时一列都没遮。只遮这份结果
@@ -246,3 +264,124 @@ async def test_column_names_the_model_wrote_are_kept(monkeypatch, masked_shop):
snapshot = artifact_store.load(out.queries[0]["artifact"])
assert snapshot["columns"] == ["buyer_phone", "13900000001 的金额"]
assert set(_cells(snapshot, "buyer_phone")) == {MASKED} and out.result["value"] == 1200
# --------------------------------------------------------------------------
# 别的查询里遮掉的值、调用方给的值:后面的查询按值遮(复核第 1 轮)
# --------------------------------------------------------------------------
async def test_values_masked_in_an_earlier_query_are_masked_in_later_ones(monkeypatch, masked_shop):
"""Q1 选出手机号(查询时整列遮掉,值并进内存);之后的查询里,没设遮罩的联系方式表、备注写着同一批号码(换了写法的
也算):整格是号码的整格换,夹在文字里的换那一截。交给模型的、落库的、通用接口给的都是遮好的。以前只按这一份结果
里的遮罩值换,Q2 原样交给模型、原样落库,质询工件事后也收不到 Q1 的值。"""
name = masked_shop
steps = [q(name, "SELECT buyer_phone, amount AS total FROM sales ORDER BY id", "c1"),
q(name, "SELECT phone_num, visits FROM contacts ORDER BY id", "c2"),
q(name, "SELECT id, note FROM sales ORDER BY id", "c3")]
script = Script(monkeypatch, steps, {"value": {"alias": "Q2", "row": 0, "column": "visits"}, "rationale": "x"})
out, _ = await ask(name)
assert out.error is None and out.result["value"] == 5
seen = script.tool_results()
assert all(_clean(seen[c]) for c in ("c1", "c2", "c3")), seen
contacts = artifact_store.load(out.queries[1]["artifact"])
assert contacts["rows"] == [[MASKED, 5], [MASKED, 2], ["13800000000", 1]]
assert contacts["masked_at_query"] == [], "按值换的不算整列遮:没对上的号码照常显示"
notes = artifact_store.load(out.queries[2]["artifact"])
assert _cells(notes, "note") == [f"客户{MASKED}来电改址", "例行回访", f"回访:联系 {MASKED}", ""]
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
for query in out.queries:
got = await client.get(f"/api/artifacts/{query['artifact']}")
assert got.status_code == 200 and _clean(got.text)
async def test_values_given_by_the_caller_are_masked_before_the_model_sees_them(monkeypatch, masked_shop):
"""调用方交进来的遮罩值(审批质询、反方审稿收的报告快照里的值):第一条查询就按它遮,模型看不到。"""
name = masked_shop
given = masking._Secrets()
given.add({"columns": ["buyer_phone"], "rows": [["13900000001"], ["13900000003"]]}, {"buyer_phone"})
script = Script(monkeypatch, [q(name, "SELECT id, note, amount AS total FROM sales ORDER BY id", "c1")],
{"value": {"alias": "Q1", "row": 0, "column": "total"}, "rationale": "x"})
out, _ = await ask(name, masked_values=given)
assert out.result["value"] == 1200 and _clean(script.tool_results()["c1"])
snapshot = artifact_store.load(out.queries[0]["artifact"])
assert _cells(snapshot, "note") == [f"客户{MASKED}来电改址", "例行回访", f"回访:联系 {MASKED}", ""]
assert not given.strong - out.masked_values.strong, "交进来的值随遮掉的值一起交回"
async def test_a_reference_to_a_cell_masked_by_value_says_so(monkeypatch, masked_shop):
"""格子引用指到按值换成「已遮罩」的那一格:照实说,不说成「不是数值」。"""
name = masked_shop
steps = [q(name, "SELECT buyer_phone FROM sales ORDER BY id", "c1"),
q(name, "SELECT phone_num, visits FROM contacts ORDER BY id", "c2")]
Script(monkeypatch, steps, {"value": {"alias": "Q2", "row": 0, "column": "phone_num"}, "rationale": "x"})
out, _ = await ask(name)
assert out.result["value"] is None
cell = out.cells[0]
assert cell["status"] == "unresolved"
assert cell["reason"] == "Q2 第 0 行「phone_num」与设置了遮罩的值相同,查询时已遮罩,不采信"
async def test_a_table_function_over_a_masked_json_column_is_masked(monkeypatch, json_shop):
"""遮罩的是 JSON 列(客户资料):用 json_each 拆开是最自然的写法,拆出来的 key、value 在交给模型、落库之前就遮掉。
以前原样交给模型并落库,masked_at_query 是空的。"""
name = json_shop
sql = "SELECT c.id AS 编号, j.key, j.value FROM customers c, json_each(c.profile) j ORDER BY c.id, j.key"
script = Script(monkeypatch, [q(name, sql, "c1")],
{"value": {"alias": "Q1", "row": 0, "column": "编号"}, "rationale": "x"})
out, _ = await ask(name)
assert out.result["value"] == 1 and _clean(script.tool_results()["c1"])
snapshot = artifact_store.load(out.queries[0]["artifact"])
assert snapshot["masked_at_query"] == ["key", "value"] and _cells(snapshot, "编号") == [1, 1, 2, 2]
assert _clean(json.dumps(snapshot, ensure_ascii=False))
async def test_a_failed_query_does_not_hand_masked_values_to_the_model(monkeypatch, masked_shop):
"""数据库的报错会带上数据里的值(Postgres 类型转换失败:invalid input syntax for type bigint: "139-0000-0001"):
数据源设了遮罩时,引号里、SQL 里没写着的那一段和不带引号的长数换成「已遮罩」;模型自己写的名字照留(它据此改写
SQL)。遮罩开关没开的(Agent 节点)照旧原话。"""
from app.tools import datasource
said = {"message": ""}
async def failing(source, sql, **kwargs):
raise RuntimeError(said["message"])
monkeypatch.setattr(datasource, "run_query", failing)
sql = "SELECT CAST(buyer_phon AS BIGINT) AS p FROM sales WHERE id = 12345"
ctx = ToolContext(run_id="run-error", node_id="n1", data_versions=Versions.CURRENT)
async with SessionLocal() as session:
[masking_tool] = await build_datasource_tools([f"db_query__{masked_shop}"], ctx, session, mask_results=True)
[plain_tool] = await build_datasource_tools([f"db_query__{masked_shop}"], ctx, session)
for message, shown in [
('invalid input syntax for type bigint: "139-0000-0001"', 'invalid input syntax for type bigint: "已遮罩"'),
("value '13900000001' is out of range for type integer", "value '已遮罩' is out of range for type integer"),
("Conversion Error: Type INT64 with value 13900000001 can't be cast to INT32",
"Conversion Error: Type INT64 with value 已遮罩 can't be cast to INT32"),
('column "buyer_phon" does not exist', 'column "buyer_phon" does not exist'),
("no rows near id 12345", "no rows near id 12345"),
]:
said["message"] = message
assert await masking_tool.ainvoke({"sql": sql}) == f"查询失败:{shown}"
assert await plain_tool.ainvoke({"sql": sql}) == f"查询失败:{message}"
def test_a_result_from_a_source_without_masks_is_masked_by_the_values_seen_so_far():
"""数据源没设遮罩(另一个库里的联系方式表),此前查询遮掉的值照样按值换;这一份结果没有可收的值。"""
known = masking._Secrets()
known.add({"columns": ["buyer_phone"], "rows": [["13900000001"]]}, {"buyer_phone"})
payload = {"sql": "SELECT phone_num, note FROM contacts", "columns": ["phone_num", "note"],
"rows": [["13900000001", "客户 139 0000 0001 来电"], ["13800000000", "例行回访"]]}
done = masking.mask_query_result(payload, [], "sqlite", known)
assert done.payload["rows"] == [[MASKED, f"客户 {MASKED} 来电"], ["13800000000", "例行回访"]]
assert done.columns == [] and not done.secrets.strong
assert masking.mask_query_result(payload, [], "sqlite").payload is payload, "什么都没收到时原样"
def test_labels_written_in_this_query_are_not_masked_by_values_from_earlier_ones():
"""分档标签是这条 SQL 自己写的字:此前查询里认不准、当遮罩值收了的同名标签,不把这一份的标签列换掉。"""
known = masking._Secrets()
known.add({"columns": ["档"], "rows": [["大单"], ["小单"]]}, {"档"})
payload = {"sql": "SELECT CASE WHEN amount > 1000 THEN '大单' ELSE '小单' END AS 档, COUNT(*) AS 笔数 FROM sales "
"GROUP BY 1", "columns": ["档", "笔数"], "rows": [["大单", 1], ["小单", 3]]}
assert masking.mask_query_result(payload, ["amount"], "sqlite", known).payload["rows"] == [["大单", 1], ["小单", 3]]