mirror of
https://github.com/ThearchyHelios/AgentLab.git
synced 2026-11-16 01:12:00 +00:00
K3 复核修正(第 1 轮):后面的查询按此前遮掉的值遮、质询工件照回答时记下的位置重放;表值函数拆开遮罩列也认;汇总遮罩列的指标不派盲算;反方理由按值遮
复核发现质询工件接口相对基线回退:agent 自己的查询快照在工具边界遮好之后,事后取质询工件收不到它另一次查询里的号码, 没设遮罩的备注、联系方式列原样给出。另有几处模型照样看得到遮罩值、或者遮多了的地方。 - 工具边界(tools/datasource、engine/masking.mask_query_result):此前已经认出的遮罩值(这个 agent 前面的查询遮掉的、 调用方交进来的)也用来按值遮后面的结果,数据源没设遮罩的也一样。审批质询、反方审稿在派 agent 之前把报告快照里的 遮罩值交进来(inquiry.snapshot_secrets,run_verify_agent 的 masked_values),模型从第一条查询起就看不到。已经摆在 读者眼前的值(提问人写的、报告指标、被审的结论原文)跟着并进来的值一起免遮(_Secrets.protected);这份结果自己写 的分档标签不按前面收的值换。 - 先查备注、后查出号码的:回答时只靠内存里的值才换掉的地方按快照记成位置(不含值,masking.view_edits),写进质询 事件的 masked_cells,质询工件接口先照着重放(replay_edits)再按原来的办法遮。工具边界按值换掉的整格照样画成遮罩列。 agent 查到的值先收、报告快照的后收:上限收满时不丢 agent 自己查到的。 - _derived_hidden 认 FROM、JOIN、LATERAL 里拆开遮罩列的表值函数(json_each、json_tree、unnest、jsonb_each_text、 regexp_split_to_table、JSON_TABLE、LATERAL VALUES):按出列名认,不认得的函数又没写列名表的整份遮。 - 汇总里的 CASE 只在条件里用到遮罩列的不算取自遮罩列(同写在 WHERE 里)。双盲复算按同一套认法认出指标取自遮罩列 (SUM(amount)、amount 设了遮罩)时,派盲算之前就记为无法复算并说清原因,不白花一次盲算(nodes/dual_check 的 _query_input,改动在文件所有权之外,只加这一处判断)。 - 反方审稿:理由正文和理由文档按值换掉遮罩值(换了写法的也认),这句结论原文里写着的不换;重新组装时引用到的格子按 遮好的快照渲染。 - 换了写法的号码另认不换行空格、窄空格、细空格、制表符、减号,最多四个分隔符;SQL 的字符串、带引号的名字、注释里 也认(不带引号的数照旧是算式),字符串里换掉的号码同一条 SQL 里写成阈值的也不留;列名同样认。 - 查询失败时报错原话里的值换掉(mask_error_text:引号里、不带引号的长数,SQL 里写着的照留)。 - 快照的 mask_columns 只记数据源设的列名,改了名字的结果列只在 masked_at_query:证据面板并各份快照的这一项从表的 字段清单里去列,混进别名会误藏同源别的表里同名的真实字段。 - * 展开数据源里的表时按冻结的表结构认它带不带同名列,没有的照常认分档标签(J2 收紧之后的多遮)。 - 格子引用指到按值换掉的格子时照实说;上万个值的合并、免遮放进线程。 测试:质询 agent 先查号码后查备注、先查备注后查号码、另一张表的号码列两种先后,质询工件、通用工件接口、事件都没有 原值;报告快照里的值在第一条查询就遮;表值函数(SQLite、Postgres、MySQL 的写法)、汇总里 CASE 的条件、* 连着的表按 表结构认标签、新的分隔符、SQL 字符串和注释和列名、报错原话、mask_columns 不混别名、遮罩列汇总的指标不派盲算、反方 理由按值遮且免遮结论原文、上限收满时 agent 的值先收。逐处去掉实现(24 处)对应的用例都会变红。
This commit is contained in:
+126
-81
@@ -13,9 +13,12 @@
|
||||
自己查询的格子,或者报告里已有的指标、运行输入;回答文档由 compose_doc 组装,裸数字被标出来(提问人自己写的数除外)。
|
||||
封存之后的追问查的是数据源现在的数据,任务里给报告生成的时间,回答末尾注明;触顶、查询失败、引用解析不了也在末尾注明
|
||||
3. 遮罩(_plan):agent 的查询结果在核查 agent 的工具边界就遮好了(engine/masking.py 的 mask_query_result):取自
|
||||
遮罩列的格子交给模型、落进快照之前就是「已遮罩」,模型看不到遮罩值。这里是第二道防线:查询时遮掉的值由 verify_agent
|
||||
交回(masked_values,只在内存里),连同报告自己快照里的遮罩值,照下面的办法按值遮;事后取质询工件时内存里的值已经
|
||||
不在,SQL 检查结果照事件里记下的 SQL 换(_recorded_swaps)。
|
||||
遮罩列的格子交给模型、落进快照之前就是「已遮罩」,模型看不到遮罩值。报告自己快照里的遮罩值在派 agent 之前就交给
|
||||
查询工具(snapshot_secrets → run_verify_agent 的 masked_values),agent 前面的查询遮掉的值也一样:后面查到的备注、
|
||||
别的表里写着同一个号码的,交给模型、落库之前就按值换掉。这里是第二道防线:查询时遮掉的值由 verify_agent 交回
|
||||
(masked_values,只在内存里),连同报告自己快照里的遮罩值,照下面的办法按值遮。事后取质询工件时内存里的值已经
|
||||
不在:回答时只靠它们才换掉的地方(先查备注、后查出号码的那份备注快照)记成位置写进事件(REPLAY_KEY,不含值,
|
||||
masking.view_edits),取工件时照着再换一遍;SQL 检查结果照事件里记下的 SQL 换(_recorded_swaps)。
|
||||
哪些列要遮照证据面板的规则(查询当时记下的,加上数据源现在设的),另按 SQL 认出改了名字的遮罩列
|
||||
(起别名、套函数、经子查询转一道、和 * 写在同一支里,_derived_hidden;认不出落在哪一列的,整份快照遮掉;按遮罩列
|
||||
分档、当分组键或套在汇总里数个数的不算,_band_only);快照按列遮,再按值遮(别的列里写着遮罩值的格子,_masked_view;质询
|
||||
@@ -24,8 +27,9 @@
|
||||
认用在哪(_literal_uses),数字照原样留着的只有范围比较的边界(没把某一个值夹住的,_pinned)和 LIMIT 的行数,而且
|
||||
同一个数在这条 SQL 别处没被换掉,封存的是实际执行的语句;在找某一个值的(等值、IN、简单 CASE、夹住某一个值的
|
||||
范围……,值换个写法、写成几个数的算式也认)和别处够长的遮罩值都换。已经摆在读者眼前的值不换:问题和选中原文里的、
|
||||
报告列出的指标和运行输入的值;SQL 里照原样留着的阈值只在回答正文里不换,格子照旧换。回答正文、质询工件的文字格里,
|
||||
换了写法的长号码(199-1474-2785、(199)1474-2785、+86 199 1474 2785、全角数字)按纯数字写法认(masking 的
|
||||
报告列出的指标和运行输入的值(agent 后面的查询遮掉的值照样免遮,_Secrets.protected);SQL 里照原样留着的阈值只在
|
||||
回答正文里不换,格子照旧换。回答正文、质询工件的文字格、SQL 的字符串、带引号的名字和注释里,换了写法的长号码
|
||||
(199-1474-2785、(199)1474-2785、+86 199 1474 2785、全角数字、不换行空格或减号隔开的)按纯数字写法认(masking 的
|
||||
_Secrets.loose_spans)。组装之后还认得出遮罩值的(换了写法的也算),这份回答不交出去
|
||||
4. 写入(_record):一条 approval.inquiry 事件,node_id 为空,用量写在数据的 usage 里(计进运行用量,见 runner 的
|
||||
_usage_of_events)。质询的查询快照只放在 inquiry_artifacts / queries 里,绝不放在 query_artifact 下:证据接口按那个键
|
||||
@@ -33,7 +37,8 @@
|
||||
append_events 不写,回答照样交回给提问的人,说明没有写进运行记录(这样的回答在进程里留一阵,_HELD)。触顶、失败
|
||||
照样记一条
|
||||
5. 读取:列表读 approval.inquiry 事件(封存前后的都算,按先后);质询工件只认质询引用过的(查询快照、回答文档、回答里
|
||||
引用的报告口径卡),遮罩照回答时的办法,快照里 SQL 检查结果的片段、说明照整条 SQL 的换法换(_masked_checks)
|
||||
引用的报告口径卡),先照事件里记下的位置重放回答时换掉的地方,再照回答时的办法遮,快照里 SQL 检查结果的片段、说明
|
||||
照整条 SQL 的换法换(_masked_checks)
|
||||
|
||||
不发 tool.start、llm.start 这类事件(run_verify_agent 不传 emit):界面从接口的返回取回答;runner 的 _stale_replay
|
||||
只看 tool.start,质询不影响恢复后的重放。
|
||||
@@ -45,17 +50,24 @@
|
||||
正好等于某一笔金额)也遮掉。这是产品取舍,不按上下文去猜
|
||||
- 遮罩数值列的 SUM / AVG / MIN / MAX 算派生自遮罩列,整列遮(COUNT 不算,见 _value_columns),CASE 里有一支取的是
|
||||
遮罩列的也算:MIN / MAX 就是某一行的原值,只有一行的分组 SUM / AVG 也是,按列认不出哪些汇总值安全,宁可多遮。
|
||||
汇总里套着结果全是写死常量的 CASE(SUM(CASE WHEN amount <= 20000 THEN 1 ELSE 0 END))只是按条件数个数,同 COUNT
|
||||
核查 agent 的查询在工具边界照这个办法遮,所以这样的指标双盲复算不了(原 SQL 里认得出的,派盲算之前就记为无法
|
||||
复算),反方审稿核对这类汇总数的结论也拿不到数。汇总里套着结果全是写死常量的 CASE(SUM(CASE WHEN amount <= 20000
|
||||
THEN 1 ELSE 0 END))只是按条件数个数,同 COUNT;汇总里的 CASE 只在条件里用到遮罩列的(SUM(CASE WHEN buyer_phone
|
||||
IS NOT NULL THEN amount ELSE 0 END))同写在 WHERE 里,不算
|
||||
- FROM、JOIN、LATERAL 里的表值函数(json_each、json_tree、unnest、jsonb_each_text、JSON_TABLE、LATERAL VALUES)拆开
|
||||
遮罩列的,按出列名认它出的列(常见函数的出列名、写的列名表、别名、函数名,_function_columns),同名的别的列一并遮
|
||||
(多遮);不认得的函数又没写列名表的,整份快照都遮
|
||||
- 按遮罩列分档的标签(CASE、IIF 的结果全是写死的常量):标签文字是 agent 写在 SQL 里的,不当遮罩值,SQL、回答正文里
|
||||
照原样;标签列当分组键(每一档一行)照常显示,不分组的逐行标签列整列遮(格子里看不出某一笔落在哪一档),可 agent
|
||||
在正文里写出「这一笔是大单」遮不住,按 id 分组再按档数个数也看得出(同按阈值查出某几行)。只有只装着标签的列不收:
|
||||
结果的这一列在 UNION 的每一支里都是这样的 CASE / IIF,或者按作用域认得出取自子查询、CTE 里只装着标签的一列;有一支
|
||||
取自数据源里的表(遮罩列原样选出、经子查询按原名传出,哪怕起了标签的名字)就照收。认不准的也照收(多遮),比如
|
||||
按这一列 USING、NATURAL 合成的同名列,递归的 CTE,UNION 里落在某一支 * 里的列,写死的字符串、和列一起算的值;
|
||||
派生来源里同名的几列(SQLite 都留着,外层按名字取排在最前面的那一列):标签前面有 * 带得出这一列(没写表名的 *
|
||||
展开了数据源里的表就算带得出),或者有没起别名、套了括号或 COLLATE 的同名列((vip_level)、vip_level COLLATE
|
||||
NOCASE),都不算只装着标签(_block_label_of、_star_label)。标签恰好和别的列里的遮罩值相同时,正文里的档名、SQL
|
||||
里写成字符串的标签也换(多遮)
|
||||
派生来源里同名的几列(SQLite 都留着,外层按名字取排在最前面的那一列):标签前面有 * 带得出这一列(* 展开了数据源里
|
||||
的表、冻结的表结构里查不到这张表或者它有同名的列,就算带得出),或者有没起别名、套了括号或 COLLATE 的同名列
|
||||
((vip_level)、vip_level COLLATE NOCASE),都不算只装着标签(_block_label_of、_star_label)。* 展开的表在表结构里
|
||||
查得到、又没有同名的列时,照常按派生来源认标签(_table_lacks)。标签恰好和别的列里的遮罩值相同时,正文里的档名、
|
||||
SQL 里写成字符串的标签也换(多遮)
|
||||
- 「夹住某一个值」(_pinned):范围比较在整条 SQL 里配对,只有 CASE 的不同分支、SELECT 里不筛行的不同项(选出列、
|
||||
GROUP BY、ORDER BY)、UNION 的不同支各算各的,两头误差都有:把找值拆进这几处(CASE WHEN age > 25 THEN … ELSE CASE
|
||||
WHEN age >= 25 …,两个选出列 SUM(age >= 25)、SUM(age > 25) 再自己相减)认不出,边界照原样留着,正文里复述也不遮;
|
||||
@@ -77,14 +89,18 @@
|
||||
- 质询工件里 SQL 检查结果的片段是检查器按语法树重新写出来的,按整条 SQL 里整个换掉了的字面量照值换:同一个值在
|
||||
片段里别处出现也换(多遮,算式里换掉的 0、1 这类也是);截断处半截的值去掉(停在字符串、带引号的名字、注释里的,
|
||||
从开头的引号、注释起整个去掉)。说明里码值表列出的码值和别的字一样按值换,恰好是遮罩值的码值也换(多遮)
|
||||
- 运行详情的「工件」页签、通用的工件接口按工件原样给:质询的查询快照落库时就遮好了(工具边界先遮),从那里打开也
|
||||
看不到取自遮罩列的值;但工具边界只认这份结果里的遮罩值,别的查询、报告快照里遮罩列的值写在这份结果没设遮罩的列里时
|
||||
(没设遮罩的联系方式列、备注),从那里打开照原样(质询工件接口按回答时的办法换)。报告自己的查询快照(写作时的查询)
|
||||
仍原样存着,从那里打开不经这里的遮罩
|
||||
- 遮罩值最多收 SECRETS_MAX 种写法,再多的按值认不出(按列遮照旧)
|
||||
- 换了写法的号码只认数字之间夹着一到三个空格、连字符、点、括号(含全角),带 +86 / 0086 / 86 前缀、全角数字:写成
|
||||
汉字(一三九……)、中间夹着别的字、拆成几处写的认不出;和别的数字只隔着一个连字符或点连着的(13900000001-2、
|
||||
19914.74)不算号码;SQL 里不这样认(199-1474-2785 在 SQL 里是一个算式)
|
||||
- 运行详情的「工件」页签、通用的工件接口按工件原样给:质询的查询快照落库时就遮好了(工具边界先遮),取自遮罩列的值、
|
||||
报告快照里的遮罩值、agent 前面的查询里遮掉的值都看不到。只有一种照原样:先查了写着号码的备注(那时还不知道这个号码
|
||||
是遮罩值),后来才查出号码——备注快照落库时原样,从通用接口打开照原样,质询工件接口照事件里记下的位置换。报告自己
|
||||
的查询快照(写作时的查询)仍原样存着,从那里打开不经这里的遮罩
|
||||
- 遮罩值最多收 SECRETS_MAX 种写法,再多的按值认不出(按列遮照旧)。agent 自己查到的值先收,报告快照里的后收
|
||||
- 换了写法的号码只认数字之间夹着一到四个分隔符(空格、制表符、全角空格、不换行空格、窄空格这类空白,连字符、减号、
|
||||
点、括号,含全角),带 +86 / 0086 / 86 前缀、全角数字:写成汉字(一三九……)、中间夹着别的字、拆成几处写的、跨行
|
||||
写的认不出;和别的数字只隔着一个连字符或点连着的(13900000001-2、19914.74)不算号码;SQL 里不带引号的数不这样认
|
||||
(199-1474-2785 在 SQL 里是三个数相减),字符串、带引号的名字、注释里照样认
|
||||
- 质询 agent 的任务里的证据摘录照 judge.prepare 按列名遮:报告取数把遮罩列改了名字选出来(buyer_phone AS 手机号)时,
|
||||
摘录里有原值,agent 看得到;它写进回答、SQL 的,按值认得出的照样换掉(报告快照里的值都收了),封存的是换过的。摘录
|
||||
按 SQL 认改了名字的遮罩列要改 judge.py,另行处理
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -113,7 +129,7 @@ from app.data.provenance_types import DOC_PROVENANCE
|
||||
from app.db.base import SessionLocal
|
||||
from app.db.models import Approval, Artifact, DataSource, Run, RunEvent
|
||||
from app.engine import judge as judging
|
||||
from app.engine.evidence import MARKER_RE, compose_doc, iter_segments, iter_units
|
||||
from app.engine.evidence import compose_doc, iter_segments, iter_units
|
||||
from app.engine.issuance import extract_numbers
|
||||
# 认遮罩列、遮罩值的共用部分在 engine/masking.py。名字照旧从这里导出:nodes/dual_check、测试按 inquiry 的名字引用
|
||||
from app.engine.masking import ( # noqa: F401
|
||||
@@ -122,8 +138,8 @@ from app.engine.masking import ( # noqa: F401
|
||||
_branch_results, _branches, _canonical, _constant, _constants, _decimal, _derived_block, _derived_hidden,
|
||||
_digits_of, _fixed, _forms, _from_sources, _is_star, _item_label, _label_columns, _leaves, _literal_types,
|
||||
_masked_view, _masked_word, _mentions, _merging, _numeric, _output_name, _pattern, _places, _plain_text,
|
||||
_point_text, _query_shaped, _result_label, _snapshot_labels, _source_label, _source_names, _star_label, _strong,
|
||||
_tainted, _value_columns, _worked_out,
|
||||
_point_text, _query_shaped, _result_label, _scrubbed_source, _snapshot_labels, _source_label, _source_names,
|
||||
_star_label, _strong, _tainted, _value_columns, _worked_out, replay_edits, view_edits,
|
||||
)
|
||||
from app.engine.merge_query import MERGE_SOURCE
|
||||
from app.engine.nodes.challenge import mark_original, with_note
|
||||
@@ -156,6 +172,9 @@ LIMITS = ("max_queries", "timeout_s", "max_cost_usd", "daily_max_usd")
|
||||
USAGE_KEYS = ("input_tokens", "output_tokens", "cost_usd", "calls", "unpriced_calls")
|
||||
#: 查询清单每条给的几项:没有行,查询结果只在快照里(取的时候按遮罩遮好)
|
||||
QUERY_KEYS = ("alias", "artifact", "sql", "row_count", "truncated", "source")
|
||||
#: 质询事件数据里记「回答时只靠 agent 查询时遮掉的值才换掉的位置」的键:{快照工件: masking.view_edits 的形状}。
|
||||
#: 不含值;事后取质询工件时内存里的值已经不在,照着位置再换一遍(inquiry_artifact)
|
||||
REPLAY_KEY = "masked_cells"
|
||||
#: 回答文档存成工件时的种类;工件引用的 meta 里记下问题的前这么多字,运行的工件列表据此认得出是哪个问题的回答
|
||||
ANSWER_KIND = "inquiry_answer"
|
||||
QUESTION_BRIEF = 40
|
||||
@@ -676,33 +695,6 @@ async def _hidden_columns(snapshot: dict[str, Any], masks: Any) -> tuple[set[str
|
||||
cols, found = await masks.of(snapshot.get("source"))
|
||||
return cols | _recorded_masks(snapshot), ([] if found or not snapshot.get("source") else [str(snapshot["source"])])
|
||||
|
||||
#: 单元格引用的前一截(Q1.r0.):后面才是列名
|
||||
_CELL_HEAD = re.compile(r"^\s*[QK]\d+\.r\d+\.")
|
||||
|
||||
|
||||
def _scrubbed_source(text: str, secrets: _Secrets) -> str:
|
||||
"""回答原文里的遮罩值换成「已遮罩」:标记之外按正文的办法换(SQL 里照原样留着的阈值不换);标记里的按段换(| 隔开
|
||||
的几段各自换),单元格引用只换列名那一截——编号、行号换不出东西,整段换的话「r0.」会让列名开头的长号码看着像小数
|
||||
的一截,反而认不出来。列名照快照、SQL 里的名字那样换(阈值也换),换得一样才对得上。
|
||||
标记里写着遮罩值的引用换完多半解析不了(列名里写着遮罩值的除外:快照的列名也换过了,正好对上),宁可弄坏一个
|
||||
引用,也不让值留在存下的原文、片段里,或者从解析不了的占位里露出来。"""
|
||||
if not text or not secrets.strong:
|
||||
return text
|
||||
out: list[str] = []
|
||||
last = 0
|
||||
for m in MARKER_RE.finditer(text):
|
||||
out.append(secrets.scrub(text[last:m.start()], prose=True))
|
||||
start, end = m.span("body")
|
||||
parts = text[start:end].split("|")
|
||||
head = _CELL_HEAD.match(parts[0]) if m.group("kind").strip() == "v" else None
|
||||
first = parts[0][:head.end()] + secrets.scrub(parts[0][head.end():]) if head \
|
||||
else secrets.scrub(parts[0], prose=True)
|
||||
parts = [first, *(secrets.scrub(p, prose=True) for p in parts[1:])]
|
||||
out.append(text[m.start():start] + "|".join(parts) + text[end:m.end()])
|
||||
last = m.end()
|
||||
out.append(secrets.scrub(text[last:], prose=True))
|
||||
return "".join(out)
|
||||
|
||||
|
||||
#: 按词认在找某一个值的比较(SQL 解析不了时的退路,见 _context):col = 'x'、col <> 'x'、col LIKE 'x'、SQLite 的
|
||||
#: col IS 'x'(IN 列表另外认)
|
||||
@@ -1404,9 +1396,12 @@ def _swap_literal(raw: str, text: str, kind: str, use: _Use, secrets: _Secrets,
|
||||
手机号)。数换个写法照样认(25.0、025、2.5e1 按 25 认,_canonical);夹住某一个值的范围按夹住的值认(age > 24
|
||||
AND age < 26 的两头都按 25 认),算式里的数按算出来的值认,同一个数在别处换掉了的阈值、行数也按等值认(见
|
||||
_literals);
|
||||
- 字符串在哪都按值换里面的遮罩值(范围边界也一样:写成字符串的长号码做边界,多半是在找那一个人);
|
||||
- 字符串在哪都按值换里面的遮罩值(范围边界也一样:写成字符串的长号码做边界,多半是在找那一个人),换了写法的号码
|
||||
也认(LIKE '%199-1474-2785%':字符串里的 199-1474-2785 不是算式);换掉的号码按纯数字写法记,同一个数写成阈值的
|
||||
据此不留;
|
||||
- 别的数:范围比较的边界、LIMIT / OFFSET 的行数照原样(agent 自己定的阈值,换了封存的就不是实际执行的语句);
|
||||
别处(选出列、函数参数、算式)认不准是不是在找某一个值,够长的遮罩值整个换(算出来的值是也换)。"""
|
||||
别处(选出列、函数参数、算式)认不准是不是在找某一个值,够长的遮罩值整个换(算出来的值是也换)。不带引号的数
|
||||
不按换了写法认:199-1474-2785 在 SQL 里是三个数相减。"""
|
||||
how, columns, points = use
|
||||
word = _masked_word()
|
||||
canonical = _canonical(text)
|
||||
@@ -1418,11 +1413,9 @@ def _swap_literal(raw: str, text: str, kind: str, use: _Use, secrets: _Secrets,
|
||||
found: list[str] = []
|
||||
|
||||
def scrub(inner: str) -> str:
|
||||
pattern = secrets.pattern(inner)
|
||||
if pattern is None:
|
||||
return inner
|
||||
found.extend(m.group() for m in pattern.finditer(inner))
|
||||
return pattern.sub(word, inner)
|
||||
out, swapped = secrets.swap(inner, loose=True)
|
||||
found.extend(swapped)
|
||||
return out
|
||||
|
||||
return _requote(raw, scrub), _number_forms(found)
|
||||
if how not in _KEPT and any(secrets.hit(v) for v in looked):
|
||||
@@ -1453,7 +1446,9 @@ def _scrub_sql(sql: str, secrets: _Secrets, compared: set[str], kind: str | None
|
||||
|
||||
def _rebuild(sql: str, tokens: list[Any], kinds: list[str], secrets: _Secrets, literal: Callable[[int, str], str]
|
||||
) -> str:
|
||||
"""按词重写一段 SQL:字面量交给 literal(第几个词、原文)定换成什么,名字(带不带引号)、注释和别的字按值换。"""
|
||||
"""按词重写一段 SQL:字面量交给 literal(第几个词、原文)定换成什么,名字(带不带引号)、注释和别的字按值换。
|
||||
词与词之间只有空白和注释,名字里的数也不是算式:都按文字的办法换(换了写法的号码也认,/* 客户 199-1474-2785 */、
|
||||
AS "199-1474-2785 的备注")。"""
|
||||
out: list[str] = []
|
||||
last = 0
|
||||
for i, token in enumerate(tokens):
|
||||
@@ -1461,16 +1456,16 @@ def _rebuild(sql: str, tokens: list[Any], kinds: list[str], secrets: _Secrets, l
|
||||
if start < last or end > len(sql):
|
||||
continue
|
||||
gap = sql[last:start]
|
||||
out.append(secrets.scrub(gap) if gap.strip() else gap)
|
||||
out.append(secrets.scrub(gap, loose=True) if gap.strip() else gap)
|
||||
raw = sql[start:end]
|
||||
if _is_literal(kinds[i]):
|
||||
raw = literal(i, raw)
|
||||
elif kinds[i] in _NAME_TOKENS:
|
||||
raw = _requote(raw, secrets.scrub)
|
||||
raw = _requote(raw, lambda inner: secrets.scrub(inner, loose=True))
|
||||
out.append(raw)
|
||||
last = end
|
||||
rest = sql[last:]
|
||||
out.append(secrets.scrub(rest) if rest.strip() else rest)
|
||||
out.append(secrets.scrub(rest, loose=True) if rest.strip() else rest)
|
||||
return "".join(out)
|
||||
|
||||
|
||||
@@ -1515,7 +1510,7 @@ def _scrub_excerpt(text: str, whole: set[str] | None, secrets: _Secrets, kind: s
|
||||
def literal(i: int, raw: str) -> str:
|
||||
if tokens[i].text in whole:
|
||||
return word if kinds[i] == "NUMBER" else _requote(raw, lambda _: word)
|
||||
return raw if kinds[i] == "NUMBER" else _requote(raw, secrets.scrub)
|
||||
return raw if kinds[i] == "NUMBER" else _requote(raw, lambda inner: secrets.scrub(inner, loose=True))
|
||||
|
||||
out = _rebuild(body, tokens, kinds, secrets, literal)
|
||||
return out + "…" if cut else out
|
||||
@@ -1603,7 +1598,8 @@ def _shown_literals(sqls: Iterable[tuple[str, str | None, set[str]]], secrets: _
|
||||
class _Plan:
|
||||
"""一次回答(或一次取质询工件)的遮罩:agent 每份查询快照遮好的样子(views)和遮掉的列(masked)、SQL 里指代
|
||||
遮罩列的名字(tainted)、数据源的类型(kinds,SQL 按它的方言分词)、遮罩列里出现过的值(secrets),以及已经不在的
|
||||
数据源(gone,按查询当时记下的遮罩处理)。"""
|
||||
数据源(gone,按查询当时记下的遮罩处理)。replay:回答时只靠 agent 查询时遮掉的值(seen,只在内存里)才换掉的
|
||||
地方,按快照记成位置(masking.view_edits),写进质询事件,事后取质询工件照着再换一遍;取工件时为空。"""
|
||||
|
||||
views: dict[str, dict[str, Any]]
|
||||
masked: dict[str, list[str]]
|
||||
@@ -1611,6 +1607,7 @@ class _Plan:
|
||||
kinds: dict[str, str | None]
|
||||
secrets: _Secrets
|
||||
gone: dict[str, list[str]]
|
||||
replay: dict[str, dict[str, Any]] = field(default_factory=dict)
|
||||
|
||||
|
||||
async def _source_kinds(names: Iterable[str]) -> dict[str, str | None]:
|
||||
@@ -1659,22 +1656,57 @@ def _settle_plan(own: dict[str, dict[str, Any]], named: dict[str, tuple[set[str]
|
||||
kind, base = kinds.get(str(snapshot.get("source") or "")), named[artifact][0]
|
||||
hidden[artifact], tainted[artifact] = _derived_hidden(snapshot, base, kind)
|
||||
secrets.add(snapshot, hidden[artifact], labels=_snapshot_labels(snapshot, kind), base=base)
|
||||
for snapshot, base in based:
|
||||
kind = kinds.get(str(snapshot.get("source") or ""))
|
||||
secrets.add(snapshot, _derived_hidden(snapshot, base, kind)[0], labels=_snapshot_labels(snapshot, kind),
|
||||
base=base)
|
||||
# agent 查询时遮掉的值先收(报告快照里的值收满了上限 SECRETS_MAX,也不丢 agent 自己查到的);不带它们的一份另算,
|
||||
# 比出事后取质询工件时认不出、要照位置重放的地方
|
||||
baseline = secrets.copy() if seen is not None and seen.strong else None
|
||||
secrets.merge(seen)
|
||||
secrets.protect(given)
|
||||
for target in (secrets, baseline):
|
||||
if target is not None:
|
||||
_add_based(target, based, kinds)
|
||||
target.protect(given)
|
||||
secrets.show(_shown_literals([(sql, kinds.get(name), _compared(tainted, artifact, live, name))
|
||||
for sql, name, artifact in shown], secrets))
|
||||
secrets.prepare()
|
||||
views: dict[str, dict[str, Any]] = {}
|
||||
masked: dict[str, list[str]] = {}
|
||||
replay: dict[str, dict[str, Any]] = {}
|
||||
for artifact, snapshot in own.items():
|
||||
views[artifact], masked[artifact] = _masked_view(snapshot, hidden[artifact], secrets,
|
||||
embedded=artifact in embedded)
|
||||
if baseline is not None and (edits := view_edits(snapshot, hidden[artifact], secrets, baseline)):
|
||||
replay[artifact] = edits
|
||||
return _Plan(views=views, masked=masked, tainted=tainted, kinds=kinds, secrets=secrets,
|
||||
gone={artifact: named[artifact][1] for artifact in own})
|
||||
gone={artifact: named[artifact][1] for artifact in own}, replay=replay)
|
||||
|
||||
|
||||
def _add_based(secrets: _Secrets, based: list[tuple[dict[str, Any], set[str]]], kinds: dict[str, str | None]) -> None:
|
||||
"""收报告自己的查询快照里遮罩列的值(based:快照和它要遮的列),按 SQL 认出改了名字的遮罩列(_derived_hidden)。"""
|
||||
for snapshot, base in based:
|
||||
kind = kinds.get(str(snapshot.get("source") or ""))
|
||||
secrets.add(snapshot, _derived_hidden(snapshot, base, kind)[0], labels=_snapshot_labels(snapshot, kind),
|
||||
base=base)
|
||||
|
||||
|
||||
async def snapshot_secrets(snapshots: Iterable[Any], given: Iterable[str | None] = ()) -> _Secrets:
|
||||
"""几份查询快照(报告自己的)里遮罩列的值,收法同回答时收报告快照(_plan),去掉已经摆在读者眼前的(given)。
|
||||
审批质询、反方审稿在派核查 agent 之前交给它的查询工具(run_verify_agent 的 masked_values):agent 查到的别的表、
|
||||
备注里写着同一个号码时,交给模型之前就换掉,模型看不到。"""
|
||||
from app.api.evidence import _Masks
|
||||
|
||||
masks = _Masks()
|
||||
based = [(snapshot, (await _hidden_columns(snapshot, masks))[0]) for snapshot in snapshots
|
||||
if _query_shaped(snapshot)]
|
||||
kinds = await _source_kinds(str(snapshot.get("source") or "") for snapshot, _ in based)
|
||||
asked = list(given)
|
||||
|
||||
def collect() -> _Secrets:
|
||||
secrets = _Secrets()
|
||||
_add_based(secrets, based, kinds)
|
||||
secrets.protect(asked)
|
||||
secrets.prepare()
|
||||
return secrets
|
||||
|
||||
return await asyncio.to_thread(collect)
|
||||
|
||||
|
||||
def _compared(tainted: dict[str, set[str]], artifact: str, live: dict[str, list[str]], name: str) -> set[str]:
|
||||
@@ -1760,21 +1792,24 @@ async def _answer(op: _Opening, ask: Ask, target: _Target | None, settings: dict
|
||||
part = settings.get("inquiry") if isinstance(settings.get("inquiry"), dict) else {}
|
||||
async with SessionLocal() as session:
|
||||
spec = await verify_model_spec(session, None, settings=settings)
|
||||
# 报告自己的查询快照:被问的那句可能直接写着遮罩列的值,agent 会照着复述;里面的遮罩值在派 agent 之前就交给它的
|
||||
# 查询工具(masked_values),agent 查到写着同一个号码的备注、别的表时,交给模型之前就换掉
|
||||
given = [ask.question, ask.quote, *_ref_values(refs)]
|
||||
others = [snapshot for entry in catalog.values() if isinstance(entry, dict) and entry.get("kind") == "query"
|
||||
and _query_shaped(snapshot := _quiet_load(op.loader, entry.get("artifact")))]
|
||||
outcome = await run_verify_agent(
|
||||
spec=spec, system=INQUIRY_SYSTEM, task=_task(op, ask, target, excerpts, refs, live, lost), sources=list(live),
|
||||
output_schema=INQUIRY_SCHEMA, max_queries=part.get("max_queries"), timeout_s=part.get("timeout_s"),
|
||||
daily_max_usd=settings.get("daily_max_usd"), run_id=op.run_id, data_versions=op.versions)
|
||||
daily_max_usd=settings.get("daily_max_usd"), run_id=op.run_id, data_versions=op.versions,
|
||||
masked_values=await snapshot_secrets(others, given))
|
||||
|
||||
# 遮罩:agent 自己的查询快照(查询时就遮好了,遮掉的值在 outcome.masked_values 里)、报告自己的查询快照(被问的
|
||||
# 那句可能直接写着遮罩列的值,agent 会照着复述)
|
||||
# 遮罩:agent 自己的查询快照(查询时就遮好了,遮掉的值在 outcome.masked_values 里)、报告自己的查询快照
|
||||
own: dict[str, dict[str, Any]] = {}
|
||||
for query in outcome.queries:
|
||||
snapshot = _quiet_load(artifact_store.load, query.get("artifact"))
|
||||
if _query_shaped(snapshot):
|
||||
own[str(query["artifact"])] = snapshot
|
||||
others = [snapshot for entry in catalog.values() if isinstance(entry, dict) and entry.get("kind") == "query"
|
||||
and _query_shaped(snapshot := _quiet_load(op.loader, entry.get("artifact")))]
|
||||
plan = await _plan(own, others, [ask.question, ask.quote, *_ref_values(refs)],
|
||||
plan = await _plan(own, others, given,
|
||||
[(str(q.get("sql") or ""), str(q.get("source") or ""), str(q.get("artifact") or ""))
|
||||
for q in outcome.queries], live, seen=outcome.masked_values)
|
||||
|
||||
@@ -1808,6 +1843,8 @@ async def _answer(op: _Opening, ask: Ask, target: _Target | None, settings: dict
|
||||
"inquiry_artifacts": list(dict.fromkeys(q["artifact"] for q in queries if q.get("artifact"))),
|
||||
"queries": queries,
|
||||
"model": outcome.model, "usage": {k: outcome.usage.get(k, 0) for k in USAGE_KEYS},
|
||||
# 只靠内存里的遮罩值才换掉的位置(不含值),质询工件照着再换一遍;没有就不加这个键,事件和以前一字不差
|
||||
**({REPLAY_KEY: plan.replay} if plan.replay else {}),
|
||||
}
|
||||
|
||||
|
||||
@@ -1865,7 +1902,8 @@ def _leaked(doc: dict[str, Any], secrets: _Secrets) -> bool:
|
||||
def _renamed_entry(entry: Any, secrets: _Secrets) -> Any:
|
||||
if not (isinstance(entry, dict) and entry.get("kind") == "query" and isinstance(entry.get("columns"), list)):
|
||||
return entry
|
||||
names = [secrets.scrub(str(c)) for c in entry["columns"]]
|
||||
# 同快照的列名一样换(_masked_view):换了写法的号码也认
|
||||
names = [secrets.scrub(str(c), loose=True) for c in entry["columns"]]
|
||||
return {**entry, "columns": names} if names != [str(c) for c in entry["columns"]] else entry
|
||||
|
||||
|
||||
@@ -2048,11 +2086,15 @@ async def inquiry_artifact(run_id: str, artifact_id: str) -> dict[str, Any]:
|
||||
raise InquiryRefused(404, "无法读取这件工件,可能已被删除", code=ARTIFACT_NOT_FOUND)
|
||||
if not _query_shaped(content):
|
||||
return {"id": artifact_id, "content": content, "masked": []}
|
||||
# 遮罩值照回答时收:引用它的那几条质询各自查到的全部快照(查询条件里写着的值,不一定在这一份的结果里),加上报告
|
||||
# 自己的查询快照;免遮的也照回答时:提问人自己写的、报告里列出的指标和运行输入的值。SQL 里照原样留着的阈值只在
|
||||
# 回答正文里不遮,格子照旧按值遮,这里不用。文字格里夹着的遮罩值按子串换(embedded):回答那边渲染出来露了就整份
|
||||
# 不交,这份快照却照样记在事件里、点得开,不换就等于在「不予展示」旁边把原值摆出来
|
||||
own: dict[str, dict[str, Any]] = {artifact_id: content}
|
||||
# agent 自己查询里遮掉的值事后已经不在(查询时就遮好了,快照里收不到):回答时只靠它们才换掉的地方记在事件里
|
||||
# (REPLAY_KEY,只记位置),先照着再换一遍。先查号码、后查备注的,备注查询时就遮好了,用不着这一步
|
||||
edits = [e for d in referenced if isinstance(d.get(REPLAY_KEY), dict) and (e := d[REPLAY_KEY].get(artifact_id))]
|
||||
replayed = replay_edits(content, edits) if edits else content
|
||||
# 再照回答时的办法遮。遮罩值照回答时收:引用它的那几条质询各自查到的全部快照(查询条件里写着的值,不一定在这一份
|
||||
# 的结果里),加上报告自己的查询快照;免遮的也照回答时:提问人自己写的、报告里列出的指标和运行输入的值。SQL 里照原样
|
||||
# 留着的阈值只在回答正文里不遮,格子照旧按值遮,这里不用。文字格里夹着的遮罩值按子串换(embedded):回答那边渲染
|
||||
# 出来露了就整份不交,这份快照却照样记在事件里、点得开,不换就等于在「不予展示」旁边把原值摆出来
|
||||
own: dict[str, dict[str, Any]] = {artifact_id: replayed}
|
||||
for data in referenced:
|
||||
for other in dict.fromkeys(str(a) for a in data.get("inquiry_artifacts") or []):
|
||||
if other not in own and _query_shaped(snapshot := _quiet_load(artifact_store.load, other)):
|
||||
@@ -2061,6 +2103,9 @@ async def inquiry_artifact(run_id: str, artifact_id: str) -> dict[str, Any]:
|
||||
asked = [t for d in referenced for t in (d.get("question"), d.get("quote")) if isinstance(t, str)]
|
||||
plan = await _plan(own, others, [*asked, *values], embedded=frozenset({artifact_id}))
|
||||
shown = plan.views[artifact_id]
|
||||
masked = list(dict.fromkeys([*plan.masked[artifact_id],
|
||||
*(m for e in edits if isinstance(e, dict) for m in e.get("masked") or []
|
||||
if isinstance(m, str))]))
|
||||
recorded = next((q["sql"] for d in referenced for q in d.get("queries") or []
|
||||
if isinstance(q, dict) and q.get("artifact") == artifact_id and isinstance(q.get("sql"), str)),
|
||||
None)
|
||||
@@ -2073,7 +2118,7 @@ async def inquiry_artifact(run_id: str, artifact_id: str) -> dict[str, Any]:
|
||||
whole, pieces = _recorded_swaps(content.get("sql"), recorded, kind) if recorded is not None else (set(), set())
|
||||
shown = {**shown, "checks": _masked_checks(shown["checks"], content.get("sql"), plan.secrets, compared, kind,
|
||||
swapped=whole, pieces=pieces)}
|
||||
out: dict[str, Any] = {"id": artifact_id, "content": shown, "masked": plan.masked[artifact_id]}
|
||||
out: dict[str, Any] = {"id": artifact_id, "content": shown, "masked": masked}
|
||||
if gone := plan.gone.get(artifact_id):
|
||||
out["mask_note"] = SOURCE_GONE.format(source="、".join(gone))
|
||||
return out
|
||||
|
||||
+465
-69
@@ -6,24 +6,29 @@
|
||||
质询(engine/inquiry.py)、双盲复算(nodes/dual_check.py,经 inquiry 的同名导出)共用:
|
||||
|
||||
- 工具边界先遮(mask_query_result):核查 agent 的查询结果交给模型、落进快照之前,取自遮罩列的格子换成「已遮罩」,
|
||||
同一份结果里别的文字格夹着的遮罩值也换掉。模型看不到遮罩值,就不会把它写进回答、理由、SQL——事后按值认每一轮
|
||||
复核都找到新的漏法,根治靠这一步;审批质询回答里的按值遮降为第二道防线
|
||||
别的格子按值换:这份结果里的遮罩值,加上这个 agent 前面的查询遮掉的、调用方交进来的(报告快照里的)。模型看不到
|
||||
遮罩值,就不会把它写进回答、理由、SQL——事后按值认每一轮复核都找到新的漏法,根治靠这一步;审批质询回答、反方
|
||||
审稿理由里的按值遮降为第二道防线。查询失败时报错原话里的值也换(mask_error_text)
|
||||
|
||||
- 哪些列(_derived_hidden):从遮罩列本身起,按 SQL 一层层认出值取自遮罩列的结果列(别名、函数、子查询、WITH、UNION 的
|
||||
另一支、和 * 写在同一支里没起别名的表达式);COUNT 只数个数、窗口的分区键只决定次序、按遮罩列分档又当分组键或套在
|
||||
汇总里数个数的不算(_value_columns、_band_only)。认不出落在哪一列的、SQL 解析不了又写着遮罩列的,整份快照的列都算
|
||||
- 只装着分档标签的列(_label_columns):标签是 agent 写在 SQL 里的字,不当遮罩值
|
||||
另一支、和 * 写在同一支里没起别名的表达式、FROM 里拆开遮罩列的表值函数);COUNT 只数个数、窗口的分区键只决定次序、
|
||||
按遮罩列分档又当分组键或套在汇总里数个数的、汇总里的 CASE 只在条件里用到的不算(_value_columns、_band_only)。认不出
|
||||
落在哪一列的、SQL 解析不了又写着遮罩列的,整份快照的列都算。双盲复算按同一套认法在派盲算之前挡下取自遮罩列的指标
|
||||
(derived_columns)
|
||||
- 只装着分档标签的列(_label_columns):标签是 agent 写在 SQL 里的字,不当遮罩值;* 展开数据源里的表时按冻结的表结构
|
||||
认它带不带同名的列(_schema_tables)
|
||||
- 哪些值(_Secrets):遮罩列里出现过的值,按值认有门槛(SECRET_MIN、SECRET_DIGITS),换的时候先按词、按开头几个字筛出
|
||||
正文里真有的那几个值,不现编上万个分支的正则;正文、文字格里换了写法的长号码(199-1474-2785、+86 …、全角数字)按
|
||||
纯数字写法在集合里查(loose_spans)
|
||||
- 快照遮好的样子(_masked_view)
|
||||
正文里真有的那几个值,不现编上万个分支的正则;正文、文字格、SQL 的字符串和注释里换了写法的长号码(199-1474-2785、
|
||||
+86 …、全角数字)按纯数字写法在集合里查(loose_spans);已经摆在读者眼前的值免遮,之后并进来的照样免(protected)
|
||||
- 快照遮好的样子(_masked_view),回答时才认得出、事后要照位置重放的地方(view_edits、replay_edits)
|
||||
|
||||
除 mask_query_result、换了写法的号码之外,这里的函数原来都在 engine/inquiry.py 里,名字不变,inquiry 照旧导出(别处按
|
||||
inquiry 的名字引用)。
|
||||
除 mask_query_result、换了写法的号码和这一轮新加的几样之外,这里的函数原来都在 engine/inquiry.py 里,名字不变,
|
||||
inquiry 照旧导出(别处按 inquiry 的名字引用)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import functools
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
@@ -32,7 +37,7 @@ from dataclasses import dataclass, field
|
||||
from decimal import Decimal
|
||||
from typing import Any
|
||||
|
||||
from app.engine.evidence import RenderError, render_cell
|
||||
from app.engine.evidence import MARKER_RE, RenderError, render_cell
|
||||
from app.engine.expressions import numeric_text
|
||||
from app.engine.merge_query import MERGE_SOURCE
|
||||
|
||||
@@ -70,7 +75,9 @@ def _masked_word() -> str:
|
||||
def _value_columns(node: Any) -> set[str]:
|
||||
"""一个选出列的值取自哪些列(列名小写)。COUNT 里的只数个数,窗口的分区键、排序键只决定次序,结果全是写死常量的
|
||||
分档(_band_only:当分组键,或者套在汇总里按条件数个数)只决定一行落在哪一档,都带不出值,不算;套在里面的子查询
|
||||
只看它选出来的列(它的条件、连接里写的列带不出值)。"""
|
||||
只看它选出来的列(它的条件、连接里写的列带不出值)。汇总里的 CASE / IIF 只看各个结果:条件只决定这一行算不算进来,
|
||||
同写在 WHERE 里(SUM(CASE WHEN buyer_phone IS NOT NULL THEN amount ELSE 0 END) 的值全部来自金额);不汇总的逐行
|
||||
CASE 照旧连条件一起算(每一行落在哪一支本身就是这一行的信息)。"""
|
||||
from sqlglot import exp
|
||||
|
||||
out: set[str] = set()
|
||||
@@ -82,6 +89,9 @@ def _value_columns(node: Any) -> set[str]:
|
||||
out.add(item.name.lower())
|
||||
elif isinstance(item, exp.Count) or (isinstance(item, (exp.Case, exp.If)) and _band_only(item)):
|
||||
continue
|
||||
elif isinstance(item, (exp.Case, exp.If)) and (results := _branch_results(item)) is not None \
|
||||
and _aggregated(item):
|
||||
stack.extend(r for r in results if r is not None)
|
||||
elif isinstance(item, exp.Window):
|
||||
stack.append(item.this)
|
||||
elif isinstance(item, exp.Select):
|
||||
@@ -91,6 +101,18 @@ def _value_columns(node: Any) -> set[str]:
|
||||
return out
|
||||
|
||||
|
||||
def _aggregated(node: Any) -> bool:
|
||||
"""node 套在同一层 SELECT 的汇总函数里(SUM(…)、AVG(…) OVER (…) 也算)。"""
|
||||
from sqlglot import exp
|
||||
|
||||
parent = node.parent
|
||||
while parent is not None and not isinstance(parent, (exp.Select, exp.Subquery)):
|
||||
if isinstance(parent, exp.AggFunc):
|
||||
return True
|
||||
parent = parent.parent
|
||||
return False
|
||||
|
||||
|
||||
def _branch_results(node: Any) -> list[Any] | None:
|
||||
"""CASE 的各个结果(每个 THEN,加上 ELSE;没写 ELSE 记 None,就是 NULL)、IIF / IF 的真假两支;别的写法为 None。
|
||||
CASE 里的每个 WHEN … THEN 在语法树里也是 If,不单算。"""
|
||||
@@ -123,11 +145,8 @@ def _band_only(node: Any) -> bool:
|
||||
results = _branch_results(node)
|
||||
if results is None or not all(_fixed(r) for r in results):
|
||||
return False
|
||||
parent = node.parent
|
||||
while parent is not None and not isinstance(parent, (exp.Select, exp.Subquery)):
|
||||
if isinstance(parent, exp.AggFunc):
|
||||
return True
|
||||
parent = parent.parent
|
||||
if _aggregated(node):
|
||||
return True
|
||||
item = node
|
||||
while isinstance(item.parent, (exp.Paren, exp.Alias)) and item.arg_key == "this":
|
||||
item = item.parent
|
||||
@@ -145,7 +164,8 @@ def _band_only(node: Any) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _label_columns(sql: Any, kind: str | None, columns: list[str]) -> dict[str, set[str]]:
|
||||
def _label_columns(sql: Any, kind: str | None, columns: list[str], tables: dict[str, set[str]] | None = None
|
||||
) -> dict[str, set[str]]:
|
||||
"""结果里只装着分档标签的列:列名(小写)→ 标签在格子里的写法。标签(CASE、IIF 写死的结果,分档的名字、档位编号)
|
||||
是 agent 自己写在 SQL 里的字,不是数据源里的值:这几列派生自遮罩列也不收进遮罩值(_Secrets.add),不然 SQL 里的
|
||||
标签、回答正文里的档名都会被换成「已遮罩」。
|
||||
@@ -154,7 +174,8 @@ def _label_columns(sql: Any, kind: str | None, columns: list[str]) -> dict[str,
|
||||
选出(SELECT vip_level)的名字就是它自己,永远算只装着标签,UNION 另一支、子查询里的遮罩列流进标签列也认不出,
|
||||
等级 1 撞上 THEN 1 就不收。取自数据源里的表的(哪怕和标签同名)、有一支装着别的、认不准的(说不清取自哪个来源、
|
||||
递归的 CTE、几支的 * 展开在哪说不清)都不算,照收(宁可多遮);同名的几列有一列不算就都不算。解析不了的一个都
|
||||
不算。"""
|
||||
不算。tables 是查询当时冻结的表结构里每张表的列名(_schema_tables):没写表名的 * 展开的表查得到、又没有同名的列
|
||||
时,这张表带不出这一列(_table_lacks),不再因为它不算。"""
|
||||
from app.data.sql_semantics import _parse, _read
|
||||
|
||||
try:
|
||||
@@ -167,7 +188,8 @@ def _label_columns(sql: Any, kind: str | None, columns: list[str]) -> dict[str,
|
||||
result = next((top for top, role, _listed in _blocks(root) if role == "result"), None)
|
||||
if result is None:
|
||||
return {}
|
||||
memo: _LabelMemo = {}
|
||||
memo = _LabelMemo()
|
||||
memo.tables = tables
|
||||
found: dict[str, list[set[str] | None]] = {}
|
||||
for place, column in enumerate(columns):
|
||||
name = column.lower()
|
||||
@@ -180,8 +202,42 @@ def _label_columns(sql: Any, kind: str | None, columns: list[str]) -> dict[str,
|
||||
|
||||
#: 标签列分析里「这个来源不出这一列」(_block_label、_source_label):和 None(出这一列,但装着别的、认不准)分开
|
||||
_ABSENT: Any = object()
|
||||
#: 标签列分析记下的结果:(id(那块查询), 列名) → 标签的写法、None 或 _ABSENT
|
||||
_LabelMemo = dict[tuple[int, str], Any]
|
||||
|
||||
|
||||
class _LabelMemo(dict): # type: ignore[type-arg]
|
||||
"""标签列分析记下的结果:(id(那块查询), 列名) → 标签的写法、None 或 _ABSENT。tables 是冻结的表结构里每张表的列名
|
||||
(小写,_schema_tables),没有为 None。"""
|
||||
|
||||
tables: dict[str, set[str]] | None = None
|
||||
|
||||
|
||||
def _schema_tables(schema: Any) -> dict[str, set[str]] | None:
|
||||
"""冻结的表结构(数据源的 schema_cache、表结构快照的内容:{"tables": {表名: {"columns": [{"name": …}]}}})→
|
||||
{表名(小写): 列名(小写)},按登记的名字和不带库名的表名各记一份(几个库里同名的表合起来算,宁可多遮)。表结构
|
||||
里没列出的表不记(截断了的表结构也一样),查不到就当说不清;整份拿不到的为 None。"""
|
||||
tables = schema.get("tables") if isinstance(schema, dict) else None
|
||||
if not isinstance(tables, dict):
|
||||
return None
|
||||
out: dict[str, set[str]] = {}
|
||||
for key, meta in tables.items():
|
||||
columns = meta.get("columns") if isinstance(meta, dict) else None
|
||||
if not isinstance(columns, list):
|
||||
continue
|
||||
names = {str(c["name"]).lower() for c in columns if isinstance(c, dict) and c.get("name") is not None}
|
||||
for name in {str(key).lower(), str(key).lower().rsplit(".", 1)[-1],
|
||||
str(meta.get("qualified") or key).lower().rsplit(".", 1)[-1]}:
|
||||
out.setdefault(name, set()).update(names)
|
||||
return out
|
||||
|
||||
|
||||
def _table_lacks(source: Any, name: str, memo: _LabelMemo) -> bool:
|
||||
"""FROM 里的这个来源是数据源里的一张表,冻结的表结构里查得到它、又没有 name 这一列:* 展开它带不出 name。"""
|
||||
from sqlglot import exp
|
||||
|
||||
if not memo.tables or not isinstance(source, exp.Table) or not isinstance(source.this, exp.Identifier):
|
||||
return False
|
||||
columns = memo.tables.get(source.name.lower())
|
||||
return columns is not None and name not in columns
|
||||
|
||||
|
||||
def _result_label(top: Any, place: int, name: str, width: int, memo: _LabelMemo) -> set[str] | None:
|
||||
@@ -247,7 +303,8 @@ def _star_label(select: Any, name: str, memo: _LabelMemo, *, before: int | None
|
||||
if not _is_star(item):
|
||||
continue
|
||||
qualifier = (item.table.lower() or None) if isinstance(item, exp.Column) else None
|
||||
if qualifier is None and any(_derived_block(source) is None for _, source in _from_sources(select)):
|
||||
if qualifier is None and any(_derived_block(source) is None and not _table_lacks(source, name, memo)
|
||||
for _, source in _from_sources(select)):
|
||||
return None
|
||||
texts = _source_label(select, qualifier, name, memo)
|
||||
if texts is None:
|
||||
@@ -275,7 +332,8 @@ def _source_label(select: Any, qualifier: str | None, name: str, memo: _LabelMem
|
||||
for _alias, source in sources:
|
||||
derived = _derived_block(source)
|
||||
if derived is None:
|
||||
opaque = True
|
||||
# 冻结的表结构里这张表没有这一列:它不出这一列,不算认不准
|
||||
opaque = opaque or not _table_lacks(source, name, memo)
|
||||
continue
|
||||
texts = _block_label(derived[0], derived[1], name, memo)
|
||||
if texts is None:
|
||||
@@ -378,6 +436,68 @@ def _derived_block(source: Any) -> tuple[Any, list[str] | None] | None:
|
||||
return node, ([c.name.lower() for c in alias.columns] if alias is not None and alias.columns else None)
|
||||
|
||||
|
||||
#: 表值函数常见的出列名:SQLite 的 json_each / json_tree(key、value、atom、fullkey、path),Postgres 的 jsonb_each
|
||||
#: (key、value),Spark 的 explode / posexplode(col、pos),Snowflake 的 FLATTEN(key、value、path、index、this)。
|
||||
#: 结构性的几列(id、parent、type)带不出值,不算——算上的话,同一条 SQL 里别的 id 列都会被遮
|
||||
_FUNCTION_COLUMNS = frozenset({"key", "value", "atom", "fullkey", "path", "col", "pos", "index", "this"})
|
||||
#: 出列名认得全的表值函数:出的是上面那几列,或者一列、名字就是别名(没起别名的是函数名)
|
||||
_TABLE_FUNCTIONS = frozenset({
|
||||
"json_each", "json_tree", "jsonb_each", "jsonb_each_text", "json_each_text", "json_array_elements",
|
||||
"json_array_elements_text", "jsonb_array_elements", "jsonb_array_elements_text", "json_object_keys",
|
||||
"jsonb_object_keys", "unnest", "explode", "explode_outer", "posexplode", "regexp_split_to_table",
|
||||
"string_to_table", "string_split", "openjson", "flatten",
|
||||
})
|
||||
|
||||
|
||||
def _function_sources(select: Any) -> list[Any]:
|
||||
"""一条 SELECT 的 FROM、JOIN、LATERAL VIEW 里说不清出哪些列、装着什么的来源:不是数据源里的表(含指向 WITH 的名字),
|
||||
也不是子查询这类派生来源——表值函数(json_each、unnest、JSON_TABLE)、LATERAL 函数、VALUES。"""
|
||||
from sqlglot import exp
|
||||
|
||||
sources = [source for _, source in _from_sources(select)] + list(select.args.get("laterals") or [])
|
||||
out: list[Any] = []
|
||||
for source in sources:
|
||||
if _derived_block(source) is not None:
|
||||
continue
|
||||
node = source.this if isinstance(source, exp.Lateral) else source
|
||||
if isinstance(node, exp.Table) and isinstance(node.this, exp.Identifier):
|
||||
continue
|
||||
out.append(source)
|
||||
return out
|
||||
|
||||
|
||||
def _function_columns(source: Any) -> set[str] | None:
|
||||
"""表值函数这类来源出的列名(小写):写的列名表、别名(Postgres 里一列的函数出的列就叫别名,没起别名的叫函数名)、
|
||||
函数名、常见的出列名(_FUNCTION_COLUMNS);JSON_TABLE 按 COLUMNS 里写的,VALUES 按 column1、column2…。不认得的函数
|
||||
又没写列名表的,说不清出哪几列,为 None。"""
|
||||
from sqlglot import exp
|
||||
|
||||
node = source.this if isinstance(source, exp.Lateral) else source
|
||||
alias = source.args.get("alias") or (node.args.get("alias") if node is not source else None)
|
||||
names: set[str] = set(_FUNCTION_COLUMNS)
|
||||
listed: list[str] = []
|
||||
if alias is not None:
|
||||
if alias.name:
|
||||
names.add(alias.name.lower())
|
||||
listed = [c.name.lower() for c in alias.args.get("columns") or [] if c.name]
|
||||
names |= set(listed)
|
||||
func = node.this if isinstance(node, exp.Table) else node
|
||||
while isinstance(func, exp.Subquery):
|
||||
func = func.this
|
||||
if isinstance(func, exp.JSONTable):
|
||||
schema = func.args.get("schema")
|
||||
defined = [str(c.name).lower() for c in (schema.expressions if schema is not None else []) if c.name]
|
||||
return names | set(defined) if defined or listed else None
|
||||
if isinstance(func, exp.Values):
|
||||
width = max((len(row.expressions) for row in func.expressions if isinstance(row, exp.Tuple)), default=1)
|
||||
return names | {f"column{i}" for i in range(1, width + 1)}
|
||||
name = (func.name if isinstance(func, exp.Anonymous) else func.sql_name() if isinstance(func, exp.Func)
|
||||
else "").lower()
|
||||
if name:
|
||||
names.add(name)
|
||||
return names if listed or name in _TABLE_FUNCTIONS else None
|
||||
|
||||
|
||||
def _merging(select: Any, name: str) -> bool:
|
||||
"""FROM 里按同名的 name 这一列连起来(USING 里写着它,或者 NATURAL):几个来源的这一列合成一列。"""
|
||||
return any(str(join.args.get("method") or "").upper() == "NATURAL"
|
||||
@@ -531,7 +651,9 @@ def _tainted(sql: str, kind: str | None, masked: set[str], width: int) -> tuple[
|
||||
不同层的也一并算上(宁可多遮)。
|
||||
位置是给没起别名的表达式用的(UPPER(buyer_phone) 的结果列名对不上任何名字):和 * 写在同一支里的,* 前面的从头数、
|
||||
后面的从末尾倒着数。认不出的——两个 * 之间的项、派生来源里没起别名的表达式(外层的 * 会把它原样带出来,列名
|
||||
又认不准)——整份快照的列都算(同解析不了)。"""
|
||||
又认不准)——整份快照的列都算(同解析不了)。
|
||||
FROM、JOIN、LATERAL 里的表值函数(json_each、unnest、JSON_TABLE、LATERAL VALUES)参数里写着这些名字的,它出的列
|
||||
都算(_function_columns):遮罩列是 JSON 时,json_each 拆开是最自然的写法。出哪几列说不清的整份快照都算。"""
|
||||
from sqlglot import exp
|
||||
|
||||
from app.data.sql_semantics import _parse, _read
|
||||
@@ -547,7 +669,9 @@ def _tainted(sql: str, kind: str | None, masked: set[str], width: int) -> tuple[
|
||||
selects = list(root.find_all(exp.Select))
|
||||
blocks = _blocks(root)
|
||||
sources = [(top, listed) for top, role, listed in blocks if role == "source"]
|
||||
for _ in range(len(selects) + len(sources) + 1):
|
||||
functions = [source for select_ in selects for source in _function_sources(select_)]
|
||||
unreadable = False
|
||||
for _ in range(len(selects) + len(sources) + len(functions) + 1):
|
||||
before = len(names)
|
||||
for select_ in selects:
|
||||
for item in select_.expressions:
|
||||
@@ -556,9 +680,14 @@ def _tainted(sql: str, kind: str | None, masked: set[str], width: int) -> tuple[
|
||||
names.add(name)
|
||||
for top, listed in sources:
|
||||
names |= _source_names(top, listed, names)[0]
|
||||
for source in functions:
|
||||
if names & _value_columns(source):
|
||||
columns = _function_columns(source)
|
||||
unreadable = unreadable or columns is None
|
||||
names |= columns or set()
|
||||
if len(names) == before:
|
||||
break
|
||||
everything = any(_source_names(top, listed, names)[1] for top, listed in sources)
|
||||
everything = unreadable or any(_source_names(top, listed, names)[1] for top, listed in sources)
|
||||
positions: set[int] = set()
|
||||
for top, role, _listed in blocks:
|
||||
if role != "result":
|
||||
@@ -598,12 +727,41 @@ def _derived_hidden(snapshot: dict[str, Any], hidden: set[str], kind: str | None
|
||||
return names | {columns[i].lower() for i in positions if i < len(columns)}, names
|
||||
|
||||
|
||||
def _snapshot_labels(snapshot: dict[str, Any], kind: str | None) -> dict[str, set[str]]:
|
||||
"""一份快照里只装着分档标签的列(_label_columns)。没有 SQL 的(合并查询)一个都不算。"""
|
||||
def derived_columns(snapshot: dict[str, Any], hidden: set[str], kind: str | None) -> set[str] | None:
|
||||
"""按 SQL 认出值取自遮罩列(hidden,小写)的结果列(小写,列名直接对上的也算)。SQL 解析不了、合并查询的快照(不看
|
||||
SQL)为 None:认不准,宽严由调用方定(_derived_hidden 整份遮;双盲复算照常派盲算,由盲算自己的查询去认)。"""
|
||||
sql = snapshot.get("sql")
|
||||
if not hidden or not isinstance(sql, str) or not sql.strip() or snapshot.get("source") == MERGE_SOURCE:
|
||||
return None
|
||||
columns = [str(c) for c in snapshot.get("columns") or []]
|
||||
found = _tainted(sql, kind, set(hidden), len(columns))
|
||||
if found is None:
|
||||
return None
|
||||
names, positions = found
|
||||
return {c.lower() for i, c in enumerate(columns) if c.lower() in names or i in positions}
|
||||
|
||||
|
||||
def _snapshot_labels(snapshot: dict[str, Any], kind: str | None, tables: dict[str, set[str]] | None = None
|
||||
) -> dict[str, set[str]]:
|
||||
"""一份快照里只装着分档标签的列(_label_columns)。没有 SQL 的(合并查询)一个都不算。tables 没给的,按快照记着
|
||||
的表结构快照(schema_artifact,查询当时冻结的)认。"""
|
||||
sql = snapshot.get("sql")
|
||||
if not isinstance(sql, str) or not sql.strip():
|
||||
return {}
|
||||
return _label_columns(sql, kind, [str(c) for c in snapshot.get("columns") or []])
|
||||
if tables is None and isinstance(snapshot.get("schema_artifact"), str):
|
||||
tables = _frozen_tables(snapshot["schema_artifact"])
|
||||
return _label_columns(sql, kind, [str(c) for c in snapshot.get("columns") or []], tables)
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=64)
|
||||
def _frozen_tables(artifact: str) -> dict[str, set[str]] | None:
|
||||
"""表结构快照工件里每张表的列名(_schema_tables)。工件按内容寻址、不会变,记下来;取不到、被改过的为 None。"""
|
||||
from app.core import artifact_store
|
||||
|
||||
try:
|
||||
return _schema_tables(artifact_store.load(artifact))
|
||||
except Exception: # noqa: BLE001 - 取不到就当不知道表结构(标签照旧当遮罩值,宁可多遮)
|
||||
return None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
@@ -728,13 +886,17 @@ _WORDS = re.compile(r"[0-9A-Za-z_]+")
|
||||
_DIGITS = re.compile(r"\d[\d,]*(?:\.\d+)?")
|
||||
#: 中文这类遮罩值按开头几个字分桶找(_Secrets._partition)
|
||||
_PREFIX = 8
|
||||
#: 正文里一串允许夹分隔符的数字(换了写法的号码,见 _Secrets.loose_spans):数字(含全角)之间夹着一到三个空格、连字符、
|
||||
#: 点、括号(含全角),开头可以带 + 和括号。逗号不算分隔符(千分位按 _DIGITS 认),冒号、斜杠也不算(时间、日期),不跨行
|
||||
_LOOSE_RUN = re.compile(r"[((]?[++]?[((]?[0-90-9]+(?:[ \u3000\-‐‑‒–—-..·・()()]{1,3}[0-90-9]+)*[))]?")
|
||||
#: 正文里一串允许夹分隔符的数字(换了写法的号码,见 _Secrets.loose_spans):数字(含全角)之间夹着一到四个分隔符,开头
|
||||
#: 可以带 + 和括号。分隔符是同一行里的各种空白(空格、制表符、全角空格、不换行空格,以及窄空格、细空格这类 U+2000 至
|
||||
#: U+200B、U+202F、U+205F)、连字符和减号(含全角、U+2010 至 U+2015、U+2212)、点(含全角、间隔号)、括号(含全角)。
|
||||
#: 逗号不算分隔符(千分位按 _DIGITS 认),冒号、斜杠也不算(时间、日期),不跨行
|
||||
_LOOSE_SEPARATORS = (" \t\u00a0\u2000-\u200b\u202f\u205f\u3000"
|
||||
"\\-\u2010-\u2015\u2212\uff0d.\uff0e\u00b7\u30fb()\uff08\uff09")
|
||||
_LOOSE_RUN = re.compile(rf"[((]?[++]?[((]?[0-90-9]+(?:[{_LOOSE_SEPARATORS}]{{1,4}}[0-90-9]+)*[))]?")
|
||||
_LOOSE_DIGITS = re.compile(r"[0-90-9]+")
|
||||
#: 两截数字之间只隔着这样一个字(不带空格):是更长的编号、小数、日期的两截(13900000001-2、13900000001.5),号码到这里
|
||||
#: 不算完——同 _pattern 里「前后不能是 [.:/-] 加数字」
|
||||
_JOINERS = frozenset("-‐‑‒–—-..")
|
||||
#: 不算完——同 _pattern 里「前后不能是 [.:/-] 加数字」。减号(U+2212)、全角连字符同连字符
|
||||
_JOINERS = frozenset("-\u2010\u2011\u2012\u2013\u2014\u2015\u2212\uff0d.\uff0e")
|
||||
_POINTS = frozenset("..")
|
||||
#: 去掉国家码(0086、86)之后至少还要有这么多位才算号码(固话号码最短 7 位):86 开头的短数不拆
|
||||
_LOCAL_MIN = 7
|
||||
@@ -857,11 +1019,13 @@ class _Secrets:
|
||||
留着的阈值、行数(去掉千分位、正负号的数字),只在回答正文里不换(scrub 的 prose)。换的时候先按词、按开头两个
|
||||
字筛出正文里真有的那几个值,再拿它们现编正则:遮罩值上万个时,现编一个上万分支的正则要一秒上下,每次都卡住整个进程。
|
||||
正文、文字格里另认换了写法的长号码(199-1474-2785、+86 199 1474 2785、全角数字,见 loose_spans):按纯数字写法
|
||||
在集合里查,不进正则。"""
|
||||
在集合里查,不进正则。protected 是 protect 收下的、已经摆在读者眼前的文字:之后并进来的值(核查 agent 后面的
|
||||
查询遮掉的)照样按它们免遮。"""
|
||||
|
||||
strong: set[str] = field(default_factory=set)
|
||||
weak: set[str] = field(default_factory=set)
|
||||
shown: set[str] = field(default_factory=set)
|
||||
protected: tuple[str, ...] = ()
|
||||
_index: Any = field(default=None, repr=False, compare=False)
|
||||
_compiled: dict[frozenset[str], re.Pattern[str]] = field(default_factory=dict, repr=False, compare=False)
|
||||
|
||||
@@ -899,8 +1063,10 @@ class _Secrets:
|
||||
self._index = None
|
||||
|
||||
def merge(self, other: _Secrets | None) -> None:
|
||||
"""并进另一份收好的遮罩值(核查 agent 查询时在工具边界遮掉的,见 mask_query_result)。合起来最多 SECRETS_MAX 种。"""
|
||||
if other is None or not other.weak:
|
||||
"""并进另一份收好的遮罩值(核查 agent 查询时在工具边界遮掉的,见 mask_query_result)。合起来最多 SECRETS_MAX 种。
|
||||
两边免遮的文字合起来,并进来的值照样按它们免遮:报告的指标正好等于某一笔金额时,agent 查出了那一笔,别的查询里
|
||||
正好是这个数的格子也不遮。"""
|
||||
if other is None:
|
||||
return
|
||||
for form in other.weak:
|
||||
if len(self.weak) >= SECRETS_MAX:
|
||||
@@ -908,14 +1074,23 @@ class _Secrets:
|
||||
self.weak.add(form)
|
||||
if form in other.strong:
|
||||
self.strong.add(form)
|
||||
self.protected = tuple(dict.fromkeys((*self.protected, *other.protected)))
|
||||
self._index, self._compiled = None, {}
|
||||
if self.protected:
|
||||
self._spare(" ".join(self.protected))
|
||||
|
||||
def protect(self, texts: Iterable[str | None]) -> None:
|
||||
"""已经摆在读者眼前的值不换:提问人自己写进问题、选中原文里的(「13900000001 这位客户」「排除 300 以下的」),
|
||||
报告里列出的指标、运行输入的值——换了只会让回答、查询对不上问题和报告。正文、格子、SQL 里都不换。"""
|
||||
报告里列出的指标、运行输入的值——换了只会让回答、查询对不上问题和报告。正文、格子、SQL 里都不换。文字记下来,
|
||||
之后并进来的值照样免遮(merge)。"""
|
||||
given = " ".join(t for t in texts if t)
|
||||
if not given:
|
||||
return
|
||||
self.protected = tuple(dict.fromkeys((*self.protected, given)))
|
||||
self._spare(given)
|
||||
|
||||
def _spare(self, given: str) -> None:
|
||||
"""去掉 given(几段免遮的文字)里写着的值。"""
|
||||
numbers = {_digits_of(n) for n in _DIGITS.findall(given)}
|
||||
# 换了写法写进来的号码(「199-1474-2785 这位客户」)按纯数字写法认
|
||||
reformatted = {key for _, _, key in self.loose_spans(given)}
|
||||
@@ -958,15 +1133,40 @@ class _Secrets:
|
||||
|
||||
def scrub(self, text: str, *, prose: bool = False, loose: bool | None = None) -> str:
|
||||
"""text 里的遮罩值换成「已遮罩」。prose:这是回答正文,SQL 里照原样留着的阈值、行数不换(show)。loose:换了写法
|
||||
的长号码也换(loose_spans),缺省同 prose——只在正文、文字格里这样认,SQL 里的 199-1474-2785 是一个算式。"""
|
||||
的长号码也换(loose_spans),缺省同 prose——正文、文字格、SQL 的字符串和注释里这样认,SQL 里不带引号的
|
||||
199-1474-2785 是一个算式。"""
|
||||
return self.swap(text, prose=prose, loose=loose)[0]
|
||||
|
||||
def swap(self, text: str, *, prose: bool = False, loose: bool | None = None) -> tuple[str, list[str]]:
|
||||
"""scrub,另交回换掉了哪些值:按值认出的照原文的写法,换了写法的号码交它的纯数字写法(SQL 里同一个数写成阈值的
|
||||
据此不留,见 inquiry._swap_literal)。"""
|
||||
hits = self._hits(text, prose=prose, loose=loose)
|
||||
word = _masked_word()
|
||||
for start, end, _ in reversed(hits):
|
||||
text = text[:start] + word + text[end:]
|
||||
return text, [value for _, _, value in hits]
|
||||
|
||||
def spans(self, text: str, *, prose: bool = False, loose: bool | None = None) -> list[tuple[int, int]]:
|
||||
"""scrub 会换掉的那几段(起, 止),按先后、不重叠。质询工件按位置重放回答时换掉的地方(view_edits)。"""
|
||||
return [(start, end) for start, end, _ in self._hits(text, prose=prose, loose=loose)]
|
||||
|
||||
def _hits(self, text: str, *, prose: bool, loose: bool | None) -> list[tuple[int, int, str]]:
|
||||
"""text 里要换掉的遮罩值:(起, 止, 值),按先后、不重叠。换了写法的号码先认(+86 19914742785 整个换掉,不留下
|
||||
一截「+86」),按值认的落在它们里面的不再算。"""
|
||||
if not text or not self.strong:
|
||||
return text
|
||||
if prose if loose is None else loose:
|
||||
# 先换换了写法的:+86 19914742785 整个换掉,不留下一截「+86」
|
||||
for start, end, _ in reversed(self.loose_spans(text, prose=prose)):
|
||||
text = text[:start] + _masked_word() + text[end:]
|
||||
return []
|
||||
hits = [(start, end, key) for start, end, key in self.loose_spans(text, prose=prose)] \
|
||||
if (prose if loose is None else loose) else []
|
||||
pattern = self.pattern(text, prose=prose)
|
||||
return pattern.sub(_masked_word(), text) if pattern is not None else text
|
||||
if pattern is not None:
|
||||
taken = list(hits)
|
||||
hits += [(m.start(), m.end(), m.group()) for m in pattern.finditer(text)
|
||||
if not any(m.start() < end and start < m.end() for start, end, _ in taken)]
|
||||
return sorted(hits)
|
||||
|
||||
def copy(self) -> _Secrets:
|
||||
"""另一份一样的遮罩值(改它不影响这一份):交给线程、按句另加免遮的值时用。"""
|
||||
return _Secrets(strong=set(self.strong), weak=set(self.weak), shown=set(self.shown), protected=self.protected)
|
||||
|
||||
def loose_spans(self, text: str, *, prose: bool = False) -> list[tuple[int, int, str]]:
|
||||
"""text 里换了写法的长号码(199-1474-2785、(199)1474-2785、+86 199 1474 2785、全角数字……):(起, 止, 纯数字
|
||||
@@ -1058,6 +1258,34 @@ class _Secrets:
|
||||
return self._index
|
||||
|
||||
|
||||
#: 单元格引用的前一截(Q1.r0.):后面才是列名
|
||||
_CELL_HEAD = re.compile(r"^\s*[QK]\d+\.r\d+\.")
|
||||
|
||||
|
||||
def _scrubbed_source(text: str, secrets: _Secrets) -> str:
|
||||
"""回答原文里的遮罩值换成「已遮罩」:标记之外按正文的办法换(SQL 里照原样留着的阈值不换);标记里的按段换(| 隔开
|
||||
的几段各自换),单元格引用只换列名那一截——编号、行号换不出东西,整段换的话「r0.」会让列名开头的长号码看着像小数
|
||||
的一截,反而认不出来。列名照快照、SQL 里的名字那样换(阈值也换,换了写法的号码也认),换得一样才对得上。
|
||||
标记里写着遮罩值的引用换完多半解析不了(列名里写着遮罩值的除外:快照的列名也换过了,正好对上),宁可弄坏一个
|
||||
引用,也不让值留在存下的原文、片段里,或者从解析不了的占位里露出来。"""
|
||||
if not text or not secrets.strong:
|
||||
return text
|
||||
out: list[str] = []
|
||||
last = 0
|
||||
for m in MARKER_RE.finditer(text):
|
||||
out.append(secrets.scrub(text[last:m.start()], prose=True))
|
||||
start, end = m.span("body")
|
||||
parts = text[start:end].split("|")
|
||||
head = _CELL_HEAD.match(parts[0]) if m.group("kind").strip() == "v" else None
|
||||
first = parts[0][:head.end()] + secrets.scrub(parts[0][head.end():], loose=True) if head \
|
||||
else secrets.scrub(parts[0], prose=True)
|
||||
parts = [first, *(secrets.scrub(p, prose=True) for p in parts[1:])]
|
||||
out.append(text[m.start():start] + "|".join(parts) + text[end:m.end()])
|
||||
last = m.end()
|
||||
out.append(secrets.scrub(text[last:], prose=True))
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def _masked_view(snapshot: dict[str, Any], hidden: set[str], secrets: _Secrets, *, embedded: bool = False,
|
||||
rename: bool = True) -> tuple[dict[str, Any], list[str]]:
|
||||
"""一份查询快照遮好的样子和遮掉的列:要遮的列(hidden,小写)每一格都换成「已遮罩」、类型记成文本(渲染时不当数);
|
||||
@@ -1068,22 +1296,30 @@ def _masked_view(snapshot: dict[str, Any], hidden: set[str], secrets: _Secrets,
|
||||
embedded 只给质询工件用:别的列里一格文字夹着遮罩值(备注里写着手机号,换了写法的也算),把那一截换成「已遮罩」,
|
||||
这一格别的字照原样,门槛同整格(_Secrets.scrub);这一列不记进遮掉的列——界面把记进去的列整列画成「已遮罩」,这一列在数据源
|
||||
里也没设遮罩。回答一路不这样换:渲染出来露了就整份不交(_leaked),不然回答被问的那一格看着像查到了别的东西。
|
||||
rename 为假时列名不换(工具边界先遮时:列名是模型自己写的,换了它引用的列名就对不上)。"""
|
||||
rename 为假时列名不换(工具边界先遮时:列名是模型自己写的,换了它引用的列名就对不上)。列名按文字的办法换(换了写法的
|
||||
号码也认,同 SQL 里带引号的名字)。
|
||||
工具边界遮过的快照(记着 masked_at_query)里,整格就是「已遮罩」的是查询时按值换掉的:这一列照样记进遮掉的列,和回答时
|
||||
按值换掉的一样画。"""
|
||||
columns = [str(c) for c in snapshot.get("columns") or []]
|
||||
index = {i for i, c in enumerate(columns) if c.lower() in hidden}
|
||||
by_value = bool(secrets.strong)
|
||||
if not index and not by_value:
|
||||
premasked = "masked_at_query" in snapshot
|
||||
if not index and not by_value and not premasked:
|
||||
return snapshot, []
|
||||
word = _masked_word()
|
||||
touched = set(index)
|
||||
edited = False
|
||||
position = {c: i for i, c in enumerate(columns)}
|
||||
names = [secrets.scrub(c) for c in columns] if by_value and rename else columns
|
||||
names = [secrets.scrub(c, loose=True) for c in columns] if by_value and rename else columns
|
||||
|
||||
def cell(i: int | None, value: Any) -> Any:
|
||||
nonlocal edited
|
||||
if i in index:
|
||||
return word
|
||||
if premasked and value == word:
|
||||
if i is not None:
|
||||
touched.add(i)
|
||||
return value
|
||||
if by_value and secrets.hit(value):
|
||||
if i is not None:
|
||||
touched.add(i)
|
||||
@@ -1119,6 +1355,122 @@ def _masked_view(snapshot: dict[str, Any], hidden: set[str], secrets: _Secrets,
|
||||
return view, [names[i] for i in sorted(touched)]
|
||||
|
||||
|
||||
def _cell_text(value: Any) -> str | None:
|
||||
"""一格按文字换的样子(_masked_view 的 embedded):文字照原样,JSON(对象、数组)按界面画出来的样子;别的为 None。"""
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
if isinstance(value, (dict, list)):
|
||||
return json.dumps(value, ensure_ascii=False, default=str)
|
||||
return None
|
||||
|
||||
|
||||
def view_edits(snapshot: dict[str, Any], hidden: set[str], secrets: _Secrets, baseline: _Secrets
|
||||
) -> dict[str, Any] | None:
|
||||
"""质询工件按回答时的办法遮(_masked_view 的 embedded)时,secrets 比 baseline 多换掉的地方,相对原快照记成位置,
|
||||
不含值:{"cells": [[行, 列], …](整格换成「已遮罩」), "spans": [[行, 列, 起, 止], …](文字格里换掉的那一截,起止按
|
||||
这一格的文字算), "names": [[列, 起, 止], …](列名里换掉的那一截), "masked": [列名, …](因此画成遮罩列的)};没有
|
||||
多换的为 None。
|
||||
agent 先查了备注(那时还不知道备注里的号码是遮罩值,原样交回、原样存下),后来又查出了手机号(查询时遮掉,值只在
|
||||
内存里):回答时按内存里的值(secrets)认得出,事后取质询工件时只剩 baseline 认得出的。这份位置记进质询事件,
|
||||
事后照着再换一遍(replay_edits)。行是字典的快照不记(核查的查询快照都是行列表)。"""
|
||||
full, full_masked = _masked_view(snapshot, hidden, secrets, embedded=True)
|
||||
base, base_masked = _masked_view(snapshot, hidden, baseline, embedded=True)
|
||||
if full is base:
|
||||
return None
|
||||
word = _masked_word()
|
||||
cells: list[list[int]] = []
|
||||
spans: list[list[int]] = []
|
||||
for r, (row, shown, plain) in enumerate(zip(snapshot.get("rows") or [], full["rows"], base["rows"])):
|
||||
if not (isinstance(row, list) and isinstance(shown, list) and isinstance(plain, list)):
|
||||
continue
|
||||
for c, (value, now, before) in enumerate(zip(row, shown, plain)):
|
||||
if now == before:
|
||||
continue
|
||||
if now == word:
|
||||
cells.append([r, c])
|
||||
elif (text := _cell_text(value)) is not None:
|
||||
spans += [[r, c, start, end] for start, end in secrets.spans(text, loose=True)]
|
||||
columns = [str(c) for c in snapshot.get("columns") or []]
|
||||
full_names, base_names = full.get("columns", columns), base.get("columns", columns)
|
||||
names = [[i, start, end] for i, (column, now, before) in enumerate(zip(columns, full_names, base_names))
|
||||
if now != before for start, end in secrets.spans(column, loose=True)]
|
||||
# 按位置比:同一列在两边可能叫不同的名字(列名里的值只有一边换了)
|
||||
already = {i for i, name in enumerate(base_names) if name in base_masked}
|
||||
masked = [name for i, name in enumerate(full_names) if name in full_masked and i not in already]
|
||||
if not (cells or spans or names or masked):
|
||||
return None
|
||||
return {"cells": cells, "spans": spans, "names": names, "masked": masked}
|
||||
|
||||
|
||||
def replay_edits(snapshot: dict[str, Any], edits: Iterable[Any]) -> dict[str, Any]:
|
||||
"""照记下的位置(view_edits,可能有好几份:同一份快照被几条质询引用)把回答时换掉的地方再换一遍。位置对不上的
|
||||
(行、列越界,起止不合理,形状不对)跳过:记录是封存的事件,坏了也不能让取工件出错。没有可换的返回原快照。"""
|
||||
rows = snapshot.get("rows")
|
||||
columns = [str(c) for c in snapshot.get("columns") or []]
|
||||
if not isinstance(rows, list):
|
||||
return snapshot
|
||||
|
||||
def ints(entry: Any, size: int) -> list[int] | None:
|
||||
ok = isinstance(entry, list) and len(entry) == size and all(
|
||||
isinstance(x, int) and not isinstance(x, bool) and x >= 0 for x in entry)
|
||||
return entry if ok else None
|
||||
|
||||
whole: set[tuple[int, int]] = set()
|
||||
parts: dict[tuple[int, int], list[tuple[int, int]]] = {}
|
||||
heads: dict[int, list[tuple[int, int]]] = {}
|
||||
for edit in edits:
|
||||
if not isinstance(edit, dict):
|
||||
continue
|
||||
for entry in edit.get("cells") or []:
|
||||
if (at := ints(entry, 2)) is not None:
|
||||
whole.add((at[0], at[1]))
|
||||
for entry in edit.get("spans") or []:
|
||||
if (at := ints(entry, 4)) is not None and at[2] < at[3]:
|
||||
parts.setdefault((at[0], at[1]), []).append((at[2], at[3]))
|
||||
for entry in edit.get("names") or []:
|
||||
if (at := ints(entry, 3)) is not None and at[1] < at[2]:
|
||||
heads.setdefault(at[0], []).append((at[1], at[2]))
|
||||
word = _masked_word()
|
||||
|
||||
def swapped(text: str, cuts: list[tuple[int, int]]) -> str:
|
||||
merged: list[list[int]] = []
|
||||
for start, end in sorted(cuts):
|
||||
if end > len(text):
|
||||
continue
|
||||
if merged and start <= merged[-1][1]:
|
||||
merged[-1][1] = max(merged[-1][1], end)
|
||||
else:
|
||||
merged.append([start, end])
|
||||
for start, end in reversed(merged):
|
||||
text = text[:start] + word + text[end:]
|
||||
return text
|
||||
|
||||
out_rows: list[Any] = []
|
||||
changed = False
|
||||
for r, row in enumerate(rows):
|
||||
if not isinstance(row, list):
|
||||
out_rows.append(row)
|
||||
continue
|
||||
new_row = list(row)
|
||||
for c, value in enumerate(row):
|
||||
if (r, c) in whole:
|
||||
new_row[c] = word
|
||||
elif (r, c) in parts and (text := _cell_text(value)) is not None:
|
||||
new_row[c] = swapped(text, parts[(r, c)])
|
||||
changed = changed or new_row != row
|
||||
out_rows.append(new_row)
|
||||
names = [swapped(name, heads[i]) if i in heads else name for i, name in enumerate(columns)]
|
||||
if not changed and names == columns:
|
||||
return snapshot
|
||||
view = {**snapshot, "rows": out_rows}
|
||||
if names != columns:
|
||||
view["columns"] = names
|
||||
if isinstance(snapshot.get("column_types"), dict):
|
||||
rename = dict(zip(columns, names))
|
||||
view["column_types"] = {rename.get(str(k), k): v for k, v in snapshot["column_types"].items()}
|
||||
return view
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 工具边界:查询结果交给核查 agent 之前先遮
|
||||
# --------------------------------------------------------------------------
|
||||
@@ -1137,33 +1489,77 @@ class QueryMask:
|
||||
secrets: _Secrets
|
||||
|
||||
|
||||
def mask_query_result(payload: dict[str, Any], masked: Iterable[str], kind: str | None) -> QueryMask:
|
||||
def mask_query_result(payload: dict[str, Any], masked: Iterable[str], kind: str | None,
|
||||
known: _Secrets | None = None, *, schema: Any = None) -> QueryMask:
|
||||
"""一份查询结果(tools/datasource 的查询工具交回的样子:columns / rows / column_types / sql)在交给核查 agent、落进
|
||||
快照之前先遮。masked 是数据源设了遮罩的列,kind 是数据源的类型(SQL 按它的方言解析)。
|
||||
快照之前先遮。masked 是数据源设了遮罩的列,kind 是数据源的类型(SQL 按它的方言解析)。known 是此前已经认出的遮罩值:
|
||||
这个 agent 前面的查询遮掉的,加上调用方交进来的(审批质询、反方审稿收的报告快照里的值)。schema 是数据源的表结构
|
||||
(schema_cache,和随快照冻结的是同一份),认分档标签时用(_label_columns 的 tables)。
|
||||
|
||||
- 整列遮:列名直接对上遮罩列的,加上按 SQL 认出值取自遮罩列的(_derived_hidden:别名、函数、子查询、WITH、
|
||||
SELECT *;COUNT、只在条件里用到、分档当分组键的不算);SQL 解析不了、又写着遮罩列的,整份结果的列都遮。这几列的
|
||||
格子换成「已遮罩」,类型记成文本
|
||||
- 别的列:整格就是这份结果里某个遮罩值的换掉,文字格里夹着的那一截换掉(换了写法的号码也认,_Secrets.loose_spans);
|
||||
门槛同按值遮(SECRET_DIGITS、SECRET_MIN),只装着分档标签的列里的标签不算遮罩值
|
||||
- 列名不换(模型自己写的,换了它引用的列名就对不上);结果里一列都没取自遮罩列的原样交回:数值的汇总、只数个数的
|
||||
查询不受影响
|
||||
SELECT *、表值函数;COUNT、只在条件里用到、分档当分组键的不算);SQL 解析不了、又写着遮罩列的,整份结果的列都遮。
|
||||
这几列的格子换成「已遮罩」,类型记成文本
|
||||
- 别的列按值换:整格就是遮罩值的换掉,文字格里夹着的那一截换掉(换了写法的号码也认,_Secrets.loose_spans)。遮罩值
|
||||
是这份结果里的加上 known:先查号码、再查写着号码的备注、没设遮罩的联系方式表(别的库里的也一样)时,后一份同样
|
||||
遮好。门槛同按值遮(SECRET_DIGITS、SECRET_MIN);只装着分档标签的列里的标签不算遮罩值,这份结果自己写的标签也不
|
||||
按 known 换(前面的查询认不准、当遮罩值收了的同名标签,不能把这一份的档位列换掉)
|
||||
- 列名不换(模型自己写的,换了它引用的列名就对不上);结果里一列都没取自遮罩列、也没有可按值换的原样交回:数值的
|
||||
汇总、只数个数的查询不受影响
|
||||
|
||||
分析本身出了意外,整份结果的列都遮:这一步挡在模型前面,分析的缺陷不能把原值放过去(宁可多遮)。"""
|
||||
交回的 secrets 只有这份结果里的遮罩值(known 调用方自己有)。分析本身出了意外,整份结果的列都遮:这一步挡在模型
|
||||
前面,分析的缺陷不能把原值放过去(宁可多遮)。"""
|
||||
base = {str(c).strip().lower() for c in masked if str(c).strip()}
|
||||
columns = [str(c) for c in payload.get("columns") or []]
|
||||
if not base or not columns or not _query_shaped(payload):
|
||||
given = known is not None and bool(known.strong)
|
||||
if not (base or given) or not columns or not _query_shaped(payload):
|
||||
return QueryMask(payload, [], _Secrets())
|
||||
try:
|
||||
hidden, _ = _derived_hidden(payload, base, kind)
|
||||
if not any(c.lower() in hidden for c in columns):
|
||||
return QueryMask(payload, [], _Secrets())
|
||||
secrets = _Secrets()
|
||||
secrets.add(payload, hidden, labels=_snapshot_labels(payload, kind), base=base)
|
||||
secrets.prepare()
|
||||
view, _ = _masked_view(payload, hidden, secrets, embedded=True, rename=False)
|
||||
hidden = _derived_hidden(payload, base, kind)[0] if base else set()
|
||||
own = _Secrets()
|
||||
labels = _snapshot_labels(payload, kind, _schema_tables(schema))
|
||||
if any(c.lower() in hidden for c in columns):
|
||||
own.add(payload, hidden, labels=labels, base=base)
|
||||
pool = own
|
||||
if given:
|
||||
pool = own.copy()
|
||||
pool.merge(known)
|
||||
# 这份结果自己写的标签(CASE 写死的档名)不当遮罩值,哪怕前面的查询把同名的字收了
|
||||
if written := [text for name, texts in labels.items() if name not in base for text in texts]:
|
||||
pool.protect(written)
|
||||
if not pool.strong and not any(c.lower() in hidden for c in columns):
|
||||
return QueryMask(payload, [], own)
|
||||
pool.prepare()
|
||||
view, _ = _masked_view(payload, hidden, pool, embedded=True, rename=False)
|
||||
except Exception: # noqa: BLE001 - 认不准就整份遮掉,原值不交给模型
|
||||
logger.warning("核查查询的遮罩分析出错,整份结果按遮罩处理", exc_info=True)
|
||||
hidden, secrets = {c.lower() for c in columns}, _Secrets()
|
||||
view, _ = _masked_view(payload, hidden, secrets, rename=False)
|
||||
return QueryMask(view, [c for c in columns if c.lower() in hidden], secrets)
|
||||
hidden, own = {c.lower() for c in columns}, _Secrets()
|
||||
view, _ = _masked_view(payload, hidden, own, rename=False)
|
||||
return QueryMask(view, [c for c in columns if c.lower() in hidden], own)
|
||||
|
||||
|
||||
#: 查询报错里引号括起来的一段(Postgres、DuckDB、MySQL 把转换失败的值写在引号里:invalid input syntax for type
|
||||
#: bigint: "139-0000-0001")。引号前后紧挨着字母数字的不算(can't、isn't 里的撇号)
|
||||
_QUOTED_ERROR = re.compile(r"(?<![0-9A-Za-z_])(\"[^\"\n]*\"|'[^'\n]*')(?![0-9A-Za-z_])")
|
||||
#: 报错里不带引号的长数(DuckDB:Type INT64 with value 13900000001 can't be cast)
|
||||
_ERROR_NUMBER = re.compile(r"(?<![0-9A-Za-z_.])\d{%d,}(?![0-9A-Za-z_])" % SECRET_DIGITS)
|
||||
|
||||
|
||||
def mask_error_text(message: str, sql: str, known: _Secrets | None = None) -> str:
|
||||
"""查询失败时交给核查 agent、写进 tool.end 的报错原话。驱动的原话是模型改写 SQL 的线索,可数据库会在报错里带上
|
||||
数据里的值(类型转换失败、越界),数据源设了遮罩时交出去之前先换:引号里的一段、不带引号的长数,SQL 里没写着的
|
||||
换成「已遮罩」——SQL 里写着的是模型自己写的表名、列名、字面量,照留,它要靠这些改写;此前查询遮掉的值(known)
|
||||
按值换。不认得是不是数据的也换(多遮):报错只是线索,说不全也不耽误改写。"""
|
||||
if not message:
|
||||
return message
|
||||
word = _masked_word()
|
||||
lowered = sql.lower()
|
||||
|
||||
def quoted(m: re.Match[str]) -> str:
|
||||
inner = m.group(1)[1:-1]
|
||||
if not inner.strip() or inner.lower() in lowered:
|
||||
return m.group()
|
||||
return m.group(1)[0] + word + m.group(1)[-1]
|
||||
|
||||
out = _QUOTED_ERROR.sub(quoted, message)
|
||||
out = _ERROR_NUMBER.sub(lambda m: m.group() if m.group() in sql else word, out)
|
||||
return known.scrub(out, loose=True) if known is not None and known.strong else out
|
||||
|
||||
@@ -9,7 +9,11 @@
|
||||
取自的查询算,合并查询按它合并的那几个源算);这句没挂依据,就在这份报告引用的数据源里查。任务里给这句结论
|
||||
和它挂的证据摘录(judge.prepare,数据源遮罩的列照旧不给;摘录里原报告的编号都标成「原报告 Qn」,和 agent 自己
|
||||
的 Q1 分开),要它先想清楚「如果这句不成立,数据里会看到什么」,写只读查询去找,最后交结论(经受住 / 被削弱 /
|
||||
被推翻 / 无法判断)和理由。理由里的数字只能引用它自己查询的格子,裸数字由 compose_doc 标出来
|
||||
被推翻 / 无法判断)和理由。理由里的数字只能引用它自己查询的格子,裸数字由 compose_doc 标出来。
|
||||
遮罩:agent 的查询结果在工具边界就遮好了;这份报告引用的查询快照里的遮罩值(按 SQL 认出改了名字的遮罩列,同审批
|
||||
质询)反查之前就交给查询工具,agent 查到写着同一个号码的别的表时也看不到。摘录按列名遮,改了名字的遮罩列原值
|
||||
agent 看得到,会照着写进理由:理由正文和理由文档按值换成「已遮罩」(换了写法的号码也认),这句结论原文里写着的
|
||||
不换(_masked_rationale)
|
||||
3. 出处:一条查询都没执行成功的,判什么都记为无法判断;判「被推翻」的,理由里还得引用到自己的查询结果——它要让
|
||||
正式出具降档,审稿记录里得查得到凭什么。不采信时系统写明原因,反方原本的结论和理由照录在后面(纯文字:解析
|
||||
不了的引用写成「(引用无效)」,裸数字注明「(无出处)」)。这句该查的数据源都不存在或已停用、核查模型本身用
|
||||
@@ -44,8 +48,10 @@
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
from collections.abc import Callable, Collection, Iterable
|
||||
from dataclasses import replace
|
||||
from typing import Any
|
||||
|
||||
from langgraph.func import task
|
||||
@@ -372,6 +378,52 @@ def _claim_of(cand: judging.Candidate, outcome: VerifyOutcome) -> tuple[dict[str
|
||||
return _claim(cand, verdict, rationale, doc=outcome.rationale, queries=outcome.queries, usage=outcome.usage), None
|
||||
|
||||
|
||||
def _spared(seed: Any, claim: str) -> Any:
|
||||
"""报告快照里的遮罩值(seed)去掉这句结论原文里写着的,交给这一句的反查。seed 本身不动(下一句另算)。"""
|
||||
known = seed.copy()
|
||||
known.protect([claim])
|
||||
return known
|
||||
|
||||
|
||||
def _masked_rationale(outcome: VerifyOutcome, secrets: Any, *, given: Iterable[str | None]) -> VerifyOutcome:
|
||||
"""反方的理由里的遮罩值换成「已遮罩」,换了写法的号码也认(199-1474-2785、+86 …)。值是这份报告引用的查询快照里的
|
||||
(secrets,反查之前就交给了查询工具),加上 agent 查询时在工具边界遮掉的(outcome.masked_values);被审的那句结论
|
||||
原文里写着的(given)已经摆在读者眼前,不换。反方 agent 的查询结果在工具边界就遮好了,可证据摘录按列名遮,改了名字
|
||||
的遮罩列(buyer_phone AS 手机号)的原值它看得到,会写进理由。
|
||||
换的办法同审批质询的回答(_scrubbed_source):原文里的值先换掉,再照原来的目录重新组装理由文档;引用到的格子按
|
||||
遮好的快照渲染(文字格里夹着的遮罩值换掉那一截)。理由里认不出遮罩值的原样交回。"""
|
||||
from app.engine.evidence import compose_doc
|
||||
from app.engine.masking import _masked_view, _scrubbed_source
|
||||
|
||||
doc, result = outcome.rationale, outcome.result
|
||||
pool = secrets.copy()
|
||||
pool.merge(outcome.masked_values)
|
||||
pool.protect(given)
|
||||
if result is None or not isinstance(doc, dict) or not pool.strong:
|
||||
return outcome
|
||||
pool.prepare()
|
||||
source = str(doc.get("source") or "")
|
||||
if not (pool.spans(str(doc.get("markdown") or ""), prose=True) or pool.spans(source, prose=True)):
|
||||
return outcome
|
||||
views: dict[str, Any] = {}
|
||||
fetch = _fetcher()
|
||||
for query in outcome.queries:
|
||||
snapshot = fetch(query.get("artifact"))
|
||||
if isinstance(snapshot, dict) and isinstance(snapshot.get("rows"), list):
|
||||
views[str(query["artifact"])] = _masked_view(snapshot, set(), pool, embedded=True)[0]
|
||||
def load(artifact: str) -> Any:
|
||||
return views[artifact] if artifact in views else artifact_store.load(artifact)
|
||||
|
||||
try:
|
||||
again = compose_doc(_scrubbed_source(source, pool), doc.get("catalog") or {},
|
||||
node_id=str(doc.get("node_id") or ""), run_id=doc.get("run_id"), loader=load)
|
||||
except Exception: # noqa: BLE001 - 重新组装出了意外,值也不能留着
|
||||
# 正文按值换掉,理由文档不要了(界面照正文显示;判「被推翻」的因此查不到出处,按无法判断记)
|
||||
said = pool.scrub(str(doc.get("markdown") or ""), prose=True)
|
||||
return replace(outcome, rationale=None, result={**result, RATIONALE: said})
|
||||
return replace(outcome, rationale=again, result={**result, RATIONALE: again["markdown"]})
|
||||
|
||||
|
||||
def _cites(doc: dict[str, Any] | None) -> bool:
|
||||
"""理由文档里有没有一处引用解析到了 agent 自己的查询结果(compose_doc 只把解析成功的引用记进 unit.cites)。"""
|
||||
return isinstance(doc, dict) and any(unit.get("cites") for _, unit in iter_units(doc))
|
||||
@@ -566,6 +618,9 @@ class _Review:
|
||||
# 这句没挂依据时在整份报告引用的数据源里查:写作者看得到的就是这几个库
|
||||
everywhere = claim_sources(self.catalog, self.catalog, self.fetch)
|
||||
usable = await _usable_sources(everywhere)
|
||||
# 报告引用的查询快照里遮罩列的值:反查之前交给查询工具(agent 查到写着同一个号码的别的表时看不到),反查之后
|
||||
# 据此换掉理由里的遮罩值(_masked_rationale)
|
||||
seed = await self._seed()
|
||||
claims: list[dict[str, Any]] = []
|
||||
usage = _zero_usage()
|
||||
model: str | None = None
|
||||
@@ -629,11 +684,14 @@ class _Review:
|
||||
if allot.tightened:
|
||||
self.run_seconds = allot.seconds
|
||||
# 收紧过的:核查 agent 到点自己发的 verify_limit 改说运行时长上限将至(deadline.relabel),不写收紧后的秒数
|
||||
# 这句结论原文里写着的值已经摆在读者眼前,不当遮罩值。上万个值过一遍要几十毫秒,放进线程
|
||||
known = await asyncio.to_thread(_spared, seed, cand.text)
|
||||
outcome = await run_verify_agent(
|
||||
spec=self.spec, system=CHALLENGE_SYSTEM, task=task_text, sources=live,
|
||||
output_schema=CHALLENGE_SCHEMA, max_queries=self.max_queries, timeout_s=allot.timeout_s,
|
||||
daily_max_usd=self.daily_max_usd, emit=deadline.relabel(ctx.emit, allot), run_id=ctx.run.run_id,
|
||||
node_id=ctx.node.id, data_versions=run_versions(ctx.run))
|
||||
node_id=ctx.node.id, data_versions=run_versions(ctx.run), masked_values=known)
|
||||
outcome = await asyncio.to_thread(_masked_rationale, outcome, known, given=[cand.text])
|
||||
if outcome.error and not outcome.model:
|
||||
broken = outcome.error # 连模型都没建成(没配好、接入出错):换一句也一样
|
||||
model = model or outcome.model
|
||||
@@ -709,6 +767,14 @@ class _Review:
|
||||
return {"artifact": artifact, "counts": counts, "usage": usage, "unreviewed": missed,
|
||||
"claims": [{k: c[k] for k in ("unit", "text", "verdict")} for c in claims]}
|
||||
|
||||
async def _seed(self) -> Any:
|
||||
"""这份报告目录里的查询快照里遮罩列的值(engine/masking 的 _Secrets)。收法同审批质询收报告快照
|
||||
(inquiry.snapshot_secrets:按 SQL 认出改了名字的遮罩列,查询当时记下的遮罩加上数据源现在设的)。"""
|
||||
from app.engine.inquiry import snapshot_secrets # inquiry 导入了这个模块,用到时再导入
|
||||
|
||||
return await snapshot_secrets(self.fetch(entry.get("artifact")) for entry in self.catalog.values()
|
||||
if isinstance(entry, dict) and entry.get("kind") == "query")
|
||||
|
||||
async def _store(self, record: dict[str, Any]) -> str | None:
|
||||
try:
|
||||
return await artifact_store.put_json(record, kind="challenge", run_id=self.ctx.run.run_id,
|
||||
|
||||
@@ -13,7 +13,9 @@
|
||||
连取值也不给;遮罩列不给。计算式只有一项来自查询、或者写了口径说明时不给列名;几项都来自查询又没写口径说明的,列名
|
||||
里看不出日期、口径的取舍时才给,好让它分得清哪一项是哪一项。
|
||||
|
||||
**遮罩列上的输入不复算。** cell() 取自遮罩列的输入不能单独复算;别的输入(计算式里写成当时代进去的值)里按位置取自
|
||||
**遮罩列上的输入不复算。** cell() 取自遮罩列的输入不能单独复算,改了名字、套了函数或汇总的遮罩列(SUM(amount) AS
|
||||
gmv,amount 设了遮罩)按 SQL 认出来也一样(engine/masking.derived_columns):盲算的查询在工具边界按同一套认法遮掉,
|
||||
派出去也取不到数,不白花一次盲算。别的输入(计算式里写成当时代进去的值)里按位置取自
|
||||
查询结果的一格(vars.res.rows[0][2]),顺着取值链认出它取自哪份快照的哪一列(recompute.input_origin,证据面板、推演、
|
||||
可验证报告同一个认法),按需算那份快照要遮的列(和证据面板同一套,见 _Sources.hidden)——它不一定是 cell() 用到的
|
||||
那几份。取自遮罩列、遮没遮核对不了的,整个指标按无法复算记:遮罩值不能当字面量写进交给盲算的计算式。伸进查询结果的行、
|
||||
@@ -75,6 +77,7 @@ from app.engine.expressions import ExpressionError, _cell_call, _chain_root, cel
|
||||
parse_expression
|
||||
from app.engine.replay import _in_graph, ask, once
|
||||
from app.engine.labels import DUAL_CHECK_FIELD_LABEL, DUAL_CHECK_ON_REJECT_LABEL
|
||||
from app.engine.masking import derived_columns
|
||||
from app.engine.schema import DUAL_CHECK_ON_REJECT, NodeType, dual_check_card
|
||||
from app.engine.state import GraphState
|
||||
|
||||
@@ -613,9 +616,15 @@ async def _query_input(item: dict[str, Any], sources: _Sources, aliases: dict[st
|
||||
value = cell_value(item.get("value"))
|
||||
if not _number(value):
|
||||
return {}, f"输入 {item['path']} 不是数值"
|
||||
if column.lower() in await sources.hidden(artifact):
|
||||
hidden = await sources.hidden(artifact)
|
||||
if column.lower() in hidden:
|
||||
return {}, f"输入 {item['path']} 取自设置了遮罩的列,不能用于复算"
|
||||
found = await sources.source(source)
|
||||
derived = derived_columns(snap, set(hidden), found.kind if found is not None else None) if hidden else None
|
||||
if derived and column.lower() in derived:
|
||||
# 改了名字、套了函数或汇总的遮罩列(SUM(amount) AS gmv):盲算的查询在工具边界按同一套认法遮掉(engine/masking),
|
||||
# 交回的格子注定取不到,不白派一次盲算。SQL 解析不了的认不准,照常派
|
||||
return {}, f"输入 {item['path']} 由设置了遮罩的列算出(经改名、函数或汇总),盲算时会被遮罩,不能用于复算"
|
||||
if found is None or not found.enabled:
|
||||
return {}, f"数据源「{source}」不存在或已停用"
|
||||
return {"path": item["path"], "key": f"{artifact}|{row}|{column}", "value": value,
|
||||
|
||||
@@ -7,11 +7,12 @@
|
||||
(tools/datasource.py)。这一路没有审批关卡,不能指望人在半路拦下一条 DELETE。
|
||||
|
||||
**先遮再交。** 建查询工具时还打开遮罩开关(tools/datasource 的 mask_results):数据源设了遮罩时,取自遮罩列的格子
|
||||
(按 SQL 认,起别名、套函数、经子查询或 WITH 转一道、SELECT * 都算)在交给模型、落进查询快照之前就换成「已遮罩」,
|
||||
同一份结果里别的文字格夹着的遮罩值也换掉(engine/masking.py 的 mask_query_result)。模型看不到遮罩值,就不会把它
|
||||
写进回答、理由、SQL;以前是事后按值认,每一轮复核都找到新的漏法。快照里的 masked_at_query 记下查询时就遮了的列,
|
||||
格子引用指到这几列的不采信。遮掉的值只在内存里交回调用方(VerifyOutcome.masked_values,审批质询按值遮的第二道防线
|
||||
用),不进快照、不交给模型、不进 as_dict。
|
||||
(按 SQL 认,起别名、套函数、经子查询或 WITH 转一道、SELECT *、表值函数拆开的都算)在交给模型、落进查询快照之前就
|
||||
换成「已遮罩」,别的格子按值换:这份结果里的遮罩值、前面的查询遮掉的、调用方交进来的(masked_values:审批质询、
|
||||
反方审稿收的报告快照里的值),文字格里夹着的换那一截(engine/masking.py 的 mask_query_result);查询失败时报错原话
|
||||
里的值也换。模型看不到遮罩值,就不会把它写进回答、理由、SQL;以前是事后按值认,每一轮复核都找到新的漏法。快照里的
|
||||
masked_at_query 记下查询时就遮了的列,格子引用指到这几列、指到按值换掉的格子的都不采信。遮掉的值只在内存里交回
|
||||
调用方(VerifyOutcome.masked_values,审批质询、反方审稿按值遮的第二道防线用),不进快照、不交给模型、不进 as_dict。
|
||||
|
||||
**数从快照取。** 每次成功的查询是一份查询快照(工具自己落的 put_json),按这个 agent 自己的顺序编 Q1、Q2…。
|
||||
最后交结果时,调用方结构里的数值字段一律改成格子引用 {alias, row, column},由代码按引用从快照里取值:模型自己
|
||||
@@ -66,7 +67,7 @@ from app.engine import spend
|
||||
from app.engine.evidence import build_catalog, cell_eid, compose_doc, query_entry_fields, schema_ledger_entry
|
||||
from app.engine.expressions import CellError, column_kind, locate_cell, numeric_text
|
||||
from app.engine.judge import FALLBACK_MAX_TOKENS, MAX_TOKENS, _json_in, _tokens, _usd
|
||||
from app.engine.masking import _Secrets
|
||||
from app.engine.masking import _masked_word, _Secrets
|
||||
from app.engine.nodes.llm import _DIGEST_BUDGET, _digest, _split_structured, _typed, explain_model_error
|
||||
from app.engine.state import message_text
|
||||
from app.engine.toolcalls import ToolTimeout, limit_fields, limit_of, run_bounded
|
||||
@@ -159,8 +160,8 @@ class VerifyOutcome:
|
||||
output_tokens, total_tokens, cost_usd, calls, unpriced_calls}。limit:触顶原因(LIMITS 之一),没触顶为 None。
|
||||
answered:模型是不是自己停下来的(没被上限掐断,也没因为输出到了上限被截断)。被截断的一律没有结果,error 是
|
||||
TRUNCATED_ERROR。
|
||||
masked_values:查询时在工具边界遮掉的值(engine/masking 的 _Secrets)。只在内存里交给调用方按值遮(审批质询的回答),
|
||||
不进 as_dict、不往外交。
|
||||
masked_values:查询时在工具边界遮掉的值(engine/masking 的 _Secrets),连同调用方交进来的。只在内存里交给调用方按值遮
|
||||
(审批质询的回答、反方审稿的理由),不进 as_dict、不往外交。
|
||||
"""
|
||||
|
||||
result: dict[str, Any] | None
|
||||
@@ -428,6 +429,12 @@ class _Cells:
|
||||
"row": row, "column": name, "artifact": query["artifact"],
|
||||
"reason": f"{alias} 的「{name}」取自设置了遮罩的列,查询时已遮罩,不采信"})
|
||||
return None
|
||||
if raw == _masked_word() and isinstance(content, dict) and "masked_at_query" in content:
|
||||
# 这一格查询时按值换掉了(和别的查询、报告里的遮罩值相同):同样照实说
|
||||
self.cells.append({"path": path, "status": "unresolved", "ref": f"{alias}.r{row}.{name}", "alias": alias,
|
||||
"row": row, "column": name, "artifact": query["artifact"],
|
||||
"reason": f"{alias} 第 {row} 行「{name}」与设置了遮罩的值相同,查询时已遮罩,不采信"})
|
||||
return None
|
||||
truth = _typed(raw, {"type": sorted(types)}, column_kind(content, name))
|
||||
cited = {"ref": f"{alias}.r{row}.{name}", "alias": alias, "row": row, "column": name,
|
||||
"artifact": query["artifact"], "eid": cell_eid(query["artifact"], row, name), "value": truth}
|
||||
@@ -663,7 +670,8 @@ class _State:
|
||||
queries: list[dict[str, Any]] = field(default_factory=list)
|
||||
ledger: list[dict[str, Any]] = field(default_factory=list)
|
||||
model: str | None = None
|
||||
#: 查询工具在工具边界遮掉的值(经 ToolContext.extra 并进来,只在内存里)
|
||||
#: 查询工具在工具边界遮掉的值(经 ToolContext.extra 并进来,只在内存里),起头是调用方交进来的。后面的查询按它们
|
||||
#: 按值遮
|
||||
seen: _Secrets = field(default_factory=_Secrets)
|
||||
|
||||
def outcome(self, **fields: Any) -> VerifyOutcome:
|
||||
@@ -695,6 +703,7 @@ async def run_verify_agent(
|
||||
run_id: str = "",
|
||||
node_id: str = "",
|
||||
data_versions: dict[str, Any] | Versions | None = Versions.UNSET,
|
||||
masked_values: Any = None,
|
||||
) -> VerifyOutcome:
|
||||
"""让一个只读 agent 查库,按 output_schema 交回有出处的结果。永远不抛。
|
||||
|
||||
@@ -709,9 +718,15 @@ async def run_verify_agent(
|
||||
tool.end(role: verify,查询带 query_artifact);不在节点里(质询接口)就不传,什么都不发。
|
||||
- run_id / node_id:查询快照、表结构快照记在谁名下。data_versions:上传表格用哪一版(运行固定的那份
|
||||
data_versions;不在运行里传 Versions.CURRENT;缺省 UNSET 时上传源一律报错,不拿当前版本顶上)。
|
||||
- masked_values:调用方已经认出的遮罩值(engine/masking 的 _Secrets,审批质询、反方审稿收的报告快照里的值)。
|
||||
查询工具从第一条查询起就按它们按值遮:agent 查到的备注、别的表里写着同一个号码时,模型看不到。随查询时遮掉的值
|
||||
一起交回(VerifyOutcome.masked_values)。
|
||||
"""
|
||||
state = _State(started=time.monotonic())
|
||||
try:
|
||||
if masked_values is not None:
|
||||
# 上万个值并一遍要几十毫秒,放进线程,别挡住同一进程里别的运行
|
||||
await asyncio.to_thread(state.seen.merge, masked_values)
|
||||
return await _verify(state, spec=spec, system=system, task=task, sources=sources,
|
||||
output_schema=output_schema, max_queries=max_queries, timeout_s=timeout_s,
|
||||
max_cost_usd=max_cost_usd, daily_max_usd=daily_max_usd, max_steps=max_steps, emit=emit,
|
||||
|
||||
@@ -468,6 +468,8 @@ def _make_query_tool(source: Any, ctx: ToolContext, *, fixed: bool = False,
|
||||
缓存键相同,会把这个源平时用的可写连接也换成只读的。
|
||||
mask_results:数据源设了遮罩时,取自遮罩列的格子(按 SQL 认,改了名字的也算)在交给模型、落进快照之前就换成
|
||||
「已遮罩」,见 engine/masking.py 的 mask_query_result;这一次遮掉的值并进 ctx.extra[MASKED_VALUES_KEY](有的话)。
|
||||
那里已经有的值(这个 agent 前面的查询遮掉的、调用方交进来的)这一次也按值换,数据源没设遮罩也一样:先查号码、
|
||||
再查写着号码的备注或别的库,后一份同样遮好。查询失败时报错原话里的值也换(masking.mask_error_text)。
|
||||
"""
|
||||
# 数据源在 options.query_timeout_s 里配了就用它,否则取缺省。数据库按它停下语句,
|
||||
# 引擎按它告诉界面上限是多少(metadata["timeout_s"])
|
||||
@@ -495,6 +497,10 @@ def _make_query_tool(source: Any, ctx: ToolContext, *, fixed: bool = False,
|
||||
if is_write(statement, dialect=(source.kind or "").lower()):
|
||||
return f"SQL 被拒绝:{FORCED_READONLY}"
|
||||
limits = QueryLimits(timeout_seconds=seconds, **({"max_rows": int(limit)} if limit else {}))
|
||||
mask = masked_columns(source.options)
|
||||
# 此前认出的遮罩值:交给线程的是一份拷贝,线程里读的时候这边不会再往里并
|
||||
sink = ctx.extra.get(MASKED_VALUES_KEY) if mask_results else None
|
||||
known = sink.copy() if sink is not None and sink.strong else None
|
||||
try:
|
||||
result = await run_query(source, sql, limits=limits)
|
||||
except SqlRejected as e:
|
||||
@@ -511,21 +517,29 @@ def _make_query_tool(source: Any, ctx: ToolContext, *, fixed: bool = False,
|
||||
|
||||
# 驱动的原话留着("no such table: x" 正是模型改写 SQL 要的线索),
|
||||
# 类名和 SQLAlchemy 的包装前缀去掉:这一行也会原样出现在运行面板上
|
||||
return f"查询失败:{first_line(e) or explain(e)[0]}"
|
||||
said = first_line(e) or explain(e)[0]
|
||||
if mask_results and (mask or known is not None):
|
||||
from app.engine.masking import mask_error_text
|
||||
|
||||
# 数据库会把转换失败的值写进报错(invalid input syntax for type bigint: "139-0000-0001")
|
||||
said = mask_error_text(said, sql, known)
|
||||
return f"查询失败:{said}"
|
||||
|
||||
payload = result.to_payload()
|
||||
payload["source"] = source.name
|
||||
mask = masked_columns(source.options)
|
||||
# 查询时就整列遮掉的结果列;没开遮罩开关、数据源没设遮罩时为 None(快照和原来一字不差)
|
||||
# 查询时就整列遮掉的结果列;没开遮罩开关、数据源没设遮罩又没有可按值换的时为 None(快照和原来一字不差)
|
||||
masked_now: list[str] | None = None
|
||||
if mask_results and mask:
|
||||
if mask_results and (mask or known is not None):
|
||||
from app.engine.masking import mask_query_result
|
||||
|
||||
# 按 SQL 认列、逐格比对是纯计算,放进线程(同 _sql_checks)
|
||||
done = await asyncio.to_thread(mask_query_result, payload, mask, source.kind)
|
||||
payload, masked_now = done.payload, done.columns
|
||||
if (sink := ctx.extra.get(MASKED_VALUES_KEY)) is not None:
|
||||
sink.merge(done.secrets)
|
||||
done = await asyncio.to_thread(mask_query_result, payload, mask, source.kind, known,
|
||||
schema=source.schema_cache)
|
||||
if mask or done.payload is not payload:
|
||||
payload, masked_now = done.payload, done.columns
|
||||
if sink is not None:
|
||||
# 上万个值并一遍要几十毫秒(免遮的文字要重新过一遍),也放进线程
|
||||
await asyncio.to_thread(sink.merge, done.secrets)
|
||||
# 对照数据目录检查这条 SQL(一对多关联后重复计算、存量跨期求和……)。结果交给模型(Agent 据此改写),
|
||||
# 也随查询快照存下:证据面板看得到,口径卡据此给指标标「存疑」、出具降档。没查出问题不加这个键,
|
||||
# 快照和以前一字不差
|
||||
@@ -541,15 +555,15 @@ def _make_query_tool(source: Any, ctx: ToolContext, *, fixed: bool = False,
|
||||
|
||||
# 只进快照、不进交给模型的结果的几样(模型看到多余的字段只会误会:以为数据被遮了,
|
||||
# 或者把一串十六进制抄进报告):
|
||||
# - 查询当时数据源设的遮罩列:数据源事后改名、删掉,证据面板仍按当时的遮。查询时就遮了的结果列(改了名字的
|
||||
# 也算)一并记进去,证据面板、质询工件按这个键把它们画成遮罩列;
|
||||
# - masked_at_query:开了遮罩开关时,查询时就整列遮掉的结果列(这几列的格子是「已遮罩」,不是数据源里的值);
|
||||
# - 查询当时数据源设的遮罩列:数据源事后改名、删掉,证据面板仍按当时的遮。只记数据源设的列名——证据面板把
|
||||
# 同源各份快照记的这一项并起来,从表的字段清单里去掉,混进结果列的别名会把同名的真实字段也去掉;
|
||||
# - masked_at_query:开了遮罩开关时,查询时就整列遮掉的结果列(这几列的格子是「已遮罩」,不是数据源里的值,
|
||||
# 改了名字的也在这里)。有这一项的快照,别的格子整格是「已遮罩」的是查询时按值换掉的;
|
||||
# - data_version:上传表格查的是哪个快照,事后据此认得出这个数出自哪一版数据。
|
||||
# 手工源、没设遮罩的,快照和原来一字不差
|
||||
extra: dict[str, Any] = {}
|
||||
if mask:
|
||||
own = {m.lower() for m in mask}
|
||||
extra["mask_columns"] = [*mask, *(c for c in masked_now or [] if c.lower() not in own)]
|
||||
extra["mask_columns"] = mask
|
||||
if masked_now is not None:
|
||||
extra["masked_at_query"] = masked_now
|
||||
if version := getattr(source, "snapshot_id", None):
|
||||
|
||||
@@ -1128,6 +1128,88 @@ async def test_the_challenge_agent_never_sees_masked_values(monkeypatch, tmp_pat
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
async def test_masked_values_written_into_the_rationale_are_masked_in_any_shape(monkeypatch, tmp_path):
|
||||
"""复核第 1 轮:报告取数把遮罩列改了名字选出来(buyer_phone AS 手机号),摘录按列名遮不住,反方 agent 看得到号码,
|
||||
把它换个写法写进理由(139-0000-0001、+86 139 0000 0002)。审稿记录里的理由正文、理由文档都换成「已遮罩」:值从这句
|
||||
挂的报告快照里按 SQL 认出来收(和审批质询收报告快照的办法一样),连同 agent 查询时在工具边界遮掉的值;同时报告
|
||||
快照里的值在反查之前就交给查询工具,agent 查到写着同一个号码的别的表时也看不到。以前理由原样存下、随运行封存。"""
|
||||
name, source_id = await _source(tmp_path, options={"mask_columns": ["buyer_phone"]})
|
||||
try:
|
||||
sql = "SELECT region AS 区域, buyer_phone AS 手机号, amount AS 金额 FROM sales ORDER BY id"
|
||||
said = "来自 139-0000-0001 号码的那笔属实,另有 +86 139 0000 0002 的一笔。"
|
||||
plays = {"华东有一笔": Play(["SELECT COUNT(*) AS 笔数 FROM sales WHERE region = '华东'"], "stands", said)}
|
||||
Models(monkeypatch, name, writer=("华东有一笔 [[v:Q1.r0.金额]] 的订单。[[see:Q1]]",), plays=plays)
|
||||
row = await finish(graph(name, sql=sql), run_class="exploratory")
|
||||
assert row.status == "succeeded", row.error
|
||||
record = artifact_store.load((await output_of(row.id, "chal"))["challenge_artifact"])
|
||||
[claim] = [c for c in record["claims"] if "华东有一笔" in c["text"]]
|
||||
assert claim["verdict"] == "stands"
|
||||
assert claim["rationale"] == "来自 已遮罩 号码的那笔属实,另有 已遮罩 的一笔。"
|
||||
assert claim["rationale_doc"]["markdown"] == claim["rationale"]
|
||||
assert claim["rationale_doc"]["source"] == claim["rationale"], "存下的原文也换掉"
|
||||
shown = json.dumps(record, ensure_ascii=False)
|
||||
assert "0000-0001" not in shown and "0000 0002" not in shown
|
||||
finally:
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
async def test_the_challenge_agent_does_not_see_report_values_in_other_tables(monkeypatch, tmp_path):
|
||||
"""报告取数选了手机号(遮罩列);反方 agent 去查回访记录,备注里写着同一个号码(备注没设遮罩)。报告快照里的遮罩值
|
||||
在反查之前就交给了查询工具:交给模型的备注里那一截已经是「已遮罩」,它的查询快照里也是。以前原样交给模型。"""
|
||||
path = tmp_path / f"{uuid.uuid4().hex[:8]}.db"
|
||||
db = sqlite3.connect(path)
|
||||
db.executescript(
|
||||
"CREATE TABLE sales (id INTEGER PRIMARY KEY, region TEXT, buyer_phone TEXT, amount INTEGER, refunded INTEGER);"
|
||||
"INSERT INTO sales VALUES (1, '华东', '13900000001', 1200, 0), (2, '华南', '13900000003', 900, 0);"
|
||||
"CREATE TABLE callbacks (id INTEGER PRIMARY KEY, region TEXT, note TEXT);"
|
||||
"INSERT INTO callbacks VALUES (1, '华东', '客户 139-0000-0001 来电改址'), (2, '华南', '例行回访');")
|
||||
db.commit()
|
||||
db.close()
|
||||
name = f"shop_{uuid.uuid4().hex[:8]}"
|
||||
async with SessionLocal() as session:
|
||||
row = DataSource(name=name, kind="sqlite", database=str(path), readonly=True, enabled=True,
|
||||
options={"mask_columns": ["buyer_phone"]})
|
||||
session.add(row)
|
||||
await session.flush()
|
||||
row.schema_cache = await introspect.introspect(row)
|
||||
await session.commit()
|
||||
source_id = row.id
|
||||
try:
|
||||
sql = "SELECT region AS 区域, buyer_phone, amount AS 金额 FROM sales ORDER BY id"
|
||||
notes = "SELECT region AS 区域, note AS 备注 FROM callbacks ORDER BY id"
|
||||
plays = {"华东有一笔": Play([notes], "stands", "回访记录里有 [[v:Q1.r0.备注]]。[[see:Q1]]")}
|
||||
models = Models(monkeypatch, name, writer=("华东有一笔 [[v:Q1.r0.金额]] 的订单。[[see:Q1]]",), plays=plays)
|
||||
row = await finish(graph(name, sql=sql), run_class="exploratory")
|
||||
assert row.status == "succeeded", row.error
|
||||
seen = [str(m.content) for _, key, messages in models.calls if key == "华东有一笔"
|
||||
for m in messages if isinstance(m, ToolMessage)]
|
||||
assert any("客户 已遮罩 来电改址" in s for s in seen) and all("0000-0001" not in s for s in seen)
|
||||
record = artifact_store.load((await output_of(row.id, "chal"))["challenge_artifact"])
|
||||
[claim] = [c for c in record["claims"] if "华东有一笔" in c["text"]]
|
||||
assert claim["rationale"] == "回访记录里有 客户 已遮罩 来电改址。"
|
||||
snapshot = artifact_store.load(claim["queries"][0]["artifact"])
|
||||
assert snapshot["rows"] == [["华东", "客户 已遮罩 来电改址"], ["华南", "例行回访"]]
|
||||
finally:
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
def test_values_written_in_the_claim_itself_are_kept_in_the_rationale():
|
||||
"""被审的那句结论原文里写着的号码已经摆在读者眼前:理由里复述它不换,别的号码照换。"""
|
||||
from app.engine.evidence import compose_doc
|
||||
from app.engine.masking import _Secrets
|
||||
from app.engine.verify_agent import VerifyOutcome
|
||||
|
||||
secrets = _Secrets()
|
||||
secrets.add({"columns": ["p"], "rows": [["13900000001"], ["13900000003"]]}, {"p"})
|
||||
doc = compose_doc("号码 13900000003 的客户在华南,139-0000-0001 不在。", {}, node_id="chal", run_id="r1")
|
||||
outcome = VerifyOutcome(result={"verdict": "stands", "rationale": doc["markdown"]}, cells=[], queries=[],
|
||||
rationale=doc, usage={})
|
||||
masked = challenge_mod._masked_rationale(outcome, secrets, given=["号码 13900000003 的客户在华南。"])
|
||||
assert masked.result["rationale"] == "号码 13900000003 的客户在华南,已遮罩 不在。"
|
||||
assert masked.rationale["markdown"] == masked.result["rationale"] and masked.rationale["node_id"] == "chal"
|
||||
assert challenge_mod._masked_rationale(outcome, _Secrets(), given=[]) is outcome, "没有遮罩值时原样"
|
||||
|
||||
|
||||
async def test_a_metric_excerpt_marks_the_report_query_numbers_too(monkeypatch, shop):
|
||||
"""句子挂的是口径卡指标(正式报告的主路径「取数 → 口径卡 → 报告」):摘录正文里「取自查询 Q1」「查询 Q1 的 SQL」
|
||||
也标成原报告的编号,和反查 agent 自己的 Q1 分开;工具按指标的输入取自的查询找源。"""
|
||||
|
||||
@@ -1055,6 +1055,38 @@ async def test_a_renamed_masked_column_in_the_blind_query_is_masked_before_the_b
|
||||
assert {row[0] for row in shown["rows"]} == {"已遮罩"}, "交给盲算模型的结果里也是「已遮罩」"
|
||||
|
||||
|
||||
async def test_a_metric_summed_over_a_masked_column_is_not_blind_computed(monkeypatch, tmp_path):
|
||||
"""复核第 1 轮:金额这一列设了遮罩,指标是 SUM(amount)。核查 agent 的查询结果在工具边界遮好,汇总遮罩列的值也遮
|
||||
(只有一行的汇总就是那一行的原值),盲算交回的格子注定取不到。所以派出盲算之前按 SQL 认出这一列由遮罩列算出,
|
||||
直接按无法复算记、说清原因,不白花一次盲算的钱。以前只按列名认,gmv 认不出,盲算照样派出去,最后才在格子引用上
|
||||
被拒,原因说成「盲算交回的格子引用不可用」。"""
|
||||
name, source_id = await add_source(build(tmp_path / "amount_masked.db"), options={"mask_columns": "amount"})
|
||||
try:
|
||||
blind = Blind(monkeypatch, name, {"销售额": (BLIND_TOTAL, "blind_total")})
|
||||
run = await start(graph(name, RIGHT))
|
||||
evs = await events(run.id)
|
||||
out = dual_output(run, evs)
|
||||
[row] = out["metrics"]
|
||||
assert row["status"] == "unchecked" and blind.runs == 0, "没派盲算"
|
||||
assert row["reason"] == ("输入 cell(nodes.fetch, 0, 'gmv') 由设置了遮罩的列算出(经改名、函数或汇总),"
|
||||
"盲算时会被遮罩,不能用于复算")
|
||||
assert not [e for e in evs if e.type == "tool.end" and e.node_id == "dual"]
|
||||
finally:
|
||||
await drop_source(source_id)
|
||||
|
||||
|
||||
async def test_a_metric_filtered_by_a_masked_column_inside_its_sum_is_still_blind_computed(monkeypatch, masked_shop):
|
||||
"""遮罩列(区域)只在汇总的 CASE 条件里(SUM(CASE WHEN region = '华东' THEN amount ELSE 0 END)):同写在 WHERE 里,
|
||||
值全部来自金额,原 SQL 和盲算的查询都不遮,照常比对。"""
|
||||
name, _ = masked_shop
|
||||
east = (f"SELECT SUM(CASE WHEN region = '华东' THEN amount ELSE 0 END) AS gmv FROM orders "
|
||||
f"WHERE status <> 'refunded' AND paid_date {WEEK}")
|
||||
blind = Blind(monkeypatch, name, {"销售额": (east, "gmv")})
|
||||
run = await start(graph(name, east))
|
||||
out = dual_output(run, await events(run.id))
|
||||
assert blind.runs == 1 and out["metrics"][0]["status"] == "agreed", out["metrics"][0]
|
||||
|
||||
|
||||
async def test_masked_group_columns_are_not_shown_to_the_blind_agent(monkeypatch, masked_shop):
|
||||
name, _ = masked_shop
|
||||
first = {**GMV, "id": "first", "name": "首组销售额", "expression": "cell(nodes.fetch, 0, 'gmv')"}
|
||||
|
||||
@@ -1474,14 +1474,22 @@ CALLBACKS = SALES + ("CREATE TABLE callbacks (id INTEGER PRIMARY KEY, region TEX
|
||||
LEAKED = "回答中含有设了遮罩的值,无法遮住,已不予展示,请换个问法重新提问"
|
||||
|
||||
|
||||
async def test_a_masked_value_inside_a_text_cell_is_cut_out_of_the_artifact(monkeypatch, tmp_path, client):
|
||||
"""备注里写着手机号(备注没设遮罩,手机号设了遮罩、被报告取数选中):agent 引用这一格,回答照旧整份不交;
|
||||
质询工件里这份快照照样记着、点得开,那一格里的号码换成「已遮罩」,别的字、别的格子照原样,不然「不予展示」
|
||||
旁边一点就能看到原值。"""
|
||||
async def test_a_masked_value_inside_a_text_cell_is_cut_out_before_the_agent_sees_it(monkeypatch, tmp_path, client):
|
||||
"""备注里写着手机号(备注没设遮罩,手机号设了遮罩、被报告取数选中):报告快照里的遮罩值在开始回答之前就交给核查
|
||||
agent 的查询工具,agent 查备注时那一截就换成「已遮罩」,模型看不到;它引用这一格,回答照常给出,渲染出来也是遮好的。
|
||||
质询工件、通用工件接口里这份快照同样遮好,别的字、别的格子照原样。以前模型看得到原值,回答整份不交。"""
|
||||
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=CALLBACKS)
|
||||
notes = "SELECT region AS 区域, note AS 备注 FROM callbacks ORDER BY id"
|
||||
plays = {"回访": Play([notes], "华东的回访备注是 [[v:Q1.r0.备注]]。[[see:Q1]]")}
|
||||
Models(monkeypatch, name, writer="金额最高的一笔 [[v:Q1.r0.金额]]。[[see:Q1]]", plays=plays)
|
||||
seen: list[str] = []
|
||||
answering = MockChatModel._decide
|
||||
|
||||
def spy(model, messages):
|
||||
seen.extend(str(m.content) for m in messages if isinstance(m, ToolMessage))
|
||||
return answering(model, messages)
|
||||
|
||||
monkeypatch.setattr(MockChatModel, "_decide", spy)
|
||||
try:
|
||||
report_sql = "SELECT buyer_phone, amount AS 金额 FROM sales ORDER BY amount DESC"
|
||||
paused = await pause(name, graph=fetching(name, report_sql))
|
||||
@@ -1491,20 +1499,114 @@ async def test_a_masked_value_inside_a_text_cell_is_cut_out_of_the_artifact(monk
|
||||
assert r.status_code == 200, r.text
|
||||
assert "1390000000" not in r.text
|
||||
item = r.json()["inquiry"]
|
||||
assert (item["answer"]["ok"], item["answer"]["text"], item["answer"]["doc_artifact"]) == (False, "", None)
|
||||
assert item["answer"]["error"] == LEAKED
|
||||
assert (item["answer"]["ok"], item["answer"]["text"]) == (True, "华东的回访备注是 客户已遮罩来电改址。")
|
||||
[query] = item["queries"]
|
||||
assert query["sql"] == notes
|
||||
[event] = await events(paused.id, "approval.inquiry")
|
||||
assert event.data["inquiry_artifacts"] == [query["artifact"]]
|
||||
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{query['artifact']}")
|
||||
assert got.status_code == 200, got.text
|
||||
assert got.json()["content"]["rows"] == [["华东", "客户已遮罩来电改址"], ["华南", "例行回访"]]
|
||||
assert got.json()["masked"] == [] and "1390000000" not in got.text
|
||||
assert any("客户已遮罩来电改址" in s for s in seen) and all("1390000000" not in s for s in seen), \
|
||||
"交给模型的查询结果里就是遮好的"
|
||||
for url in (f"/api/runs/{paused.id}/inquiries/artifacts/{query['artifact']}",
|
||||
f"/api/artifacts/{query['artifact']}"):
|
||||
got = await client.get(url)
|
||||
assert got.status_code == 200, got.text
|
||||
assert got.json()["content"]["rows"] == [["华东", "客户已遮罩来电改址"], ["华南", "例行回访"]]
|
||||
assert "1390000000" not in got.text
|
||||
assert got.json().get("masked", []) == [], "备注只换了一截,不画成遮罩列"
|
||||
finally:
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
#: 质询 agent 的两条查询:一条选出手机号(查询时遮掉),一条查回访备注(备注没设遮罩,里面写着手机号)
|
||||
PHONES_EAST = "SELECT buyer_phone AS 号码, amount AS 金额 FROM sales WHERE region = '华东' ORDER BY amount DESC"
|
||||
NOTES = "SELECT region AS 区域, note AS 备注 FROM callbacks ORDER BY id"
|
||||
|
||||
|
||||
async def test_a_number_from_the_agents_other_query_is_masked_when_the_number_came_first(monkeypatch, tmp_path, client):
|
||||
"""复核第 1 轮(对基线回退):报告取数没选手机号,号码只出现在 agent 自己的另一次查询里。先查号码、后查备注:号码
|
||||
查询时就遮掉、值并进内存,后面那次查询的备注在交给模型、落库之前就按值换掉,回答照常给出;质询工件、通用工件接口
|
||||
给的都是遮好的。以前事后取质询工件时收不到这个号码,备注原样给出。"""
|
||||
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=CALLBACKS)
|
||||
plays = {"回访": Play([PHONES_EAST, NOTES], "华东的回访备注是 [[v:Q2.r0.备注]]。[[see:Q2]]")}
|
||||
Models(monkeypatch, name, writer="华东销售额 [[v:Q1.r0.销售额]]。[[see:Q1]]", plays=plays)
|
||||
try:
|
||||
paused = await pause(name, graph=fetching(name, QUERY))
|
||||
[approval] = await pending(paused.id)
|
||||
r = await ask(client, paused.id, report="write", unit=None, question="华东的回访说了什么?",
|
||||
approval_id=approval.id)
|
||||
assert r.status_code == 200, r.text
|
||||
item = r.json()["inquiry"]
|
||||
assert (item["answer"]["ok"], item["answer"]["text"]) == (True, "华东的回访备注是 客户已遮罩来电改址。")
|
||||
phones, notes = item["queries"]
|
||||
for url in (f"/api/runs/{paused.id}/inquiries/artifacts/{notes['artifact']}",
|
||||
f"/api/artifacts/{notes['artifact']}"):
|
||||
got = await client.get(url)
|
||||
assert got.status_code == 200, got.text
|
||||
assert got.json()["content"]["rows"] == [["华东", "客户已遮罩来电改址"], ["华南", "例行回访"]]
|
||||
assert "1390000000" not in got.text
|
||||
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{phones['artifact']}")
|
||||
assert got.json()["content"]["rows"] == [["已遮罩", 1200], ["已遮罩", 300]] and got.json()["masked"] == ["号码"]
|
||||
[event] = await events(paused.id, "approval.inquiry")
|
||||
assert "1390000000" not in json.dumps(event.data, ensure_ascii=False)
|
||||
finally:
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
async def test_a_number_from_the_agents_other_query_is_masked_when_the_number_came_last(monkeypatch, tmp_path, client):
|
||||
"""先查备注、后查号码:查备注时还不知道里面写着遮罩值,原样交给模型、原样存下;回答引用这一格,渲染出来才露,整份
|
||||
不交。回答时按内存里的值换掉的位置记进事件(不含值),事后取质询工件照着再换一遍:质询工件里那一截是「已遮罩」,
|
||||
和回答时一样。"""
|
||||
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=CALLBACKS)
|
||||
plays = {"回访": Play([NOTES, PHONES_EAST], "华东的回访备注是 [[v:Q1.r0.备注]]。[[see:Q1]]")}
|
||||
Models(monkeypatch, name, writer="华东销售额 [[v:Q1.r0.销售额]]。[[see:Q1]]", plays=plays)
|
||||
try:
|
||||
paused = await pause(name, graph=fetching(name, QUERY))
|
||||
[approval] = await pending(paused.id)
|
||||
r = await ask(client, paused.id, report="write", unit=None, question="华东的回访说了什么?",
|
||||
approval_id=approval.id)
|
||||
assert r.status_code == 200, r.text
|
||||
assert "1390000000" not in r.text
|
||||
item = r.json()["inquiry"]
|
||||
assert (item["answer"]["ok"], item["answer"]["error"]) == (False, LEAKED)
|
||||
notes, phones = item["queries"]
|
||||
[event] = await events(paused.id, "approval.inquiry")
|
||||
assert "1390000000" not in json.dumps(event.data, ensure_ascii=False)
|
||||
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{notes['artifact']}")
|
||||
assert got.status_code == 200, got.text
|
||||
assert got.json()["content"]["rows"] == [["华东", "客户已遮罩来电改址"], ["华南", "例行回访"]]
|
||||
assert "1390000000" not in got.text
|
||||
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{phones['artifact']}")
|
||||
assert got.json()["content"]["rows"] == [["已遮罩", 1200], ["已遮罩", 300]]
|
||||
finally:
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
async def test_a_number_written_in_another_table_is_masked_whichever_query_came_first(monkeypatch, tmp_path, client):
|
||||
"""另一张表(联系方式)的号码列没设遮罩,存的是同一批号码:先查手机号的,查询时就按值换;后查的,事后取质询工件
|
||||
照回答时记下的位置换。两种先后都整列画成遮罩列。"""
|
||||
phones = "SELECT buyer_phone AS 手机号, amount AS 金额 FROM sales ORDER BY id"
|
||||
visits = "SELECT phone_num AS 号码, visits AS 次数 FROM contacts ORDER BY id"
|
||||
for order in ([phones, visits], [visits, phones]):
|
||||
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=CONTACTS)
|
||||
at = order.index(visits) + 1
|
||||
plays = {"常来": Play(order, f"最常来的客户来过 [[v:Q{at}.r0.次数]] 次。[[see:Q{at}]]")}
|
||||
Models(monkeypatch, name, writer="华东销售额 [[v:Q1.r0.销售额]]。[[see:Q1]]", plays=plays)
|
||||
try:
|
||||
paused = await pause(name, graph=fetching(name, QUERY))
|
||||
[approval] = await pending(paused.id)
|
||||
r = await ask(client, paused.id, report="write", unit=None, question="常来的客户来过几次?",
|
||||
approval_id=approval.id)
|
||||
assert r.status_code == 200, r.text
|
||||
assert r.json()["inquiry"]["answer"]["text"] == "最常来的客户来过 5 次。"
|
||||
query = r.json()["inquiry"]["queries"][at - 1]
|
||||
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{query['artifact']}")
|
||||
assert got.status_code == 200, got.text
|
||||
assert got.json()["content"]["rows"] == [["已遮罩", 5], ["已遮罩", 2]]
|
||||
assert got.json()["masked"] == ["号码"] and "1390000000" not in got.text
|
||||
finally:
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
async def test_sql_check_excerpts_in_the_artifact_are_masked_like_the_sql(monkeypatch, masked_shop, client):
|
||||
"""agent 的查询带着 SQL 检查结果(对照数据目录查出的问题,片段摘自这条 SQL):质询工件里片段照查询清单的 SQL
|
||||
换掉遮罩值,给模型的那句改法照抄了片段,不交出去(同证据面板)。"""
|
||||
@@ -1727,8 +1829,10 @@ REWRITTEN_CALLBACKS = SALES + ("CREATE TABLE callbacks (id INTEGER PRIMARY KEY,
|
||||
"(2, '华南', '例行回访');")
|
||||
|
||||
|
||||
async def test_a_reformatted_number_inside_a_text_cell_stops_the_answer(monkeypatch, tmp_path, client):
|
||||
"""备注里夹着换了写法的号码,agent 引用这一格:渲染出来才露,最后一道关认得出,回答整份不交;质询工件里只换掉那一截。"""
|
||||
async def test_a_reformatted_number_inside_a_text_cell_is_cut_out_before_the_agent_sees_it(monkeypatch, tmp_path,
|
||||
client):
|
||||
"""备注里夹着换了写法的号码(报告取数选了手机号):开始回答之前就认得出,agent 查备注时那一截已经换掉,引用这一格
|
||||
回答照常给出;质询工件里同样只换掉那一截。"""
|
||||
name, source_id = await _source(tmp_path, options={"mask_columns": "buyer_phone"}, script=REWRITTEN_CALLBACKS)
|
||||
notes = "SELECT region AS 区域, note AS 备注 FROM callbacks ORDER BY id"
|
||||
plays = {"回访": Play([notes], "华东的回访备注是 [[v:Q1.r0.备注]]。[[see:Q1]]")}
|
||||
@@ -1740,12 +1844,12 @@ async def test_a_reformatted_number_inside_a_text_cell_stops_the_answer(monkeypa
|
||||
approval_id=approval.id)
|
||||
assert r.status_code == 200, r.text
|
||||
item = r.json()["inquiry"]
|
||||
assert (item["answer"]["ok"], item["answer"]["error"]) == (False, LEAKED)
|
||||
assert (item["answer"]["ok"], item["answer"]["text"]) == (True, "华东的回访备注是 客户 已遮罩 来电改址。")
|
||||
[query] = item["queries"]
|
||||
got = await client.get(f"/api/runs/{paused.id}/inquiries/artifacts/{query['artifact']}")
|
||||
assert got.status_code == 200, got.text
|
||||
assert got.json()["content"]["rows"] == [["华东", "客户 已遮罩 来电改址"], ["华南", "例行回访"]]
|
||||
assert "0000-0001" not in got.text
|
||||
assert "0000-0001" not in got.text and "0000-0001" not in r.text
|
||||
finally:
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
@@ -402,6 +402,66 @@ def test_a_label_ahead_of_a_star_or_alone_behind_one_is_still_a_label_column(sql
|
||||
assert inquiry_mod._label_columns(sql, "sqlite", ["档"]) == {"档": {"0", "1"}}
|
||||
|
||||
|
||||
#: 没写表名的 * 连着数据源里的表和装着标签的派生来源
|
||||
STAR_JOIN = ("SELECT * FROM sales s JOIN (SELECT id AS bid, CASE WHEN amount > 1000 THEN '大单' ELSE '小单' END AS 档 "
|
||||
"FROM sales) b ON s.id = b.bid")
|
||||
STAR_COLUMNS = ["id", "region", "amount", "bid", "档"]
|
||||
|
||||
|
||||
def test_a_star_over_a_table_known_to_lack_the_column_keeps_the_label():
|
||||
"""复核第 1 轮(收紧之后的多遮):没写表名的 * 展开数据源里的表,以前一律算「说不清带不带这一列」,标签照收成
|
||||
遮罩值,回答里的「大单」、SQL 里的 '大单' 都换成「已遮罩」。冻结的表结构里查得到这张表、又没有同名的列时,照旧
|
||||
按派生来源认标签;表里有同名的列、查不到表结构的,照旧不算(宁可多遮)。"""
|
||||
from app.engine import masking
|
||||
|
||||
lacking = {"tables": {"sales": {"columns": [{"name": "id"}, {"name": "region"}, {"name": "amount"}]}}}
|
||||
holding = {"tables": {"sales": {"columns": [{"name": "id"}, {"name": "档"}, {"name": "amount"}]}}}
|
||||
assert masking._label_columns(STAR_JOIN, "sqlite", STAR_COLUMNS, masking._schema_tables(lacking)) == {
|
||||
"档": {"大单", "小单"}}
|
||||
assert masking._label_columns(STAR_JOIN, "sqlite", STAR_COLUMNS, masking._schema_tables(holding)) == {}
|
||||
assert masking._label_columns(STAR_JOIN, "sqlite", STAR_COLUMNS) == {}
|
||||
truncated = {"tables": {"orders": {"columns": [{"name": "id"}]}}, "truncated": True}
|
||||
assert masking._label_columns(STAR_JOIN, "sqlite", STAR_COLUMNS, masking._schema_tables(truncated)) == {}
|
||||
# 写了表名的 s.* 展开的是数据源里的表:同样按表结构认它带不带这一列
|
||||
nested = ("SELECT x.档 FROM (SELECT s.*, b.* FROM sales s JOIN (SELECT id AS bid, CASE WHEN amount > 1000 "
|
||||
"THEN '大单' ELSE '小单' END AS 档 FROM sales) b ON s.id = b.bid) x")
|
||||
assert masking._label_columns(nested, "sqlite", ["档"], masking._schema_tables(lacking)) == {"档": {"大单", "小单"}}
|
||||
assert masking._label_columns(nested, "sqlite", ["档"], masking._schema_tables(holding)) == {}
|
||||
assert masking._label_columns(nested, "sqlite", ["档"]) == {}
|
||||
|
||||
|
||||
async def test_a_snapshot_reads_labels_with_its_frozen_schema():
|
||||
"""快照记着查询当时冻结的表结构(schema_artifact):按它认,* 连着的表没有同名的列时标签不当遮罩值,回答正文、
|
||||
SQL 里的档名照原样;表结构取不到的照旧不算。"""
|
||||
from app.core import artifact_store
|
||||
from app.engine import masking
|
||||
|
||||
lacking = {"source": "shop", "tables": {"sales": {"columns": [{"name": "id"}, {"name": "region"},
|
||||
{"name": "amount"}]}}}
|
||||
schema = await artifact_store.put_json(lacking, kind="schema_snapshot", run_id="r-labels", node_id="n1")
|
||||
snapshot = {"source": "shop", "sql": STAR_JOIN, "columns": STAR_COLUMNS, "schema_artifact": schema,
|
||||
"rows": [[1, "华东", 1200, 1, "大单"], [2, "华南", 300, 2, "小单"]]}
|
||||
assert masking._snapshot_labels(snapshot, "sqlite") == {"档": {"大单", "小单"}}
|
||||
plan = plan_of(snapshot, {"amount"})
|
||||
assert plan.secrets.scrub("这一笔属于大单") == "这一笔属于大单"
|
||||
sql = "SELECT COUNT(*) FROM (SELECT CASE WHEN amount > 1000 THEN '大单' ELSE '小单' END AS 档 FROM sales) WHERE 档 = '大单'"
|
||||
assert inquiry_mod._scrub_sql(sql, plan.secrets, plan.tainted["a1"], "sqlite") == sql
|
||||
assert masking._snapshot_labels({**snapshot, "schema_artifact": "missing"}, "sqlite") == {}
|
||||
|
||||
|
||||
def test_the_tool_boundary_reads_labels_with_the_sources_frozen_schema():
|
||||
"""工具边界按数据源的表结构认标签:* 连着的表没有同名的列,标签不收成遮罩值,同一份结果里别的文字格里的「大单」照常。"""
|
||||
from app.engine import masking
|
||||
|
||||
payload = {"sql": STAR_JOIN, "columns": [*STAR_COLUMNS, "备注"],
|
||||
"rows": [[1, "华东", 1200, 1, "大单", "大单客户"], [2, "华南", 300, 2, "小单", "例行回访"]]}
|
||||
schema = {"tables": {"sales": {"columns": [{"name": "id"}, {"name": "region"}, {"name": "amount"},
|
||||
{"name": "备注"}]}}}
|
||||
done = masking.mask_query_result(payload, ["amount"], "sqlite", schema=schema)
|
||||
assert done.columns == ["amount", "档"] and not {"大单", "小单"} & done.secrets.strong
|
||||
assert [row[-1] for row in done.payload["rows"]] == ["大单客户", "例行回访"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 按值认:门槛、边界
|
||||
# --------------------------------------------------------------------------
|
||||
@@ -1373,3 +1433,56 @@ def test_hundreds_of_band_edges_are_read_within_a_budget():
|
||||
assert "已遮罩" in inquiry_mod._scrub_sql(bins(600), every, {"age"}, "sqlite")
|
||||
elapsed = time.perf_counter() - started
|
||||
assert elapsed < 5, f"用了 {elapsed:.2f} 秒"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 回答时才认得出的值:记下位置,事后取质询工件时照着再换一遍
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_values_known_only_at_answer_time_are_recorded_as_positions_and_replayed():
|
||||
"""agent 先查了备注(那时还不知道备注里的号码是遮罩值,查询时原样交回、原样存下),后来又查出了手机号(查询时
|
||||
遮掉,值只在内存里)。回答时按内存里的值换掉的地方记成位置(不含值);事后取质询工件时内存里的值已经不在,照着
|
||||
位置再换一遍,和回答时看到的一样。"""
|
||||
from app.engine import masking
|
||||
|
||||
note = "客户13900000001来电改址,备用 139-0000-0001"
|
||||
snapshot = {"columns": ["区域", "备注", "号码", "13900000001 的单"],
|
||||
"rows": [["华东", note, "13900000001", 1], ["华南", "例行回访", "13800000000", 2]]}
|
||||
seen = secrets_of({"columns": ["phone"], "rows": [["13900000001"]]}, {"phone"})
|
||||
edits = masking.view_edits(snapshot, set(), seen, inquiry_mod._Secrets())
|
||||
first, second = note.index("13900000001"), note.index("139-0000-0001")
|
||||
assert edits == {"cells": [[0, 2]], "spans": [[0, 1, first, first + 11], [0, 1, second, second + 13]],
|
||||
"names": [[3, 0, 11]], "masked": ["号码"]}
|
||||
assert "1390000000" not in json.dumps(edits, ensure_ascii=False), "只记位置,不记值"
|
||||
replayed = masking.replay_edits(snapshot, [edits])
|
||||
assert replayed["columns"] == ["区域", "备注", "号码", f"{MASKED} 的单"]
|
||||
assert replayed["rows"] == [["华东", f"客户{MASKED}来电改址,备用 {MASKED}", MASKED, 1],
|
||||
["华南", "例行回访", "13800000000", 2]]
|
||||
# 不靠内存里的值也认得出的(报告快照里就有),事后照样认得出,不记
|
||||
assert masking.view_edits(snapshot, set(), seen, seen) is None
|
||||
# 记下的位置对不上(越界、坏形状)的跳过,不出错
|
||||
broken = {"cells": [[9, 9], "x"], "spans": [[0, 1, 50, 60], [0, 9, 0, 1]], "names": [[7, 0, 1]], "masked": [3]}
|
||||
assert masking.replay_edits(snapshot, [broken, None, "x"]) == snapshot
|
||||
|
||||
|
||||
def test_the_plan_records_what_only_the_values_seen_by_the_agent_masked():
|
||||
"""_settle_plan 带上 agent 查询时遮掉的值(seen)时,另记下只靠它们才换掉的位置(replay);不带 seen 的(事后取
|
||||
质询工件)不记。"""
|
||||
notes = {"source": "shop", "sql": "SELECT region AS 区域, note AS 备注 FROM callbacks", "columns": ["区域", "备注"],
|
||||
"rows": [["华东", "客户13900000001来电改址"], ["华南", "例行回访"]]}
|
||||
seen = secrets_of({"columns": ["phone"], "rows": [["13900000001"]]}, {"phone"})
|
||||
plan = inquiry_mod._settle_plan({"a1": notes}, {"a1": ({"buyer_phone"}, [])}, [], {"shop": "sqlite"}, [], [], {},
|
||||
frozenset(), seen)
|
||||
assert plan.replay == {"a1": {"cells": [], "spans": [[0, 1, 2, 13]], "names": [], "masked": []}}
|
||||
later = inquiry_mod._settle_plan({"a1": notes}, {"a1": ({"buyer_phone"}, [])}, [], {"shop": "sqlite"}, [], [], {},
|
||||
frozenset({"a1"}))
|
||||
assert later.replay == {} and later.views["a1"]["rows"][0][1] == "客户13900000001来电改址"
|
||||
|
||||
|
||||
def test_values_the_agent_saw_survive_a_report_that_fills_the_cap():
|
||||
"""报告快照里的遮罩值收满了上限(SECRETS_MAX):agent 查询时遮掉的值先收,照样认得出(以前先收报告的,满了就丢)。"""
|
||||
report = {"columns": ["buyer_phone"], "rows": [[str(13800000000 + i)] for i in range(inquiry_mod.SECRETS_MAX)]}
|
||||
seen = secrets_of({"columns": ["phone"], "rows": [["19914742785"]]}, {"phone"})
|
||||
plan = inquiry_mod._settle_plan({}, {}, [(report, {"buyer_phone"})], {}, [], [], {}, frozenset(), seen)
|
||||
assert plan.secrets.hit("19914742785")
|
||||
|
||||
@@ -137,3 +137,73 @@ def test_many_masked_numbers_are_matched_without_stalling():
|
||||
out = secrets.scrub(text, prose=True)
|
||||
assert time.perf_counter() - began < 2.0
|
||||
assert "199-0000" not in out and out.count(MASKED) == text.count("199-0000")
|
||||
|
||||
|
||||
#: 复核第 1 轮:别的空白、减号作分隔符的写法(模型输出里不换行空格、窄空格并不少见)
|
||||
OTHER_SEPARATORS = [
|
||||
pytest.param("199 1474 2785", id="不换行空格"),
|
||||
pytest.param("199 1474 2785", id="窄不换行空格"),
|
||||
pytest.param("199 1474 2785", id="细空格"),
|
||||
pytest.param("199 1474 2785", id="发丝空格"),
|
||||
pytest.param("199\t1474\t2785", id="制表符"),
|
||||
pytest.param("199−1474−2785", id="减号"),
|
||||
pytest.param("199 1474 2785", id="四个空格"),
|
||||
pytest.param("199 - 1474 - 2785", id="连字符两边带空格"),
|
||||
pytest.param("+86 199 1474 2785", id="+86 加不换行空格"),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("written", OTHER_SEPARATORS)
|
||||
def test_numbers_split_by_other_spaces_and_minus_signs_are_masked_too(written):
|
||||
secrets = secrets_of([PHONE])
|
||||
assert secrets.scrub(f"客户 {written} 来电", prose=True) == f"客户 {MASKED} 来电"
|
||||
assert inquiry_mod._leaked({"markdown": f"来电的是 {written}。", "blocks": []}, secrets)
|
||||
assert secrets.hit(written), "整格就是这个号码的,格子按值认"
|
||||
|
||||
|
||||
def test_a_line_break_still_ends_a_number():
|
||||
"""不跨行:两行各写一截的不拼成一个号码。"""
|
||||
secrets = secrets_of([PHONE])
|
||||
assert secrets.scrub("编号 199\n1474 2785", prose=True) == "编号 199\n1474 2785"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("sql, sealed", [
|
||||
pytest.param("SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note LIKE '%199-1474-2785%'",
|
||||
f"SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note LIKE '%{MASKED}%'", id="LIKE 里的一截"),
|
||||
pytest.param("SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note = '客户 199 1474 2785 来电'",
|
||||
f"SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note = '客户 {MASKED} 来电'", id="字符串里夹着"),
|
||||
pytest.param('SELECT note AS "199-1474-2785 的备注" FROM callbacks',
|
||||
f'SELECT note AS "{MASKED} 的备注" FROM callbacks', id="带引号的列名"),
|
||||
pytest.param("SELECT COUNT(*) AS 笔数 FROM callbacks /* 客户 199-1474-2785 */",
|
||||
f"SELECT COUNT(*) AS 笔数 FROM callbacks /* 客户 {MASKED} */", id="块注释"),
|
||||
pytest.param("SELECT COUNT(*) AS 笔数 FROM callbacks -- 客户 199 1474 2785\n WHERE id = 1",
|
||||
f"SELECT COUNT(*) AS 笔数 FROM callbacks -- 客户 {MASKED}\n WHERE id = 1", id="行注释"),
|
||||
])
|
||||
def test_a_reformatted_number_in_sql_strings_names_and_comments_is_masked(sql, sealed):
|
||||
"""SQL 里字符串、带引号的名字、注释里换了写法的号码也认(以前只有整个字面量就是号码的才换)。"""
|
||||
assert inquiry_mod._scrub_sql(sql, secrets_of([PHONE]), set(), "sqlite") == sealed
|
||||
|
||||
|
||||
def test_a_bare_number_expression_in_sql_is_still_left_alone():
|
||||
"""不带引号的 199-1474-2785 在 SQL 里是一个算式(三个数相减),照旧不按号码认。"""
|
||||
sql = "SELECT 199-1474-2785 AS x FROM callbacks"
|
||||
assert inquiry_mod._scrub_sql(sql, secrets_of([PHONE]), set(), "sqlite") == sql
|
||||
|
||||
|
||||
def test_a_threshold_matching_a_number_swapped_in_a_string_is_not_kept():
|
||||
"""字符串里按换了写法认出、换掉的号码,同一条 SQL 里写成阈值的同一个数也不留(不然阈值把它露了)。"""
|
||||
sql = f"SELECT COUNT(*) AS 笔数 FROM callbacks WHERE note LIKE '%199-1474-2785%' AND phone_no >= {PHONE}"
|
||||
sealed = inquiry_mod._scrub_sql(sql, secrets_of([PHONE]), set(), "sqlite")
|
||||
assert PHONE not in sealed and "1474" not in sealed
|
||||
|
||||
|
||||
def test_a_reformatted_number_in_a_column_name_is_masked_like_the_sql():
|
||||
"""agent 把换了写法的号码写进列名(AS "139-0000-0001 的单"):质询工件的列名、回答里引用这一列的标记照 SQL 里带引号
|
||||
的名字一样换,三处对得上。"""
|
||||
secrets = secrets_of(["13900000001"])
|
||||
snapshot = {"columns": ["139-0000-0001 的单"], "rows": [[3]]}
|
||||
view, _ = _masked_view(snapshot, set(), secrets)
|
||||
assert view["columns"] == [f"{MASKED} 的单"]
|
||||
assert inquiry_mod._scrub_sql('SELECT COUNT(*) AS "139-0000-0001 的单" FROM sales', secrets, set(), "sqlite") == (
|
||||
f'SELECT COUNT(*) AS "{MASKED} 的单" FROM sales')
|
||||
assert inquiry_mod._scrubbed_source("共 [[v:Q1.r0.139-0000-0001 的单]] 单。", secrets) == f"共 [[v:Q1.r0.{MASKED} 的单]] 单。"
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
"""按 SQL 认取自遮罩列的结果列(engine/masking.py 的 _derived_hidden):FROM、JOIN、LATERAL 里的表值函数,汇总里的 CASE。
|
||||
|
||||
核查 agent 的查询在工具边界按这套认法整列遮掉(mask_query_result),审批质询、双盲复算也用它:
|
||||
|
||||
- 表值函数(SQLite 的 json_each / json_tree,Postgres 的 unnest、jsonb_each_text、regexp_split_to_table、LATERAL
|
||||
VALUES,MySQL 的 JSON_TABLE)把遮罩列拆开时,它出的列都算取自遮罩列:遮罩列是 JSON 时,用 json_each 拆开是最自然的
|
||||
写法,以前只追 SELECT 项、子查询和 *,这些写法的原值原样交给模型、落进快照。出列名认得全的(常见的几个函数、写了
|
||||
列名表的)按名字认,认不全的整份遮
|
||||
- 汇总里的 CASE / IIF 只在条件里用到遮罩列的(SUM(CASE WHEN buyer_phone IS NOT NULL THEN amount ELSE 0 END)),同
|
||||
写在 WHERE 里,不算;结果的某一支取的是遮罩列、不汇总的逐行 CASE 照旧算
|
||||
|
||||
数据都是虚构的:手机号、区域、金额。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from app.engine import masking
|
||||
|
||||
#: 设了遮罩的列:手机号,和装着手机号、姓名的 JSON 列
|
||||
MASKED = {"buyer_phone", "profile"}
|
||||
SALES = ["id", "region", "buyer_phone", "amount", "profile"]
|
||||
#: SQLite 的 json_each / json_tree 出的列
|
||||
JSON_EACH = ["key", "value", "type", "atom", "id", "parent", "fullkey", "path"]
|
||||
|
||||
|
||||
def hidden_of(sql: str, columns: list[str], kind: str = "sqlite", masked: set[str] = MASKED) -> set[str]:
|
||||
snapshot = {"source": "shop", "sql": sql, "columns": columns, "rows": []}
|
||||
return masking._derived_hidden(snapshot, set(masked), kind)[0]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind, sql, columns, shown", [
|
||||
pytest.param("sqlite", "SELECT j.key, j.value FROM sales, json_each(sales.profile) AS j", ["key", "value"],
|
||||
["key", "value"], id="json_each 拆遮罩的 JSON 列"),
|
||||
pytest.param("sqlite", "SELECT t.atom AS 值 FROM sales, json_tree(sales.profile) AS t WHERE t.type = 'text'",
|
||||
["值"], ["值"], id="json_tree"),
|
||||
pytest.param("sqlite", "SELECT value FROM sales, json_each(json_array(buyer_phone))", ["value"], ["value"],
|
||||
id="json_each 套 json_array"),
|
||||
pytest.param("sqlite", "SELECT r.value AS 号码 FROM sales s JOIN json_each(s.profile) r ON 1 = 1", ["号码"],
|
||||
["号码"], id="JOIN 里的 json_each"),
|
||||
pytest.param("sqlite", "SELECT value FROM json_each((SELECT json_group_array(buyer_phone) FROM sales))",
|
||||
["value"], ["value"], id="参数是子查询"),
|
||||
pytest.param("sqlite", "SELECT c.value AS 号码 FROM (SELECT json_group_array(buyer_phone) AS arr FROM sales) t, "
|
||||
"json_each(t.arr) c", ["号码"], ["号码"], id="参数取自派生来源里改了名字的遮罩列"),
|
||||
pytest.param("sqlite", "SELECT * FROM sales, json_each(sales.profile)", [*SALES, *JSON_EACH],
|
||||
["buyer_phone", "profile", "key", "value", "atom", "fullkey", "path"], id="* 带出 json_each 的列"),
|
||||
pytest.param("sqlite", "SELECT x FROM (SELECT j.value AS x FROM sales, json_each(sales.profile) j) t", ["x"],
|
||||
["x"], id="子查询里拆开再改名"),
|
||||
pytest.param("postgres", "SELECT u FROM sales, unnest(ARRAY[buyer_phone]) AS u", ["u"], ["u"], id="unnest"),
|
||||
pytest.param("postgres", "SELECT x.p FROM sales s CROSS JOIN LATERAL unnest(ARRAY[s.buyer_phone]) AS x(p)",
|
||||
["p"], ["p"], id="LATERAL unnest 带列名表"),
|
||||
pytest.param("postgres", "SELECT x FROM sales, LATERAL unnest(ARRAY[buyer_phone]) AS x", ["x"], ["x"],
|
||||
id="LATERAL unnest"),
|
||||
pytest.param("postgres", "SELECT u FROM sales CROSS JOIN LATERAL (VALUES (buyer_phone)) AS v(u)", ["u"], ["u"],
|
||||
id="LATERAL VALUES"),
|
||||
pytest.param("postgres", "SELECT d FROM sales, regexp_split_to_table(buyer_phone, '-') AS d", ["d"], ["d"],
|
||||
id="regexp_split_to_table"),
|
||||
pytest.param("postgres", "SELECT e.key, e.value FROM sales, jsonb_each_text(profile) AS e", ["key", "value"],
|
||||
["key", "value"], id="jsonb_each_text"),
|
||||
pytest.param("postgres", "SELECT x FROM sales, unnest(string_to_array(buyer_phone, ',')) AS x", ["x"], ["x"],
|
||||
id="unnest 套 string_to_array"),
|
||||
pytest.param("postgres", "SELECT * FROM sales, unnest(ARRAY[buyer_phone])", [*SALES, "unnest"],
|
||||
["buyer_phone", "profile", "unnest"], id="没起别名的 unnest 按函数名出列"),
|
||||
pytest.param("mysql", "SELECT j.p FROM sales s, JSON_TABLE(JSON_ARRAY(s.buyer_phone), '$[*]' "
|
||||
"COLUMNS (p VARCHAR(20) PATH '$')) j", ["p"], ["p"], id="JSON_TABLE"),
|
||||
])
|
||||
def test_a_table_function_over_a_masked_column_hides_what_it_returns(kind, sql, columns, shown):
|
||||
assert {c.lower() for c in shown} <= hidden_of(sql, columns, kind)
|
||||
|
||||
|
||||
def test_columns_a_table_function_does_not_return_stay_visible():
|
||||
"""拆的是遮罩列,别的列(区域、金额)照常;拆的不是遮罩列的,一列都不遮。"""
|
||||
hidden = hidden_of("SELECT s.region, s.amount, j.value AS 值 FROM sales s, json_each(s.profile) j",
|
||||
["region", "amount", "值"])
|
||||
assert "值" in hidden and not {"region", "amount"} & hidden
|
||||
assert hidden_of("SELECT s.region, j.value AS 标签 FROM sales s, json_each(s.tags) j", ["region", "标签"]) == MASKED
|
||||
|
||||
|
||||
def test_a_table_function_that_cannot_be_read_hides_the_whole_snapshot():
|
||||
"""不认得的函数拆遮罩列、又没写列名表:说不清出哪几列,整份快照的列都遮(同 SQL 解析不了)。"""
|
||||
columns = ["region", "piece"]
|
||||
assert hidden_of("SELECT s.region, t.piece FROM sales s, my_split(s.buyer_phone) t", columns, "postgres") == {
|
||||
*MASKED, "region", "piece"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("sql", [
|
||||
"SELECT SUM(CASE WHEN buyer_phone IS NOT NULL THEN amount ELSE 0 END) AS total FROM sales",
|
||||
"SELECT SUM(CASE buyer_phone WHEN '13900000001' THEN amount END) AS total FROM sales",
|
||||
"SELECT AVG(IIF(buyer_phone LIKE '139%', amount, NULL)) AS total FROM sales",
|
||||
"SELECT MAX(CASE WHEN buyer_phone IS NULL THEN 0 ELSE amount END) AS total FROM sales",
|
||||
"SELECT SUM(CASE WHEN buyer_phone IS NULL THEN 0 ELSE CASE WHEN region = '华东' THEN amount END END) AS total "
|
||||
"FROM sales",
|
||||
])
|
||||
def test_a_masked_column_only_in_the_condition_of_an_aggregated_case_is_like_a_filter(sql):
|
||||
"""值全部来自金额,手机号只决定这一行算不算进来:同写在 WHERE 里,不算取自遮罩列。"""
|
||||
assert hidden_of(sql, ["total"]) == MASKED
|
||||
assert hidden_of(sql.replace("CASE WHEN buyer_phone IS NOT NULL THEN amount ELSE 0 END", "amount")
|
||||
+ " WHERE buyer_phone IS NOT NULL", ["total"]) == MASKED
|
||||
|
||||
|
||||
@pytest.mark.parametrize("sql", [
|
||||
# 结果的某一支取的就是遮罩列
|
||||
"SELECT SUM(CASE WHEN region = '华东' THEN buyer_phone END) AS total FROM sales",
|
||||
"SELECT MAX(IIF(amount > 1000, buyer_phone, NULL)) AS total FROM sales",
|
||||
# 不汇总的逐行 CASE:每一行各自落在哪一支
|
||||
"SELECT CASE WHEN buyer_phone IS NULL THEN 0 ELSE amount END AS total FROM sales",
|
||||
])
|
||||
def test_a_case_returning_the_masked_value_or_not_aggregated_is_still_derived(sql):
|
||||
assert "total" in hidden_of(sql, ["total"])
|
||||
@@ -34,12 +34,18 @@ from tests.test_verify_agent import SPEC, Script, q
|
||||
|
||||
MASKED = "已遮罩"
|
||||
#: 遮罩列里的值(和备注里换了写法的号码):交给模型的、存下的、接口给的都不能出现
|
||||
SECRETS = ("1390000000", "张伟", "李娜", "王芳", "刘洋", "139-0000-0003")
|
||||
SECRETS = ("1390000000", "张伟", "李娜", "王芳", "刘洋", "139-0000-0003", "139 0000 0003")
|
||||
SALES = ("CREATE TABLE sales (id INTEGER PRIMARY KEY, region TEXT, buyer_phone TEXT, buyer_name TEXT, amount INTEGER,"
|
||||
" note TEXT);"
|
||||
"INSERT INTO sales VALUES (1, '华东', '13900000001', '张伟', 1200, '客户13900000001来电改址'),"
|
||||
" (2, '华东', '13900000002', '李娜', 300, '例行回访'), (3, '华南', '13900000003', '王芳', 900, '回访:联系 139-0000-0003'),"
|
||||
" (4, '华北', '13900000004', '刘洋', 400, '');")
|
||||
" (4, '华北', '13900000004', '刘洋', 400, '');"
|
||||
# 联系方式表的号码列、客户资料的 JSON 列都没设遮罩(json_shop 另设了 profile):同一批号码、姓名写在这里
|
||||
"CREATE TABLE contacts (id INTEGER PRIMARY KEY, phone_num TEXT, visits INTEGER);"
|
||||
"INSERT INTO contacts VALUES (1, '13900000001', 5), (2, '139 0000 0003', 2), (3, '13800000000', 1);"
|
||||
"CREATE TABLE customers (id INTEGER PRIMARY KEY, region TEXT, profile TEXT);"
|
||||
"INSERT INTO customers VALUES (1, '华东', '{\"phone\": \"13900000001\", \"name\": \"张伟\"}'),"
|
||||
" (2, '华南', '{\"phone\": \"13900000002\", \"name\": \"李娜\"}');")
|
||||
VALUE_SCHEMA = {"type": "object", "properties": {"value": {"type": "number", "description": "要核对的数"}},
|
||||
"required": ["value"]}
|
||||
|
||||
@@ -76,6 +82,14 @@ async def masked_shop(tmp_path):
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def json_shop(tmp_path):
|
||||
"""同一个库,客户资料(JSON:手机号、姓名)这一列设了遮罩。返回源名。"""
|
||||
name, source_id = await _source(tmp_path, {"mask_columns": "profile"})
|
||||
yield name
|
||||
await engines.invalidate(source_id)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def plain_shop(tmp_path):
|
||||
name, source_id = await _source(tmp_path, None)
|
||||
@@ -137,7 +151,9 @@ async def test_masked_columns_are_masked_before_the_model_sees_them(monkeypatch,
|
||||
assert snapshot.get("column_types", {}).get(column, "text") == "text"
|
||||
assert _cells(snapshot, amount) == [1200, 300, 900, 400]
|
||||
assert snapshot["masked_at_query"] == masked
|
||||
assert {c.lower() for c in snapshot["mask_columns"]} == {"buyer_phone", "buyer_name", *(c.lower() for c in masked)}
|
||||
# mask_columns 照记查询当时数据源设的遮罩列(证据面板按它从表的字段清单里去掉列),改了名字的结果列只记在
|
||||
# masked_at_query:混进去的话,同源别的表里同名的真实字段(name、region)会从字段清单里消失
|
||||
assert snapshot["mask_columns"] == ["buyer_phone", "buyer_name"]
|
||||
|
||||
# 运行详情「工件」页签走的通用接口:给的就是遮好的
|
||||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
|
||||
@@ -161,6 +177,8 @@ async def test_masked_columns_are_masked_before_the_model_sees_them(monkeypatch,
|
||||
("SELECT SUM(amount) AS total FROM sales WHERE buyer_phone = '13900000001'", 1200),
|
||||
("SELECT region, SUM(amount) AS total FROM sales GROUP BY region ORDER BY total DESC", 1500),
|
||||
("SELECT region, note, amount AS total FROM sales ORDER BY id", 1200),
|
||||
# 汇总里的 CASE 只在条件里用到手机号:同写在 WHERE 里,值全部来自金额
|
||||
("SELECT SUM(CASE WHEN buyer_phone LIKE '139%' THEN amount ELSE 0 END) AS total FROM sales", 2800),
|
||||
])
|
||||
async def test_queries_that_do_not_read_masked_values_are_untouched(monkeypatch, masked_shop, sql, value):
|
||||
"""没碰遮罩列的值(汇总、只数个数、只在条件里用到):结果原样,取值照常,快照记下查询时一列都没遮。只遮这份结果
|
||||
@@ -246,3 +264,124 @@ async def test_column_names_the_model_wrote_are_kept(monkeypatch, masked_shop):
|
||||
snapshot = artifact_store.load(out.queries[0]["artifact"])
|
||||
assert snapshot["columns"] == ["buyer_phone", "13900000001 的金额"]
|
||||
assert set(_cells(snapshot, "buyer_phone")) == {MASKED} and out.result["value"] == 1200
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# 别的查询里遮掉的值、调用方给的值:后面的查询按值遮(复核第 1 轮)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def test_values_masked_in_an_earlier_query_are_masked_in_later_ones(monkeypatch, masked_shop):
|
||||
"""Q1 选出手机号(查询时整列遮掉,值并进内存);之后的查询里,没设遮罩的联系方式表、备注写着同一批号码(换了写法的
|
||||
也算):整格是号码的整格换,夹在文字里的换那一截。交给模型的、落库的、通用接口给的都是遮好的。以前只按这一份结果
|
||||
里的遮罩值换,Q2 原样交给模型、原样落库,质询工件事后也收不到 Q1 的值。"""
|
||||
name = masked_shop
|
||||
steps = [q(name, "SELECT buyer_phone, amount AS total FROM sales ORDER BY id", "c1"),
|
||||
q(name, "SELECT phone_num, visits FROM contacts ORDER BY id", "c2"),
|
||||
q(name, "SELECT id, note FROM sales ORDER BY id", "c3")]
|
||||
script = Script(monkeypatch, steps, {"value": {"alias": "Q2", "row": 0, "column": "visits"}, "rationale": "x"})
|
||||
out, _ = await ask(name)
|
||||
assert out.error is None and out.result["value"] == 5
|
||||
seen = script.tool_results()
|
||||
assert all(_clean(seen[c]) for c in ("c1", "c2", "c3")), seen
|
||||
contacts = artifact_store.load(out.queries[1]["artifact"])
|
||||
assert contacts["rows"] == [[MASKED, 5], [MASKED, 2], ["13800000000", 1]]
|
||||
assert contacts["masked_at_query"] == [], "按值换的不算整列遮:没对上的号码照常显示"
|
||||
notes = artifact_store.load(out.queries[2]["artifact"])
|
||||
assert _cells(notes, "note") == [f"客户{MASKED}来电改址", "例行回访", f"回访:联系 {MASKED}", ""]
|
||||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
|
||||
for query in out.queries:
|
||||
got = await client.get(f"/api/artifacts/{query['artifact']}")
|
||||
assert got.status_code == 200 and _clean(got.text)
|
||||
|
||||
|
||||
async def test_values_given_by_the_caller_are_masked_before_the_model_sees_them(monkeypatch, masked_shop):
|
||||
"""调用方交进来的遮罩值(审批质询、反方审稿收的报告快照里的值):第一条查询就按它遮,模型看不到。"""
|
||||
name = masked_shop
|
||||
given = masking._Secrets()
|
||||
given.add({"columns": ["buyer_phone"], "rows": [["13900000001"], ["13900000003"]]}, {"buyer_phone"})
|
||||
script = Script(monkeypatch, [q(name, "SELECT id, note, amount AS total FROM sales ORDER BY id", "c1")],
|
||||
{"value": {"alias": "Q1", "row": 0, "column": "total"}, "rationale": "x"})
|
||||
out, _ = await ask(name, masked_values=given)
|
||||
assert out.result["value"] == 1200 and _clean(script.tool_results()["c1"])
|
||||
snapshot = artifact_store.load(out.queries[0]["artifact"])
|
||||
assert _cells(snapshot, "note") == [f"客户{MASKED}来电改址", "例行回访", f"回访:联系 {MASKED}", ""]
|
||||
assert not given.strong - out.masked_values.strong, "交进来的值随遮掉的值一起交回"
|
||||
|
||||
|
||||
async def test_a_reference_to_a_cell_masked_by_value_says_so(monkeypatch, masked_shop):
|
||||
"""格子引用指到按值换成「已遮罩」的那一格:照实说,不说成「不是数值」。"""
|
||||
name = masked_shop
|
||||
steps = [q(name, "SELECT buyer_phone FROM sales ORDER BY id", "c1"),
|
||||
q(name, "SELECT phone_num, visits FROM contacts ORDER BY id", "c2")]
|
||||
Script(monkeypatch, steps, {"value": {"alias": "Q2", "row": 0, "column": "phone_num"}, "rationale": "x"})
|
||||
out, _ = await ask(name)
|
||||
assert out.result["value"] is None
|
||||
cell = out.cells[0]
|
||||
assert cell["status"] == "unresolved"
|
||||
assert cell["reason"] == "Q2 第 0 行「phone_num」与设置了遮罩的值相同,查询时已遮罩,不采信"
|
||||
|
||||
|
||||
async def test_a_table_function_over_a_masked_json_column_is_masked(monkeypatch, json_shop):
|
||||
"""遮罩的是 JSON 列(客户资料):用 json_each 拆开是最自然的写法,拆出来的 key、value 在交给模型、落库之前就遮掉。
|
||||
以前原样交给模型并落库,masked_at_query 是空的。"""
|
||||
name = json_shop
|
||||
sql = "SELECT c.id AS 编号, j.key, j.value FROM customers c, json_each(c.profile) j ORDER BY c.id, j.key"
|
||||
script = Script(monkeypatch, [q(name, sql, "c1")],
|
||||
{"value": {"alias": "Q1", "row": 0, "column": "编号"}, "rationale": "x"})
|
||||
out, _ = await ask(name)
|
||||
assert out.result["value"] == 1 and _clean(script.tool_results()["c1"])
|
||||
snapshot = artifact_store.load(out.queries[0]["artifact"])
|
||||
assert snapshot["masked_at_query"] == ["key", "value"] and _cells(snapshot, "编号") == [1, 1, 2, 2]
|
||||
assert _clean(json.dumps(snapshot, ensure_ascii=False))
|
||||
|
||||
|
||||
async def test_a_failed_query_does_not_hand_masked_values_to_the_model(monkeypatch, masked_shop):
|
||||
"""数据库的报错会带上数据里的值(Postgres 类型转换失败:invalid input syntax for type bigint: "139-0000-0001"):
|
||||
数据源设了遮罩时,引号里、SQL 里没写着的那一段和不带引号的长数换成「已遮罩」;模型自己写的名字照留(它据此改写
|
||||
SQL)。遮罩开关没开的(Agent 节点)照旧原话。"""
|
||||
from app.tools import datasource
|
||||
|
||||
said = {"message": ""}
|
||||
|
||||
async def failing(source, sql, **kwargs):
|
||||
raise RuntimeError(said["message"])
|
||||
|
||||
monkeypatch.setattr(datasource, "run_query", failing)
|
||||
sql = "SELECT CAST(buyer_phon AS BIGINT) AS p FROM sales WHERE id = 12345"
|
||||
ctx = ToolContext(run_id="run-error", node_id="n1", data_versions=Versions.CURRENT)
|
||||
async with SessionLocal() as session:
|
||||
[masking_tool] = await build_datasource_tools([f"db_query__{masked_shop}"], ctx, session, mask_results=True)
|
||||
[plain_tool] = await build_datasource_tools([f"db_query__{masked_shop}"], ctx, session)
|
||||
for message, shown in [
|
||||
('invalid input syntax for type bigint: "139-0000-0001"', 'invalid input syntax for type bigint: "已遮罩"'),
|
||||
("value '13900000001' is out of range for type integer", "value '已遮罩' is out of range for type integer"),
|
||||
("Conversion Error: Type INT64 with value 13900000001 can't be cast to INT32",
|
||||
"Conversion Error: Type INT64 with value 已遮罩 can't be cast to INT32"),
|
||||
('column "buyer_phon" does not exist', 'column "buyer_phon" does not exist'),
|
||||
("no rows near id 12345", "no rows near id 12345"),
|
||||
]:
|
||||
said["message"] = message
|
||||
assert await masking_tool.ainvoke({"sql": sql}) == f"查询失败:{shown}"
|
||||
assert await plain_tool.ainvoke({"sql": sql}) == f"查询失败:{message}"
|
||||
|
||||
|
||||
def test_a_result_from_a_source_without_masks_is_masked_by_the_values_seen_so_far():
|
||||
"""数据源没设遮罩(另一个库里的联系方式表),此前查询遮掉的值照样按值换;这一份结果没有可收的值。"""
|
||||
known = masking._Secrets()
|
||||
known.add({"columns": ["buyer_phone"], "rows": [["13900000001"]]}, {"buyer_phone"})
|
||||
payload = {"sql": "SELECT phone_num, note FROM contacts", "columns": ["phone_num", "note"],
|
||||
"rows": [["13900000001", "客户 139 0000 0001 来电"], ["13800000000", "例行回访"]]}
|
||||
done = masking.mask_query_result(payload, [], "sqlite", known)
|
||||
assert done.payload["rows"] == [[MASKED, f"客户 {MASKED} 来电"], ["13800000000", "例行回访"]]
|
||||
assert done.columns == [] and not done.secrets.strong
|
||||
assert masking.mask_query_result(payload, [], "sqlite").payload is payload, "什么都没收到时原样"
|
||||
|
||||
|
||||
def test_labels_written_in_this_query_are_not_masked_by_values_from_earlier_ones():
|
||||
"""分档标签是这条 SQL 自己写的字:此前查询里认不准、当遮罩值收了的同名标签,不把这一份的标签列换掉。"""
|
||||
known = masking._Secrets()
|
||||
known.add({"columns": ["档"], "rows": [["大单"], ["小单"]]}, {"档"})
|
||||
payload = {"sql": "SELECT CASE WHEN amount > 1000 THEN '大单' ELSE '小单' END AS 档, COUNT(*) AS 笔数 FROM sales "
|
||||
"GROUP BY 1", "columns": ["档", "笔数"], "rows": [["大单", 1], ["小单", 3]]}
|
||||
assert masking.mask_query_result(payload, ["amount"], "sqlite", known).payload["rows"] == [["大单", 1], ["小单", 3]]
|
||||
|
||||
Reference in New Issue
Block a user