ThearchyHelios
7794794278
评测的两种分析期望(贡献、判定):期望 SQL 的探针快照上用平台的算子(app.analysis.run,随数据变化的调用错误记成算不了)算出期望的判定,评分沿用五种结论
- 题目(cases.py):contribution 贡献恒为变化拆解(可加、比率、乘积)、另写 top_n(期望点名主要来源的前几段,缺省 1);verdict 判定是波动基线、趋势判断、差异检验、可比性检查。参数同分析工具(不写输入、总数、下钻,不写阈值),文字参数可以带 {{ schedule.* }};来源、方向、判定不许人填(HUMAN_VALUE_KEYS 加 segment、subjects、direction、verdict 等);口径题、期望反问的题配不了
- 算期望(gold.py、judgements.py):遮罩列、目录事实、现查表的日期范围和分析工具同一套取法;阈值用这次评测的分析设置快照(Context.analysis),试算时现读设置;期望 SQL 只返回两期时可以不写基期、本期;算不了的这一项照实记(评分时不能评分),参数和结果对不上的整题没有可用的值;截断的快照照旧拒收。作答前后两次期望、和上一次比的「数据变了」都不比运行当天、现查时刻、输入快照的工件(stable_sets)
- 评分(score.py、judgements.py,SCORER_VERSION 3):判定码归到同一套判定类再比;答案引用了(判定语、具名的数、结果表的格、句末依据)判定对得上的分析结论是对且可追溯,引用它的那句却写反了不算;没引用但正文按说法表(engine/analysis_claims 同一个 scan、同一套绑定规则)写对了是对但没引用;判定、来源、方向对不上或写成相反、根本没做也没说是错。答案的分析结论从作答运行的台账读、引用从报告文档读,有判断词的句子一并读出;逐题 E1 引用分析结论时按它的输入查询比(下钻顺着上一层找);判定语不再当成答案里的数;老结果的明细里没有这两项按空读
- 执行(runner.py、answer.py):有分析期望的题集开评时拍一份分析设置,算期望、各臂作答的运行(Run.options 的 analysis)都按它,记进指纹(compare 的「分析设置」一项),重试沿用;别的题集的指纹、作答运行的选项不变。助手提示词的指纹加上分析方法(guide 的 id 和全文)、写作者的分析规则和说法表版本
- 总表(stats.py):各期望的结论另按期望的种类各一份(outcomes_by_kind)
2026-10-21 18:11:57 +08:00
..
2026-10-21 16:58:45 +08:00
2026-10-21 15:47:30 +08:00
2026-10-19 10:07:06 +08:00
2026-10-21 16:03:10 +08:00
2026-10-19 16:09:19 +08:00
2026-10-19 12:06:31 +08:00
2026-10-19 09:52:09 +08:00
2026-10-21 15:49:06 +08:00
2026-10-21 16:11:00 +08:00
2026-10-21 16:06:40 +08:00
2026-10-21 17:10:43 +08:00
2026-10-21 13:29:23 +08:00
2026-10-21 15:56:49 +08:00
2026-10-21 15:51:30 +08:00
2026-10-21 15:50:15 +08:00
2026-10-21 16:00:29 +08:00
2026-10-21 16:09:21 +08:00
2026-10-21 15:56:16 +08:00
2026-10-21 16:03:10 +08:00
2026-10-21 14:08:56 +08:00
2026-10-21 16:06:40 +08:00
2026-10-21 16:58:45 +08:00
2026-10-21 16:10:01 +08:00
2026-10-21 13:49:09 +08:00
2026-10-21 16:00:44 +08:00
2026-10-21 16:56:48 +08:00
2026-10-21 17:12:48 +08:00
2026-10-21 15:57:51 +08:00
2026-10-21 13:08:14 +08:00
2026-10-21 16:16:40 +08:00
2026-10-21 13:12:05 +08:00
2026-10-21 16:02:44 +08:00
2026-10-21 16:00:29 +08:00
2026-10-21 13:03:12 +08:00
2026-10-21 16:03:10 +08:00
2026-10-18 15:56:29 +08:00
2026-10-19 15:16:54 +08:00
2026-10-21 13:13:04 +08:00
2026-10-19 12:07:31 +08:00
2026-10-19 18:48:22 +08:00
2026-10-21 13:13:04 +08:00
2026-10-21 13:13:04 +08:00
2026-10-21 13:13:04 +08:00
2026-10-21 13:13:04 +08:00
2026-10-19 15:29:00 +08:00
2026-10-17 12:00:58 +08:00
2026-10-19 12:07:31 +08:00
2026-10-19 17:55:02 +08:00
2026-10-17 08:51:51 +08:00
2026-10-21 09:08:10 +08:00
2026-10-21 09:08:10 +08:00
2026-10-21 09:08:10 +08:00
2026-10-17 11:55:53 +08:00
2026-10-17 09:12:29 +08:00
2026-10-21 13:02:30 +08:00
2026-10-19 15:16:54 +08:00
2026-10-19 09:08:45 +08:00
2026-10-19 22:48:23 +08:00
2026-10-18 18:07:03 +08:00
2026-10-19 22:48:23 +08:00
2026-10-19 15:08:39 +08:00
2026-10-19 15:07:17 +08:00
2026-10-19 09:03:37 +08:00
2026-10-21 16:03:10 +08:00
2026-10-21 13:13:04 +08:00
2026-10-19 22:48:23 +08:00
2026-10-20 10:38:30 +08:00
2026-10-19 22:48:23 +08:00
2026-10-19 22:48:23 +08:00
2026-10-21 13:13:04 +08:00
2026-10-19 08:47:53 +08:00
2026-10-19 22:48:23 +08:00
2026-10-19 15:16:54 +08:00
2026-10-19 08:08:51 +08:00
2026-10-19 09:52:09 +08:00
2026-10-19 15:09:10 +08:00
2026-10-19 12:06:31 +08:00
2026-10-19 14:06:20 +08:00
2026-10-19 16:15:21 +08:00
2026-10-17 08:17:12 +08:00
2026-10-17 08:17:12 +08:00
2026-10-20 10:33:34 +08:00
2026-10-21 18:11:57 +08:00
2026-10-21 17:17:02 +08:00
2026-10-19 15:16:54 +08:00
2026-10-17 12:29:20 +08:00
2026-10-19 11:40:33 +08:00
2026-10-17 11:40:27 +08:00
2026-10-19 15:16:54 +08:00
2026-10-16 21:41:30 +08:00
2026-10-21 13:12:15 +08:00
2026-10-19 17:55:02 +08:00
2026-10-19 09:52:09 +08:00
2026-10-19 17:55:02 +08:00
2026-10-19 17:55:02 +08:00
2026-10-16 20:46:18 +08:00
2026-10-21 16:08:22 +08:00
2026-10-21 13:13:04 +08:00
2026-10-21 13:13:04 +08:00
2026-10-19 11:40:33 +08:00
2026-10-19 11:40:33 +08:00
2026-10-21 13:12:15 +08:00
2026-10-19 12:07:31 +08:00
2026-10-17 09:19:24 +08:00
2026-10-17 13:31:25 +08:00
2026-10-17 12:15:07 +08:00
2026-10-17 13:07:19 +08:00
2026-10-16 22:16:58 +08:00
2026-10-19 12:57:38 +08:00
2026-10-19 10:01:05 +08:00
2026-10-19 11:54:08 +08:00
2026-10-19 10:05:38 +08:00
2026-10-19 10:01:05 +08:00
2026-10-17 12:31:13 +08:00
2026-10-17 11:53:54 +08:00
2026-10-16 22:02:09 +08:00
2026-10-21 13:13:04 +08:00
2026-10-21 16:06:40 +08:00
2026-10-17 12:05:12 +08:00
2026-10-20 10:04:53 +08:00
2026-10-20 10:39:53 +08:00
2026-10-20 10:04:53 +08:00
2026-10-19 13:15:56 +08:00
2026-10-19 15:16:54 +08:00
2026-10-16 22:30:09 +08:00
2026-10-19 15:16:54 +08:00
2026-10-19 16:12:07 +08:00
2026-10-19 12:03:35 +08:00
2026-10-19 11:58:45 +08:00
2026-10-17 07:59:19 +08:00
2026-10-21 13:12:15 +08:00
2026-10-21 13:13:04 +08:00
2026-10-21 13:13:04 +08:00
2026-10-20 10:58:20 +08:00
2026-10-16 21:41:30 +08:00
2026-10-19 12:03:35 +08:00
2026-10-20 11:41:28 +08:00
2026-10-20 11:41:28 +08:00
2026-10-19 15:16:54 +08:00
2026-10-20 11:41:28 +08:00
2026-10-18 18:25:24 +08:00
2026-10-19 08:59:15 +08:00
2026-10-18 15:56:29 +08:00
2026-10-19 17:55:02 +08:00
2026-10-21 16:04:42 +08:00
2026-10-19 17:00:32 +08:00
2026-10-21 13:12:15 +08:00
2026-10-19 17:00:32 +08:00
2026-10-19 17:55:02 +08:00
2026-10-19 16:47:07 +08:00
2026-10-19 17:55:02 +08:00
2026-10-21 13:12:15 +08:00
2026-10-19 17:06:08 +08:00
2026-10-19 09:10:27 +08:00
2026-10-21 16:02:46 +08:00
2026-10-19 11:40:33 +08:00
2026-10-19 09:52:09 +08:00
2026-10-19 09:52:09 +08:00
2026-10-19 09:52:09 +08:00
2026-10-17 08:31:17 +08:00
2026-10-19 14:06:20 +08:00
2026-10-18 18:09:20 +08:00
2026-10-19 11:45:55 +08:00
2026-10-18 16:15:28 +08:00
2026-10-18 15:53:08 +08:00
2026-10-17 09:19:24 +08:00
2026-10-17 08:51:51 +08:00
2026-10-19 11:40:33 +08:00
2026-10-17 11:49:31 +08:00
2026-10-17 11:47:26 +08:00
2026-10-17 13:31:25 +08:00
2026-10-17 08:17:12 +08:00
2026-10-18 18:28:17 +08:00
2026-10-18 18:15:37 +08:00
2026-10-19 11:43:41 +08:00
2026-10-19 15:16:54 +08:00
2026-10-19 09:52:09 +08:00
2026-10-19 12:03:35 +08:00
2026-10-19 13:18:23 +08:00
2026-10-17 11:36:50 +08:00
2026-10-17 07:59:19 +08:00
2026-10-21 13:13:04 +08:00
2026-10-20 08:18:55 +08:00