MojiDiff

← experiments

openmoji-g1-derived-threshold-v13-7c73fab-1687c462-9b9b1699

completed partially-falsified

Hypothesis

v12 beat the identity baseline, 0.6567 against 0.6502, but only with a decode threshold fitted on a disjoint half of the held-out icons; at its own argmax decode it lost, 0.6086 against 0.6515. That threshold is derivable rather than arbitrary: keeping a retained field is always right and changing one is right only if the value head re-predicts the same token, so changing pays exactly above p = 1/(1+q) in the value head's accuracy q on corrupted fields. Estimating q on TRAIN icons and decoding at that threshold should beat identity with no calibration on held-out data at all.

Why run it

It converts the project's first win over the trivial policy from one that needed a post-hoc threshold into one the model simply produces. That is the difference between a result that holds up and a result that needs an asterisk, and it costs one run.

Scope limits

Fixed-topology and geometry-only, single seed, marginal-respecting corruption at 0.35, token accuracy rather than renders. The threshold is derived from a first-order break-even argument that assumes changing a retained field is never right; that is nearly but not exactly true, so the derived threshold is principled rather than optimal.

Measured behaviour

Held-out loss
lossoptimizer step11.21.41.61.8010002000selected
Table view
optimizer stepheld-out loss
01.7456
601.3075
1201.2794
1801.2635
2401.2579
3001.2418
3601.2186
4201.1976
4801.1826
5401.1721
6001.1582
6601.1461
7201.1345
7801.1225
8401.1049
9001.0950
9601.0905
10201.0784
10801.0782
11401.0685
12001.0643
12601.0622
13201.0564
13801.0533
14401.0512
15001.0468
15601.0484
16201.0512
16801.0444
17401.0429
18001.0435
18601.0400
19201.0370
19801.0373
20401.0380
21001.0378
21601.0326
22201.0353
22801.0389
23401.0352
24001.0401
24601.0406
25201.0370
25801.0357
26401.0419
Held-out token accuracy
aggregatechanged fieldsretained fields
accuracyoptimizer step00.250.50.751010002000selected
Table view
optimizer stepaggregatechanged fieldsretained fields
00.33410.00130.5121
600.62550.00120.9594
1200.64370.00160.9871
1800.64730.00270.9921
2400.64820.00220.9937
3000.64760.00300.9923
3600.62890.00630.9619
4200.60960.01430.9279
4800.59060.01670.8975
5400.57570.02400.8708
6000.58270.02600.8804
6600.58670.03460.8819
7200.60040.03360.9035
7800.59490.04110.8910
8400.60870.04400.9107
9000.60670.04640.9064
9600.60290.05150.8978
10200.61380.05140.9146
10800.60700.05430.9026
11400.61080.05690.9070
12000.60860.06000.9019
12600.60690.06210.8983
13200.60810.06440.8989
13800.60420.06430.8930
14400.60550.06730.8933
15000.61160.06590.9035
15600.60800.06570.8981
16200.60870.07030.8967
16800.60200.07150.8857
17400.61000.07150.8979
18000.60580.07280.8909
18600.60330.07300.8869
19200.61000.07180.8978
19800.60660.07120.8929
20400.59920.07530.8793
21000.60080.07450.8823
21600.60860.07380.8947
22200.60710.07480.8917
22800.60060.07700.8806
23400.60050.07500.8815
24000.60210.07400.8845
24600.60180.07680.8826
25200.60290.07730.8840
25800.60440.07410.8880
26400.59910.08180.8757
Training token accuracy (per batch)
accuracyoptimizer step00.20.40.60.810002000selected
Table view
optimizer steptrain accuracy
10.3187
20.3975
30.5010
40.5082
50.5148
60.5540
70.5224
80.5394
90.5473
100.5155
110.5012
120.4833
130.4709
140.4672
150.4734
160.5177
170.5300
180.5586
190.5445
200.5629
210.5410
220.5685
230.5791
240.5906
250.5677
260.5893
270.5475
280.5895
290.5687
300.5671
310.5765
320.5660
330.5575
340.5747
350.5800
360.5951
370.5767
380.5887
390.5978
400.6188
410.6092
420.6228
430.5846
440.6257
450.6311
460.6147
470.6138
480.6178
490.6034
500.6230
510.6152
520.6210
530.6163
540.6163
550.6118
560.6051
570.6248
580.6245
590.6274
600.6143
610.6211
620.6114
630.6262
640.6372
650.6455
660.6428
670.6226
680.6447
690.6204
700.6365
710.6446
720.6361
730.6472
740.6248
750.6399
760.6305
770.6321
780.6294
790.6015
800.6416
810.6292
820.6351
830.6209
840.6371
850.6386
860.6285
870.6286
880.6233
890.6438
900.6423
910.6414
920.6347
930.6048
940.6499
950.6464
960.6358
970.6359
980.6515
990.6491
1000.6406
1010.6494
1020.6413
1030.6472
1040.6406
1050.6602
1060.6432
1070.6414
1080.6342
1090.6615
1100.6478
1110.6485
1120.6466
1130.6421
1140.6479
1150.6456
1160.6482
1170.6404
1180.6439
1190.6418
1200.6604
1210.6438
1220.6583
1230.6383
1240.6464
1250.6556
1260.6460
1270.6414
1280.6505
1290.6420
1300.6457
1310.6346
1320.6373
1330.6361
1340.6430
1350.6218
1360.6357
1370.6368
1380.6501
1390.6434
1400.6349
1410.6443
1420.6471
1430.6435
1440.6362
1450.6439
1460.6452
1470.6495
1480.6373
1490.6441
1500.6354
1510.6454
1520.6394
1530.6402
1540.6369
1550.6645
1560.6427
1570.6382
1580.6474
1590.6401
1600.6490
1610.6432
1620.6414
1630.6487
1640.6432
1650.6422
1660.6432
1670.6506
1680.6417
1690.6573
1700.6531
1710.6518
1720.6556
1730.6488
1740.6426
1750.6489
1760.6377
1770.6438
1780.6396
1790.6562
1800.6434
1810.6417
1820.6476
1830.6464
1840.6535
1850.6613
1860.6541
1870.6443
1880.6389
1890.6374
1900.6475
1910.6467
1920.6393
1930.6539
1940.6381
1950.6528
1960.6527
1970.6411
1980.6402
1990.6429
2000.6552
2010.6453
2020.6438
2030.6591
2040.6495
2050.6439
2060.6491
2070.6530
2080.6627
2090.6542
2100.6547
2110.6369
2120.6559
2130.6491
2140.6395
2150.6512
2160.6355
2170.6463
2180.6392
2190.6558
2200.6407
2210.6478
2220.6429
2230.6524
2240.6577
2250.6609
2260.6431
2270.6381
2280.6413
2290.6524
2300.6485
2310.6343
2320.6448
2330.6515
2340.6494
2350.6339
2360.6527
2370.6366
2380.6522
2390.6582
2400.6537
2410.6395
2420.6450
2430.6617
2440.6402
2450.6516
2460.6493
2470.6386
2480.6547
2490.6453
2500.6488
2510.6569
2520.6536
2530.6530
2540.6598
2550.6522
2560.6527
2570.6570
2580.6451
2590.6629
2600.6416
2610.6313
2620.6474
2630.6548
2640.6458
2650.6580
2660.6440
2670.6545
2680.6448
2690.6489
2700.6504
2710.6365
2720.6579
2730.6431
2740.6350
2750.6283
2760.6441
2770.6509
2780.6390
2790.6597
2800.6395
2810.6499
2820.6586
2830.6427
2840.6561
2850.6520
2860.6547
2870.6445
2880.6415
2890.6566
2900.6591
2910.6352
2920.6493
2930.6465
2940.6558
2950.6481
2960.6538
2970.6540
2980.6587
2990.6468
3000.6510
3010.6561
3020.6544
3030.6421
3040.6473
3050.6356
3060.6414
3070.6411
3080.6489
3090.6479
3100.6419
3110.6359
3120.6619
3130.6522
3140.6480
3150.6393
3160.6447
3170.6509
3180.6482
3190.6468
3200.6596
3210.6446
3220.6413
3230.6371
3240.6434
3250.6374
3260.6471
3270.6476
3280.6454
3290.6531
3300.6393
3310.6389
3320.6440
3330.6495
3340.6471
3350.6406
3360.6564
3370.6383
3380.6539
3390.6480
3400.6379
3410.6412
3420.6358
3430.6405
3440.6461
3450.6478
3460.6421
3470.6486
3480.6305
3490.6499
3500.6558
3510.6528
3520.6479
3530.6370
3540.6538
3550.6460
3560.6421
3570.6506
3580.6341
3590.6221
3600.6362
3610.6373
3620.6279
3630.6372
3640.6228
3650.6354
3660.6364
3670.6482
3680.6297
3690.6328
3700.6286
3710.6480
3720.6228
3730.6278
3740.6237
3750.6256
3760.6301
3770.6346
3780.6193
3790.6316
3800.6099
3810.6068
3820.6230
3830.6216
3840.6298
3850.6127
3860.6176
3870.6186
3880.6223
3890.6090
3900.6100
3910.6097
3920.6217
3930.5968
3940.6377
3950.6175
3960.6363
3970.6287
3980.6283
3990.6195
4000.6139
4010.6268
4020.6226
4030.6290
4040.6285
4050.6164
4060.6163
4070.6118
4080.6125
4090.6154
4100.6002
4110.6048
4120.6168
4130.6171
4140.6079
4150.6112
4160.6178
4170.6119
4180.6261
4190.6057
4200.6187
4210.6151
4220.6278
4230.6094
4240.6299
4250.6256
4260.6253
4270.6169
4280.6020
4290.5979
4300.5960
4310.5924
4320.6372
4330.6148
4340.6149
4350.6209
4360.6154
4370.6260
4380.6062
4390.6065
4400.6135
4410.5979
4420.6021
4430.6060
4440.6230
4450.6142
4460.6177
4470.6096
4480.5968
4490.5864
4500.5799
4510.5773
4520.5916
4530.6036
4540.6119
4550.6306
4560.6277
4570.6202
4580.6159
4590.5774
4600.5975
4610.5778
4620.6000
4630.6188
4640.6116
4650.6297
4660.6024
4670.6014
4680.6048
4690.6019
4700.6006
4710.6162
4720.6042
4730.6358
4740.5887
4750.6237
4760.6146
4770.5970
4780.6002
4790.5698
4800.6034
4810.5902
4820.6255
4830.6316
4840.6263
4850.6140
4860.6225
4870.6309
4880.5935
4890.5972
4900.5950
4910.5924
4920.5808
4930.5930
4940.5970
4950.6084
4960.5943
4970.5976
4980.5854
4990.5990
5000.6236
5010.6017
5020.6078
5030.6062
5040.6080
5050.6012
5060.5905
5070.5975
5080.5867
5090.5812
5100.5994
5110.5950
5120.6279
5130.6043
5140.5731
5150.6248
5160.5985
5170.6142
5180.6125
5190.6017
5200.6160
5210.5882
5220.6310
5230.6176
5240.5982
5250.6037
5260.6257
5270.5895
5280.6281
5290.6040
5300.6089
5310.6031
5320.6094
5330.6096
5340.6259
5350.6206
5360.5942
5370.6204
5380.5971
5390.5931
5400.5704
5410.5971
5420.5949
5430.5880
5440.6356
5450.6086
5460.5814
5470.6029
5480.5568
5490.5951
5500.5802
5510.5887
5520.6059
5530.5799
5540.6116
5550.5823
5560.5639
5570.5957
5580.5804
5590.6037
5600.5885
5610.6264
5620.6121
5630.5974
5640.6068
5650.6071
5660.6077
5670.5761
5680.5864
5690.6110
5700.5885
5710.6160
5720.6261
5730.6210
5740.6340
5750.6049
5760.6181
5770.5893
5780.5929
5790.6028
5800.5836
5810.6164
5820.5992
5830.6112
5840.5997
5850.6224
5860.6154
5870.6056
5880.6256
5890.5954
5900.6124
5910.6060
5920.6036
5930.6224
5940.6218
5950.6011
5960.5865
5970.5751
5980.5917
5990.5926
6000.6026
6010.6059
6020.6019
6030.6289
6040.6265
6050.6387
6060.6089
6070.6140
6080.6073
6090.5928
6100.5834
6110.6103
6120.6156
6130.5856
6140.6488
6150.6192
6160.6167
6170.6061
6180.6135
6190.5863
6200.5829
6210.5713
6220.6145
6230.5789
6240.6035
6250.6126
6260.6164
6270.6033
6280.6007
6290.6077
6300.6221
6310.6338
6320.6112
6330.6345
6340.5782
6350.6248
6360.6035
6370.5975
6380.6056
6390.6244
6400.5866
6410.6189
6420.5988
6430.6139
6440.6019
6450.6215
6460.5959
6470.6066
6480.6079
6490.6176
6500.6197
6510.6184
6520.6017
6530.6434
6540.6197
6550.6214
6560.6129
6570.5946
6580.6082
6590.5767
6600.5724
6610.5953
6620.6136
6630.5883
6640.6037
6650.6111
6660.6065
6670.6216
6680.5822
6690.5732
6700.6155
6710.6085
6720.6132
6730.6108
6740.6643
6750.6029
6760.6062
6770.6011
6780.6050
6790.6082
6800.6035
6810.5751
6820.5971
6830.6140
6840.6268
6850.6342
6860.6134
6870.6083
6880.6113
6890.6436
6900.6321
6910.5974
6920.6167
6930.6174
6940.6095
6950.6111
6960.6041
6970.6053
6980.6054
6990.6039
7000.6029
7010.6138
7020.6318
7030.6144
7040.6425
7050.6088
7060.6316
7070.6119
7080.5948
7090.6119
7100.6073
7110.5933
7120.6342
7130.5955
7140.6231
7150.6329
7160.6144
7170.6051
7180.5911
7190.6262
7200.6086
7210.6211
7220.6238
7230.5877
7240.6077
7250.6118
7260.6233
7270.6224
7280.6107
7290.6328
7300.6152
7310.6107
7320.6020
7330.6408
7340.5831
7350.6187
7360.6307
7370.6022
7380.6126
7390.6401
7400.6089
7410.6325
7420.6543
7430.6064
7440.6226
7450.5834
7460.6049
7470.6228
7480.6249
7490.6022
7500.5973
7510.6305
7520.6139
7530.6291
7540.6096
7550.6202
7560.6309
7570.6363
7580.5979
7590.6247
7600.6292
7610.6634
7620.6403
7630.6162
7640.5941
7650.6131
7660.6199
7670.6299
7680.6147
7690.6174
7700.6201
7710.6204
7720.6152
7730.6261
7740.6413
7750.6156
7760.6084
7770.6396
7780.6341
7790.6412
7800.6127
7810.6147
7820.6111
7830.6014
7840.6307
7850.5992
7860.6285
7870.6168
7880.6432
7890.6119
7900.6207
7910.6190
7920.6128
7930.6036
7940.5991
7950.6188
7960.6127
7970.6187
7980.6381
7990.6386
8000.6482
8010.6160
8020.6184
8030.6284
8040.6187
8050.6014
8060.6305
8070.6182
8080.6374
8090.6193
8100.6116
8110.6295
8120.6232
8130.6353
8140.5988
8150.6306
8160.6000
8170.6218
8180.6136
8190.6234
8200.6324
8210.6470
8220.6395
8230.6493
8240.6141
8250.6058
8260.6306
8270.5757
8280.5985
8290.6022
8300.6185
8310.6070
8320.6316
8330.6230
8340.6283
8350.6459
8360.6016
8370.5942
8380.6242
8390.6001
8400.6262
8410.6436
8420.6420
8430.6194
8440.6209
8450.6405
8460.6250
8470.6408
8480.6380
8490.5910
8500.6709
8510.6200
8520.6428
8530.6508
8540.6296
8550.6399
8560.6396
8570.6521
8580.6241
8590.6314
8600.6034
8610.6301
8620.6085
8630.6350
8640.6321
8650.6264
8660.6401
8670.6331
8680.6476
8690.6262
8700.6527
8710.6164
8720.6593
8730.6257
8740.6337
8750.6312
8760.6262
8770.6204
8780.6000
8790.6550
8800.6388
8810.6106
8820.6620
8830.6315
8840.6296
8850.5950
8860.6404
8870.6413
8880.6148
8890.6240
8900.6165
8910.6069
8920.6293
8930.6535
8940.5935
8950.6432
8960.6274
8970.6383
8980.6292
8990.6457
9000.6303
9010.6522
9020.6475
9030.6346
9040.6486
9050.6130
9060.6472
9070.6331
9080.6355
9090.6442
9100.6513
9110.6117
9120.6339
9130.5900
9140.6263
9150.6110
9160.6319
9170.6318
9180.6206
9190.6433
9200.6337
9210.6516
9220.6347
9230.6434
9240.6631
9250.6145
9260.6113
9270.6173
9280.6516
9290.6578
9300.6211
9310.6152
9320.6064
9330.6381
9340.6276
9350.6370
9360.6261
9370.6080
9380.6672
9390.6452
9400.6571
9410.6064
9420.6604
9430.6413
9440.6284
9450.6203
9460.6536
9470.6344
9480.6076
9490.6567
9500.6309
9510.6179
9520.6319
9530.6422
9540.6231
9550.6339
9560.6217
9570.6381
9580.5911
9590.6316
9600.6216
9610.6088
9620.6220
9630.6409
9640.6271
9650.6403
9660.6576
9670.6517
9680.6693
9690.5898
9700.6471
9710.6071
9720.6305
9730.6181
9740.6705
9750.6124
9760.6423
9770.6101
9780.6477
9790.6445
9800.6512
9810.6404
9820.6211
9830.6180
9840.6528
9850.6389
9860.6573
9870.6191
9880.6342
9890.6530
9900.6009
9910.6391
9920.6084
9930.6491
9940.6144
9950.6149
9960.6348
9970.6356
9980.6054
9990.6282
10000.6296
10010.6118
10020.6398
10030.6174
10040.5968
10050.6321
10060.6187
10070.6431
10080.6253
10090.6577
10100.6548
10110.6019
10120.6266
10130.6383
10140.6282
10150.6434
10160.6129
10170.6140
10180.6536
10190.6303
10200.6708
10210.6469
10220.6342
10230.6403
10240.6674
10250.6837
10260.6232
10270.6258
10280.6575
10290.6287
10300.6368
10310.6424
10320.6464
10330.6369
10340.6481
10350.6285
10360.6460
10370.6357
10380.6499
10390.6212
10400.6493
10410.6464
10420.6467
10430.6397
10440.6363
10450.6265
10460.6231
10470.6586
10480.6171
10490.6529
10500.6461
10510.6259
10520.6286
10530.6187
10540.6336
10550.6412
10560.6199
10570.6305
10580.6029
10590.6239
10600.6354
10610.6497
10620.6486
10630.6251
10640.6439
10650.6247
10660.6249
10670.6344
10680.6393
10690.6250
10700.6518
10710.6331
10720.6048
10730.6205
10740.6610
10750.6415
10760.6501
10770.6699
10780.6310
10790.6909
10800.6159
10810.6151
10820.6435
10830.6360
10840.6214
10850.6357
10860.6532
10870.6327
10880.6802
10890.6352
10900.6540
10910.6532
10920.6441
10930.6071
10940.6508
10950.6476
10960.6323
10970.6915
10980.6248
10990.6063
11000.6190
11010.6555
11020.6248
11030.6406
11040.6282
11050.6391
11060.6375
11070.6213
11080.6414
11090.6538
11100.6310
11110.6381
11120.6257
11130.6558
11140.6620
11150.6587
11160.6405
11170.6633
11180.6293
11190.6454
11200.6393
11210.6394
11220.6217
11230.6275
11240.6281
11250.6324
11260.6167
11270.6388
11280.6274
11290.6262
11300.6470
11310.6177
11320.6267
11330.6379
11340.6635
11350.6611
11360.6569
11370.6321
11380.6018
11390.6414
11400.6287
11410.6584
11420.6339
11430.6463
11440.6557
11450.6154
11460.6625
11470.6400
11480.6423
11490.6100
11500.6370
11510.6326
11520.6449
11530.6245
11540.6421
11550.6284
11560.6562
11570.6442
11580.6271
11590.6304
11600.6053
11610.6348
11620.6251
11630.6098
11640.6377
11650.6338
11660.6145
11670.6410
11680.6225
11690.6353
11700.6681
11710.6231
11720.6023
11730.6641
11740.5991
11750.6333
11760.6413
11770.6606
11780.6187
11790.6063
11800.6321
11810.6444
11820.6587
11830.6185
11840.6059
11850.6681
11860.6353
11870.6807
11880.6815
11890.6401
11900.6384
11910.6044
11920.6854
11930.6533
11940.6149
11950.6331
11960.6635
11970.6400
11980.6703
11990.6500
12000.6297
12010.6792
12020.6185
12030.6522
12040.6515
12050.6627
12060.6238
12070.6703
12080.6498
12090.6727
12100.6074
12110.6595
12120.6251
12130.6305
12140.6679
12150.6591
12160.6139
12170.6634
12180.6469
12190.6395
12200.6167
12210.6500
12220.6663
12230.6162
12240.6488
12250.6378
12260.5895
12270.6458
12280.6719
12290.5924
12300.6406
12310.6225
12320.6521
12330.6368
12340.6639
12350.6242
12360.6688
12370.6406
12380.6782
12390.6636
12400.5962
12410.6752
12420.6380
12430.6626
12440.6591
12450.6572
12460.6503
12470.6395
12480.6238
12490.6547
12500.6102
12510.6538
12520.5981
12530.6295
12540.6291
12550.6391
12560.6493
12570.6376
12580.6397
12590.6554
12600.6213
12610.6304
12620.6436
12630.6527
12640.6682
12650.6676
12660.5982
12670.6022
12680.6319
12690.6505
12700.6475
12710.6219
12720.6114
12730.6422
12740.6704
12750.6423
12760.6437
12770.6470
12780.6538
12790.6262
12800.6237
12810.6492
12820.6603
12830.6115
12840.6843
12850.6089
12860.6172
12870.6177
12880.6568
12890.6148
12900.6370
12910.6307
12920.6565
12930.5972
12940.6261
12950.6702
12960.6233
12970.6275
12980.6305
12990.6175
13000.6265
13010.6623
13020.6368
13030.6852
13040.6059
13050.6509
13060.6334
13070.6354
13080.6416
13090.6835
13100.6304
13110.6642
13120.6210
13130.6586
13140.6524
13150.6745
13160.6530
13170.6239
13180.6439
13190.6288
13200.6474
13210.6370
13220.6359
13230.6667
13240.6359
13250.6335
13260.6583
13270.6320
13280.6358
13290.6166
13300.6165
13310.6325
13320.6331
13330.6270
13340.6242
13350.6465
13360.6169
13370.6511
13380.6607
13390.6119
13400.6616
13410.6448
13420.6458
13430.6273
13440.6625
13450.6588
13460.6175
13470.6182
13480.6458
13490.6269
13500.6698
13510.6300
13520.6171
13530.6907
13540.6511
13550.6597
13560.6844
13570.6340
13580.6603
13590.6416
13600.7106
13610.6172
13620.6216
13630.6523
13640.6397
13650.6731
13660.6717
13670.6295
13680.6556
13690.6726
13700.6225
13710.6603
13720.6609
13730.6411
13740.6189
13750.6871
13760.6545
13770.6621
13780.6444
13790.6337
13800.6390
13810.6019
13820.6744
13830.6376
13840.6326
13850.6517
13860.6236
13870.6296
13880.6328
13890.6427
13900.6572
13910.6204
13920.6528
13930.6183
13940.6434
13950.6448
13960.6552
13970.6218
13980.6426
13990.6256
14000.6513
14010.6382
14020.6537
14030.6601
14040.6468
14050.6602
14060.6652
14070.6586
14080.6036
14090.6968
14100.6359
14110.6451
14120.6762
14130.6508
14140.6800
14150.6537
14160.6404
14170.6373
14180.6379
14190.6340
14200.6365
14210.6449
14220.6109
14230.6875
14240.6294
14250.6450
14260.6670
14270.6556
14280.6137
14290.6345
14300.6485
14310.6454
14320.6988
14330.6297
14340.6025
14350.6126
14360.6487
14370.6479
14380.6478
14390.6329
14400.6465
14410.6235
14420.6378
14430.6674
14440.6512
14450.6667
14460.6441
14470.6070
14480.6598
14490.6658
14500.6702
14510.6246
14520.6703
14530.6304
14540.6333
14550.6244
14560.6500
14570.6228
14580.6329
14590.6311
14600.6567
14610.6390
14620.6374
14630.6575
14640.6207
14650.6442
14660.6355
14670.6223
14680.6523
14690.6536
14700.6806
14710.6507
14720.6259
14730.6549
14740.6376
14750.6351
14760.6702
14770.6473
14780.6525
14790.6542
14800.6398
14810.6559
14820.6519
14830.6907
14840.6282
14850.6515
14860.6262
14870.6500
14880.6440
14890.6455
14900.6167
14910.6900
14920.6305
14930.6259
14940.6529
14950.6213
14960.6454
14970.6174
14980.6321
14990.6562
15000.6348
15010.6252
15020.6592
15030.6144
15040.6387
15050.6568
15060.6307
15070.6259
15080.6654
15090.6166
15100.6611
15110.6464
15120.6837
15130.6255
15140.6058
15150.6360
15160.6803
15170.6539
15180.6632
15190.5947
15200.6563
15210.6634
15220.6775
15230.6736
15240.6578
15250.6522
15260.6525
15270.6938
15280.7052
15290.6336
15300.6423
15310.6732
15320.6330
15330.6750
15340.6609
15350.6399
15360.6706
15370.6532
15380.6543
15390.6536
15400.6463
15410.6257
15420.6915
15430.6455
15440.6556
15450.6447
15460.6423
15470.6126
15480.6495
15490.6736
15500.6547
15510.6029
15520.6760
15530.6666
15540.6302
15550.6261
15560.6412
15570.6710
15580.6180
15590.6480
15600.6326
15610.5989
15620.6438
15630.6624
15640.6007
15650.6477
15660.6349
15670.6736
15680.6511
15690.6508
15700.6602
15710.6721
15720.6384
15730.6774
15740.6603
15750.5876
15760.6612
15770.6349
15780.6462
15790.6709
15800.6694
15810.6642
15820.6871
15830.6339
15840.6465
15850.6300
15860.6564
15870.6033
15880.6379
15890.6384
15900.6470
15910.6604
15920.6536
15930.6741
15940.6758
15950.6553
15960.6317
15970.6050
15980.6842
15990.6640
16000.6887
16010.6230
16020.6052
16030.6370
16040.6675
16050.6549
16060.6160
16070.6357
16080.6571
16090.6655
16100.6657
16110.6153
16120.6795
16130.6546
16140.6222
16150.6293
16160.6845
16170.6721
16180.6356
16190.6670
16200.6555
16210.6233
16220.6370
16230.6522
16240.6395
16250.6422
16260.6409
16270.6666
16280.6132
16290.6413
16300.6680
16310.6340
16320.6299
16330.6495
16340.6343
16350.6383
16360.6859
16370.6452
16380.6851
16390.5888
16400.6619
16410.6443
16420.6493
16430.6298
16440.6819
16450.6150
16460.6924
16470.6338
16480.6576
16490.6401
16500.6729
16510.6650
16520.6216
16530.6512
16540.6440
16550.6662
16560.6537
16570.6478
16580.6610
16590.6756
16600.6537
16610.6595
16620.6220
16630.6306
16640.6432
16650.6059
16660.6458
16670.6357
16680.6203
16690.5936
16700.6592
16710.6299
16720.6428
16730.6659
16740.6329
16750.6235
16760.6575
16770.6143
16780.6567
16790.6746
16800.6684
16810.6091
16820.6116
16830.6575
16840.6151
16850.6655
16860.6322
16870.6005
16880.7009
16890.6365
16900.6902
16910.6817
16920.6431
16930.6581
16940.6551
16950.7011
16960.6457
16970.6350
16980.6393
16990.6742
17000.6619
17010.6679
17020.6426
17030.6782
17040.6629
17050.6492
17060.6691
17070.6695
17080.6561
17090.6115
17100.6882
17110.6542
17120.6683
17130.6521
17140.6369
17150.6410
17160.6108
17170.6797
17180.6709
17190.6090
17200.6888
17210.6282
17220.6273
17230.6278
17240.6671
17250.6574
17260.6158
17270.6718
17280.6283
17290.6379
17300.6516
17310.6704
17320.6454
17330.6273
17340.6512
17350.6684
17360.6586
17370.6552
17380.6560
17390.6587
17400.6535
17410.6571
17420.6766
17430.6096
17440.6972
17450.6255
17460.6603
17470.6923
17480.6340
17490.6737
17500.6517
17510.6376
17520.6530
17530.6226
17540.6353
17550.6456
17560.6303
17570.6281
17580.6785
17590.6545
17600.6635
17610.6898
17620.6445
17630.6455
17640.6552
17650.6401
17660.6751
17670.6934
17680.6642
17690.5936
17700.6171
17710.6535
17720.6323
17730.6630
17740.6234
17750.6525
17760.6834
17770.6560
17780.6752
17790.6464
17800.6590
17810.6661
17820.6402
17830.6239
17840.6941
17850.6515
17860.6385
17870.6880
17880.6305
17890.6395
17900.6382
17910.6590
17920.6469
17930.6472
17940.6443
17950.6949
17960.6370
17970.6686
17980.6552
17990.6381
18000.6437
18010.6293
18020.6575
18030.6394
18040.6724
18050.6829
18060.6456
18070.6133
18080.6954
18090.6628
18100.6404
18110.6583
18120.6852
18130.6555
18140.6503
18150.6410
18160.6692
18170.6459
18180.6934
18190.6253
18200.6369
18210.6468
18220.6619
18230.6502
18240.6499
18250.6375
18260.6753
18270.6387
18280.6411
18290.6471
18300.6026
18310.6368
18320.6152
18330.6228
18340.6518
18350.6504
18360.6396
18370.6367
18380.6515
18390.6479
18400.6813
18410.6532
18420.6206
18430.6731
18440.6217
18450.6502
18460.6278
18470.7029
18480.6478
18490.6041
18500.6462
18510.6618
18520.6604
18530.6491
18540.6256
18550.6469
18560.6736
18570.6800
18580.6860
18590.6924
18600.6424
18610.6600
18620.6935
18630.6981
18640.6203
18650.6428
18660.6705
18670.6349
18680.6610
18690.6720
18700.6590
18710.6783
18720.6541
18730.6603
18740.6318
18750.6543
18760.6670
18770.6751
18780.6287
18790.7058
18800.6402
18810.6553
18820.6276
18830.6585
18840.6501
18850.6911
18860.6024
18870.6646
18880.6808
18890.6237
18900.6173
18910.6543
18920.6824
18930.6524
18940.6590
18950.6436
18960.6115
18970.6197
18980.6526
18990.6496
19000.6468
19010.6186
19020.6729
19030.6433
19040.6630
19050.6445
19060.6830
19070.6221
19080.6863
19090.6628
19100.6438
19110.6419
19120.6703
19130.6676
19140.6640
19150.7178
19160.6573
19170.6804
19180.6228
19190.6233
19200.6545
19210.6544
19220.6641
19230.6268
19240.6537
19250.6535
19260.6880
19270.6412
19280.6771
19290.6770
19300.6446
19310.6522
19320.6220
19330.6674
19340.6836
19350.6973
19360.6176
19370.6062
19380.6539
19390.6499
19400.6724
19410.6584
19420.6447
19430.6498
19440.6727
19450.6761
19460.6560
19470.6850
19480.6637
19490.6611
19500.6350
19510.6741
19520.6811
19530.6655
19540.6593
19550.6483
19560.6363
19570.6574
19580.6400
19590.6502
19600.6454
19610.6563
19620.6330
19630.6506
19640.6495
19650.6613
19660.6520
19670.6211
19680.6679
19690.6350
19700.6169
19710.6547
19720.6614
19730.6945
19740.6054
19750.6541
19760.6517
19770.6337
19780.6416
19790.6861
19800.6573
19810.6696
19820.6626
19830.6589
19840.6590
19850.6627
19860.6741
19870.6252
19880.6567
19890.6483
19900.6610
19910.6539
19920.6486
19930.6645
19940.6487
19950.6515
19960.6643
19970.6267
19980.6073
19990.6560
20000.6186
20010.6438
20020.6423
20030.6401
20040.6085
20050.6646
20060.6312
20070.6669
20080.6674
20090.6387
20100.6217
20110.6913
20120.6248
20130.6554
20140.6701
20150.6787
20160.6144
20170.6181
20180.6798
20190.6486
20200.6837
20210.6356
20220.6217
20230.7272
20240.6519
20250.6856
20260.7022
20270.6643
20280.6526
20290.6686
20300.7213
20310.6827
20320.6527
20330.6345
20340.6684
20350.6711
20360.6669
20370.6474
20380.6722
20390.6907
20400.6483
20410.6842
20420.6537
20430.6802
20440.6253
20450.6990
20460.6621
20470.6857
20480.6609
20490.6726
20500.6672
20510.6035
20520.6967
20530.6778
20540.6168
20550.7052
20560.6352
20570.6381
20580.6375
20590.6565
20600.6961
20610.6161
20620.6787
20630.6275
20640.6262
20650.6571
20660.6576
20670.6141
20680.6550
20690.6504
20700.7011
20710.6282
20720.6698
20730.6542
20740.6782
20750.6646
20760.6798
20770.6835
20780.6313
20790.7170
20800.6373
20810.6625
20820.6979
20830.6736
20840.6629
20850.6728
20860.6392
20870.6632
20880.6228
20890.6728
20900.6453
20910.6499
20920.6330
20930.6770
20940.6589
20950.6514
20960.6875
20970.6494
20980.6360
20990.6531
21000.6470
21010.6606
21020.6922
21030.6679
21040.6070
21050.5988
21060.6732
21070.6453
21080.6638
21090.6248
21100.6069
21110.6739
21120.6861
21130.6771
21140.6529
21150.6673
21160.6771
21170.6305
21180.6494
21190.6871
21200.6709
21210.6136
21220.7126
21230.6291
21240.6256
21250.6574
21260.6650
21270.6331
21280.6352
21290.6383
21300.6828
21310.6333
21320.6523
21330.6437
21340.6455
21350.6176
21360.6541
21370.6411
21380.6337
21390.6811
21400.6722
21410.6903
21420.6020
21430.6932
21440.6518
21450.6403
21460.6539
21470.7131
21480.6290
21490.6821
21500.6313
21510.6686
21520.6785
21530.7004
21540.6540
21550.6379
21560.6482
21570.6797
21580.6459
21590.6763
21600.6485
21610.6976
21620.6525
21630.6433
21640.6693
21650.6247
21660.6635
21670.6230
21680.6300
21690.6587
21700.6538
21710.6348
21720.6404
21730.6530
21740.6358
21750.6534
21760.6656
21770.6257
21780.6794
21790.6405
21800.6437
21810.6253
21820.6947
21830.6657
21840.6180
21850.6450
21860.6543
21870.6573
21880.6709
21890.6301
21900.6295
21910.6912
21920.6826
21930.7041
21940.6879
21950.6579
21960.6551
21970.7177
21980.7177
21990.6324
22000.6482
22010.6812
22020.6504
22030.6693
22040.6569
22050.6659
22060.6833
22070.6737
22080.6485
22090.6757
22100.6625
22110.6664
22120.6656
22130.6663
22140.6870
22150.6477
22160.6777
22170.6505
22180.6591
22190.6651
22200.7137
22210.6254
22220.6843
22230.6670
22240.6291
22250.6561
22260.6282
22270.6843
22280.6520
22290.6562
22300.6589
22310.6131
22320.6497
22330.6782
22340.6811
22350.6509
22360.6420
22370.6662
22380.6595
22390.6767
22400.6466
22410.6897
22420.6109
22430.6784
22440.6684
22450.6387
22460.6408
22470.6857
22480.6446
22490.6776
22500.7026
22510.6449
22520.7114
22530.6342
22540.6510
22550.6574
22560.6456
22570.6605
22580.6436
22590.6629
22600.6596
22610.7018
22620.6386
22630.6820
22640.6736
22650.6577
22660.6291
22670.6413
22680.6725
22690.6731
22700.7046
22710.6264
22720.6142
22730.6434
22740.6795
22750.6519
22760.6506
22770.6514
22780.6674
22790.6848
22800.6785
22810.6735
22820.6793
22830.6650
22840.6530
22850.6444
22860.6781
22870.6731
22880.6782
22890.6459
22900.6780
22910.6493
22920.6730
22930.6362
22940.6702
22950.6531
22960.6566
22970.6478
22980.6605
22990.6487
23000.6542
23010.6723
23020.6291
23030.6626
23040.6328
23050.6239
23060.6638
23070.6731
23080.7022
23090.6575
23100.6453
23110.6399
23120.6456
23130.6373
23140.7027
23150.6698
23160.6806
23170.6724
23180.6347
23190.6830
23200.6683
23210.6886
23220.6115
23230.6636
23240.6260
23250.6798
23260.6532
23270.6698
23280.6864
23290.6702
23300.6643
23310.6528
23320.6542
23330.6291
23340.6647
23350.6184
23360.6421
23370.6555
23380.6385
23390.6156
23400.6535
23410.6410
23420.6492
23430.6999
23440.6600
23450.6343
23460.6968
23470.6319
23480.6567
23490.6845
23500.6913
23510.6172
23520.6229
23530.6464
23540.6711
23550.6697
23560.6411
23570.6370
23580.6922
23590.6624
23600.7320
23610.7086
23620.6646
23630.6724
23640.6543
23650.7328
23660.6735
23670.6431
23680.6364
23690.6807
23700.6447
23710.6799
23720.6554
23730.6715
23740.7060
23750.6526
23760.6853
23770.6622
23780.6980
23790.6470
23800.7162
23810.6647
23820.6734
23830.6507
23840.6890
23850.6439
23860.6395
23870.6851
23880.6861
23890.6227
23900.6978
23910.6401
23920.6381
23930.6123
23940.6775
23950.6919
23960.6200
23970.6672
23980.6523
23990.6208
24000.6909
24010.7019
24020.5936
24030.6583
24040.6458
24050.6977
24060.6662
24070.6834
24080.6323
24090.7057
24100.6578
24110.6871
24120.6763
24130.6111
24140.6949
24150.6604
24160.6581
24170.6931
24180.6761
24190.6727
24200.6697
24210.6341
24220.6660
24230.6216
24240.6897
24250.6605
24260.6176
24270.6422
24280.6559
24290.6737
24300.6744
24310.6880
24320.6912
24330.6469
24340.6528
24350.6536
24360.6845
24370.7060
24380.6850
24390.6053
24400.6131
24410.6560
24420.6625
24430.6614
24440.6404
24450.5982
24460.6891
24470.7089
24480.6947
24490.6572
24500.6792
24510.6901
24520.6447
24530.6605
24540.7051
24550.6795
24560.6142
24570.7086
24580.6184
24590.6393
24600.6546
24610.6719
24620.6244
24630.6546
24640.6453
24650.6903
24660.6328
24670.6628
24680.6724
24690.6398
24700.6171
24710.6659
24720.6283
24730.6363
24740.7057
24750.6760
24760.7002
24770.6031
24780.6942
24790.6138
24800.6526
24810.6495
24820.7112
24830.6132
24840.6883
24850.6351
24860.6720
24870.6893
24880.7023
24890.6596
24900.6554
24910.6548
24920.6793
24930.6535
24940.6744
24950.6523
24960.7037
24970.6689
24980.6443
24990.7003
25000.6517
25010.6695
25020.6352
25030.6335
25040.6591
25050.6577
25060.6497
25070.6386
25080.6600
25090.6314
25100.6699
25110.6572
25120.6365
25130.6858
25140.6538
25150.6571
25160.6564
25170.7132
25180.6932
25190.6417
25200.6518
25210.6599
25220.6768
25230.6803
25240.6384
25250.6227
25260.7036
25270.6704
25280.6949
25290.7021
25300.6703
25310.6768
25320.7093
25330.7142
25340.6396
25350.6474
25360.6749
25370.6644
25380.6733
25390.6890
25400.6408
25410.6939
25420.6902
25430.6458
25440.6850
25450.6778
25460.6617
25470.6245
25480.6920
25490.6564
25500.6649
25510.6822
25520.6638
25530.6633
25540.6315
25550.7130
25560.6473
25570.6763
25580.6967
25590.6256
25600.6588
25610.6457
25620.6689
25630.6782
25640.6429
25650.6528
25660.6362
25670.6460
25680.6826
25690.6693
25700.6294
25710.6575
25720.6524
25730.6754
25740.6725
25750.6437
25760.6717
25770.6578
25780.6735
25790.6686
25800.6729
25810.6340
25820.7143
25830.6386
25840.6833
25850.7058
25860.6497
25870.7224
25880.6500
25890.6630
25900.6515
25910.6617
25920.6582
25930.6550
25940.6417
25950.6544
25960.7142
25970.6658
25980.6693
25990.6934
26000.6946
26010.6430
26020.6661
26030.6882
26040.6906
26050.7213
26060.6289
26070.6177
26080.6380
26090.6913
26100.6431
26110.6776
26120.6455
26130.6818
26140.6619
26150.6506
26160.6829
26170.6757
26180.6882
26190.6751
26200.6404
26210.6691
26220.7080
26230.6987
26240.6300
26250.7022
26260.6433
26270.6570
26280.6460
26290.6866
26300.6373
26310.6786
26320.6453
26330.6749
26340.6504
26350.6742
26360.6758
26370.6391
26380.6607
26390.6506
26400.6099

Measured result

Verbatim from the run's summary.json.

bucket
bucket-p32-t128
bucket_icons
3359
checkpoint_bytes
6463494
checkpoint_round_trip
true
checkpoint_sha256
584565f5646ef154…
config_sha256
1687c4625ce85296…
corruption
factorized_marginal_respecting_geometry
cuda_version
13.4
decision_threshold
0.7842316797885812
deterministic_algorithms
true
device
cuda
eval_every
60
final_train
loss
0.9516589045524597
step
2640
train_token_accuracy
0.6099060014461316
group_vocabulary_size
12
locked_path_exact
true
metrics_sha256
445e5c763f340a77…
model_parameters
525152
schema_version
1
scope
dominant-bucket fixed-topology geometry pilot; not unconditional generation
selected_train_rows
color/svg/1F3F4-E0064-E0065-E0062-E0065-E007F.svg, color/svg/1F468-1F3FC-200D-1F9BC.svg, color/svg/1F561.svg, color/svg/1F469-1F3FE-200D-1F9BC.svg, color/svg/1F469-1F3FD-200D-2764-FE0F-200D-1F48B-200D-1F469-1F3FE.svg, color/svg/1F6B5-1F3FB-200D-2642-FE0F.svg … and 2675 more
selected_validation_rows
color/svg/1F994.svg, color/svg/E30A.svg, color/svg/1F6BE.svg, color/svg/1F3CA-1F3FB-200D-2642-FE0F.svg, color/svg/1F3CB-1F3FC-200D-2642-FE0F.svg, color/svg/1F93D-1F3FC.svg … and 122 more
selection
completed_steps
2640
evals_without_improvement
8
min_delta
0.0
objective
held_out_loss
patience_evals
8
selected_held_out_loss
1.0326451063156128
selected_step
2160
stopped_early
true
steps
6300
study_version
openmoji-g1-derived-threshold-v13
subgroup_vocabulary_size
118
torch_version
2.14.0a0+4fdf77b940.nv26.08
train_value_accuracy
0.27513338949733224
validation
accuracy
0.651869626074785
changed_accuracy
0.04935083566458647
changed_total
13941
loss
1.0326451063156128
retained_accuracy
0.9741051904707101
retained_total
26067
validation_final_step
accuracy
0.5990801839632074
changed_accuracy
0.08184491786815867
changed_total
13941
loss
1.0418919324874878
retained_accuracy
0.8757049142594084
retained_total
26067
validation_trace_sha256
1e3e2c5c755065cf…
validation_untrained
accuracy
0.3340831833633273
changed_accuracy
0.0012911555842479018
changed_total
13941
loss
1.7456328868865967
retained_accuracy
0.5120650631066099
retained_total
26067

State transitions

  1. planned2026-09-21T03:10:00Z
  2. completed2026-09-21T03:20:00Zbreak_even_threshold_beats_identity_with_no_calibration_but_trained_on_q_is_inflated

Run record

Verbatim from runs/openmoji-g1-derived-threshold-v13-7c73fab-1687c462-9b9b1699/run.yaml, the record committed before launch.

schema_version
1
run_id
openmoji-g1-derived-threshold-v13-7c73fab-1687c462-9b9b1699
state
completed
parent_run
openmoji-g1-balanced-objective-v12-9bf3fe9-499c2436-9b9b1699
hypothesis
v12 beat the identity baseline, 0.6567 against 0.6502, but only with a decode threshold fitted on a disjoint half of the held-out icons; at its own argmax decode it lost, 0.6086 against 0.6515. That threshold is derivable rather than arbitrary: keeping a retained field is always right and changing one is right only if the value head re-predicts the same token, so changing pays exactly above p = 1/(1+q) in the value head's accuracy q on corrupted fields. Estimating q on TRAIN icons and decoding at that threshold should beat identity with no calibration on held-out data at all.
expected_information_gain
It converts the project's first win over the trivial policy from one that needed a post-hoc threshold into one the model simply produces. That is the difference between a result that holds up and a result that needs an asterisk, and it costs one run.
scope_limits
Fixed-topology and geometry-only, single seed, marginal-respecting corruption at 0.35, token accuracy rather than renders. The threshold is derived from a first-order break-even argument that assumes changing a retained field is never right; that is nearly but not exactly true, so the derived threshold is principled rather than optimal.
changed_factors
primary
training.derive_decision_threshold false -> true. After training, the value head's accuracy on corrupted TRAIN fields is measured under a corruption draw disjoint from the held-out one, the decode threshold is set to 1/(1+q), and the held-out evaluation is repeated with it.
parameters
unchanged at 525,152
held_fixed
v12 exactly otherwise, including the entropy-balanced value loss weight.
no_training_change
This alters decoding and reporting only. The trained weights are identical to v12's; the checkpoint should match.
code
git_commit
7c73fab
execution_mode
native-local
config
path
configs/learning/openmoji-g1-derived-threshold-v13.yaml
sha256
1687c4625ce85296…
dataset
hybrid_sha256
9b9b1699677a6f97…
bucket
bucket-p32-t128
train_samples
2681
validation_samples
128
baselines
identity_on_the_full_held_out_draw
0.6515
v12_own_argmax_decode
0.6086
v12_with_a_threshold_fitted_on_held_out_icons
calibration_half
0.6568
evaluation_half
0.6567
identity_there
0.6502
v12_value_head_accuracy_on_corrupted_fields
0.1233
predeclared_criteria
evaluated_at
the checkpoint selected by held-out loss, decoded at the derived threshold
primary
id
beats_identity_without_calibration
statement
Held-out aggregate token accuracy at the DERIVED threshold exceeds the identity policy's, computed as retained_total / (changed_total + retained_total) from this run's own totals. No threshold is fitted on held-out data anywhere in this run.
id
improves_on_argmax
statement
Aggregate accuracy exceeds v12's own argmax decode, 0.6086.
id
threshold_is_in_the_expected_region
statement
The derived threshold lies between 0.85 and 0.95. The held-out value accuracy was 0.1233, giving 0.890; a train-split estimate should land near it, and a wildly different value would mean the estimator is measuring something else.
id
structural_safety
statement
Locked-path exactness holds and the canonical checkpoint round-trips.
id
reproducibility
statement
A second complete invocation returns an identical JSON result.
reported_not_gated
the train-split value accuracy, the derived threshold, and the resulting flag rate, changed and retained accuracy, which will move in opposite directions by design
falsification_meaning
If the derived threshold does not beat identity while a fitted one does, the break-even argument is too crude - most likely because changing a retained field is not quite never right, or because the value head's accuracy on train icons does not transfer. Either would be worth knowing, and the fallback is to derive the threshold from a held-out-of-train split rather than from train itself.
outputs
local_metadata
runs/openmoji-g1-derived-threshold-v13-7c73fab-1687c462-9b9b1699
report_root
reports/learning/openmoji-g1-derived-threshold-v13
durable_artifacts
/home/dev/.cache/openmoji-g1-derived-threshold-v13-7c73fab-1687c462-9b9b1699
planned_at
2026-09-21 03:10:00+00:00
completed_at
2026-09-21 03:20:00+00:00
result
device
cuda
train_value_accuracy
0.2751
derived_threshold
0.7842
held_out
aggregate
0.6519
changed
0.0494
retained
0.9741
identity_baseline
0.6515
margin
0.0003
checkpoint_identical_to_v12
true
identical_rerun
value
true
caveat
Its in-place idempotency rerun failed closed at a later commit, because v14 added `calibration_samples` and `calibration_withheld_from_training` to the summary schema after v13 had run, and the artifact writer refuses to replace a differing file. Re-running to a fresh report root at the current commit reproduces every other field exactly, so the run is deterministic and only the schema moved. The failure is recorded rather than papered over.
predeclared_outcome
overall
partially-falsified
beats_identity_without_calibration
passed
true
observed
0.6519
threshold
0.6515
improves_on_argmax
passed
true
observed
0.6519
threshold
0.6086
threshold_is_in_the_expected_region
passed
false
observed
0.7842
expected
0.85, 0.95
structural_safety
passed
true
reproducibility
passed
true
see
the caveat above
conclusion
The break-even threshold works - this beats identity with nothing fitted on held-out data - but by only 0.0003, and the threshold criterion is falsified for an informative reason. The value head scores 0.2751 on icons the model trained on against 0.1233 held out, so estimating q there gives 0.784 where the held-out accuracy implies 0.890, and the model edits more than pays. The fix is the one this run's falsification meaning named: estimate q on icons withheld from training.