MojiDiff

← experiments

openmoji-g1-marginal-corruption-v9-74dc5fd-0ef00ff4-9b9b1699

completed partially-falsified

Hypothesis

v8's detector reached 2.806 precision lift, but a zero-parameter detector that knows only the corpus marginal reaches 2.831 on the same data, so v8 matched a density test rather than learning geometry. Factorized corruption draws replacements uniformly from the full legal vocabulary, so many land on tokens real icons never use. Drawing them from the corpus distribution for the field's role instead removes that shortcut: the marginal detector falls to 0.771 while the local-continuity signal survives at 1.893. Under this leak-free process the model must use the surrounding geometry, and this run measures for the first time how much of that signal it can actually learn.

Why run it

This is the first honest detection benchmark in the project. Every earlier detector number - v6 1.234, v7 1.365, v8 2.806, and the free references at 2.13 and 2.83 - was measured on a process whose corruption is identifiable from a token marginal alone. A trained model that beats the continuity statistic HERE has learned geometry; one that does not has not, and no amount of the earlier numbers would have told us which.

Scope limits

Detection-only, so it reconstructs nothing and held-out accuracy is meaningless here. Single seed, probability 0.35. Not comparable with v8 in absolute terms: the task is strictly harder because the easy fraction of corruptions has been removed, so a lower lift than v8's 2.806 is expected and is not a regression.

Measured behaviour

Held-out loss
lossoptimizer step0.60.811.21.41.6010002000selected
Table view
optimizer stepheld-out loss
01.4055
600.6704
1200.6533
1800.6493
2400.6493
3000.6487
3600.6469
4200.6472
4800.6451
5400.6458
6000.6456
6600.6446
7200.6449
7800.6436
8400.6429
9000.6427
9600.6434
10200.6412
10800.6412
11400.6402
12000.6393
12600.6405
13200.6392
13800.6395
14400.6390
15000.6361
15600.6369
16200.6396
16800.6372
17400.6352
18000.6369
18600.6349
19200.6350
19800.6333
20400.6365
21000.6362
21600.6319
22200.6323
22800.6329
23400.6305
24000.6328
24600.6325
25200.6323
25800.6330
26400.6334
27000.6322
27600.6351
28200.6339
Held-out token accuracy
aggregatechanged fieldsretained fields
accuracyoptimizer step00.250.50.751010002000selected
Table view
optimizer stepaggregatechanged fieldsretained fields
00.26750.00210.4095
600.59550.00040.9138
1200.63460.00010.9740
1800.64450.00000.9891
2400.64910.00000.9962
3000.64880.00000.9957
3600.64900.00000.9960
4200.64760.00000.9940
4800.64840.00000.9952
5400.64340.00000.9875
6000.64550.00000.9907
6600.64120.00000.9841
7200.64540.00010.9906
7800.64660.00000.9925
8400.64500.00000.9899
9000.64420.00000.9887
9600.64330.00010.9872
10200.64140.00010.9844
10800.63970.00010.9818
11400.63660.00010.9770
12000.63660.00020.9770
12600.62900.00030.9653
13200.62560.00020.9601
13800.62700.00010.9623
14400.62070.00010.9525
15000.62470.00040.9586
15600.61900.00020.9499
16200.61210.00030.9393
16800.60140.00010.9230
17400.61260.00020.9400
18000.60630.00030.9304
18600.60510.00040.9285
19200.60280.00050.9250
19800.61260.00020.9402
20400.58420.00040.8965
21000.58800.00040.9023
21600.59690.00060.9157
22200.59660.00050.9154
22800.59750.00020.9169
23400.60260.00030.9247
24000.59130.00060.9072
24600.58870.00040.9033
25200.57760.00040.8863
25800.57730.00060.8858
26400.57770.00050.8864
27000.58530.00040.8981
27600.57330.00080.8794
28200.57010.00060.8747
Training token accuracy (per batch)
accuracyoptimizer step00.20.40.60.810002000selected
Table view
optimizer steptrain accuracy
10.2793
20.2790
30.3806
40.4000
50.4351
60.4741
70.5008
80.5185
90.5284
100.4963
110.4852
120.5032
130.4913
140.4837
150.4645
160.4499
170.4399
180.4599
190.4378
200.4559
210.4292
220.4376
230.4500
240.4659
250.4654
260.4710
270.4789
280.4998
290.4977
300.4929
310.5275
320.5186
330.4926
340.5145
350.5280
360.5219
370.5152
380.5190
390.5241
400.5338
410.5548
420.5574
430.5436
440.5688
450.5438
460.5471
470.5597
480.5418
490.5326
500.5473
510.5526
520.5578
530.5504
540.5623
550.5717
560.5594
570.5836
580.5872
590.6005
600.5905
610.5897
620.6065
630.5987
640.6012
650.6129
660.6041
670.5892
680.6060
690.5996
700.5997
710.6203
720.6118
730.6228
740.6023
750.6211
760.6194
770.6171
780.6233
790.5843
800.6206
810.6156
820.6122
830.6028
840.6101
850.6190
860.6132
870.6062
880.6128
890.6258
900.6257
910.6284
920.6263
930.6145
940.6409
950.6349
960.6240
970.6276
980.6402
990.6346
1000.6262
1010.6424
1020.6309
1030.6368
1040.6317
1050.6471
1060.6318
1070.6358
1080.6294
1090.6546
1100.6407
1110.6407
1120.6394
1130.6348
1140.6441
1150.6458
1160.6444
1170.6344
1180.6444
1190.6317
1200.6538
1210.6308
1220.6557
1230.6318
1240.6423
1250.6528
1260.6444
1270.6380
1280.6443
1290.6406
1300.6419
1310.6325
1320.6308
1330.6318
1340.6360
1350.6186
1360.6221
1370.6212
1380.6440
1390.6358
1400.6281
1410.6399
1420.6425
1430.6377
1440.6342
1450.6413
1460.6452
1470.6469
1480.6386
1490.6422
1500.6308
1510.6451
1520.6339
1530.6376
1540.6302
1550.6601
1560.6402
1570.6320
1580.6423
1590.6365
1600.6461
1610.6388
1620.6414
1630.6519
1640.6382
1650.6343
1660.6379
1670.6420
1680.6386
1690.6520
1700.6514
1710.6499
1720.6497
1730.6488
1740.6399
1750.6452
1760.6368
1770.6391
1780.6341
1790.6512
1800.6372
1810.6371
1820.6419
1830.6422
1840.6496
1850.6567
1860.6498
1870.6414
1880.6374
1890.6398
1900.6477
1910.6467
1920.6405
1930.6494
1940.6381
1950.6497
1960.6460
1970.6337
1980.6343
1990.6412
2000.6497
2010.6437
2020.6438
2030.6598
2040.6525
2050.6426
2060.6482
2070.6513
2080.6607
2090.6530
2100.6530
2110.6359
2120.6537
2130.6472
2140.6347
2150.6433
2160.6314
2170.6396
2180.6401
2190.6520
2200.6386
2210.6454
2220.6429
2230.6500
2240.6559
2250.6602
2260.6433
2270.6367
2280.6372
2290.6468
2300.6469
2310.6294
2320.6412
2330.6479
2340.6445
2350.6269
2360.6499
2370.6375
2380.6504
2390.6551
2400.6533
2410.6393
2420.6427
2430.6581
2440.6397
2450.6488
2460.6482
2470.6350
2480.6520
2490.6401
2500.6464
2510.6536
2520.6528
2530.6500
2540.6562
2550.6498
2560.6516
2570.6526
2580.6442
2590.6631
2600.6412
2610.6343
2620.6457
2630.6520
2640.6427
2650.6538
2660.6405
2670.6536
2680.6386
2690.6440
2700.6469
2710.6375
2720.6563
2730.6397
2740.6329
2750.6293
2760.6443
2770.6468
2780.6368
2790.6540
2800.6374
2810.6477
2820.6547
2830.6414
2840.6547
2850.6495
2860.6524
2870.6418
2880.6413
2890.6551
2900.6561
2910.6357
2920.6488
2930.6475
2940.6521
2950.6446
2960.6487
2970.6506
2980.6534
2990.6449
3000.6506
3010.6562
3020.6537
3030.6419
3040.6455
3050.6270
3060.6390
3070.6392
3080.6484
3090.6474
3100.6356
3110.6340
3120.6578
3130.6548
3140.6547
3150.6407
3160.6441
3170.6513
3180.6471
3190.6432
3200.6587
3210.6462
3220.6425
3230.6343
3240.6409
3250.6344
3260.6444
3270.6463
3280.6426
3290.6549
3300.6389
3310.6407
3320.6413
3330.6495
3340.6399
3350.6337
3360.6572
3370.6344
3380.6533
3390.6480
3400.6406
3410.6438
3420.6422
3430.6471
3440.6495
3450.6506
3460.6454
3470.6523
3480.6333
3490.6500
3500.6545
3510.6526
3520.6455
3530.6362
3540.6485
3550.6454
3560.6452
3570.6504
3580.6382
3590.6339
3600.6467
3610.6509
3620.6431
3630.6551
3640.6376
3650.6402
3660.6530
3670.6512
3680.6379
3690.6384
3700.6415
3710.6608
3720.6409
3730.6536
3740.6477
3750.6482
3760.6460
3770.6471
3780.6381
3790.6518
3800.6485
3810.6404
3820.6450
3830.6392
3840.6396
3850.6350
3860.6416
3870.6431
3880.6491
3890.6348
3900.6420
3910.6462
3920.6576
3930.6306
3940.6596
3950.6356
3960.6550
3970.6592
3980.6458
3990.6498
4000.6402
4010.6477
4020.6497
4030.6463
4040.6509
4050.6481
4060.6469
4070.6353
4080.6542
4090.6509
4100.6459
4110.6373
4120.6475
4130.6447
4140.6398
4150.6408
4160.6412
4170.6457
4180.6437
4190.6401
4200.6485
4210.6523
4220.6538
4230.6425
4240.6539
4250.6467
4260.6467
4270.6472
4280.6417
4290.6422
4300.6517
4310.6462
4320.6582
4330.6458
4340.6499
4350.6500
4360.6365
4370.6522
4380.6533
4390.6404
4400.6587
4410.6477
4420.6431
4430.6439
4440.6455
4450.6477
4460.6384
4470.6453
4480.6406
4490.6425
4500.6259
4510.6352
4520.6487
4530.6513
4540.6435
4550.6528
4560.6583
4570.6561
4580.6370
4590.6356
4600.6483
4610.6450
4620.6442
4630.6422
4640.6459
4650.6449
4660.6407
4670.6383
4680.6514
4690.6455
4700.6533
4710.6546
4720.6453
4730.6619
4740.6313
4750.6478
4760.6491
4770.6440
4780.6409
4790.6445
4800.6556
4810.6418
4820.6582
4830.6552
4840.6556
4850.6401
4860.6581
4870.6544
4880.6416
4890.6537
4900.6479
4910.6353
4920.6395
4930.6425
4940.6415
4950.6427
4960.6466
4970.6463
4980.6398
4990.6464
5000.6619
5010.6539
5020.6515
5030.6422
5040.6525
5050.6363
5060.6437
5070.6413
5080.6544
5090.6469
5100.6380
5110.6443
5120.6558
5130.6502
5140.6446
5150.6443
5160.6531
5170.6459
5180.6442
5190.6441
5200.6519
5210.6362
5220.6545
5230.6506
5240.6440
5250.6494
5260.6611
5270.6302
5280.6556
5290.6511
5300.6499
5310.6405
5320.6603
5330.6541
5340.6562
5350.6458
5360.6367
5370.6468
5380.6431
5390.6432
5400.6349
5410.6454
5420.6422
5430.6339
5440.6504
5450.6475
5460.6317
5470.6481
5480.6273
5490.6530
5500.6321
5510.6358
5520.6410
5530.6364
5540.6474
5550.6365
5560.6350
5570.6458
5580.6314
5590.6584
5600.6390
5610.6486
5620.6425
5630.6399
5640.6459
5650.6484
5660.6383
5670.6409
5680.6414
5690.6517
5700.6379
5710.6454
5720.6610
5730.6542
5740.6481
5750.6484
5760.6526
5770.6347
5780.6446
5790.6496
5800.6346
5810.6549
5820.6426
5830.6506
5840.6381
5850.6413
5860.6460
5870.6451
5880.6579
5890.6339
5900.6552
5910.6480
5920.6366
5930.6504
5940.6474
5950.6397
5960.6477
5970.6372
5980.6412
5990.6460
6000.6525
6010.6482
6020.6605
6030.6471
6040.6493
6050.6615
6060.6456
6070.6575
6080.6432
6090.6349
6100.6292
6110.6358
6120.6466
6130.6449
6140.6620
6150.6467
6160.6484
6170.6384
6180.6427
6190.6449
6200.6447
6210.6381
6220.6432
6230.6302
6240.6463
6250.6509
6260.6486
6270.6478
6280.6408
6290.6487
6300.6623
6310.6559
6320.6432
6330.6496
6340.6445
6350.6442
6360.6446
6370.6443
6380.6409
6390.6464
6400.6344
6410.6477
6420.6398
6430.6526
6440.6387
6450.6442
6460.6367
6470.6443
6480.6463
6490.6445
6500.6521
6510.6491
6520.6556
6530.6623
6540.6460
6550.6467
6560.6424
6570.6462
6580.6492
6590.6359
6600.6352
6610.6413
6620.6440
6630.6375
6640.6484
6650.6479
6660.6450
6670.6447
6680.6335
6690.6323
6700.6548
6710.6458
6720.6437
6730.6466
6740.6553
6750.6386
6760.6490
6770.6421
6780.6360
6790.6486
6800.6445
6810.6355
6820.6411
6830.6410
6840.6537
6850.6476
6860.6414
6870.6491
6880.6389
6890.6436
6900.6440
6910.6420
6920.6572
6930.6406
6940.6452
6950.6482
6960.6392
6970.6371
6980.6475
6990.6374
7000.6333
7010.6454
7020.6487
7030.6381
7040.6519
7050.6393
7060.6529
7070.6435
7080.6304
7090.6536
7100.6420
7110.6491
7120.6410
7130.6468
7140.6492
7150.6559
7160.6561
7170.6529
7180.6392
7190.6443
7200.6465
7210.6479
7220.6464
7230.6405
7240.6410
7250.6419
7260.6526
7270.6388
7280.6484
7290.6466
7300.6473
7310.6413
7320.6448
7330.6533
7340.6298
7350.6446
7360.6538
7370.6456
7380.6446
7390.6508
7400.6435
7410.6492
7420.6483
7430.6428
7440.6460
7450.6279
7460.6494
7470.6527
7480.6501
7490.6273
7500.6361
7510.6498
7520.6463
7530.6325
7540.6428
7550.6422
7560.6501
7570.6471
7580.6330
7590.6531
7600.6395
7610.6643
7620.6465
7630.6517
7640.6427
7650.6530
7660.6554
7670.6591
7680.6443
7690.6394
7700.6366
7710.6418
7720.6341
7730.6516
7740.6554
7750.6368
7760.6361
7770.6620
7780.6560
7790.6526
7800.6378
7810.6430
7820.6441
7830.6454
7840.6558
7850.6382
7860.6409
7870.6543
7880.6587
7890.6379
7900.6399
7910.6482
7920.6396
7930.6211
7940.6435
7950.6448
7960.6467
7970.6490
7980.6427
7990.6569
8000.6435
8010.6353
8020.6425
8030.6466
8040.6445
8050.6330
8060.6344
8070.6423
8080.6472
8090.6412
8100.6544
8110.6395
8120.6419
8130.6397
8140.6393
8150.6387
8160.6343
8170.6449
8180.6460
8190.6442
8200.6447
8210.6565
8220.6449
8230.6536
8240.6415
8250.6332
8260.6524
8270.6274
8280.6327
8290.6420
8300.6497
8310.6464
8320.6500
8330.6431
8340.6403
8350.6425
8360.6300
8370.6349
8380.6374
8390.6492
8400.6538
8410.6512
8420.6460
8430.6382
8440.6536
8450.6432
8460.6475
8470.6492
8480.6609
8490.6375
8500.6588
8510.6339
8520.6474
8530.6420
8540.6519
8550.6406
8560.6546
8570.6344
8580.6374
8590.6465
8600.6397
8610.6329
8620.6298
8630.6508
8640.6456
8650.6464
8660.6378
8670.6455
8680.6484
8690.6436
8700.6446
8710.6458
8720.6520
8730.6402
8740.6393
8750.6444
8760.6376
8770.6398
8780.6424
8790.6439
8800.6404
8810.6394
8820.6472
8830.6523
8840.6489
8850.6340
8860.6472
8870.6439
8880.6473
8890.6374
8900.6356
8910.6400
8920.6489
8930.6497
8940.6291
8950.6496
8960.6481
8970.6451
8980.6507
8990.6514
9000.6521
9010.6460
9020.6619
9030.6303
9040.6488
9050.6447
9060.6462
9070.6446
9080.6529
9090.6363
9100.6432
9110.6370
9120.6417
9130.6329
9140.6382
9150.6476
9160.6337
9170.6547
9180.6447
9190.6551
9200.6291
9210.6523
9220.6430
9230.6413
9240.6505
9250.6362
9260.6285
9270.6287
9280.6475
9290.6498
9300.6328
9310.6499
9320.6461
9330.6325
9340.6437
9350.6471
9360.6414
9370.6482
9380.6608
9390.6353
9400.6446
9410.6451
9420.6495
9430.6445
9440.6428
9450.6430
9460.6451
9470.6359
9480.6484
9490.6470
9500.6514
9510.6400
9520.6373
9530.6392
9540.6454
9550.6450
9560.6353
9570.6423
9580.6338
9590.6453
9600.6464
9610.6316
9620.6487
9630.6503
9640.6497
9650.6444
9660.6382
9670.6485
9680.6485
9690.6414
9700.6369
9710.6274
9720.6399
9730.6359
9740.6483
9750.6386
9760.6385
9770.6380
9780.6475
9790.6393
9800.6408
9810.6481
9820.6461
9830.6285
9840.6560
9850.6433
9860.6512
9870.6392
9880.6303
9890.6505
9900.6394
9910.6418
9920.6439
9930.6580
9940.6382
9950.6530
9960.6453
9970.6469
9980.6279
9990.6459
10000.6487
10010.6337
10020.6422
10030.6267
10040.6413
10050.6351
10060.6376
10070.6507
10080.6444
10090.6438
10100.6417
10110.6253
10120.6421
10130.6450
10140.6392
10150.6365
10160.6396
10170.6411
10180.6431
10190.6311
10200.6530
10210.6365
10220.6423
10230.6484
10240.6487
10250.6500
10260.6458
10270.6367
10280.6515
10290.6408
10300.6386
10310.6390
10320.6329
10330.6411
10340.6408
10350.6448
10360.6419
10370.6378
10380.6536
10390.6471
10400.6365
10410.6567
10420.6407
10430.6353
10440.6458
10450.6399
10460.6403
10470.6411
10480.6433
10490.6430
10500.6446
10510.6388
10520.6341
10530.6298
10540.6276
10550.6388
10560.6385
10570.6281
10580.6342
10590.6393
10600.6405
10610.6490
10620.6431
10630.6384
10640.6433
10650.6410
10660.6249
10670.6437
10680.6410
10690.6313
10700.6286
10710.6353
10720.6234
10730.6572
10740.6400
10750.6557
10760.6528
10770.6393
10780.6455
10790.6572
10800.6411
10810.6327
10820.6441
10830.6468
10840.6354
10850.6483
10860.6528
10870.6455
10880.6533
10890.6457
10900.6487
10910.6385
10920.6304
10930.6349
10940.6492
10950.6347
10960.6207
10970.6546
10980.6421
10990.6437
11000.6493
11010.6376
11020.6335
11030.6465
11040.6328
11050.6389
11060.6432
11070.6368
11080.6478
11090.6434
11100.6385
11110.6370
11120.6358
11130.6523
11140.6398
11150.6473
11160.6457
11170.6473
11180.6476
11190.6529
11200.6403
11210.6455
11220.6408
11230.6355
11240.6318
11250.6408
11260.6338
11270.6431
11280.6279
11290.6381
11300.6463
11310.6318
11320.6494
11330.6245
11340.6462
11350.6285
11360.6425
11370.6387
11380.6261
11390.6318
11400.6451
11410.6399
11420.6308
11430.6315
11440.6367
11450.6284
11460.6411
11470.6327
11480.6281
11490.6386
11500.6391
11510.6499
11520.6465
11530.6398
11540.6381
11550.6305
11560.6444
11570.6388
11580.6368
11590.6349
11600.6228
11610.6337
11620.6422
11630.6345
11640.6385
11650.6367
11660.6442
11670.6464
11680.6493
11690.6388
11700.6543
11710.6440
11720.6239
11730.6409
11740.6219
11750.6227
11760.6383
11770.6257
11780.6246
11790.6261
11800.6319
11810.6323
11820.6410
11830.6305
11840.6439
11850.6414
11860.6474
11870.6513
11880.6429
11890.6447
11900.6250
11910.6294
11920.6374
11930.6437
11940.6438
11950.6374
11960.6494
11970.6457
11980.6454
11990.6319
12000.6416
12010.6417
12020.6303
12030.6369
12040.6528
12050.6346
12060.6416
12070.6394
12080.6531
12090.6472
12100.6320
12110.6509
12120.6344
12130.6411
12140.6375
12150.6498
12160.6392
12170.6450
12180.6475
12190.6438
12200.6369
12210.6528
12220.6437
12230.6408
12240.6423
12250.6371
12260.6364
12270.6524
12280.6460
12290.6276
12300.6422
12310.6265
12320.6415
12330.6343
12340.6431
12350.6363
12360.6408
12370.6414
12380.6404
12390.6439
12400.6364
12410.6391
12420.6378
12430.6475
12440.6277
12450.6416
12460.6306
12470.6291
12480.6390
12490.6469
12500.6327
12510.6423
12520.6184
12530.6400
12540.6277
12550.6266
12560.6338
12570.6316
12580.6282
12590.6217
12600.6272
12610.6223
12620.6257
12630.6304
12640.6348
12650.6408
12660.6372
12670.6362
12680.6374
12690.6422
12700.6435
12710.6239
12720.6409
12730.6245
12740.6427
12750.6291
12760.6472
12770.6240
12780.6325
12790.6343
12800.6336
12810.6355
12820.6359
12830.6188
12840.6458
12850.6255
12860.6288
12870.6159
12880.6390
12890.6248
12900.6368
12910.6329
12920.6290
12930.6180
12940.6311
12950.6426
12960.6345
12970.6463
12980.6254
12990.6310
13000.6250
13010.6339
13020.6251
13030.6249
13040.6271
13050.6211
13060.6454
13070.6307
13080.6468
13090.6382
13100.6390
13110.6334
13120.6294
13130.6429
13140.6418
13150.6464
13160.6364
13170.6417
13180.6404
13190.6350
13200.6324
13210.6377
13220.6450
13230.6311
13240.6258
13250.6311
13260.6293
13270.6324
13280.6301
13290.6240
13300.6442
13310.6278
13320.6310
13330.6366
13340.6341
13350.6440
13360.6308
13370.6381
13380.6467
13390.6403
13400.6469
13410.6398
13420.6442
13430.6310
13440.6341
13450.6296
13460.6290
13470.6276
13480.6387
13490.6273
13500.6375
13510.6506
13520.6374
13530.6485
13540.6423
13550.6247
13560.6386
13570.6353
13580.6303
13590.6291
13600.6370
13610.6273
13620.6285
13630.6453
13640.6397
13650.6483
13660.6467
13670.6333
13680.6251
13690.6511
13700.6370
13710.6365
13720.6283
13730.6229
13740.6369
13750.6415
13760.6340
13770.6323
13780.6185
13790.6303
13800.6263
13810.6248
13820.6365
13830.6325
13840.6190
13850.6185
13860.6283
13870.6185
13880.6176
13890.6232
13900.6326
13910.6380
13920.6258
13930.6334
13940.6451
13950.6357
13960.6391
13970.6211
13980.6339
13990.6132
14000.6354
14010.6248
14020.6262
14030.6192
14040.6343
14050.6376
14060.6331
14070.6418
14080.6369
14090.6430
14100.6429
14110.6362
14120.6181
14130.6412
14140.6131
14150.6344
14160.6318
14170.6298
14180.6298
14190.6300
14200.6264
14210.6303
14220.6365
14230.6195
14240.6275
14250.6234
14260.6268
14270.6337
14280.6311
14290.6145
14300.6188
14310.6293
14320.6330
14330.6187
14340.6345
14350.6316
14360.6259
14370.6382
14380.6218
14390.6270
14400.6184
14410.6248
14420.6182
14430.6372
14440.6321
14450.6231
14460.6267
14470.6194
14480.6443
14490.6267
14500.6326
14510.6286
14520.6180
14530.6390
14540.6138
14550.6326
14560.6323
14570.6308
14580.6316
14590.6199
14600.6229
14610.6399
14620.6354
14630.6381
14640.6334
14650.6330
14660.6272
14670.6233
14680.6241
14690.6341
14700.6281
14710.6085
14720.6382
14730.6352
14740.6205
14750.6399
14760.6261
14770.6366
14780.6224
14790.6352
14800.6273
14810.6282
14820.6260
14830.6232
14840.6325
14850.6456
14860.6233
14870.6192
14880.6154
14890.6269
14900.6189
14910.6165
14920.6129
14930.6285
14940.6404
14950.6303
14960.6245
14970.6265
14980.6440
14990.6339
15000.6242
15010.6292
15020.6333
15030.6286
15040.6293
15050.6270
15060.6347
15070.6394
15080.6294
15090.6447
15100.6422
15110.6384
15120.6205
15130.6345
15140.6144
15150.6328
15160.6432
15170.6284
15180.6292
15190.6109
15200.6344
15210.6275
15220.6322
15230.6249
15240.6306
15250.6403
15260.6371
15270.6291
15280.6439
15290.6320
15300.6321
15310.6364
15320.6328
15330.6354
15340.6196
15350.6306
15360.6194
15370.6280
15380.6194
15390.6113
15400.6140
15410.6186
15420.6163
15430.6280
15440.6126
15450.6397
15460.6166
15470.6152
15480.6097
15490.6220
15500.6257
15510.6150
15520.6349
15530.6205
15540.6329
15550.6352
15560.6328
15570.6311
15580.6256
15590.6270
15600.6242
15610.6281
15620.6254
15630.6211
15640.6146
15650.6207
15660.6152
15670.6301
15680.6359
15690.6334
15700.6313
15710.6279
15720.6311
15730.6348
15740.6287
15750.6112
15760.6138
15770.6052
15780.6048
15790.6175
15800.6152
15810.6230
15820.6389
15830.6310
15840.6285
15850.6415
15860.6284
15870.6358
15880.6363
15890.6280
15900.6219
15910.6187
15920.6125
15930.6250
15940.6096
15950.6271
15960.6141
15970.6126
15980.6238
15990.6254
16000.6300
16010.6366
16020.6276
16030.6377
16040.6350
16050.6304
16060.6220
16070.6331
16080.6267
16090.6230
16100.6332
16110.6125
16120.6287
16130.6219
16140.6127
16150.6335
16160.6364
16170.6277
16180.6153
16190.6263
16200.6462
16210.6234
16220.6106
16230.6289
16240.6313
16250.6233
16260.6276
16270.6225
16280.6311
16290.6388
16300.6138
16310.6314
16320.6372
16330.6260
16340.6362
16350.6336
16360.6133
16370.6205
16380.6041
16390.6095
16400.6267
16410.6253
16420.6177
16430.6250
16440.6245
16450.6171
16460.6253
16470.6338
16480.6143
16490.6207
16500.6217
16510.6231
16520.6261
16530.6329
16540.6240
16550.6381
16560.6455
16570.6367
16580.6239
16590.6288
16600.6373
16610.6164
16620.6173
16630.6104
16640.6270
16650.6142
16660.6272
16670.6195
16680.6271
16690.6197
16700.6316
16710.6222
16720.6088
16730.6245
16740.6369
16750.6221
16760.6103
16770.6261
16780.6218
16790.6097
16800.6064
16810.6089
16820.5994
16830.6145
16840.6110
16850.6206
16860.6274
16870.6250
16880.6168
16890.6240
16900.6284
16910.6152
16920.6260
16930.6142
16940.6231
16950.6130
16960.6146
16970.6219
16980.6335
16990.6225
17000.6133
17010.6423
17020.6191
17030.6354
17040.6211
17050.6271
17060.6193
17070.6185
17080.6192
17090.6190
17100.6284
17110.6355
17120.6234
17130.6148
17140.6032
17150.5952
17160.6272
17170.6302
17180.6323
17190.6124
17200.6149
17210.6183
17220.6156
17230.6246
17240.6137
17250.6249
17260.6370
17270.6355
17280.6331
17290.6295
17300.6329
17310.6307
17320.6311
17330.6182
17340.6186
17350.6325
17360.6307
17370.6188
17380.6272
17390.6247
17400.6170
17410.6211
17420.6395
17430.6224
17440.6072
17450.6268
17460.6246
17470.6150
17480.6148
17490.6159
17500.6180
17510.6362
17520.6137
17530.6102
17540.6063
17550.6232
17560.6209
17570.6310
17580.6220
17590.6190
17600.6248
17610.6274
17620.6240
17630.6196
17640.6263
17650.6182
17660.6166
17670.6230
17680.6315
17690.6122
17700.6269
17710.6067
17720.6093
17730.6259
17740.6224
17750.6352
17760.6266
17770.6111
17780.6189
17790.6123
17800.6266
17810.6241
17820.6186
17830.6085
17840.6105
17850.6014
17860.6295
17870.6342
17880.6327
17890.6337
17900.6192
17910.6294
17920.6214
17930.6299
17940.6165
17950.6179
17960.6313
17970.6305
17980.6311
17990.6360
18000.6239
18010.6330
18020.6250
18030.6270
18040.6186
18050.6285
18060.6039
18070.6340
18080.6314
18090.6069
18100.6305
18110.6236
18120.6198
18130.6029
18140.6169
18150.6212
18160.6214
18170.6118
18180.6165
18190.6004
18200.6236
18210.6285
18220.6107
18230.6278
18240.6258
18250.6239
18260.6046
18270.5988
18280.6118
18290.6111
18300.5964
18310.6061
18320.6045
18330.6178
18340.6186
18350.6239
18360.6339
18370.6189
18380.6275
18390.6229
18400.6223
18410.6277
18420.6303
18430.6229
18440.6175
18450.6182
18460.6250
18470.6141
18480.6210
18490.6021
18500.6216
18510.6113
18520.6249
18530.6122
18540.6236
18550.6068
18560.6199
18570.6132
18580.5974
18590.6196
18600.6133
18610.6114
18620.6054
18630.6241
18640.6244
18650.6293
18660.6156
18670.6127
18680.6214
18690.6117
18700.6239
18710.6158
18720.6087
18730.6137
18740.5999
18750.6119
18760.6250
18770.6126
18780.6167
18790.6009
18800.6255
18810.6216
18820.6024
18830.6025
18840.6332
18850.6055
18860.6098
18870.6110
18880.6221
18890.6189
18900.6215
18910.6289
18920.6307
18930.6206
18940.6259
18950.6139
18960.6276
18970.6004
18980.6214
18990.6233
19000.6061
19010.5980
19020.6129
19030.6077
19040.6198
19050.6127
19060.6134
19070.6224
19080.6163
19090.6233
19100.6243
19110.6092
19120.6085
19130.6184
19140.6121
19150.6157
19160.6111
19170.6209
19180.6213
19190.6088
19200.6234
19210.6208
19220.6258
19230.6156
19240.6257
19250.6185
19260.6279
19270.6200
19280.6190
19290.6152
19300.6127
19310.6255
19320.6135
19330.6184
19340.6210
19350.6127
19360.6208
19370.6256
19380.6260
19390.6078
19400.6276
19410.6277
19420.6345
19430.5981
19440.6277
19450.6436
19460.6151
19470.6158
19480.6277
19490.6251
19500.6138
19510.6239
19520.6126
19530.6138
19540.6256
19550.6131
19560.6285
19570.6232
19580.6338
19590.6219
19600.6208
19610.5935
19620.5959
19630.6184
19640.6174
19650.6163
19660.6395
19670.6234
19680.6372
19690.6283
19700.6231
19710.5932
19720.6152
19730.5985
19740.5926
19750.6105
19760.6001
19770.5983
19780.6218
19790.6129
19800.6252
19810.6237
19820.6349
19830.6207
19840.6181
19850.6104
19860.6035
19870.6230
19880.6139
19890.6182
19900.6124
19910.6300
19920.6224
19930.6095
19940.6058
19950.6083
19960.5967
19970.6115
19980.5930
19990.6072
20000.5987
20010.6105
20020.6326
20030.6183
20040.6242
20050.6240
20060.6209
20070.6269
20080.6191
20090.6387
20100.6272
20110.6122
20120.6159
20130.6120
20140.6157
20150.6006
20160.6142
20170.5873
20180.6137
20190.6322
20200.6106
20210.6320
20220.6375
20230.6256
20240.6284
20250.6332
20260.6068
20270.6220
20280.5948
20290.6295
20300.6075
20310.6393
20320.6243
20330.6332
20340.6090
20350.6115
20360.6278
20370.6001
20380.6170
20390.5969
20400.5969
20410.6115
20420.6027
20430.6162
20440.6235
20450.6234
20460.6437
20470.6378
20480.6240
20490.6271
20500.6167
20510.6106
20520.6148
20530.6132
20540.6052
20550.6127
20560.6056
20570.6137
20580.6227
20590.6149
20600.6318
20610.6327
20620.6234
20630.6227
20640.6125
20650.6158
20660.6112
20670.6125
20680.6113
20690.6082
20700.6331
20710.6187
20720.6121
20730.6133
20740.6131
20750.6193
20760.6066
20770.6290
20780.6443
20790.6125
20800.6216
20810.6238
20820.6187
20830.6121
20840.6140
20850.6016
20860.6275
20870.6176
20880.6046
20890.6105
20900.6284
20910.6172
20920.6330
20930.6082
20940.6160
20950.6078
20960.6131
20970.5958
20980.5983
20990.5996
21000.6087
21010.5976
21020.6052
21030.6029
21040.6228
21050.5990
21060.6127
21070.6085
21080.6122
21090.5930
21100.6114
21110.6073
21120.6038
21130.6099
21140.6210
21150.6106
21160.6231
21170.5987
21180.6106
21190.5974
21200.5992
21210.6013
21220.6109
21230.6101
21240.6107
21250.6153
21260.6118
21270.5922
21280.6180
21290.5998
21300.5960
21310.6176
21320.6079
21330.6150
21340.6002
21350.5960
21360.6181
21370.6046
21380.6165
21390.6034
21400.6254
21410.6193
21420.6317
21430.6231
21440.5991
21450.6252
21460.6227
21470.6078
21480.6038
21490.5959
21500.6162
21510.5969
21520.6006
21530.6046
21540.6205
21550.6277
21560.6334
21570.6281
21580.6130
21590.6303
21600.6370
21610.6062
21620.5966
21630.6169
21640.6137
21650.6107
21660.6215
21670.6046
21680.6339
21690.6211
21700.6280
21710.6263
21720.6271
21730.6212
21740.6036
21750.6090
21760.5995
21770.6175
21780.6100
21790.6173
21800.6151
21810.6249
21820.6108
21830.6087
21840.6205
21850.6024
21860.5941
21870.6000
21880.6036
21890.6088
21900.6095
21910.6127
21920.6261
21930.6161
21940.6075
21950.6197
21960.6034
21970.6089
21980.5934
21990.6120
22000.6089
22010.6135
22020.6205
22030.6251
22040.6150
22050.6114
22060.6285
22070.5998
22080.5931
22090.5981
22100.6023
22110.6126
22120.6153
22130.6161
22140.6244
22150.6136
22160.6337
22170.6224
22180.6162
22190.6459
22200.6138
22210.6158
22220.6100
22230.6110
22240.6151
22250.6018
22260.5994
22270.6119
22280.6185
22290.6163
22300.6177
22310.6152
22320.6120
22330.6296
22340.6162
22350.5940
22360.6065
22370.6106
22380.6049
22390.6157
22400.6070
22410.6064
22420.6128
22430.6052
22440.6063
22450.6060
22460.6086
22470.6122
22480.6050
22490.5949
22500.5968
22510.5971
22520.6029
22530.6041
22540.6194
22550.6252
22560.6127
22570.6188
22580.6171
22590.6220
22600.6087
22610.6157
22620.6065
22630.6155
22640.5920
22650.6030
22660.5976
22670.6018
22680.6020
22690.6145
22700.5894
22710.5877
22720.6240
22730.6050
22740.6132
22750.6119
22760.6013
22770.6124
22780.6075
22790.6161
22800.6191
22810.6239
22820.6006
22830.6155
22840.6238
22850.6182
22860.6116
22870.5988
22880.6092
22890.6012
22900.6116
22910.6224
22920.6212
22930.6316
22940.6300
22950.6262
22960.5969
22970.6065
22980.6154
22990.6188
23000.6155
23010.6238
23020.6130
23030.6253
23040.6124
23050.6205
23060.6009
23070.6150
23080.6085
23090.6130
23100.6133
23110.5937
23120.6024
23130.6050
23140.6138
23150.6233
23160.6174
23170.6151
23180.6197
23190.6048
23200.6073
23210.6003
23220.6145
23230.6201
23240.6072
23250.6023
23260.6317
23270.6289
23280.6234
23290.6154
23300.6068
23310.6065
23320.6270
23330.6038
23340.6080
23350.5919
23360.6142
23370.5999
23380.6103
23390.6234
23400.6075
23410.6390
23420.6100
23430.6194
23440.6374
23450.6298
23460.6008
23470.6175
23480.5985
23490.6099
23500.6087
23510.6147
23520.5982
23530.6042
23540.6055
23550.6146
23560.6102
23570.6143
23580.6157
23590.6215
23600.6184
23610.6086
23620.6117
23630.6203
23640.6037
23650.6055
23660.6231
23670.6163
23680.6279
23690.6054
23700.6162
23710.6060
23720.5940
23730.6076
23740.6046
23750.6118
23760.6008
23770.6117
23780.6160
23790.6424
23800.6320
23810.6150
23820.6106
23830.6189
23840.6135
23850.6117
23860.5996
23870.5986
23880.6178
23890.6128
23900.6261
23910.6011
23920.6122
23930.6156
23940.6208
23950.6125
23960.6121
23970.6075
23980.6100
23990.6096
24000.6344
24010.6062
24020.6192
24030.6087
24040.6122
24050.6114
24060.6103
24070.6090
24080.6163
24090.6078
24100.6083
24110.6109
24120.5959
24130.6062
24140.6024
24150.6041
24160.5915
24170.5915
24180.5991
24190.5950
24200.6121
24210.6120
24220.6161
24230.6135
24240.6111
24250.6138
24260.6003
24270.5992
24280.5802
24290.6086
24300.5980
24310.6145
24320.6091
24330.6138
24340.6064
24350.6105
24360.6160
24370.6116
24380.6082
24390.6126
24400.6021
24410.6054
24420.6030
24430.6023
24440.5964
24450.6029
24460.6182
24470.6214
24480.6150
24490.6400
24500.5968
24510.6211
24520.6164
24530.6142
24540.6129
24550.5893
24560.5918
24570.6039
24580.6014
24590.6100
24600.6224
24610.6290
24620.6135
24630.6097
24640.6108
24650.6052
24660.6181
24670.6163
24680.6093
24690.6071
24700.6147
24710.6193
24720.6065
24730.6035
24740.6029
24750.6061
24760.5906
24770.5983
24780.6116
24790.5929
24800.6022
24810.6036
24820.6018
24830.5984
24840.6087
24850.6275
24860.6042
24870.6125
24880.6097
24890.6034
24900.6369
24910.6177
24920.6077
24930.6107
24940.6239
24950.6179
24960.6040
24970.6022
24980.5940
24990.6046
25000.6187
25010.6144
25020.5982
25030.6270
25040.6068
25050.6145
25060.6283
25070.6123
25080.6099
25090.6036
25100.5989
25110.6071
25120.6311
25130.6182
25140.6214
25150.6214
25160.6370
25170.6217
25180.6082
25190.6282
25200.6044
25210.5947
25220.6255
25230.6006
25240.6234
25250.6026
25260.5960
25270.6085
25280.5929
25290.5927
25300.6172
25310.6050
25320.6024
25330.5970
25340.6087
25350.6081
25360.5960
25370.6164
25380.6155
25390.6063
25400.6023
25410.5941
25420.5969
25430.5907
25440.5896
25450.5980
25460.5962
25470.6101
25480.6022
25490.6057
25500.6099
25510.6160
25520.6170
25530.6105
25540.6175
25550.6019
25560.6079
25570.5993
25580.6038
25590.5988
25600.6132
25610.6121
25620.6052
25630.6098
25640.5969
25650.5972
25660.6183
25670.6063
25680.6100
25690.6102
25700.5819
25710.6094
25720.6042
25730.6038
25740.5957
25750.6052
25760.5944
25770.6146
25780.6087
25790.5947
25800.6017
25810.6189
25820.6147
25830.6018
25840.5912
25850.5896
25860.6065
25870.6057
25880.6196
25890.6242
25900.6028
25910.6142
25920.6121
25930.6006
25940.5860
25950.6083
25960.6015
25970.6136
25980.6131
25990.6019
26000.6285
26010.6300
26020.6120
26030.6121
26040.6128
26050.6000
26060.5925
26070.6249
26080.6231
26090.6158
26100.6177
26110.6206
26120.6071
26130.6153
26140.6012
26150.6036
26160.5948
26170.6152
26180.6039
26190.6234
26200.6241
26210.6189
26220.6251
26230.6244
26240.6240
26250.5892
26260.6171
26270.6138
26280.6146
26290.5948
26300.6119
26310.6006
26320.6070
26330.6221
26340.6223
26350.6304
26360.6042
26370.6116
26380.6072
26390.6101
26400.5953
26410.6000
26420.6152
26430.6027
26440.6121
26450.6024
26460.6034
26470.5932
26480.6182
26490.5914
26500.6226
26510.6092
26520.6097
26530.6263
26540.6015
26550.6197
26560.6080
26570.6082
26580.6030
26590.6142
26600.5930
26610.6058
26620.6013
26630.5959
26640.5956
26650.6078
26660.6060
26670.6123
26680.6135
26690.6050
26700.5988
26710.6212
26720.5934
26730.5935
26740.6065
26750.6015
26760.6105
26770.6059
26780.6003
26790.6179
26800.6188
26810.6069
26820.6066
26830.6109
26840.6063
26850.5996
26860.6130
26870.5902
26880.6008
26890.6152
26900.6231
26910.6208
26920.6205
26930.6177
26940.5987
26950.6026
26960.5982
26970.5802
26980.5997
26990.6082
27000.6004
27010.6035
27020.6047
27030.6102
27040.6051
27050.6185
27060.6026
27070.6080
27080.6228
27090.6124
27100.6165
27110.5970
27120.5996
27130.5919
27140.6047
27150.5948
27160.6011
27170.5928
27180.6219
27190.6073
27200.6158
27210.6144
27220.6167
27230.6102
27240.6269
27250.6174
27260.5962
27270.5970
27280.6034
27290.5941
27300.5923
27310.5878
27320.5943
27330.6088
27340.6173
27350.6169
27360.5962
27370.6070
27380.5831
27390.6060
27400.6011
27410.6025
27420.6106
27430.6077
27440.6112
27450.6174
27460.6036
27470.6183
27480.6118
27490.6070
27500.6057
27510.6002
27520.6075
27530.6051
27540.6000
27550.6055
27560.6159
27570.6272
27580.6133
27590.5990
27600.6181
27610.6148
27620.6105
27630.6041
27640.6133
27650.5987
27660.6159
27670.5859
27680.6203
27690.5965
27700.5982
27710.6089
27720.6045
27730.6080
27740.6123
27750.6174
27760.5942
27770.6079
27780.6109
27790.6061
27800.6303
27810.6106
27820.6036
27830.6129
27840.6229
27850.6025
27860.6104
27870.6114
27880.6216
27890.6044
27900.5979
27910.6249
27920.6073
27930.6151
27940.6302
27950.6164
27960.6129
27970.6111
27980.6026
27990.6021
28000.5997
28010.5930
28020.6070
28030.6160
28040.5957
28050.6173
28060.6087
28070.6047
28080.5827
28090.5987
28100.6067
28110.5994
28120.6056
28130.6142
28140.6157
28150.5938
28160.6039
28170.6109
28180.5980
28190.5987
28200.6014

Measured result

Verbatim from the run's summary.json.

bucket
bucket-p32-t128
bucket_icons
3359
checkpoint_bytes
4715161
checkpoint_round_trip
true
checkpoint_sha256
cff49ce8e8f2988d…
config_sha256
0ef00ff401f00797…
corruption
factorized_marginal_respecting_geometry
cuda_version
13.4
deterministic_algorithms
true
device
cuda
eval_every
60
final_train
loss
0.6135861277580261
step
2820
train_token_accuracy
0.6014218009478673
group_vocabulary_size
12
locked_path_exact
true
metrics_sha256
9276f6ee9ed29e00…
model_parameters
579872
schema_version
1
scope
dominant-bucket fixed-topology geometry pilot; not unconditional generation
selected_train_rows
color/svg/1F3F4-E0064-E0065-E0062-E0065-E007F.svg, color/svg/1F468-1F3FC-200D-1F9BC.svg, color/svg/1F561.svg, color/svg/1F469-1F3FE-200D-1F9BC.svg, color/svg/1F469-1F3FD-200D-2764-FE0F-200D-1F48B-200D-1F469-1F3FE.svg, color/svg/1F6B5-1F3FB-200D-2642-FE0F.svg … and 2675 more
selected_validation_rows
color/svg/1F994.svg, color/svg/E30A.svg, color/svg/1F6BE.svg, color/svg/1F3CA-1F3FB-200D-2642-FE0F.svg, color/svg/1F3CB-1F3FC-200D-2642-FE0F.svg, color/svg/1F93D-1F3FC.svg … and 122 more
selection
completed_steps
2820
evals_without_improvement
8
min_delta
0.0
objective
held_out_loss
patience_evals
8
selected_held_out_loss
0.6305077075958252
selected_step
2340
stopped_early
true
steps
6300
study_version
openmoji-g1-marginal-corruption-v9
subgroup_vocabulary_size
118
torch_version
2.14.0a0+4fdf77b940.nv26.08
validation
accuracy
0.6025794841031794
changed_accuracy
0.0002869234631662004
changed_total
13941
loss
0.6305077075958252
retained_accuracy
0.9246940576207465
retained_total
26067
validation_final_step
accuracy
0.5701359728054389
changed_accuracy
0.0005738469263324008
changed_total
13941
loss
0.6339353919029236
retained_accuracy
0.8747458472398051
retained_total
26067
validation_trace_sha256
69b9d3422e328aad…
validation_untrained
accuracy
0.2675214957008598
changed_accuracy
0.002080195107954953
changed_total
13941
loss
1.4054925441741943
retained_accuracy
0.4094832546898377
retained_total
26067

Written result

v8 reached 2.806 precision lift, which looked like a breakthrough until a judge pointed out that a detector knowing only the corpus token marginal reaches 2.831 on the same data with zero parameters and zero context. Factorized corruption draws replacements uniformly from the full legal vocabulary, so a large share land on tokens real icons essentially never use, and spotting them needs no geometry at all.

This run removes that shortcut. Replacements are drawn from the corpus distribution for the field's own role — keyed by segment kind as well as slot, since slot 0 is an endpoint for a LINE and a control handle for a CUBIC — so every corrupted token is in-distribution by construction. The marginal is estimated from train programs only. Nothing else changes from v8.

Measured on 128 held-out icons before the run, at the matched 8.573% flag rate:

free detectorunder factorizedunder marginal
marginal density (0 params, no context)2.8400.771
local continuity (0 params, neighbours)2.1261.893

The density shortcut is gone — below chance, because corrupted tokens are now typical — while the genuine geometric signal survives.

Result

criterionthresholdobservedoutcome
beats_the_free_geometric_detector> 1.8931.506falsified
not_explained_by_density> 1.201.506pass
structural_safetylocked-path exact, round-tripsboth truepass
reproducibilityidentical rerunidenticalpass

Selected step 2,340 of 2,820 run, early stopped — against v8's full 6,300 with no stop. The harder task converges sooner.

What 1.506 means

It is the first honest detection number this project has produced, and it says two things at once.

The model does learn geometry. At 1.506 it is far above the 0.771 a density detector manages here, and above the 1.20 bar set to rule that out. Whatever it is using, it is not the marginal, because the marginal no longer helps.

It still loses to a zero-parameter statistic. The local-continuity heuristic reaches 1.893 on the identical corrupted inputs. A 579,872-parameter transformer, trained under sound optimisation for 2,340 steps, extracts less of the geometric signal than the mean absolute distance to the same slot in adjacent segments.

And v8's 2.806 is now properly read: mostly the leak. The honest ranking under leak-free corruption is continuity 1.893, v9 1.506, density 0.771.

What this settles and what it opens

Settled: the corruption process was leaking, v8's apparent success was largely that leak, and the fix is real and cheap. Every detector number measured under factorized corruption — v6's 1.234, v7's 1.365, v8's 2.806, and the 2.13 and 2.83 free references — describes a task that is partly a density test. This run's 1.506 replaces them as the reference point.

Opened, and exactly as this run's predeclared falsification meaning anticipated: the model cannot beat a trivial geometric statistic even with the shortcut removed and the optimisation fixed, so the remaining candidates are representational. The diagnostic panel measured two, independently:

The representation lens measured the first in a controlled simulation: with the current encoding a probe on the neighbour-difference target scores AUC 0.5024, chance; replacing the categorical lookup with a numeric code of the same value gives 0.9981. That is the next run, and it is a single factor.

Scope

Detection-only, single seed, probability 0.35. Not comparable with v8 in absolute terms: the task is strictly harder because the easy fraction of corruptions was removed, so 1.506 against 2.806 is not a regression.

State transitions

  1. planned2026-09-21T00:40:00Z
  2. completed2026-09-21T01:00:00Zleak_closed_the_model_learns_geometry_but_still_loses_to_a_zero_parameter_continuity_statistic

Run record

Verbatim from runs/openmoji-g1-marginal-corruption-v9-74dc5fd-0ef00ff4-9b9b1699/run.yaml, the record committed before launch.

schema_version
1
run_id
openmoji-g1-marginal-corruption-v9-74dc5fd-0ef00ff4-9b9b1699
state
completed
parent_run
openmoji-g1-sound-optimisation-v8-949e0e5-4feee442-9b9b1699
hypothesis
v8's detector reached 2.806 precision lift, but a zero-parameter detector that knows only the corpus marginal reaches 2.831 on the same data, so v8 matched a density test rather than learning geometry. Factorized corruption draws replacements uniformly from the full legal vocabulary, so many land on tokens real icons never use. Drawing them from the corpus distribution for the field's role instead removes that shortcut: the marginal detector falls to 0.771 while the local-continuity signal survives at 1.893. Under this leak-free process the model must use the surrounding geometry, and this run measures for the first time how much of that signal it can actually learn.
expected_information_gain
This is the first honest detection benchmark in the project. Every earlier detector number - v6 1.234, v7 1.365, v8 2.806, and the free references at 2.13 and 2.83 - was measured on a process whose corruption is identifiable from a token marginal alone. A trained model that beats the continuity statistic HERE has learned geometry; one that does not has not, and no amount of the earlier numbers would have told us which.
scope_limits
Detection-only, so it reconstructs nothing and held-out accuracy is meaningless here. Single seed, probability 0.35. Not comparable with v8 in absolute terms: the task is strictly harder because the easy fraction of corruptions has been removed, so a lower lift than v8's 2.806 is expected and is not a regression.
changed_factors
primary
training.corruption_process factorized -> marginal.
parameters
unchanged at 579,872
held_fixed
v8 exactly otherwise: pool_loss_over_fields, mask_padding, detection_only, slot binding, edit-mask heads, the full 2,681-icon split, the 128-icon validation draw and its corruption seeds, seed 3101, probability 0.35, batch 16, lr 0.001, eval_every 60, the 6,300-step cap, held-out-loss selection with patience 8.
marginal_estimated_from
train programs only, so no held-out information reaches the corruption applied to held-out icons.
code
git_commit
74dc5fd
execution_mode
native-local
config
path
configs/learning/openmoji-g1-marginal-corruption-v9.yaml
sha256
0ef00ff401f00797…
dataset
hybrid_sha256
9b9b1699677a6f97…
bucket
bucket-p32-t128
train_samples
2681
validation_samples
128
free_references_under_each_process
note
Measured before this run, on 128 held-out icons, at the matched 8.573% flag rate. The marginal column is what this run must be judged against.
factorized_uniform
marginal_density_lift
2.84
local_continuity_lift
2.126
marginal_respecting
marginal_density_lift
0.771
local_continuity_lift
1.893
trained_under_factorized
v6
1.234
v7
1.365
v8
2.806
predeclared_criteria
note
Measured by scripts/detector_report.py at the matched 8.573% flag rate.
evaluated_at
the checkpoint selected by held-out loss
primary
id
beats_the_free_geometric_detector
statement
Precision lift exceeds 1.893, the local-continuity statistic measured under this same corruption process. This is the first criterion in the project that a model can only pass by using geometry, because the density shortcut is gone.
id
not_explained_by_density
statement
Precision lift exceeds 1.20. The marginal-density detector scores 0.771 here, so anything at or below about 1.0 would mean the model learned nothing this process did not hand it.
id
structural_safety
statement
Locked-path exactness holds and the canonical checkpoint round-trips.
id
reproducibility
statement
A second complete invocation returns an identical JSON result.
reported_not_gated
detector recall and precision at the model's own threshold and at the base flag rate, the selected step and whether early stopping or the cap ended the run, held-out field-pooled loss, not comparable across corruption processes
falsification_meaning
If the model cannot beat a zero-parameter continuity statistic once the density shortcut is removed, then v8's apparent success was entirely the leak, and the conclusion is that this model class does not learn the geometric relation even under sound optimisation. That would make the representational interventions - per-field tokenisation, or ordinal coordinate features - the next runs, and both were measured and argued for by the diagnostic panel.
outputs
local_metadata
runs/openmoji-g1-marginal-corruption-v9-74dc5fd-0ef00ff4-9b9b1699
report_root
reports/learning/openmoji-g1-marginal-corruption-v9
durable_artifacts
/home/dev/.cache/openmoji-g1-marginal-corruption-v9-74dc5fd-0ef00ff4-9b9b1699
planned_at
2026-09-21 00:40:00+00:00
completed_at
2026-09-21 01:00:00+00:00
result
device
cuda
model_parameters
579872
selected_step
2340
completed_steps
2820
ended_by
early_stopping
identical_rerun
true
detector_at_matched_flag_rate
precision
0.5248
lift_over_base_rate
1.506
recall
0.1291
detector_at_base_flag_rate
precision
0.4295
lift_over_base_rate
1.232
predeclared_outcome
overall
partially-falsified
beats_the_free_geometric_detector
passed
false
observed
1.506
threshold
1.893
not_explained_by_density
passed
true
observed
1.506
threshold
1.2
structural_safety
passed
true
reproducibility
passed
true
conclusion
The first honest detection number in the project, and it says two things. The model does learn geometry: 1.506 is far above the 0.771 a density detector manages under this process, so it is not using the marginal, which no longer helps. But it still loses to a zero-parameter local-continuity statistic at 1.893 on the identical corrupted inputs, under sound optimisation and 2,340 steps.
reinterprets_v8
v8's 2.806 was mostly the leak. The honest ranking under leak-free corruption is continuity 1.893, v9 1.506, density 0.771. Every detector number measured under factorized corruption - v6 1.234, v7 1.365, v8 2.806, and the 2.13 and 2.83 free references - describes a task that is partly a density test.
opens
The predeclared falsification meaning applies: the model cannot beat a trivial geometric statistic with the shortcut removed and the optimisation fixed, so the remaining candidates are representational. The measured one is that coordinate values enter as identities rather than magnitudes - the trained input embedding is statistically indistinguishable from random init at Spearman -0.136 and adjacent-bin distance 0.989 of the all-pairs mean, while the output head, which gets a direct gradient from the corpus marginal, does show order at +0.143 and 0.882.