MojiDiff

← experiments

openmoji-g1-edit-mask-v6-0228c3b-0f6ce115-9b9b1699

completed falsified

Hypothesis

Every Gate G model so far scores below the identity policy on held-out token accuracy because the objective makes identity expensive: each field is an independent softmax over a 289- or 417-way vocabulary, so "leave this one alone" costs as much as inventing a new value, and 26,030 of 40,008 held-out fields are uncorrupted. Adding a per-field keep-or-change decision, and copying the input wherever it says keep, makes identity reachable as predict-keep-everywhere. The model should then clear the identity baseline rather than fall 0.27 short of it.

Why run it

This is the first Gate G criterion set against the trivial policy rather than against noise, and it tests the last candidate standing after data volume, capacity, the corruption regime, noise-level information and encoder slot-blindness were each eliminated by a predeclared comparison. Clearing identity would be the first evidence in the sequence that the model does something useful rather than something harmful.

Scope limits

Fixed-topology and geometry-only, single seed. Exactly v5 plus 1,552 parameters of keep heads, +0.27%, which is not a plausible capacity effect given v3 showed a 3.53x increase does nothing. Beating identity in token space would not mean the output is a recognizable icon; no run in this sequence has produced one.

Measured behaviour

Held-out loss
lossoptimizer step56789050010001500selected
Table view
optimizer stepheld-out loss
08.8510
606.8898
1206.4304
1806.2172
2406.1594
3006.1218
3606.0671
4206.0760
4806.0354
5406.0083
6006.0837
6606.0541
7206.0342
7806.0002
8405.9622
9005.9990
9606.0107
10205.9645
10806.0326
11406.0267
12005.9434
12606.0142
13205.9188
13805.8578
14405.9599
15005.9830
15605.9382
16205.9975
16805.8980
17405.8948
18005.9402
18605.8923
Held-out token accuracy
aggregatechanged fieldsretained fields
accuracyoptimizer step00.250.50.751050010001500selected
Table view
optimizer stepaggregatechanged fieldsretained fields
00.26390.00120.4050
600.50000.00380.7665
1200.55620.00470.8523
1800.59710.00360.9158
2400.59450.00390.9117
3000.60300.00420.9246
3600.60170.00510.9221
4200.58990.00670.9031
4800.61360.00590.9398
5400.61620.00600.9439
6000.61260.00490.9390
6600.60640.00590.9289
7200.61370.00520.9405
7800.62380.00520.9559
8400.60310.00970.9218
9000.61080.00870.9340
9600.62660.00830.9587
10200.61300.01100.9362
10800.62590.00760.9580
11400.61030.01140.9318
12000.61320.01090.9366
12600.60520.01290.9232
13200.61620.01360.9398
13800.61190.01320.9335
14400.61980.01020.9471
15000.60530.01240.9237
15600.61530.01410.9381
16200.61620.01470.9392
16800.60300.01370.9194
17400.61030.01610.9293
18000.60740.01720.9244
18600.61960.01390.9448
Training token accuracy (per batch)
accuracyoptimizer step00.20.40.60.850010001500selected
Table view
optimizer steptrain accuracy
10.2863
20.2722
30.3461
40.3707
50.3830
60.4289
70.4364
80.4647
90.4590
100.4669
110.4592
120.4886
130.4845
140.4959
150.4828
160.4721
170.4691
180.4745
190.4618
200.4788
210.4516
220.4571
230.4605
240.4532
250.4495
260.4478
270.4344
280.4565
290.4523
300.4494
310.4805
320.4680
330.4671
340.4662
350.4632
360.4536
370.4652
380.4569
390.4619
400.4630
410.4915
420.4726
430.4821
440.4843
450.4745
460.4893
470.4993
480.4684
490.4703
500.4845
510.4944
520.5062
530.4757
540.4889
550.4897
560.4775
570.4941
580.4842
590.4930
600.4977
610.5052
620.4954
630.5138
640.5187
650.5158
660.5132
670.4976
680.5267
690.5187
700.5219
710.5446
720.5375
730.5336
740.5188
750.5532
760.5495
770.5114
780.5233
790.4911
800.5176
810.5128
820.5020
830.5107
840.4844
850.5138
860.4986
870.4850
880.5051
890.5217
900.5208
910.5164
920.5258
930.5205
940.5139
950.5281
960.5255
970.5389
980.5467
990.5338
1000.5500
1010.5694
1020.5519
1030.5549
1040.5338
1050.5717
1060.5467
1070.5538
1080.5444
1090.5842
1100.5541
1110.5548
1120.5505
1130.5341
1140.5487
1150.5687
1160.5540
1170.5432
1180.5306
1190.5650
1200.5435
1210.5525
1220.5764
1230.5496
1240.5558
1250.5617
1260.5573
1270.5547
1280.5744
1290.5742
1300.5848
1310.5631
1320.5701
1330.5734
1340.5772
1350.5535
1360.5736
1370.5521
1380.5893
1390.5730
1400.5616
1410.5647
1420.5752
1430.5508
1440.5523
1450.5581
1460.5726
1470.5667
1480.5539
1490.5548
1500.5619
1510.5685
1520.5619
1530.5702
1540.5652
1550.5920
1560.5676
1570.5685
1580.5911
1590.5823
1600.5791
1610.5715
1620.5890
1630.5833
1640.5712
1650.5581
1660.5558
1670.5697
1680.5598
1690.5774
1700.5918
1710.5782
1720.5942
1730.5891
1740.5882
1750.6001
1760.5843
1770.6008
1780.5888
1790.6103
1800.6137
1810.5939
1820.6061
1830.5902
1840.5867
1850.6064
1860.6045
1870.5914
1880.5913
1890.5662
1900.5822
1910.5575
1920.5706
1930.5668
1940.5492
1950.5630
1960.5728
1970.5456
1980.5540
1990.5710
2000.5892
2010.5870
2020.5899
2030.6053
2040.5853
2050.5804
2060.6061
2070.6025
2080.6134
2090.6003
2100.6047
2110.5639
2120.6122
2130.5840
2140.5803
2150.5873
2160.5852
2170.5793
2180.5672
2190.5973
2200.5872
2210.5777
2220.5622
2230.5887
2240.5881
2250.5788
2260.5741
2270.5776
2280.5810
2290.5766
2300.5887
2310.5918
2320.5872
2330.6057
2340.6130
2350.6013
2360.6172
2370.6002
2380.6026
2390.6215
2400.6132
2410.6111
2420.6066
2430.6137
2440.5899
2450.5864
2460.6073
2470.5866
2480.5961
2490.5783
2500.6089
2510.5975
2520.6018
2530.6086
2540.5943
2550.5974
2560.5975
2570.6019
2580.5978
2590.6116
2600.6073
2610.5867
2620.5990
2630.6101
2640.5919
2650.6158
2660.5962
2670.6159
2680.6173
2690.6303
2700.6107
2710.6065
2720.6255
2730.6030
2740.6095
2750.5873
2760.6084
2770.6065
2780.5850
2790.6137
2800.5937
2810.6085
2820.6151
2830.5942
2840.6088
2850.5998
2860.6148
2870.6276
2880.6115
2890.6373
2900.6265
2910.6046
2920.6077
2930.6240
2940.6148
2950.6080
2960.6191
2970.6220
2980.6321
2990.6114
3000.6189
3010.6302
3020.6068
3030.5946
3040.6168
3050.5996
3060.6083
3070.6145
3080.6018
3090.6047
3100.6002
3110.5873
3120.6008
3130.6123
3140.6111
3150.6055
3160.6051
3170.6081
3180.6192
3190.6239
3200.6132
3210.6041
3220.6119
3230.6145
3240.6054
3250.6011
3260.6168
3270.6116
3280.6030
3290.6083
3300.5937
3310.5986
3320.5828
3330.5779
3340.5870
3350.5774
3360.6135
3370.5946
3380.5969
3390.5936
3400.5982
3410.6130
3420.6013
3430.5999
3440.6019
3450.5973
3460.6001
3470.6162
3480.6060
3490.6149
3500.6336
3510.6179
3520.6087
3530.6045
3540.6302
3550.6031
3560.6055
3570.5999
3580.6034
3590.5832
3600.6082
3610.6183
3620.5976
3630.6242
3640.6019
3650.6061
3660.6114
3670.6243
3680.6123
3690.6181
3700.6219
3710.6303
3720.5969
3730.6084
3740.6027
3750.6217
3760.6145
3770.6120
3780.6013
3790.6155
3800.6198
3810.6012
3820.6084
3830.6076
3840.6156
3850.5903
3860.6035
3870.5997
3880.6058
3890.6011
3900.6209
3910.6130
3920.6302
3930.5867
3940.6328
3950.5990
3960.6188
3970.6016
3980.6047
3990.6148
4000.6052
4010.6203
4020.6234
4030.6209
4040.6278
4050.6194
4060.6302
4070.6112
4080.6376
4090.6207
4100.6078
4110.6051
4120.6107
4130.6109
4140.6039
4150.6017
4160.6018
4170.6047
4180.6134
4190.6057
4200.6174
4210.5976
4220.6096
4230.5963
4240.6034
4250.6075
4260.6113
4270.6044
4280.6033
4290.6114
4300.6159
4310.6022
4320.6189
4330.6154
4340.6020
4350.6277
4360.6175
4370.6306
4380.6233
4390.6070
4400.6392
4410.6232
4420.6181
4430.6225
4440.6201
4450.6169
4460.6167
4470.6301
4480.6230
4490.6182
4500.6139
4510.6181
4520.6163
4530.6132
4540.6136
4550.6211
4560.6190
4570.6145
4580.6019
4590.5922
4600.5972
4610.5974
4620.6080
4630.6063
4640.6025
4650.6239
4660.6004
4670.6011
4680.6163
4690.5986
4700.6176
4710.6162
4720.6136
4730.6398
4740.6120
4750.6386
4760.6134
4770.6235
4780.6132
4790.6046
4800.6267
4810.6137
4820.6273
4830.6178
4840.6353
4850.6167
4860.6220
4870.6347
4880.6203
4890.6264
4900.6205
4910.6171
4920.6234
4930.6357
4940.6273
4950.6301
4960.6373
4970.6267
4980.6197
4990.6193
5000.6425
5010.6179
5020.6175
5030.6140
5040.6233
5050.6018
5060.6158
5070.6265
5080.6185
5090.6097
5100.6172
5110.5980
5120.6199
5130.6196
5140.6104
5150.6388
5160.6308
5170.6219
5180.6158
5190.6279
5200.6365
5210.6218
5220.6347
5230.6287
5240.6130
5250.6311
5260.6355
5270.6123
5280.6474
5290.6227
5300.6319
5310.6287
5320.6397
5330.6259
5340.6381
5350.6300
5360.6286
5370.6324
5380.6301
5390.6284
5400.6013
5410.6172
5420.6278
5430.6175
5440.6225
5450.6172
5460.5966
5470.6129
5480.5810
5490.6119
5500.6021
5510.6071
5520.6116
5530.5918
5540.6270
5550.6076
5560.6196
5570.6239
5580.6099
5590.6361
5600.6203
5610.6213
5620.6184
5630.6079
5640.6149
5650.6221
5660.6240
5670.6047
5680.6197
5690.6150
5700.6056
5710.6251
5720.6171
5730.6196
5740.6419
5750.6196
5760.6306
5770.6177
5780.6195
5790.6164
5800.6162
5810.6398
5820.6245
5830.6250
5840.6193
5850.6213
5860.6212
5870.6296
5880.6256
5890.6247
5900.6280
5910.6242
5920.6146
5930.6258
5940.6352
5950.6225
5960.6201
5970.6083
5980.6209
5990.6112
6000.6292
6010.6167
6020.6203
6030.6380
6040.6321
6050.6389
6060.6232
6070.6368
6080.6257
6090.6153
6100.6181
6110.6123
6120.6319
6130.6181
6140.6520
6150.6199
6160.6175
6170.6310
6180.6194
6190.6228
6200.6335
6210.6001
6220.6302
6230.5941
6240.6225
6250.6337
6260.6292
6270.6278
6280.6215
6290.6240
6300.6408
6310.6426
6320.6334
6330.6358
6340.6186
6350.6276
6360.6274
6370.6173
6380.6159
6390.6225
6400.5996
6410.6363
6420.6208
6430.6190
6440.6190
6450.6274
6460.6138
6470.6198
6480.6222
6490.6307
6500.6313
6510.6229
6520.6250
6530.6452
6540.6234
6550.6266
6560.6369
6570.6175
6580.6194
6590.6079
6600.6049
6610.6157
6620.6195
6630.6218
6640.6233
6650.6182
6660.6158
6670.6165
6680.6126
6690.6058
6700.6130
6710.6098
6720.6321
6730.6136
6740.6380
6750.6175
6760.6372
6770.6224
6780.6227
6790.6274
6800.6209
6810.6058
6820.6298
6830.6238
6840.6240
6850.6262
6860.6117
6870.6234
6880.6073
6890.6245
6900.6291
6910.6122
6920.6187
6930.6233
6940.6165
6950.6227
6960.6248
6970.6339
6980.6203
6990.6327
7000.6075
7010.6106
7020.6217
7030.6080
7040.6294
7050.6120
7060.6239
7070.6198
7080.5897
7090.6330
7100.6148
7110.6286
7120.6156
7130.6192
7140.6194
7150.6228
7160.6285
7170.6153
7180.6182
7190.6169
7200.6240
7210.6250
7220.6289
7230.6207
7240.6169
7250.6184
7260.6318
7270.6139
7280.6278
7290.6186
7300.6148
7310.6136
7320.6003
7330.6326
7340.6065
7350.6264
7360.6173
7370.6185
7380.6268
7390.6468
7400.6273
7410.6307
7420.6483
7430.6287
7440.6315
7450.6235
7460.6322
7470.6490
7480.6327
7490.6161
7500.6152
7510.6309
7520.6211
7530.6148
7540.6101
7550.6220
7560.6218
7570.6393
7580.6117
7590.6352
7600.6286
7610.6424
7620.6249
7630.6278
7640.6136
7650.6257
7660.6255
7670.6235
7680.6256
7690.6187
7700.6194
7710.6318
7720.6343
7730.6297
7740.6313
7750.6222
7760.6200
7770.6394
7780.6254
7790.6385
7800.6254
7810.6229
7820.6319
7830.6249
7840.6360
7850.6115
7860.6198
7870.6301
7880.6254
7890.6218
7900.6318
7910.6264
7920.6303
7930.6204
7940.6290
7950.6300
7960.6260
7970.6290
7980.6209
7990.6402
8000.6265
8010.6263
8020.6286
8030.6350
8040.6199
8050.6175
8060.6164
8070.6269
8080.6303
8090.6367
8100.6135
8110.6168
8120.6237
8130.6267
8140.6097
8150.6204
8160.6177
8170.6166
8180.6213
8190.6238
8200.6143
8210.6345
8220.6296
8230.6326
8240.6273
8250.6100
8260.6380
8270.6166
8280.6166
8290.6208
8300.6310
8310.6257
8320.6196
8330.6290
8340.6017
8350.6192
8360.5948
8370.5969
8380.6157
8390.6213
8400.6239
8410.6164
8420.6307
8430.6108
8440.6234
8450.6273
8460.6094
8470.6208
8480.6286
8490.6220
8500.6372
8510.6316
8520.6287
8530.6296
8540.6204
8550.6334
8560.6300
8570.6176
8580.6156
8590.6114
8600.6160
8610.6161
8620.6136
8630.6275
8640.6269
8650.6187
8660.6152
8670.6208
8680.6265
8690.6166
8700.6265
8710.6215
8720.6339
8730.6295
8740.6214
8750.6107
8760.6145
8770.6196
8780.6230
8790.6405
8800.6194
8810.6161
8820.6284
8830.6397
8840.6233
8850.6175
8860.6277
8870.6285
8880.6211
8890.6252
8900.6177
8910.6127
8920.6278
8930.6286
8940.6139
8950.6237
8960.6290
8970.6178
8980.6077
8990.6275
9000.6139
9010.6319
9020.6392
9030.6172
9040.6295
9050.6026
9060.6307
9070.6201
9080.6398
9090.6272
9100.6334
9110.6260
9120.6299
9130.6289
9140.6165
9150.6198
9160.6267
9170.6407
9180.6060
9190.6262
9200.6177
9210.6286
9220.6197
9230.6212
9240.6282
9250.6115
9260.6042
9270.6059
9280.6274
9290.6326
9300.6096
9310.6193
9320.6234
9330.6062
9340.6243
9350.6295
9360.6326
9370.6373
9380.6507
9390.6277
9400.6463
9410.6271
9420.6503
9430.6332
9440.6210
9450.6180
9460.6326
9470.5989
9480.6248
9490.6331
9500.6242
9510.6253
9520.6244
9530.6191
9540.6261
9550.6266
9560.6243
9570.6341
9580.6196
9590.6395
9600.6412
9610.6354
9620.6413
9630.6454
9640.6398
9650.6256
9660.6326
9670.6380
9680.6405
9690.6287
9700.6164
9710.6048
9720.6259
9730.6214
9740.6357
9750.6172
9760.6296
9770.6205
9780.6213
9790.6055
9800.6270
9810.6240
9820.6236
9830.6122
9840.6380
9850.6341
9860.6386
9870.6314
9880.6251
9890.6431
9900.6220
9910.6268
9920.6282
9930.6376
9940.6018
9950.6248
9960.6328
9970.6297
9980.6124
9990.6260
10000.6237
10010.6046
10020.6215
10030.6136
10040.6239
10050.6115
10060.6194
10070.6220
10080.6211
10090.6272
10100.6309
10110.6147
10120.6271
10130.6222
10140.6076
10150.6082
10160.6256
10170.6300
10180.6306
10190.6131
10200.6481
10210.6203
10220.6212
10230.6296
10240.6313
10250.6432
10260.6314
10270.6269
10280.6381
10290.6317
10300.6259
10310.6362
10320.6233
10330.6255
10340.6345
10350.6167
10360.6236
10370.6194
10380.6466
10390.6283
10400.6295
10410.6356
10420.6280
10430.6178
10440.6333
10450.6204
10460.6221
10470.6295
10480.6168
10490.6299
10500.6318
10510.6296
10520.6100
10530.6142
10540.6186
10550.6232
10560.6185
10570.6212
10580.6124
10590.6179
10600.6229
10610.6311
10620.6245
10630.6132
10640.6224
10650.6231
10660.6168
10670.6206
10680.6272
10690.6347
10700.6332
10710.6286
10720.6134
10730.6319
10740.6323
10750.6466
10760.6315
10770.6432
10780.6341
10790.6514
10800.6438
10810.6233
10820.6354
10830.6389
10840.6226
10850.6357
10860.6309
10870.6297
10880.6407
10890.6220
10900.6285
10910.6117
10920.6095
10930.5810
10940.6263
10950.6061
10960.6093
10970.6475
10980.6246
10990.6261
11000.6244
11010.6348
11020.6270
11030.6387
11040.6275
11050.6328
11060.6355
11070.6343
11080.6367
11090.6313
11100.6210
11110.6367
11120.6240
11130.6218
11140.6338
11150.6221
11160.6333
11170.6337
11180.6345
11190.6419
11200.6284
11210.6272
11220.6205
11230.6027
11240.6190
11250.6370
11260.6245
11270.6427
11280.6225
11290.6169
11300.6343
11310.6168
11320.6314
11330.6104
11340.6393
11350.6314
11360.6497
11370.6224
11380.6173
11390.6120
11400.6287
11410.6338
11420.6188
11430.6326
11440.6275
11450.6127
11460.6145
11470.6174
11480.6133
11490.6077
11500.6174
11510.6204
11520.6274
11530.6130
11540.6205
11550.6130
11560.6296
11570.6265
11580.6126
11590.6408
11600.6151
11610.6286
11620.6285
11630.6239
11640.6371
11650.6242
11660.6342
11670.6405
11680.6367
11690.6338
11700.6402
11710.6272
11720.6091
11730.6314
11740.6172
11750.6220
11760.6256
11770.6265
11780.6221
11790.6185
11800.6208
11810.6116
11820.6232
11830.6141
11840.6265
11850.6293
11860.6330
11870.6408
11880.6551
11890.6264
11900.6229
11910.6121
11920.6510
11930.6440
11940.6233
11950.6149
11960.6443
11970.6280
11980.6474
11990.6156
12000.6260
12010.6293
12020.6172
12030.6257
12040.6426
12050.6325
12060.6359
12070.6333
12080.6493
12090.6456
12100.6134
12110.6407
12120.6277
12130.6375
12140.6357
12150.6412
12160.6246
12170.6442
12180.6342
12190.6311
12200.6189
12210.6430
12220.6311
12230.6205
12240.6238
12250.6183
12260.6124
12270.6280
12280.6449
12290.6203
12300.6288
12310.6007
12320.6204
12330.6162
12340.6405
12350.6100
12360.6328
12370.6315
12380.6361
12390.6356
12400.6287
12410.6449
12420.6277
12430.6439
12440.6323
12450.6431
12460.6369
12470.6302
12480.6338
12490.6497
12500.6183
12510.6355
12520.6222
12530.6299
12540.6170
12550.6371
12560.6249
12570.6238
12580.6207
12590.6238
12600.6173
12610.6082
12620.6160
12630.6385
12640.6430
12650.6331
12660.6279
12670.6258
12680.6258
12690.6371
12700.6327
12710.6194
12720.6217
12730.6344
12740.6491
12750.6340
12760.6464
12770.6311
12780.6314
12790.6379
12800.6239
12810.6287
12820.6185
12830.6071
12840.6404
12850.6316
12860.6284
12870.6175
12880.6335
12890.6216
12900.6207
12910.6144
12920.6344
12930.6095
12940.6307
12950.6376
12960.6240
12970.6351
12980.6235
12990.6300
13000.6192
13010.6393
13020.6290
13030.6511
13040.6259
13050.6298
13060.6379
13070.6235
13080.6334
13090.6352
13100.6325
13110.6325
13120.6292
13130.6315
13140.6353
13150.6597
13160.6359
13170.6407
13180.6355
13190.6316
13200.6390
13210.6185
13220.6298
13230.6313
13240.6179
13250.6305
13260.6246
13270.6145
13280.6188
13290.5992
13300.6196
13310.6158
13320.6166
13330.6323
13340.6226
13350.6362
13360.6286
13370.6450
13380.6331
13390.6279
13400.6337
13410.6317
13420.6444
13430.6273
13440.6374
13450.6433
13460.6263
13470.6278
13480.6378
13490.6156
13500.6205
13510.6397
13520.6294
13530.6447
13540.6342
13550.6345
13560.6464
13570.6342
13580.6360
13590.6365
13600.6555
13610.6242
13620.6285
13630.6350
13640.6224
13650.6499
13660.6356
13670.6254
13680.6212
13690.6343
13700.6189
13710.6293
13720.6336
13730.6264
13740.6267
13750.6453
13760.6165
13770.6306
13780.6297
13790.6338
13800.6163
13810.6201
13820.6351
13830.6227
13840.6239
13850.6302
13860.6300
13870.6200
13880.6267
13890.6273
13900.6348
13910.6342
13920.6204
13930.6194
13940.6349
13950.6206
13960.6200
13970.6223
13980.6285
13990.6081
14000.6338
14010.6248
14020.6236
14030.6289
14040.6284
14050.6234
14060.6359
14070.6323
14080.6100
14090.6427
14100.6317
14110.6410
14120.6421
14130.6340
14140.6441
14150.6488
14160.6470
14170.6367
14180.6488
14190.6399
14200.6288
14210.6269
14220.6219
14230.6338
14240.6257
14250.6314
14260.6325
14270.6325
14280.6064
14290.6032
14300.6178
14310.6041
14320.6315
14330.6114
14340.6217
14350.6040
14360.6265
14370.6222
14380.6248
14390.6207
14400.6360
14410.6322
14420.6226
14430.6413
14440.6286
14450.6322
14460.6253
14470.6178
14480.6429
14490.6416
14500.6253
14510.6159
14520.6334
14530.6295
14540.6193
14550.6339
14560.6396
14570.6191
14580.6174
14590.6145
14600.6363
14610.6275
14620.6425
14630.6406
14640.6330
14650.6331
14660.6345
14670.6376
14680.6357
14690.6383
14700.6393
14710.6324
14720.6309
14730.6300
14740.6302
14750.6127
14760.6265
14770.6201
14780.6167
14790.6296
14800.6194
14810.6179
14820.6225
14830.6393
14840.6223
14850.6351
14860.6289
14870.6289
14880.6358
14890.6457
14900.6310
14910.6463
14920.6271
14930.6240
14940.6361
14950.6064
14960.6206
14970.5992
14980.6171
14990.6174
15000.6148
15010.6284
15020.6257
15030.6300
15040.6112
15050.6301
15060.6279
15070.6264
15080.6178
15090.6163
15100.6269
15110.6185
15120.6321
15130.6277
15140.6075
15150.6341
15160.6426
15170.6314
15180.6311
15190.6076
15200.6480
15210.6393
15220.6431
15230.6430
15240.6363
15250.6403
15260.6410
15270.6391
15280.6602
15290.6310
15300.6332
15310.6485
15320.6179
15330.6303
15340.6220
15350.6205
15360.6219
15370.6303
15380.6244
15390.6297
15400.6316
15410.6225
15420.6295
15430.6345
15440.6282
15450.6266
15460.6161
15470.6150
15480.6158
15490.6279
15500.6143
15510.6160
15520.6297
15530.6242
15540.6143
15550.6365
15560.6332
15570.6395
15580.6201
15590.6249
15600.6290
15610.6240
15620.6243
15630.6281
15640.6276
15650.6212
15660.6198
15670.6414
15680.6103
15690.6247
15700.6152
15710.6406
15720.6113
15730.6285
15740.6269
15750.6029
15760.6339
15770.6160
15780.6350
15790.6440
15800.6362
15810.6444
15820.6538
15830.6236
15840.6227
15850.6347
15860.6364
15870.6365
15880.6339
15890.6222
15900.6357
15910.6460
15920.6257
15930.6470
15940.6351
15950.6390
15960.6113
15970.6061
15980.6429
15990.6415
16000.6408
16010.6317
16020.6249
16030.6199
16040.6336
16050.6334
16060.6160
16070.6226
16080.6370
16090.6343
16100.6390
16110.6209
16120.6381
16130.6326
16140.6209
16150.6320
16160.6480
16170.6308
16180.6188
16190.6375
16200.6549
16210.6206
16220.6250
16230.6376
16240.6341
16250.6251
16260.6148
16270.6332
16280.6261
16290.6290
16300.6362
16310.6235
16320.6197
16330.6367
16340.6271
16350.6322
16360.6435
16370.6371
16380.6397
16390.6214
16400.6363
16410.6288
16420.6282
16430.6177
16440.6431
16450.6100
16460.6466
16470.6219
16480.6170
16490.6145
16500.6365
16510.6304
16520.6256
16530.6291
16540.6359
16550.6326
16560.6358
16570.6334
16580.6320
16590.6336
16600.6385
16610.6255
16620.6138
16630.6077
16640.6237
16650.6216
16660.6378
16670.6155
16680.6182
16690.6170
16700.6350
16710.6241
16720.6268
16730.6318
16740.6174
16750.6250
16760.6155
16770.6223
16780.6266
16790.6263
16800.6325
16810.6180
16820.6108
16830.6362
16840.6000
16850.6319
16860.6219
16870.6352
16880.6506
16890.6329
16900.6430
16910.6485
16920.6332
16930.6379
16940.6330
16950.6468
16960.6165
16970.6290
16980.6273
16990.6448
17000.6322
17010.6441
17020.6162
17030.6445
17040.6302
17050.6400
17060.6295
17070.6445
17080.6280
17090.6077
17100.6480
17110.6414
17120.6357
17130.6184
17140.6133
17150.6216
17160.6293
17170.6425
17180.6489
17190.6103
17200.6425
17210.6282
17220.6383
17230.6311
17240.6291
17250.6445
17260.6321
17270.6283
17280.6202
17290.6206
17300.6276
17310.6406
17320.6471
17330.6324
17340.6267
17350.6327
17360.6278
17370.6248
17380.6265
17390.6316
17400.6275
17410.6279
17420.6375
17430.6107
17440.6453
17450.6354
17460.6332
17470.6492
17480.6229
17490.6388
17500.6228
17510.6397
17520.6244
17530.5932
17540.6128
17550.6191
17560.6048
17570.6079
17580.6316
17590.6200
17600.6376
17610.6352
17620.6243
17630.6274
17640.6274
17650.6285
17660.6472
17670.6499
17680.6394
17690.6079
17700.6084
17710.6209
17720.6137
17730.6201
17740.6137
17750.6308
17760.6427
17770.6182
17780.6368
17790.6301
17800.6519
17810.6444
17820.6291
17830.6268
17840.6441
17850.6184
17860.6378
17870.6499
17880.6265
17890.6368
17900.6326
17910.6400
17920.6336
17930.6261
17940.6205
17950.6394
17960.6263
17970.6396
17980.6291
17990.6323
18000.6365
18010.6307
18020.6298
18030.6296
18040.6443
18050.6441
18060.6258
18070.6323
18080.6576
18090.6462
18100.6305
18110.6399
18120.6515
18130.6259
18140.6282
18150.6323
18160.6332
18170.6244
18180.6515
18190.6038
18200.6250
18210.6341
18220.6264
18230.6427
18240.6233
18250.6348
18260.6475
18270.6115
18280.6274
18290.6240
18300.6068
18310.6268
18320.6194
18330.6068
18340.6263
18350.6199
18360.6168
18370.6227
18380.6366
18390.6284
18400.6361
18410.6238
18420.6326
18430.6374
18440.6331
18450.6385
18460.6219
18470.6325
18480.6272
18490.6056
18500.6336
18510.6211
18520.6123
18530.6159
18540.6252
18550.6283
18560.6369
18570.6327
18580.6413
18590.6595
18600.6307

Measured result

Verbatim from the run's summary.json.

bucket
bucket-p32-t128
bucket_icons
3359
checkpoint_bytes
7120113
checkpoint_round_trip
true
checkpoint_sha256
fc656537e4e749a6…
config_sha256
0f6ce115c1409288…
corruption
factorized_role_uniform_geometry
cuda_version
13.4
deterministic_algorithms
true
device
cuda
eval_every
60
final_train
loss
4.848937511444092
step
1860
train_token_accuracy
0.6306564690885914
group_vocabulary_size
12
locked_path_exact
true
metrics_sha256
f9f464ccbf425d22…
model_parameters
579872
schema_version
1
scope
dominant-bucket fixed-topology geometry pilot; not unconditional generation
selected_train_rows
color/svg/1F3F4-E0064-E0065-E0062-E0065-E007F.svg, color/svg/1F468-1F3FC-200D-1F9BC.svg, color/svg/1F561.svg, color/svg/1F469-1F3FE-200D-1F9BC.svg, color/svg/1F469-1F3FD-200D-2764-FE0F-200D-1F48B-200D-1F469-1F3FE.svg, color/svg/1F6B5-1F3FB-200D-2642-FE0F.svg … and 2675 more
selected_validation_rows
color/svg/1F994.svg, color/svg/E30A.svg, color/svg/1F6BE.svg, color/svg/1F3CA-1F3FB-200D-2642-FE0F.svg, color/svg/1F3CB-1F3FC-200D-2642-FE0F.svg, color/svg/1F93D-1F3FC.svg … and 122 more
selection
completed_steps
1860
evals_without_improvement
8
min_delta
0.0
objective
held_out_loss
patience_evals
8
selected_held_out_loss
5.8577985763549805
selected_step
1380
stopped_early
true
steps
6300
study_version
openmoji-g1-edit-mask-v6
subgroup_vocabulary_size
118
torch_version
2.14.0a0+4fdf77b940.nv26.08
validation
accuracy
0.6119276144771045
changed_accuracy
0.013163542709972815
changed_total
13978
loss
5.8577985763549805
retained_accuracy
0.9334613907030349
retained_total
26030
validation_final_step
accuracy
0.6195510897820435
changed_accuracy
0.013878952639862642
changed_total
13978
loss
5.892281532287598
retained_accuracy
0.9447944679216289
retained_total
26030
validation_trace_sha256
8029c7e25f188a3b…
validation_untrained
accuracy
0.2639222155568886
changed_accuracy
0.0012161968808127057
changed_total
13978
loss
8.85101318359375
retained_accuracy
0.40499423741836343
retained_total
26030

Written result

Every trained Gate G model scored below the identity policy — emit x_t unchanged — which is legal under argmax decoding and scores 0.6506 aggregate held-out accuracy. The objective was the suspected reason: each field is an independent softmax over a 289- or 417-way vocabulary, so "leave this one alone" costs as much as inventing a value, and 26,030 of 40,008 held-out fields are uncorrupted.

This run adds a per-field keep-or-change head and copies the input wherever the model says keep, making identity reachable as predict-keep-everywhere. 1,552 parameters on top of v5, +0.27%. Everything else is v5 exactly.

Result

modelaggregatechangedretained
v5 slot-bound0.37930.07870.5408
v6 edit-mask0.61190.01320.9335
identity0.65060.00001.0000
criterionthresholdobservedoutcome
beats_identity> 0.65060.6119falsified
retains_what_identity_retains>= 0.950.9335falsified
recovers_more_than_identity> 0.07870.0132falsified
stops_damaging_light_corruptionrecovery at 0.10 >= 0-0.1087falsified
structural_safetylocked-path exact, round-tripsboth truepass
reproducibilityidentical rerunidenticalpass

Overall: falsified, and the predeclared degenerate-collapse watch has fired. The model predicts keep on 91.4% of held-out fields. It moved most of the way to the trivial policy and stopped just short of it, arriving somewhere strictly worse than both v5 and identity: it recovers six times fewer corrupted fields than v5 while still not keeping as reliably as doing nothing.

The renders say the same thing. Mean x_hat_0 error now tracks the input almost exactly — 0.0580, 0.0895, 0.1362, 0.1767 against inputs of 0.0510, 0.0874, 0.1359, 0.1796 — and recovery is near zero everywhere:

corruption pv2v5v6identity
0.05-4.6517-3.1470-0.34730.0000
0.10-1.2024-0.7607-0.10870.0000
0.20-0.1315-0.0253-0.01010.0000
0.35+0.1800+0.2098+0.01470.0000

The edit mask did what it was built to do: it stopped the model destroying its input. It converted an actively harmful model into a nearly inert one.

Why it collapsed: corrupted fields are not detectable

The keep head is a corrupted-field detector, and measuring it directly answers the question this project has been circling since the corruption sweep.

quantityvalue
fields predicted keep0.9143
corrupted-field recall0.1058 (1,479 of 13,978)
corrupted-field precision0.4312 (1,479 of 3,430 flagged)
base rate of corrupted fields0.3494
false-alarm rate on uncorrupted fields0.0750

The detector finds one corrupted field in ten, and when it does flag one it is right 43% of the time against a 34.9% base rate — a lift of 1.23x. It is barely better than guessing.

That is not an architectural shortcoming, and it is why collapsing to keep is the rational thing for this model to do. A coordinate resampled uniformly from a 289-value legal vocabulary lands on a perfectly plausible coordinate. Telling it apart from a legitimate one requires already knowing what the icon should look like — which is the denoising problem itself. Detection is not an easier sub-problem than denoising; it is the same problem. With 65% of fields uncorrupted and no reliable way to find the other 35%, predicting keep is the loss-minimising policy, and the model found it.

What this closes

Seven candidate explanations have now been eliminated by predeclared comparison:

candidateeliminated by
too little datav2: 10.5x data, changed recovery 1.307x against a 1.5x bar
too little capacityv3: 3.53x parameters, same loss floor reached sooner
wrong corruption levelsweep: output near-independent of input, r = 0.91
no noise-level informationv4: every figure moved, none enough to matter
encoder slot-blindnessv5: real defect, 768 parameters, still r = 0.8621
objective cannot express identityv6: it can now, and the model collapses onto it
—detection is not separable from denoising

The predeclared falsification meaning for this run said that if the model could not clear the trivial policy with identity one prediction away, the formulation itself is in question and it should be written up as a Gate G-level negative result rather than patched. That is the situation, and this is that write-up.

Factorized role-uniform categorical corruption at p=0.35 over a 289/417-value coordinate vocabulary produces states from which the corruption is not identifiable, and therefore not invertible, by a model of this class. The project's working hypothesis names structure-aware categorical corruption; Gate F selected the factorized process as primary on four-icon fixtures where held-out recovery ran at 96–99% because the model had memorized the fixtures. That selection does not survive contact with 2,681 icons, and the Gate F comparison should be re-read as a memorization comparison rather than a denoising one.

Scope

Fixed-topology and geometry-only, single seed, one corruption draw per icon per level. This is a negative result about one corruption process at one probability with one model class. It does not show that the typed-SVG representation is unlearnable — Gate C and Gate D stand, the codec is exact, and v5 showed the encoder fix was worth 768 parameters. It shows that this corruption process does not define an invertible problem.

State transitions

  1. planned2026-09-20T21:00:00Z
  2. completed2026-09-20T21:20:00Zedit_mask_stops_the_damage_but_the_model_collapses_to_keep_because_corrupted_fields_are_not_detectable

Run record

Verbatim from runs/openmoji-g1-edit-mask-v6-0228c3b-0f6ce115-9b9b1699/run.yaml, the record committed before launch.

schema_version
1
run_id
openmoji-g1-edit-mask-v6-0228c3b-0f6ce115-9b9b1699
state
completed
parent_run
openmoji-g1-slot-bound-v5-f502df1-d8af55ea-9b9b1699
hypothesis
Every Gate G model so far scores below the identity policy on held-out token accuracy because the objective makes identity expensive: each field is an independent softmax over a 289- or 417-way vocabulary, so "leave this one alone" costs as much as inventing a new value, and 26,030 of 40,008 held-out fields are uncorrupted. Adding a per-field keep-or-change decision, and copying the input wherever it says keep, makes identity reachable as predict-keep-everywhere. The model should then clear the identity baseline rather than fall 0.27 short of it.
expected_information_gain
This is the first Gate G criterion set against the trivial policy rather than against noise, and it tests the last candidate standing after data volume, capacity, the corruption regime, noise-level information and encoder slot-blindness were each eliminated by a predeclared comparison. Clearing identity would be the first evidence in the sequence that the model does something useful rather than something harmful.
scope_limits
Fixed-topology and geometry-only, single seed. Exactly v5 plus 1,552 parameters of keep heads, +0.27%, which is not a plausible capacity effect given v3 showed a 3.53x increase does nothing. Beating identity in token space would not mean the output is a recognizable icon; no run in this sequence has produced one.
changed_factors
primary
model.edit_mask false -> true. 578,320 -> 579,872 parameters.
objective_change
Training loss becomes keep-or-change cross-entropy plus value cross-entropy on changed fields only, and decoding copies the input token wherever the model predicts keep. Accuracy is measured on the gated prediction, so it stays directly comparable with v1 through v5 and with the identity baseline.
held_fixed
slot binding on, the full 2,681-icon split, the 128-icon validation draw and its corruption seeds, seed 3101, corruption fixed at 0.35, batch size 16, learning rate 0.001, eval_every 60, the 6,300-step cap, the held-out-loss selection policy with patience 8, the codec, and the bucket. Noise conditioning stays off, since v4 falsified it.
code
git_commit
0228c3be2b9ab81399842f14a438557e796d10df
execution_mode
native-local
config
path
configs/learning/openmoji-g1-edit-mask-v6.yaml
sha256
0f6ce115c1409288…
dataset
hybrid_sha256
9b9b1699677a6f97…
bucket
bucket-p32-t128
train_samples
2681
validation_samples
128
baselines
identity
note
Emit x_t unchanged. A legal policy under argmax decoding.
aggregate
0.6506198760247951
changed
0.0
retained
1.0
render_recovery_fraction
0.0
v5_slot_bound
aggregate
0.3793491301739652
changed
0.07869509228788095
retained
0.5407990779869382
held_out_loss
7.178236961364746
mean_recovery_fraction_72px
0.05
-3.147
0.10
-0.7607
0.20
-0.0253
0.35
0.2098
per_icon_pearson_r_p005_vs_p035
0.8621
execution
seed_set
3101
deterministic_algorithms
true
cublas_workspace_config
:4096:8
optimizer_step_cap
6300
early_stopping
objective
held_out_loss
patience_evals
8
min_delta
0.0
resource_cap
max_steps
20000
max_storage_gb
50
network
none
predeclared_criteria
note
Declared before launch. No preflight. Held-out loss is not comparable across this change, because the edit-mask objective is a different loss; it is reported, and still used for checkpoint selection, but not gated.
evaluated_at
the checkpoint selected by held-out loss at corruption 0.35
primary
id
beats_identity
statement
Held-out aggregate token accuracy exceeds 0.6506198760247951, the identity policy. This is the criterion every earlier run should have carried.
id
retains_what_identity_retains
statement
Held-out retained-token accuracy is at least 0.95, against v5's 0.5408 and identity's 1.0. A model that can say keep should keep almost everything that was never corrupted.
id
recovers_more_than_identity
statement
Held-out changed-token accuracy exceeds v5's 0.07869509228788095, so the gain is not bought purely by copying. Identity scores 0.0 here, so beating identity on aggregate while scoring 0 on changed would be a degenerate pass and this criterion exists to catch it.
id
stops_damaging_light_corruption
statement
Mean per-icon recovery fraction at corruption 0.10 and 72 px is at least 0, against v5's -0.7607. Identity scores exactly 0, so this is the render-space form of "no worse than doing nothing".
id
structural_safety
statement
Locked-path exactness holds and the canonical checkpoint round-trips.
id
reproducibility
statement
A second complete invocation returns an identical JSON result.
standing_gate_not_a_prediction
id
retained_preservation_full
statement
Held-out retained-token accuracy is at least 0.90.
note
The original Gate G bar, unmet since v1 at 0.3256 through 0.5408. This run is the first with a mechanism that could plausibly meet it, which is why the primary criterion above is set higher still at 0.95.
degenerate_pass_watch
A model that predicts keep everywhere scores exactly identity: 0.6506 aggregate, 1.0 retained, 0.0 changed. That would pass beats_identity and retains_what_identity_retains while failing recovers_more_than_identity, and must be reported as a degenerate collapse to the trivial policy rather than as a win. The fraction of fields predicted keep is reported for exactly this reason.
falsification_meaning
If the model still cannot clear the trivial policy even when identity is one prediction away, the problem is not the objective's expressiveness and the fixed-topology geometry-denoising formulation itself is in question. That would be a Gate G-level negative result and should be written up as one rather than patched.
reported_not_gated
the fraction of held-out fields the model predicts keep, to detect collapse, held-out loss, which is not comparable across the objective change, the full four-level recovery sweep against v2, v4, v5 and identity, the selected step, wall time, and peak memory
outputs
local_metadata
runs/openmoji-g1-edit-mask-v6-0228c3b-0f6ce115-9b9b1699
report_root
reports/learning/openmoji-g1-edit-mask-v6
durable_artifacts
/home/dev/.cache/openmoji-g1-edit-mask-v6-0228c3b-0f6ce115-9b9b1699
planned_at
2026-09-20 21:00:00+00:00
completed_at
2026-09-20 21:20:00+00:00
result
device
cuda
model_parameters
579872
selected_step
1380
completed_steps
1860
ended_by
early_stopping
held_out_loss
5.8577985763549805
held_out_loss_note
not comparable with v1-v5; the edit-mask objective is a different loss
held_out_aggregate_accuracy
0.6119276144771045
held_out_changed_accuracy
0.013163542709972815
held_out_retained_accuracy
0.9334613907030349
identical_rerun
true
keep_head_diagnostics
fields_predicted_keep
0.9143
corrupted_field_recall
0.1058
corrupted_field_precision
0.4312
base_rate_of_corrupted_fields
0.3494
precision_lift_over_base_rate
1.23
false_alarm_rate_on_uncorrupted
0.075
mean_recovery_fraction_72px
0.05
-0.3473
0.10
-0.1087
0.20
-0.0101
0.35
0.0147
mean_x_hat_0_error_72px
0.05
0.058
0.10
0.08947
0.20
0.13619
0.35
0.17673
predeclared_outcome
overall
falsified
beats_identity
passed
false
observed
0.6119276144771045
threshold
0.6506198760247951
retains_what_identity_retains
passed
false
observed
0.9334613907030349
threshold
0.95
recovers_more_than_identity
passed
false
observed
0.013163542709972815
threshold
0.07869509228788095
stops_damaging_light_corruption
passed
false
observed
-0.1087
threshold
0.0
structural_safety
passed
true
reproducibility
passed
true
degenerate_pass_watch
fired
true
statement
The model predicts keep on 91.4% of held-out fields. It moved most of the way to the trivial policy and stopped just short, landing strictly worse than both v5 and identity: six times fewer corrupted fields recovered than v5, while still not keeping as reliably as doing nothing.
conclusion
The edit mask did what it was built to do and stopped the model destroying its input, converting an actively harmful model into a nearly inert one. It collapsed onto keep because corrupted fields are not detectable: recall 0.1058, precision 0.4312 against a 0.3494 base rate, a lift of 1.23x. A coordinate resampled uniformly from a 289-value legal vocabulary is a plausible coordinate, so telling it apart requires already knowing the icon - detection is not an easier sub-problem than denoising, it is the same problem. With 65% of fields uncorrupted and no reliable way to find the rest, predicting keep is loss-minimising and the model found it.
gate_level_finding
Factorized role-uniform categorical corruption at p=0.35 over a 289/417-value coordinate vocabulary produces states from which the corruption is not identifiable, and therefore not invertible, by a model of this class. Gate F selected this process as primary on four-icon fixtures at 96-99% held-out recovery, which does not survive contact with 2,681 icons; that comparison should be re-read as a memorization comparison rather than a denoising one.