openmoji-g1-detection-only-v7-856230a-9d2db303-9b9b1699

completed falsified

Hypothesis

The v6 keep head reached only 1.23x precision lift because the value objective dominated the shared encoder, not because the detection signal is unlearnable. The value term is a 289- or 417-class problem the model demonstrably cannot solve; the keep head is a 2-class problem a zero-parameter local-continuity statistic solves at 2.13x lift. Removing the value term should let the encoder learn representations that serve detection, and the trained detector should then at least match the free one.

Why run it

The registered corpus-scale process comparison assumes a trained model can learn detection on some corruption process. That assumption has never been tested. If the model cannot match a zero-parameter statistic even with nothing competing for the representation, comparing corruption processes is premature and the conclusion is about the model class. This costs one training run and blocks a two-arm comparison.

Scope limits

Fixed-topology and geometry-only, single seed, factorized corruption at p=0.35. A detection-only model cannot reconstruct anything, so it is a diagnostic and not a candidate design; held-out accuracy is meaningless here because the value head is untrained. The gated quantity is detector quality alone.

Measured behaviour

Held-out loss

Table view

optimizer step	held-out loss
0	1.4640
60	0.8373
120	0.7097
180	0.6862
240	0.6655
300	0.6721
360	0.7311
420	0.7186
480	0.6560
540	0.6983
600	0.6786
660	0.6171
720	0.6432
780	0.6581
840	0.6436
900	0.6808
960	0.6565
1020	0.6586
1080	0.6682
1140	0.6181

Held-out token accuracy

aggregatechanged fieldsretained fields

Table view

optimizer step	aggregate	changed fields	retained fields
0	0.2639	0.0012	0.4050
60	0.5020	0.0004	0.7714
120	0.5568	0.0003	0.8557
180	0.5947	0.0004	0.9139
240	0.5896	0.0004	0.9061
300	0.5825	0.0004	0.8951
360	0.5928	0.0001	0.9110
420	0.5889	0.0002	0.9050
480	0.6044	0.0001	0.9289
540	0.5942	0.0001	0.9132
600	0.5973	0.0003	0.9179
660	0.6035	0.0003	0.9274
720	0.6050	0.0002	0.9297
780	0.5981	0.0003	0.9191
840	0.6170	0.0004	0.9481
900	0.6087	0.0003	0.9355
960	0.6053	0.0004	0.9301
1020	0.5843	0.0005	0.8977
1080	0.5956	0.0006	0.9151
1140	0.6065	0.0002	0.9321

Training token accuracy (per batch)

Table view

optimizer step	train accuracy
1	0.2863
2	0.2784
3	0.3628
4	0.3907
5	0.4085
6	0.4616
7	0.4666
8	0.5007
9	0.4970
10	0.4941
11	0.4805
12	0.5089
13	0.4992
14	0.4981
15	0.4786
16	0.4589
17	0.4487
18	0.4509
19	0.4368
20	0.4561
21	0.4281
22	0.4271
23	0.4459
24	0.4346
25	0.4393
26	0.4371
27	0.4345
28	0.4538
29	0.4553
30	0.4528
31	0.4819
32	0.4712
33	0.4724
34	0.4686
35	0.4668
36	0.4587
37	0.4728
38	0.4574
39	0.4648
40	0.4707
41	0.4910
42	0.4705
43	0.4796
44	0.4883
45	0.4815
46	0.4968
47	0.5026
48	0.4742
49	0.4752
50	0.4941
51	0.4957
52	0.5155
53	0.4835
54	0.5048
55	0.4937
56	0.4865
57	0.4913
58	0.4920
59	0.5008
60	0.4998
61	0.5059
62	0.4978
63	0.5130
64	0.5147
65	0.5171
66	0.5138
67	0.5038
68	0.5317
69	0.5132
70	0.5217
71	0.5434
72	0.5395
73	0.5437
74	0.5188
75	0.5519
76	0.5489
77	0.5132
78	0.5247
79	0.5026
80	0.5248
81	0.5130
82	0.5112
83	0.5130
84	0.4987
85	0.5124
86	0.4979
87	0.4940
88	0.5005
89	0.5211
90	0.5114
91	0.5079
92	0.5126
93	0.5136
94	0.5187
95	0.5240
96	0.5251
97	0.5352
98	0.5446
99	0.5375
100	0.5401
101	0.5675
102	0.5486
103	0.5505
104	0.5333
105	0.5709
106	0.5433
107	0.5670
108	0.5539
109	0.5779
110	0.5571
111	0.5497
112	0.5541
113	0.5353
114	0.5411
115	0.5666
116	0.5471
117	0.5432
118	0.5211
119	0.5577
120	0.5403
121	0.5536
122	0.5748
123	0.5598
124	0.5632
125	0.5612
126	0.5638
127	0.5690
128	0.5716
129	0.5706
130	0.5734
131	0.5596
132	0.5634
133	0.5589
134	0.5706
135	0.5377
136	0.5533
137	0.5388
138	0.5737
139	0.5618
140	0.5572
141	0.5525
142	0.5630
143	0.5445
144	0.5355
145	0.5475
146	0.5624
147	0.5590
148	0.5517
149	0.5435
150	0.5544
151	0.5690
152	0.5532
153	0.5600
154	0.5615
155	0.5792
156	0.5563
157	0.5621
158	0.5698
159	0.5694
160	0.5666
161	0.5529
162	0.5712
163	0.5632
164	0.5590
165	0.5452
166	0.5523
167	0.5610
168	0.5513
169	0.5730
170	0.5857
171	0.5631
172	0.5908
173	0.5889
174	0.5818
175	0.5877
176	0.5781
177	0.5929
178	0.5825
179	0.6009
180	0.6050
181	0.5878
182	0.6008
183	0.5843
184	0.5900
185	0.6054
186	0.6049
187	0.5874
188	0.5824
189	0.5615
190	0.5827
191	0.5575
192	0.5711
193	0.5563
194	0.5382
195	0.5483
196	0.5413
197	0.5251
198	0.5229
199	0.5336
200	0.5513
201	0.5549
202	0.5554
203	0.5767
204	0.5764
205	0.5770
206	0.6069
207	0.6054
208	0.6225
209	0.6013
210	0.6042
211	0.5698
212	0.6012
213	0.5873
214	0.5721
215	0.5855
216	0.5693
217	0.5803
218	0.5633
219	0.5843
220	0.5801
221	0.5745
222	0.5634
223	0.5810
224	0.5724
225	0.5769
226	0.5613
227	0.5617
228	0.5710
229	0.5566
230	0.5680
231	0.5729
232	0.5624
233	0.5846
234	0.5920
235	0.5863
236	0.6012
237	0.5837
238	0.5881
239	0.6076
240	0.6044
241	0.6013
242	0.6008
243	0.6078
244	0.5887
245	0.5843
246	0.6148
247	0.5917
248	0.5911
249	0.5864
250	0.6058
251	0.6033
252	0.6091
253	0.5948
254	0.5917
255	0.5950
256	0.5937
257	0.5973
258	0.5960
259	0.6015
260	0.5899
261	0.5826
262	0.5883
263	0.5910
264	0.5792
265	0.5954
266	0.5769
267	0.5889
268	0.5928
269	0.6121
270	0.5879
271	0.5940
272	0.5995
273	0.5892
274	0.5915
275	0.5639
276	0.5826
277	0.5839
278	0.5775
279	0.5988
280	0.5879
281	0.6004
282	0.6058
283	0.5779
284	0.5964
285	0.6014
286	0.6139
287	0.6240
288	0.6049
289	0.6306
290	0.6214
291	0.6051
292	0.5996
293	0.6226
294	0.6107
295	0.6018
296	0.6118
297	0.6093
298	0.6163
299	0.5977
300	0.5957
301	0.6031
302	0.5896
303	0.5740
304	0.5948
305	0.5723
306	0.5896
307	0.5852
308	0.5760
309	0.5839
310	0.5787
311	0.5664
312	0.5841
313	0.5929
314	0.5892
315	0.5980
316	0.6028
317	0.6117
318	0.6130
319	0.6139
320	0.6115
321	0.5959
322	0.6056
323	0.6041
324	0.5972
325	0.5865
326	0.5986
327	0.5889
328	0.5847
329	0.5844
330	0.5792
331	0.5736
332	0.5684
333	0.5578
334	0.5823
335	0.5785
336	0.6130
337	0.5913
338	0.6023
339	0.5782
340	0.5952
341	0.6052
342	0.5924
343	0.5924
344	0.5980
345	0.5969
346	0.5959
347	0.6125
348	0.5939
349	0.6088
350	0.6221
351	0.6067
352	0.6054
353	0.5967
354	0.6144
355	0.5853
356	0.5871
357	0.5945
358	0.5844
359	0.5735
360	0.5885
361	0.6031
362	0.5902
363	0.6165
364	0.5943
365	0.5994
366	0.6116
367	0.6091
368	0.6047
369	0.6015
370	0.6023
371	0.6166
372	0.5767
373	0.5992
374	0.5871
375	0.6031
376	0.5886
377	0.5799
378	0.5788
379	0.5866
380	0.5960
381	0.5779
382	0.5849
383	0.5829
384	0.5966
385	0.5733
386	0.5793
387	0.5851
388	0.5970
389	0.6039
390	0.6139
391	0.6139
392	0.6251
393	0.5828
394	0.6297
395	0.6004
396	0.6116
397	0.5949
398	0.5926
399	0.6109
400	0.5903
401	0.6152
402	0.6073
403	0.6025
404	0.6138
405	0.6122
406	0.6181
407	0.6035
408	0.6277
409	0.6159
410	0.6080
411	0.6054
412	0.6097
413	0.5935
414	0.5907
415	0.5829
416	0.5890
417	0.5931
418	0.5975
419	0.5921
420	0.5930
421	0.5868
422	0.6010
423	0.5864
424	0.6016
425	0.5976
426	0.6021
427	0.5995
428	0.5957
429	0.6183
430	0.6157
431	0.6022
432	0.6059
433	0.6098
434	0.6035
435	0.6165
436	0.6060
437	0.6235
438	0.6217
439	0.6031
440	0.6323
441	0.6218
442	0.6141
443	0.6182
444	0.6106
445	0.6095
446	0.6046
447	0.6114
448	0.6140
449	0.6095
450	0.6081
451	0.6050
452	0.6094
453	0.6066
454	0.6070
455	0.6122
456	0.6160
457	0.6064
458	0.5950
459	0.5975
460	0.5987
461	0.5948
462	0.6053
463	0.5970
464	0.5991
465	0.6123
466	0.6006
467	0.5978
468	0.6099
469	0.5934
470	0.6063
471	0.6111
472	0.6047
473	0.6184
474	0.5964
475	0.6241
476	0.5962
477	0.6153
478	0.5890
479	0.5849
480	0.6001
481	0.5895
482	0.6228
483	0.6079
484	0.6315
485	0.6117
486	0.6069
487	0.6170
488	0.5994
489	0.6173
490	0.6020
491	0.5961
492	0.6079
493	0.6104
494	0.6112
495	0.6176
496	0.6191
497	0.6058
498	0.6011
499	0.5989
500	0.6303
501	0.6111
502	0.6021
503	0.6014
504	0.6184
505	0.5885
506	0.6002
507	0.6122
508	0.6084
509	0.5985
510	0.5909
511	0.5842
512	0.6057
513	0.6119
514	0.6061
515	0.6292
516	0.6263
517	0.6176
518	0.6110
519	0.6226
520	0.6255
521	0.6172
522	0.6162
523	0.6091
524	0.5998
525	0.6178
526	0.6220
527	0.5946
528	0.6268
529	0.6074
530	0.6220
531	0.6143
532	0.6340
533	0.6276
534	0.6249
535	0.6212
536	0.6129
537	0.6161
538	0.6105
539	0.6144
540	0.5861
541	0.5934
542	0.6049
543	0.5872
544	0.6084
545	0.6118
546	0.6014
547	0.6049
548	0.5896
549	0.6157
550	0.6021
551	0.6059
552	0.6074
553	0.5837
554	0.6014
555	0.5933
556	0.5967
557	0.6055
558	0.5751
559	0.6058
560	0.6038
561	0.6051
562	0.6107
563	0.6070
564	0.6086
565	0.6119
566	0.6170
567	0.5948
568	0.6094
569	0.6150
570	0.5966
571	0.6039
572	0.6142
573	0.6184
574	0.6238
575	0.6161
576	0.6224
577	0.6128
578	0.6211
579	0.6125
580	0.6048
581	0.6275
582	0.6013
583	0.6148
584	0.6168
585	0.6035
586	0.6078
587	0.6161
588	0.6058
589	0.6031
590	0.6118
591	0.6119
592	0.6002
593	0.6157
594	0.6284
595	0.6106
596	0.6177
597	0.5985
598	0.5987
599	0.5981
600	0.6121
601	0.6084
602	0.6124
603	0.6112
604	0.6330
605	0.6228
606	0.6226
607	0.6285
608	0.6208
609	0.6241
610	0.6119
611	0.5980
612	0.6214
613	0.6073
614	0.6171
615	0.5866
616	0.6006
617	0.6162
618	0.6052
619	0.6152
620	0.6343
621	0.5993
622	0.6300
623	0.5896
624	0.6184
625	0.6305
626	0.6188
627	0.6043
628	0.6043
629	0.5995
630	0.6124
631	0.6277
632	0.6196
633	0.6184
634	0.6163
635	0.6238
636	0.6159
637	0.6097
638	0.6147
639	0.6243
640	0.5924
641	0.6219
642	0.6036
643	0.5935
644	0.5819
645	0.5998
646	0.5832
647	0.5999
648	0.6047
649	0.6231
650	0.6228
651	0.6168
652	0.6206
653	0.6279
654	0.5993
655	0.6022
656	0.6123
657	0.6006
658	0.6037
659	0.5964
660	0.5807
661	0.6102
662	0.6164
663	0.6139
664	0.6150
665	0.6117
666	0.6083
667	0.6031
668	0.5945
669	0.5868
670	0.5907
671	0.5842
672	0.6048
673	0.5965
674	0.6016
675	0.6043
676	0.6157
677	0.6034
678	0.5972
679	0.6085
680	0.5992
681	0.5913
682	0.6007
683	0.6002
684	0.5894
685	0.5900
686	0.5842
687	0.6038
688	0.5837
689	0.6083
690	0.6089
691	0.6032
692	0.6093
693	0.6089
694	0.6081
695	0.6057
696	0.5950
697	0.5926
698	0.5778
699	0.5953
700	0.5729
701	0.5756
702	0.5919
703	0.5827
704	0.6087
705	0.5993
706	0.6177
707	0.6156
708	0.5959
709	0.6178
710	0.6013
711	0.6081
712	0.5890
713	0.5866
714	0.5716
715	0.5820
716	0.5934
717	0.6017
718	0.5982
719	0.5943
720	0.6120
721	0.6087
722	0.6037
723	0.6018
724	0.5888
725	0.5889
726	0.5991
727	0.5880
728	0.5848
729	0.5877
730	0.5976
731	0.6005
732	0.5941
733	0.6227
734	0.6102
735	0.6227
736	0.6265
737	0.6129
738	0.6135
739	0.6367
740	0.6193
741	0.6110
742	0.6280
743	0.5998
744	0.6006
745	0.5787
746	0.5798
747	0.5940
748	0.5836
749	0.5886
750	0.5971
751	0.6214
752	0.6289
753	0.6112
754	0.6170
755	0.6106
756	0.6155
757	0.6099
758	0.5767
759	0.5888
760	0.5744
761	0.6026
762	0.5879
763	0.6082
764	0.6101
765	0.6231
766	0.6292
767	0.6224
768	0.6128
769	0.6151
770	0.6079
771	0.6152
772	0.6142
773	0.6060
774	0.6021
775	0.5961
776	0.5902
777	0.6137
778	0.5968
779	0.6088
780	0.6003
781	0.5968
782	0.6142
783	0.5987
784	0.6066
785	0.5803
786	0.5837
787	0.6016
788	0.6117
789	0.5979
790	0.6067
791	0.6177
792	0.6182
793	0.6042
794	0.6121
795	0.6049
796	0.6011
797	0.6135
798	0.5917
799	0.6171
800	0.6096
801	0.6010
802	0.6132
803	0.5977
804	0.5935
805	0.5899
806	0.5774
807	0.5981
808	0.6071
809	0.6172
810	0.5900
811	0.5838
812	0.5971
813	0.6070
814	0.5778
815	0.6003
816	0.5931
817	0.5946
818	0.6027
819	0.6162
820	0.6020
821	0.6163
822	0.5950
823	0.6106
824	0.6005
825	0.5907
826	0.6089
827	0.5972
828	0.5905
829	0.6040
830	0.5857
831	0.5713
832	0.5569
833	0.5781
834	0.5605
835	0.5866
836	0.5923
837	0.6093
838	0.6211
839	0.6377
840	0.6297
841	0.6234
842	0.6176
843	0.5982
844	0.6036
845	0.5675
846	0.5590
847	0.5707
848	0.5786
849	0.5867
850	0.6160
851	0.6169
852	0.6155
853	0.6190
854	0.6130
855	0.6083
856	0.6105
857	0.5853
858	0.5826
859	0.5766
860	0.5850
861	0.5816
862	0.5755
863	0.5990
864	0.5931
865	0.5907
866	0.5957
867	0.6080
868	0.6051
869	0.5963
870	0.5993
871	0.6004
872	0.5944
873	0.5987
874	0.6035
875	0.5923
876	0.6040
877	0.5942
878	0.6037
879	0.6164
880	0.5934
881	0.5838
882	0.5847
883	0.6032
884	0.5883
885	0.5787
886	0.6019
887	0.6050
888	0.6010
889	0.6132
890	0.6076
891	0.6087
892	0.6247
893	0.6115
894	0.5967
895	0.5937
896	0.6048
897	0.5972
898	0.5969
899	0.6071
900	0.6126
901	0.6182
902	0.6320
903	0.6042
904	0.6224
905	0.6015
906	0.6130
907	0.6014
908	0.6083
909	0.5884
910	0.5891
911	0.5833
912	0.5876
913	0.5870
914	0.5854
915	0.5950
916	0.5970
917	0.6227
918	0.6048
919	0.6214
920	0.6056
921	0.6150
922	0.6114
923	0.6061
924	0.6163
925	0.5927
926	0.5718
927	0.5541
928	0.5646
929	0.5984
930	0.5906
931	0.6185
932	0.6215
933	0.6119
934	0.6239
935	0.6203
936	0.6124
937	0.6059
938	0.6140
939	0.6016
940	0.6041
941	0.5844
942	0.6261
943	0.6027
944	0.6137
945	0.6015
946	0.6090
947	0.5733
948	0.6091
949	0.6042
950	0.6074
951	0.6109
952	0.6031
953	0.5977
954	0.6093
955	0.6112
956	0.6091
957	0.6062
958	0.5970
959	0.6153
960	0.6151
961	0.6071
962	0.6187
963	0.6229
964	0.6126
965	0.6073
966	0.6093
967	0.6136
968	0.6135
969	0.5981
970	0.5880
971	0.5848
972	0.6072
973	0.6096
974	0.6262
975	0.5983
976	0.6222
977	0.6153
978	0.6085
979	0.5798
980	0.6128
981	0.6123
982	0.5988
983	0.5789
984	0.6157
985	0.6133
986	0.6117
987	0.5961
988	0.5899
989	0.5990
990	0.5692
991	0.5932
992	0.5941
993	0.6044
994	0.5761
995	0.6004
996	0.6163
997	0.6103
998	0.6068
999	0.6185
1000	0.6172
1001	0.5968
1002	0.6028
1003	0.5771
1004	0.5783
1005	0.5568
1006	0.5478
1007	0.5503
1008	0.5664
1009	0.5831
1010	0.5923
1011	0.5915
1012	0.6145
1013	0.6141
1014	0.6022
1015	0.6064
1016	0.6152
1017	0.6112
1018	0.6024
1019	0.5776
1020	0.6029
1021	0.5833
1022	0.5908
1023	0.6066
1024	0.6099
1025	0.6213
1026	0.6107
1027	0.6028
1028	0.6150
1029	0.6087
1030	0.5915
1031	0.5902
1032	0.5725
1033	0.5768
1034	0.5945
1035	0.5916
1036	0.5872
1037	0.5826
1038	0.6086
1039	0.5983
1040	0.5953
1041	0.6185
1042	0.6087
1043	0.6067
1044	0.6152
1045	0.6099
1046	0.5995
1047	0.5858
1048	0.5754
1049	0.5679
1050	0.5842
1051	0.5827
1052	0.5724
1053	0.5774
1054	0.5724
1055	0.5880
1056	0.5940
1057	0.6027
1058	0.6097
1059	0.5938
1060	0.6006
1061	0.6085
1062	0.6001
1063	0.5847
1064	0.5953
1065	0.5906
1066	0.5855
1067	0.5872
1068	0.5988
1069	0.6186
1070	0.6081
1071	0.6004
1072	0.6000
1073	0.6147
1074	0.6094
1075	0.6204
1076	0.6098
1077	0.6149
1078	0.6121
1079	0.6161
1080	0.6167
1081	0.5969
1082	0.6103
1083	0.6021
1084	0.5933
1085	0.6001
1086	0.6065
1087	0.6091
1088	0.6135
1089	0.6043
1090	0.6038
1091	0.5966
1092	0.5933
1093	0.5924
1094	0.6289
1095	0.5948
1096	0.5942
1097	0.6133
1098	0.6020
1099	0.5992
1100	0.5937
1101	0.5862
1102	0.5813
1103	0.5970
1104	0.5942
1105	0.5955
1106	0.6158
1107	0.6194
1108	0.6242
1109	0.6222
1110	0.6172
1111	0.6279
1112	0.6053
1113	0.6012
1114	0.5989
1115	0.5940
1116	0.6005
1117	0.5986
1118	0.5922
1119	0.6042
1120	0.6017
1121	0.6007
1122	0.6113
1123	0.5892
1124	0.6064
1125	0.6277
1126	0.6127
1127	0.6156
1128	0.5833
1129	0.5833
1130	0.5905
1131	0.5736
1132	0.6061
1133	0.5773
1134	0.6145
1135	0.5997
1136	0.6151
1137	0.5893
1138	0.5935
1139	0.5802
1140	0.6086

Measured result

Verbatim from the run's summary.json.

bucket

bucket-p32-t128

bucket_icons

3359

checkpoint_bytes

4715160

checkpoint_round_trip

true

checkpoint_sha256

ddf5de529a84e690…

config_sha256

9d2db303814d609a…

corruption

factorized_role_uniform_geometry

cuda_version

13.4

deterministic_algorithms

true

device

cuda

eval_every

final_train

loss

0.5561261773109436

step

1140

train_token_accuracy

0.6086463730569949

group_vocabulary_size

locked_path_exact

true

metrics_sha256

05d96b2d0c5ef0ef…

model_parameters

579872

schema_version

scope

dominant-bucket fixed-topology geometry pilot; not unconditional generation

selected_train_rows

color/svg/1F3F4-E0064-E0065-E0062-E0065-E007F.svg, color/svg/1F468-1F3FC-200D-1F9BC.svg, color/svg/1F561.svg, color/svg/1F469-1F3FE-200D-1F9BC.svg, color/svg/1F469-1F3FD-200D-2764-FE0F-200D-1F48B-200D-1F469-1F3FE.svg, color/svg/1F6B5-1F3FB-200D-2642-FE0F.svg … and 2675 more

selected_validation_rows

color/svg/1F994.svg, color/svg/E30A.svg, color/svg/1F6BE.svg, color/svg/1F3CA-1F3FB-200D-2642-FE0F.svg, color/svg/1F3CB-1F3FC-200D-2642-FE0F.svg, color/svg/1F93D-1F3FC.svg … and 122 more

selection

completed_steps

1140

evals_without_improvement

min_delta

0.0

objective

held_out_loss

patience_evals

selected_held_out_loss

0.6170586347579956

selected_step

660

stopped_early

true

steps

6300

study_version

openmoji-g1-detection-only-v7

subgroup_vocabulary_size

118

torch_version

2.14.0a0+4fdf77b940.nv26.08

validation

accuracy

0.603504299140172

changed_accuracy

0.00028616397195593076

changed_total

13978

loss

0.6170586347579956

retained_accuracy

0.9274298885900883

retained_total

26030

validation_final_step

accuracy

0.606503699260148

changed_accuracy

0.00021462297896694807

changed_total

13978

loss

0.6180558800697327

retained_accuracy

0.9320783711102574

retained_total

26030

validation_trace_sha256

d6d3f5593938c58b…

validation_untrained

accuracy

0.2639222155568886

changed_accuracy

0.0012161968808127057

changed_total

13978

loss

1.4640209674835205

retained_accuracy

0.40499423741836343

retained_total

26030

Written result

The registered corpus-scale process comparison assumes a trained model can learn corrupted-field detection on some corruption process. That assumption had never been tested, and the evidence against it was stark: a zero-parameter local-continuity statistic reaches 2.13x precision lift on factorized corruption, and the v6 keep head reached 1.23x.

The suspected reason was objective balance. The edit-mask loss is a 2-class keep decision plus a 289- or 417-class value prediction, sharing one encoder, so the encoder is shaped almost entirely by a value task the model demonstrably cannot do. This run drops the value term entirely. Nothing else changes.

Result

Precision lift at a matched 8.573% flag rate, over a 0.3494 base rate:

detector	parameters	precision	lift	recall
local-continuity statistic	0	0.7451	2.13	0.1824
v6 keep head, joint objective	579,872	0.4312	1.234	0.1058
v7 keep head, detection only	579,872	0.4770	1.365	0.1170

criterion	threshold	observed	outcome
beats_the_previous_trained_detector	> 1.234	1.365	pass
matches_the_free_detector	>= 2.13	1.365	falsified
reproducibility	identical rerun	identical	pass

The shared-encoder explanation is real but small. Removing a 289-class objective that was consuming the entire representation bought 0.13 of lift, closing 15% of the gap to a heuristic with no parameters at all. The model still cannot see what the statistic sees.

Held-out loss selected step 660 of a 6,300 cap and early stopping ended the run at 1,140, so it is not budget-limited: the model converged to this.

What it settles

The predeclared falsification meaning applies. The failure is not about objective balance and it is not about any particular corruption process, so comparing corruption processes is premature and the registered two-arm comparison is deferred rather than run. A comparison between processes is only meaningful once a trained model can learn the signal on at least one of them.

The conclusion is about this model class and training setup. That is a narrower claim than the one briefly recorded and withdrawn earlier — that the corruption is not identifiable — and it is better supported: the signal is demonstrably there, extractable by a statistic with no parameters, and this network trained this way does not find it.

Scope

One corruption process at one probability, one seed, one architecture. A detection-only model reconstructs nothing, so it is a diagnostic rather than a candidate design; held-out accuracy is meaningless here because the value head is untrained.

State transitions

planned2026-09-20T22:30:00Z
completed2026-09-20T22:45:00Zdetection_only_helps_a_little_but_the_model_still_loses_to_a_zero_parameter_statistic

Run record

Verbatim from runs/openmoji-g1-detection-only-v7-856230a-9d2db303-9b9b1699/run.yaml, the record committed before launch.

schema_version

run_id

openmoji-g1-detection-only-v7-856230a-9d2db303-9b9b1699

state

completed

parent_run

openmoji-g1-edit-mask-v6-0228c3b-0f6ce115-9b9b1699

blocks

openmoji-g1-corruption-process-corpus-76f41a3-2arms-9b9b1699

hypothesis

The v6 keep head reached only 1.23x precision lift because the value objective dominated the shared encoder, not because the detection signal is unlearnable. The value term is a 289- or 417-class problem the model demonstrably cannot solve; the keep head is a 2-class problem a zero-parameter local-continuity statistic solves at 2.13x lift. Removing the value term should let the encoder learn representations that serve detection, and the trained detector should then at least match the free one.

expected_information_gain

scope_limits

changed_factors

primary

training.detection_only false -> true, dropping the value term from the loss.

held_fixed

v6 exactly otherwise: slot binding and edit mask on, the full 2,681-icon split, the 128-icon validation draw and its corruption seeds, seed 3101, corruption 0.35, batch size 16, learning rate 0.001, eval_every 60, the 6,300-step cap, and the held-out-loss selection policy with patience 8. Model size is unchanged at 579,872.

code

git_commit

856230a

execution_mode

native-local

config

path

configs/learning/openmoji-g1-detection-only-v7.yaml

sha256

9d2db303814d609a…

dataset

hybrid_sha256

9b9b1699677a6f97…

bucket

bucket-p32-t128

train_samples

2681

validation_samples

128

baselines

measured_at_a_matched_8.573_percent_flag_rate

training_free_continuity_statistic

precision

0.7451

lift

2.13

recall

0.1824

v6_trained_keep_head

precision

0.4312

lift

1.234

recall

0.1058

base_rate_of_corrupted_fields

0.3494

execution

seed_set

3101

deterministic_algorithms

true

cublas_workspace_config

:4096:8

optimizer_step_cap

6300

early_stopping

objective

held_out_loss

patience_evals

min_delta

0.0

resource_cap

max_steps

20000

max_storage_gb

network

none

predeclared_criteria

note

Declared before launch. Measured by scripts/detector_report.py, which reproduces v6's published precision and recall exactly.

evaluated_at

the checkpoint selected by held-out loss, at a matched 8.573% flag rate

primary

beats_the_previous_trained_detector

statement

Precision lift over the base rate exceeds v6's 1.234.

matches_the_free_detector

statement

Precision lift reaches at least 2.13, the training-free local-continuity statistic's lift on the same corruption at the same flag rate. This is the real bar: a trained model that cannot match a zero-parameter heuristic on a signal the heuristic extracts is not learning the structure of the problem.

reproducibility

statement

A second complete invocation returns an identical JSON result.

falsification_meaning

If detection-only still cannot match the free statistic, the shared-encoder explanation is wrong and the failure is not about objective balance. The conclusion then concerns this model class and training setup rather than any corruption process, the registered two-arm comparison is premature and should be deferred, and the highest-value remaining move is PROJECT_PLAN.md section 12 branch 4 - the cached autoregressive model on the same codec, which is also Gate I and which tests whether the representation supports generation independent of any corruption process.

reported_not_gated

recall and precision at the model's own threshold and at the base flag rate, the selected step, whether the cap or early stopping ended it, wall time

outputs

local_metadata

runs/openmoji-g1-detection-only-v7-856230a-9d2db303-9b9b1699

report_root

reports/learning/openmoji-g1-detection-only-v7

durable_artifacts

/home/dev/.cache/openmoji-g1-detection-only-v7-856230a-9d2db303-9b9b1699

planned_at

2026-09-20 22:30:00+00:00

completed_at

2026-09-20 22:45:00+00:00

result

device

cuda

model_parameters

579872

selected_step

660

completed_steps

1140

ended_by

early_stopping

held_out_keep_loss

0.6170586347579956

identical_rerun

true

detector_at_matched_flag_rate

precision

0.47696793002915455

lift_over_base_rate

1.3651833556021185

recall

0.11703390738474746

detector_at_base_flag_rate

precision

0.41293461153240807

lift_over_base_rate

1.1819064199591203

recall

0.41293461153240807

predeclared_outcome

overall

falsified

beats_the_previous_trained_detector

passed

true

observed

1.3651833556021185

threshold

1.234

matches_the_free_detector

passed

false

observed

1.3651833556021185

threshold

2.13

reproducibility

passed

true

conclusion

The shared-encoder explanation is real but small: removing a 289-class objective that was consuming the entire representation bought 0.13 of lift and closed 15% of the gap to a zero-parameter heuristic. The model converged to this - held-out loss selected step 660 of a 6,300 cap - so it is not budget-limited. The failure is not about objective balance and not about any particular corruption process.

consequence

The registered two-arm corruption-process comparison openmoji-g1-corruption-process-corpus-76f41a3-2arms-9b9b1699 is deferred, not run. Comparing processes is only meaningful once a trained model can learn the signal on at least one of them.