Page of Shame

Papers whose reported model values disagree with best reported performance (today) and at-publication bests

Page of Shame: Papers Using Wrong Model Values

This page lists papers whose reported primary-metric value for a model disagrees with best reported performance (today’s global best on the leaderboard) by more than the reported std, or differs at all when no std is given.

Each row also compares against the best known at publication (best result for that model on or before the viewing paper’s publish date). That split matches the paper pages:

Best at pub — reported value was already worse than the best known when the paper came out (fair criticism; red on paper pages).
Best today (surpassed since publication only) — OK vs the at-publication best, but today’s leaderboard best is better (orange on paper pages).
Best reported performance — current leaderboard best (used for rank and the green value).

Summary

Statistic	Count
Flagged papers	845
Flagged rows	9379
Baseline rows	9182
Best at pub	6613
Best today only (not best at pub)	2762
At-pub best ≠ today’s best (different source papers)	5544
At-pub best = today’s best (same source paper)	3835

Today’s best source timing

When the best reported holder published relative to the viewing paper:

Timing	Rows
Already known at publication	4173
Published later (field moved on)	5206
Same paper	0
Unknown date	0

Most affected datasets

Dataset	Wrong Rows	Best at pub	Best today only	Papers	Models	Worst Rank Δ
PubMed	731	550	181	154	106	858
CiteSeer	628	477	151	149	110	860
Cora	627	481	146	159	103	890
Amazon-Photo	521	324	197	115	83	571
Actor	505	393	112	101	79	430
Amazon-Computers	449	268	181	95	67	494
Texas	427	292	135	101	73	536
Cornell	384	289	95	100	72	532
NCI1	317	238	79	98	71	578
COLLAB	246	178	68	70	56	326
MUTAG	237	173	64	111	65	745
Squirrel	228	164	64	43	51	232
Chameleon	212	164	48	47	57	281
WikiCS	188	123	65	40	56	263
Coauthor-Physics	175	121	54	50	36	267
ogbn-products	173	113	58	60	45	342
BBBP	160	120	40	40	40	259
MD17	153	118	35	35	40	113
Coauthor-CS	150	91	59	39	26	202
Tox21	143	124	19	43	36	305
PROTEINS	119	84	35	57	41	471
ToxCast	115	95	20	30	29	174
ENZYMES	106	68	38	40	39	274
DBLP	105	76	29	24	36	180
BACE	96	71	25	35	32	268

Most affected models

Model	Wrong Rows	Best at pub	Best today only	Papers	Datasets	Worst Rank Δ
GCN	1325	1029	296	452	77	890
GAT	902	721	181	319	64	868
MLP	312	209	103	114	32	665
GIN	308	239	69	149	52	718
GraphSAGE	303	206	97	151	46	793
APPNP	233	184	49	80	19	706
DGI	226	134	92	86	23	828
SGC	195	150	45	85	21	742
GRACE	184	104	80	70	21	654
MVGRL	170	98	72	69	23	723
GCNII	169	105	64	73	17	733
H2GCN	147	112	35	60	19	772
GraphCL	138	99	39	55	22	620
GPRGNN	137	94	43	52	19	749
BGRL	112	92	20	42	19	681
FAGCN	108	88	20	42	14	660
DeepWalk	98	65	33	39	16	638
JOAO	97	76	21	29	13	281
GatedGCN	94	66	28	44	14	175
WL	91	84	7	45	6	631
GAE	88	62	26	30	17	521
JKNet	83	63	20	32	13	821
SAGE	78	39	39	44	21	740
GCA	73	55	18	37	15	366
InfoGraph	72	59	13	39	11	224

Detailed table: 845 rows (loaded below).

Flagged Rows

Detailed table: 500 rows (loaded below).

*Today’s best source differs from the at-publication best for that model.

Loading detailed tables…