bananamind-research-community/bananamind-2.0.1

Architecture

All four models use the same BananaMind 2.0-derived decoder-only language-model architecture and exactly the same physical parameters. The experiment changes only which physical decoder layers are reused during a forward pass and, for Partial HQ, the training dataset.

Property	Value
Total trainable parameters	1,480,516
Transformer core parameters, including token embedding	980,547 (66.23%)
Three physical decoder blocks alone	718,275 (48.52%)
Parameters per physical decoder block	239,425
Causal trigram embedding module	499,969 (33.77%)
Vocabulary	2,048 tokens
Tokenizer	BananaMind 2 Micro
Hidden width	128
MLP intermediate width	384
Physical decoder layers	3
Query heads	4
Key/value heads	1
Attention head dimension	32
Context length	3,072 tokens
RoPE theta	100,000
N-gram hash buckets	3,906
Refresh convolution	Kernel 9, causal depthwise

Models

Short name	Architecture execution	Training dataset
Partial HQ	L1 -> L2 -> L2 -> L3	epfml/FineWeb-HQ
No-loop Edu	L1 -> L2 -> L3	HuggingFaceFW/fineweb-edu
Partial Edu	L1 -> L2 -> L2 -> L3	HuggingFaceFW/fineweb-edu
All-looped Edu	L1 -> L2 -> L3 -> L1 -> L2 -> L3	HuggingFaceFW/fineweb-edu

Primary comparison

The public multiple-choice values below are acc_norm. ARC mean is the mean of ARC-Easy and ARC-Challenge before chance normalization.

Metric	Partial HQ	No-loop Edu	Partial Edu	All-looped Edu
Base Bench 1.1 overall Elo	889	884	885	875
Base Bench raw accuracy	35.43%	33.71%	34.57%	33.14%
Base Bench weighted accuracy	34.14%	33.57%	33.65%	32.54%
ARC-Easy acc_norm	29.50%	30.35%	29.50%	30.98%
ARC-Challenge acc_norm	22.18%	22.10%	22.53%	21.33%
ARC mean acc_norm	25.84%	26.22%	26.01%	26.15%
PIQA acc_norm	53.65%	52.56%	54.30%	52.45%
HellaSwag acc_norm	27.04%	26.93%	27.04%	27.28%
ArithMark-3 acc_norm	28.10%	30.80%	33.00%	30.40%
Intelligence Index	3.79	3.93	5.37	3.88

Raw public-benchmark accuracy

Metric	Partial HQ	No-loop Edu	Partial Edu	All-looped Edu
ARC-Easy acc	28.62%	30.01%	29.00%	29.92%
ARC-Challenge acc	18.86%	17.75%	17.49%	16.89%
PIQA acc	53.75%	54.08%	55.50%	54.46%
HellaSwag acc	26.89%	26.53%	26.80%	26.94%
ArithMark-3 acc	27.90%	30.30%	33.00%	29.80%

Chance-normalized components

For a benchmark score s and random-chance score c:

N(s, c) = 100 * (s - c) / (100 - c)

The Intelligence Index is:

(HellaSwag + ARC + PIQA + 0.65 * ArithMark-3) / 3.65

Normalized component	Partial HQ	No-loop Edu	Partial Edu	All-looped Edu
HellaSwag, chance 25%	2.72	2.57	2.72	3.03
ARC mean, chance 25%	1.13	1.63	1.35	1.54
PIQA, chance 50%	7.29	5.11	8.60	4.90
ArithMark-3, chance 25%	4.13	7.73	10.67	7.20
Intelligence Index	3.79	3.93	5.37	3.88

Base Bench 1.1 category Elo

Category	Partial HQ	No-loop Edu	Partial Edu	All-looped Edu
Language completion	827	850	864	817
Commonsense	921	843	831	877
World knowledge	842	852	892	820
Context tracking	846	862	879	862
Quantitative	852	815	806	815
Logical reasoning	990	1010	965	975
Code completion	957	964	970	972

Base Bench 1.1 category raw accuracy

Each category contains 50 examples.

Category	Partial HQ	No-loop Edu	Partial Edu	All-looped Edu
Language completion	40% (20/50)	44% (22/50)	46% (23/50)	38% (19/50)
Commonsense	46% (23/50)	34% (17/50)	34% (17/50)	40% (20/50)
World knowledge	36% (18/50)	34% (17/50)	40% (20/50)	32% (16/50)
Context tracking	32% (16/50)	34% (17/50)	36% (18/50)	34% (17/50)
Quantitative	28% (14/50)	22% (11/50)	22% (11/50)	22% (11/50)
Logical reasoning	36% (18/50)	38% (19/50)	32% (16/50)	34% (17/50)
Code completion	30% (15/50)	30% (15/50)	32% (16/50)	32% (16/50)

Base Bench 1.1 category weighted accuracy

Category	Partial HQ	No-loop Edu	Partial Edu	All-looped Edu
Language completion	39.49%	42.68%	44.59%	38.22%
Commonsense	45.43%	35.02%	33.44%	39.43%
World knowledge	34.80%	36.05%	41.38%	31.97%
Context tracking	29.94%	31.85%	34.08%	31.85%
Quantitative	24.92%	21.14%	20.19%	21.14%
Logical reasoning	35.42%	37.93%	32.29%	33.54%
Code completion	31.85%	32.80%	33.44%	33.76%