SUB-MCQ EDUCATION
NILEAGI-SUB · Swahili language understanding · Education sector
About
Multiple-choice questions testing Swahili language understanding across primary and ordinary secondary level topics This release is the education sector of NILEAGI-SUB, a programme that measures Swahili understanding sector by sector under one controlled protocol. Items come from the Swahili-subject curriculum (language arts), not from other subjects taught in Swahili medium.
Evaluation highlight · August 2026
Level I evaluates eight open models (~2-5B parameters) on all 2,569 items with deterministic decoding. Exact-match accuracy ranges from 27.8% to 54.8% (chance baseline 22.4%). Gemma 4 E4B leads; architecture and adaptation outweigh modest parameter differences in this band.
This release is featured in the Cohere Labs Open Science Community, alongside our full technical report.
Data details
2601
Validated items
8
Grade bands
22.4%
Chance baseline
Items by grade band
- Standard 366
- Standard 4116
- Standard 5274
- Standard 6651
- Standard 7517
- Form 2399
- Form 3125
- Form 4421
Option counts
1,408 five-option, 1,125 four-option, 31 three-option, and 5 two-option items (98.6% have four or five options). The chance baseline averages 1/K per item (22.4%), not a fixed 1/4 or 1/5.
Record format
- swali
- Question stem (multiple choice).
- hatua
- School-grade field (Standard 3-7, Form 2-4).
- machaguo
- Two to five answer options, in display order.
- jibu
- Gold label: letter A-E matching the keyed option.
Samples
Two representative items for format review.
- Sample 1Standard 3
Neno kitanda limeundwa na silabi ngapi?
- AMoja
- BTatu
- CSaba
- DTano
- Sample 2Standard 3
Mlima huu ni sana.
- AKirefu
- BVirefu
- CWarefu
- DMrefu
Collaborate
Interested in extending NILEAGI-SUB to new sectors, model scales, or joint evaluation work? Contact us.