SUB-MCQ EDUCATION
NILEAGI-SUB · Swahili language understanding · Education sector
About
Multiple-choice questions testing Swahili language understanding across primary and ordinary secondary level topics This release is the education sector of NILEAGI-SUB, a programme that measures Swahili understanding sector by sector under one controlled protocol. Items come from the Swahili-subject curriculum (language arts), not from other subjects taught in Swahili medium.
Evaluation highlight
Level I evaluates eight open models (~2-5B parameters) on all 2,569 items with deterministic decoding. Exact-match accuracy ranges from 27.8% to 54.8% (chance baseline 22.4%). Gemma 4 E4B leads; architecture and adaptation outweigh modest parameter differences in this band.
Read moreData details
2601
Validated items
8
Grade bands
22.4%
Chance baseline
Items by grade band
- Standard 366
- Standard 4116
- Standard 5274
- Standard 6651
- Standard 7517
- Form 2399
- Form 3125
- Form 4421
Option counts
1,408 five-option, 1,125 four-option, 31 three-option, and 5 two-option items (98.6% have four or five options). The chance baseline averages 1/K per item (22.4%), not a fixed 1/4 or 1/5.
Record format
- swali
- Question stem (multiple choice).
- hatua
- School-grade field (Standard 3-7, Form 2-4).
- machaguo
- Two to five answer options, in display order.
- jibu
- Gold label: letter A-E matching the keyed option.
Samples
Two representative items for format review.
- Sample 1Standard 3
Neno kitanda limeundwa na silabi ngapi?
- AMoja
- BTatu
- CSaba
- DTano
- Sample 2Standard 3
Mlima huu ni sana.
- AKirefu
- BVirefu
- CWarefu
- DMrefu
Collaborate
Interested in extending NILEAGI-SUB to new sectors, model scales, or joint evaluation work? Contact us.