# ARealMe English Vocabulary Size Test - technical method (version 3)

Published 26 September 2026 · scoring model `curve-3.0.0` · question bank `bank-2026.09.26` ·
word list `lexicon-2026.09.26`

This document describes every step between your answers and your result, so the number
can be checked and reproduced. The test page summarises it in plain language.

## 1. What the test measures

The test estimates **receptive vocabulary size**: how many English words you can recognise
the meaning of. It counts dictionary headwords (lemmas): *run*, *runs*, *ran* and
*running* are one word; *runner* is another. Recognising a meaning among four options is
easier than using a word yourself, so this is an estimate of the words you understand,
not of the words you use.

## 2. The word list

Inputs:

* **Princeton WordNet 3.0** (Miller, 1995; Fellbaum, 1998) - the list of English headwords,
  their parts of speech, meanings, spellings as written, and inflection rules.
* **Google Web 1T unigram counts** (Brants & Franz, 2006) - word-form frequencies in about
  one trillion words of English web text, from the word-frequency list kept by
  WordleSolver.pro (https://wordlesolver.pro/): Peter Norvig's list of the 333,333 most
  frequent forms, with 69 valid words added at estimated counts and one misspelling removed
  (333,401 forms). Our thanks to WordleSolver.pro for the list.
* **SUBTLEX-US** (Brysbaert & New, 2009) - word-form frequencies in 51 million words of
  American film and TV subtitles, which follow everyday speech (Ghent University, CC BY-SA).
  Only lower-case occurrences are used, so character names (*Jack*, *Grace*) do not inflate
  ordinary words.

Only derived ranks are published here, not the counts.

Steps (`tools/build-lexicon.mjs`):

1. Candidate headwords: single-word, alphabetic WordNet lemmas of any part of speech.
2. Names are removed: 13,791 lemmas that WordNet only ever writes with a capital letter.
3. Closed-class words are removed (292 entries in `tools/data/function-words.txt`:
   articles, pronouns, prepositions, auxiliaries, numbers, weekdays…). They make up 39.6%
   of running text and are assumed known.
4. Spelling variants are merged under the more frequent spelling (1,412 merges):
   lemmas with identical senses and an edit distance of at most 2, and British/American
   pairs whose senses are a subset of the other spelling's (*centre*/*center*).
5. Each word form's count is credited to its headword(s) using WordNet's morphological
   rules and exception lists (the "morphy" algorithm), separately for web and subtitle
   counts. A form that could belong to several headwords is split evenly between them.
   SUBTLEX splits contractions at the apostrophe (*we're* → *we* + *re*, *don't* → *don* +
   *t*), so those fragments are skipped, and the seven headwords spelled like one (*re*,
   *don*, *haven*, *ain*, *won*, *em*, *ma*) are ranked by web counts alone.
6. Headwords are ranked by **everyday frequency**: the mean of their two Zipf values
   (log10 of occurrences per billion words, with one occurrence added so that unseen forms
   stay finite). Web text alone overrates web and business vocabulary: it ranks *page* 3rd,
   *click* 38th and *mortgage* 719th, but *milk* 2,520th and *afraid* 4,122nd. With the
   subtitles averaged in, *afraid* is 886th, *milk* 1,435th, *click* 544th and *mortgage*
   1,718th. Subtitle frequencies predict word recognition better than written corpora
   (Brysbaert & New, 2009). Headwords with no attested web form are not counted. Result:
   **42,719 ranked headwords** (the same headwords as the 2026-09-25 web-ranked list, in a
   new order).

Text coverage of the ranked list, measured on web text (function words included, names and
numbers excluded). Coverage is kept on written text because the reading research behind the
levels (§7) is about written text; ranking by everyday frequency makes the list slightly less
efficient for web text than a web-only ranking would be:

| Coverage of running text | Reached by the top… headwords | Counted words (see §3) |
| --- | --- | --- |
| 80% | 1,772 | ≈ 1,700 |
| 90% | 4,145 | ≈ 3,800 |
| 95% | 7,465 | ≈ 6,800 |
| 98% | 13,399 | ≈ 11,800 |
| 99% | 18,505 | ≈ 16,200 |
| 99.5% | 23,443 | ≈ 20,400 |

## 3. Frequency groups, sampling and eligibility

The ranked list is split into ten groups. Every headword has a **permanent random number**
between 0 and 1 (`tools/data/random-numbers.tsv`), and each group's candidates are its words
in order of that number (`tools/sample-candidates.mjs`), so the first candidates of a group are
a random sample of it. The numbers were fixed by the first sample (a seeded shuffle of each
web-ranked group, seed `arealme-vocabulary-size-test-v3-2026-09-25`, reproduced by
`tools/make-random-numbers.mjs`). When the groups changed with the new ranking, the same
numbers kept every word's chance of selection equal while reusing as many reviewed words as a
fair sample allows (sample coordination with permanent random numbers; Ohlsson, 1995): 465 of
the earlier 516 words stayed in the bank (the 251 of them in groups 1–3 were rewritten in the
short format, §4). Question writers took words strictly in that
order. Every word they looked at received one decision (`tools/data/item-review.json`):

* `item` - a question was written;
* `unsuitable` - a real word that is offensive or distressing to test; skipped but counted;
* `name`, `abbreviation`, `form`, `nonword` - not real vocabulary (a name or brand, or a
  word whose frequency mainly comes from one; an abbreviation; an inflected form with no
  meaning of its own; a fragment or foreign word).

The share of candidates that are real vocabulary estimates the same share for the whole
group, and the group's size is multiplied by it (the eligibility factor):

| Group | Ranks | Headwords | Candidates reviewed | Not vocabulary | Counted words |
| --- | --- | --- | --- | --- | --- |
| 1 | 1–1,000 | 1,000 | 156 | 3% | 974 |
| 2 | 1,001–2,000 | 1,000 | 170 | 10% | 900 |
| 3 | 2,001–3,000 | 1,000 | 174 | 13% | 874 |
| 4 | 3,001–5,000 | 2,000 | 57 | 5% | 1,895 |
| 5 | 5,001–8,000 | 3,000 | 61 | 11% | 2,656 |
| 6 | 8,001–12,000 | 4,000 | 65 | 17% | 3,323 |
| 7 | 12,001–17,000 | 5,000 | 47 | 13% | 4,362 |
| 8 | 17,001–23,000 | 6,000 | 49 | 16% | 5,020 |
| 9 | 23,001–31,000 | 8,000 | 48 | 15% | 6,834 |
| 10 | 31,001–42,719 | 11,719 | 43 | 5% | 11,174 |
| | | **42,719** | **870** | **10%** | **38,011** |

Groups 1–3 were reviewed further than the others (156–174 candidates instead of 43–65)
because they hold 150 questions each (§4); their factors are correspondingly more precise.

The highest possible result is therefore about 38,000 words.

## 4. Questions

The bank has **774 questions** (150, 150, 150, 54, 54, 54, 41, 41, 40, 40 per group). The
three easiest groups have the most questions because every test starts there (§5), so people
who take the test again meet new easy words for many attempts. The format
follows the Vocabulary Size Test (Nation & Beglar, 2007; Beglar, 2010): the headword and four
meanings written in simple, common words (one correct, three meanings of other words of the
same part of speech), plus "I don't know". In groups 1–3, which supply the first 30
questions, the meanings are one to four words from the 2,000 most everyday headwords and
there is no example sentence unless a word needs one to show which meaning is tested
(*page*: part of a car / part of a house / part of a shoe / part of a book). Groups 4–10 add a short example sentence that
shows the part of speech without revealing the meaning. The rules are in
`docs/item-writing.md`; the validator (`tools/build-bank.mjs`) checks the format and the
frequency of every word used in the options. When the corpus counts barely separate a word's
senses, the everyday meaning is tested. The correct option is not systematically the longest
or the shortest: across the bank, always choosing the longest option would be right 23% of
the time and always avoiding it 28%, close to the 25% of blind guessing (many easy questions
have options of equal length, which counts as a tie). Option order is shuffled per person.
Questions were drafted with AI assistance, checked by the validator and answered blind by
independent AI reviewers (§8); they have not yet been reviewed by an independent panel of
language teachers or calibrated on response data.

## 5. Test design (multistage)

Every attempt has 80 questions in three parts (`src/v3/form.ts`):

* **Part 1** (questions 1–20): ten words from each of groups 1–2 - a very easy start.
* **Part 2** (21–40): five words from each of groups 3–6.
* **Part 3** (41–80): chosen by the estimate after 40 answers (multistage testing; Yan,
  von Davier & Lewis, 2014):

| Route | Estimate after 40 answers | Questions per group 1–10 |
| --- | --- | --- |
| L | under 5,000 | 0, 0, 6, 8, 8, 8, 6, 4, 0, 0 |
| M | 5,000–14,999 | 0, 0, 0, 2, 6, 8, 8, 8, 6, 2 |
| H | 15,000 and above | 0, 0, 0, 0, 2, 4, 8, 8, 9, 9 |

Words are drawn at random within each group, preferring words this browser has not shown
before, then the words shown longest ago. Part 1 shows its ten group-1 words and then its ten
group-2 words, each in random order, so tests do not always open with the same few most
common words; parts 2 and 3 go from the most to the least common word.

## 6. Scoring model

For a word of rank *r* the probability that the person knows it is

    P(r) = 1 / (1 + exp(s · (ln r − ln m)))

with *m* the rank at which the person knows half of the words and *s* the steepness. Using
word frequency to predict difficulty follows the linear logistic test model (Fischer, 1973).

Each answer is one of *correct*, *wrong* or *I don't know*. A known word is answered
correctly except for a slip (3%). For an unknown word the person either chooses "I don't
know" or guesses (rate *g*, estimated per person); a guess is correct one time in four:

    Pr(correct) = P(r)·0.97 + (1 − P(r))·g·0.25
    Pr(wrong)   = P(r)·0.03 + (1 − P(r))·g·0.75
    Pr(unknown) = (1 − P(r))·(1 − g)

Wrong answers therefore identify guessing, and lucky guesses are discounted (compare
formula scoring, Frary, 1988; and the "I don't know" option, Stoeckel, Bennett & McLean, 2016).

The posterior over (*m*, *s*, *g*) is computed exactly on a grid: 111 values of ln *m* from
ln 20 to ln 400,000; *s* from 0.4 to 3.2 in steps of 0.2; *g* from 0 to 1 in steps of 0.1.
Prior: ln *m* ~ Normal(ln 6,000, 2.5²); *s* and *g* uniform.

For every grid curve the vocabulary size is

    V(m, s) = Σ over all ranked headwords  e(r) · P(r)

where *e(r)* is the eligibility factor of the word's group (§3); the sum is evaluated over
200 log-rank bins. The reported **vocabulary size** is the posterior median of *V*, rounded
to 10 words below 10,000 and to 100 words above; the **95% range** is the 2.5th–97.5th
posterior percentile, rounded outwards. Other results:

* **group shares** - posterior median share of each group known (the chart);
* **text coverage** - posterior median of `F·P(1) + Σ t(r)·P(r)`, where *F* is the share of
  function words (39.6%) and *t(r)* the share of running text taken by headword *r*;
* **guessing** - posterior mean of *g*.

The calculation is deterministic: the same answers always give the same result.

## 7. Levels

Levels use the person's own estimated text coverage (§6), because two people with the same
vocabulary size can know different words: someone who misses common words understands less
of a text than someone who knows exactly the most common ones.

| Level | Requirement | Basis |
| --- | --- | --- |
| Starter | coverage below 80% | |
| Everyday | coverage 80–90% | |
| Conversational | coverage 90–95% | 90–95%: reading with frequent help |
| Independent reader | coverage 95–98% | 95%: adequate comprehension (Laufer, 1989, as reviewed by Schmitt, Jiang & Grabe, 2011) |
| Fluent reader | coverage 98–99% | 98%: the usual target for comfortable, unassisted reading (Schmitt, Jiang & Grabe, 2011; Nation, 2006) |
| Advanced | coverage ≥ 99% | |
| Expert | coverage ≥ 99% and ≥ 17,900 words | 17,900 counted words (about 47% of all) reach about 99.2% coverage when they are the most common ones |
| Wordsmith | coverage ≥ 99% and ≥ 25,000 words | two thirds of all counted words |
| Lexicographer | coverage ≥ 99% and ≥ 32,000 words | about 85% of all counted words |

For comparison, Brysbaert, Stevens, Mandera and Keuleers (2016) estimated from 221,268
participants that a typical 20-year-old native speaker of American English knows about
42,000 of 61,800 lemmas, and a typical 60-year-old about 48,200. Their list contains many
more rare words than ours, so their numbers are not on the same scale.

## 8. Validation

**Simulation** (`tools/simulate.mjs`, 600 simulated test takers with known vocabularies,
word-to-word variation in difficulty, individual guessing rates and a 3% slip rate):

| Profile | 95% range contains the true size | Mean absolute error | Mean bias |
| --- | --- | --- | --- |
| Knowledge fading steadily with rarity (412) | 95.9% | 17% | +3% |
| Extra knowledge of rare words, e.g. cognates (188) | 85.1% | 21% | −16% |

For steadily fading profiles the typical error is about 6% above 24,000 words, 11% between
12,000 and 24,000, and 24–25% between 2,500 and 12,000 (about ±700 words below 2,500).
Pure random guessers receive a median result of about 140 words.

**Answer key check.** Every question written or rewritten for the 2026-09-26 bank (560 of
the 774) was answered blind by an independent AI reviewer who did not see the key; the
reviewer chose the keyed answer every time, and the options they flagged as guessable from
their form or length were rewritten.

**Software tests** (`npm test`): bank format, test assembly, routing, form validation,
determinism, monotonicity (a correct answer never lowers the result), extreme answer
patterns, random guessing and level thresholds.

## 9. Limitations

* A multiple-choice format measures recognition, and elimination can help on some items
  (Gyllstad, Vilkaitė & Schmitt, 2015; Stoeckel, McLean & Nation, 2021).
* Everyday frequency (web and subtitle counts) only approximates difficulty; items have not
  yet been calibrated on response data. People who know many rare words through another
  language can be underestimated (§8).
* The test is unsupervised; certificates record a self-administered result.
* Results of the earlier ARealMe vocabulary test (before September 2026) used a different
  list and scale and are not comparable.

## 10. Reproducing the numbers

```sh
node tools/build-lexicon.mjs --wordnet <WordNet-3.0/dict> --unigrams <unigram_freq.csv> --subtitles <SUBTLEXus74286wordstextversion.txt>
node tools/sample-candidates.mjs --wordnet <WordNet-3.0/dict>
node tools/build-bank.mjs
node tools/simulate.mjs --people 600 --seed 9 --json tools/data/simulation.json
node tools/page-numbers.mjs
npm test
```

`tools/make-random-numbers.mjs` documents how the permanent random numbers were derived from
the first (2026-09-25) sample.

## References

* Beglar, D. (2010). A Rasch-based validation of the Vocabulary Size Test. *Language Testing, 27*(1), 101–118. https://doi.org/10.1177/0265532209340194
* Brants, T., & Franz, A. (2006). *Web 1T 5-gram Version 1* (LDC2006T13). Linguistic Data Consortium. https://doi.org/10.35111/cqpa-a498
* Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. *Behavior Research Methods, 41*(4), 977–990. https://doi.org/10.3758/BRM.41.4.977
* Brysbaert, M., Mandera, P., McCormick, S. F., & Keuleers, E. (2019). Word prevalence norms for 62,000 English lemmas. *Behavior Research Methods, 51*(2), 467–479. https://doi.org/10.3758/s13428-018-1077-9
* Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). How many words do we know? *Frontiers in Psychology, 7*, 1116. https://doi.org/10.3389/fpsyg.2016.01116
* Fellbaum, C. (Ed.). (1998). *WordNet: An Electronic Lexical Database*. MIT Press. https://doi.org/10.7551/mitpress/7287.001.0001
* Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. *Acta Psychologica, 37*(6), 359–374. https://doi.org/10.1016/0001-6918(73)90003-6
* Frary, R. B. (1988). Formula scoring of multiple-choice tests (correction for guessing). *Educational Measurement: Issues and Practice, 7*(2), 33–38. https://doi.org/10.1111/j.1745-3992.1988.tb00434.x
* Goulden, R., Nation, P., & Read, J. (1990). How large can a receptive vocabulary be? *Applied Linguistics, 11*(4), 341–363. https://doi.org/10.1093/applin/11.4.341
* Gyllstad, H., Vilkaitė, L., & Schmitt, N. (2015). Assessing vocabulary size through multiple-choice formats. *ITL - International Journal of Applied Linguistics, 166*(2), 278–306. https://doi.org/10.1075/itl.166.2.04gyl
* Keuleers, E., Stevens, M., Mandera, P., & Brysbaert, M. (2015). Word knowledge in the crowd. *Quarterly Journal of Experimental Psychology, 68*(8), 1665–1692. https://doi.org/10.1080/17470218.2015.1022560
* McLean, S., Stewart, J., & Kramer, B. (2015). An empirical examination of the effect of guessing on vocabulary size test scores. *Vocabulary Learning and Instruction, 4*(1), 26–35. https://doi.org/10.7820/vli.v04.1.mclean.et.al
* Miller, G. A. (1995). WordNet: A lexical database for English. *Communications of the ACM, 38*(11), 39–41. https://doi.org/10.1145/219717.219748
* Nation, I. S. P. (2006). How large a vocabulary is needed for reading and listening? *The Canadian Modern Language Review, 63*(1), 59–82. https://doi.org/10.3138/cmlr.63.1.59
* Nation, P., & Beglar, D. (2007). A vocabulary size test. *The Language Teacher, 31*(7), 9–13. (no DOI)
* Ohlsson, E. (1995). Coordination of samples using permanent random numbers. In *Business Survey Methods*. Wiley. https://doi.org/10.1002/9781118150504.ch9
* Schmitt, N., Jiang, X., & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. *The Modern Language Journal, 95*(1), 26–43. https://doi.org/10.1111/j.1540-4781.2011.01146.x
* Stoeckel, T., Bennett, P., & McLean, S. (2016). Is "I don't know" a viable answer choice on the Vocabulary Size Test? *TESOL Quarterly, 50*(4), 965–975. https://doi.org/10.1002/tesq.325
* Stoeckel, T., McLean, S., & Nation, P. (2021). Limitations of size and levels tests of written receptive vocabulary knowledge. *Studies in Second Language Acquisition, 43*(1), 181–203. https://doi.org/10.1017/S027226312000025X
* Yan, D., von Davier, A. A., & Lewis, C. (Eds.). (2014). *Computerized Multistage Testing: Theory and Applications*. Chapman and Hall/CRC. https://doi.org/10.1201/b16858

WordNet 3.0 © 2006 Princeton University, used under the WordNet License. SUBTLEX-US word
frequencies © Marc Brysbaert and Boris New, Ghent University, used under CC BY-SA; the ranked
list derived from them (`tools/data/lemma-ranks.txt`) is shared under the same licence.