This week the playground handed us the other half of the Marvel crawl: the text of every hero's Wikipedia page. We add those pages to a corpus one at a time and ask how the vocabulary grows — then let you pick the order and watch the merged word-frequency curve against the idealized Zipf curve (slope −1).
The question. If we add the prepared Wikipedia pages one at a time, most-linked character first, how quickly does Marvel's vocabulary grow? Do the famous, well-connected heroes supply most of the words, or do minor characters keep contributing genuinely new vocabulary instead of repeating what we've already met? And does the order we read in change the path of that growth, even though everyone who reads all the pages ends up with the same vocabulary? That relationship between corpus size and vocabulary size is Heaps' law.
The week 1 network told us who mentions whom; the week 5 release is the actual prose — the full plain text of the same 303 Wikipedia articles (4.5 MB frozen on 2026-08-26). Tokenizing everything — lowercase, keep apostrophes, strip punctuation — gives 727,440 tokens containing 26,952 distinct words. The median page is only 1,238 tokens, but the corpus is front-heavy: Spider-Man's article alone is 10,293 tokens, over 8× the median.
Everything numeric below was computed once, offline, in analysis/build_week5.py
(stdlib only) and frozen into data/marvel_pages/week5_summary.json — the
browser only looks numbers up and draws them.
The metrics of this week answer different questions, so we turned them into ways of building the corpus — via one dropdown, not a wall of buttons. The Pick by dropdown drives both the manual + One step button and auto-play at once. Pick one of five strategies:
+ One step adds one page by the dropdown's choice, ▶ Play keeps adding at ~10 a second until the corpus is full (switch the dropdown live mid-run); or type any character's name in the Or type a character box and press Add to add it by hand.
Each addition draws that character's own Zipf curve on the chart (Axes: Log–log by default, Linear on demand), behind the merged vocabulary curve of everything you've built so far, with the idealized Zipf curve (slope −1) as the dashed reference. The layers are checkbox toggles; the left panel summarizes the page you just added, including its live marginal gain ΔV — the words it deposits on top of everything you already own.
Watch for the reversal: buying Zipf outliers (spiky minor stubs) one after another keeps your merged curve far below the ideal line — you accumulate vocabulary that is rare in the aggregate but strangely concentrated page by page. The Heaps setting, by contrast, buys words that show up nowhere else in what you already own. Which strategy pulls the merged curve closest to the ideal? Press Play and see.
Built in the browser from the duel section of
data/marvel_pages/week5_summary.json: the global vocabulary and per-page
(word, count) pairs are frozen output of analysis/build_week5.py; the
marginal Heaps gain is computed live, exactly, for every remaining candidate on each add.
Weird doesn't mean funny, obscure, or badly written. Here, weird has a precise meaning: a page is weird when an unusually large share of its words are words that rarely appear elsewhere in the Marvel corpus. A word is rare if it occurs in at most 3 of the 303 pages (about 1%); a page's weirdness is the fraction of its running text made of such words. Dividing by the page's own length keeps long pages from being weird just for being long. It is a statement about words only, not about the character or the quality of the writing.
Share of each page's words that occur in at most 3 of the 303 pages; the dashed line is the median page. Click a page to see its rare words.
Every page, once. Horizontally: the new words it contributes, meaning the distinct words no earlier page had used when the pages are read linked-first (most-linked page first), so the number belongs to that order. Vertically: weirdness. Look for pages that add many words but are not weird, and for short pages that add few words yet are very weird. The selection is shared with the chart above.