Benchmarks and memory tests¶
LangCheck has three sets of performance checks besides its unit and end-to-end tests: the core’s own (Rust), the VS Code extension’s end-to-end benchmarks, and the browser extension’s benchmarks, memory bounds and soak. None of them run in CI: timings depend on the machine and on what else it is doing. They are a record of what each case costs, and a ceiling far above a normal run that only something broken (a quadratic step, a cache that never hits, a hang) reaches.
The two editor harnesses share their generated documents and their recorder: extension/src/test/e2e/bench/corpus.ts (seeded generators, so each run checks text the core has never seen) and extension/src/test/e2e/bench/measure.ts (median, spread, ceilings, one JSON line per metric).
Running them¶
What |
Command |
Results |
|---|---|---|
VS Code: documents and workspaces |
|
|
Browser: documents on Overleaf and Typst |
|
|
Browser: memory at the edges |
|
|
Browser: soak |
|
|
Browser e2e scale and memory fixtures |
|
|
Core: malformed input |
|
the test’s output |
Core: server memory over 100 000 checks |
|
the test’s output |
BENCH_REPS=N sets the repetitions per metric (default 5, after one discarded warm-up) and BENCH_GATE=off records without failing on a ceiling. Wrap long runs in a memory guard (pm run --max-mem 16G -- …): the large cases hold a lot of text in several processes at once.
The browser benchmarks drive the whole path in Chromium: the page script, the content script, the background, the native host and the release core, on the stand-in Overleaf and Typst pages of the end-to-end tests. Each test gets its own browser, native host and core server. They run one test at a time, so the timings and process memory are this machine’s, with nothing else of LangCheck’s running.
Each result line carries the commit, the machine, the suite, the subject and metric, every sample, and median, min, max, spread and unit (ms, MiB or count). Lines written before 2026-10-14 name these medianMs, samplesMs and so on.
What the browser benchmarks measure¶
For each document, generated afresh for every repetition:
Metric |
What it is |
|---|---|
open + cold check |
From opening text the core has never seen to the last change in the page’s findings. |
first answer |
To the first check that finished: for a progressive check, its first window. |
reopen (cached) |
The same text opened as another file: what the core answers from its caches. |
type to findings |
A sentence typed at the end, to its findings. Includes the 500 ms typing pause. |
main thread held (TBT) |
Total blocking time of the page while the findings were drawn: time past 50 ms in each long task. |
page heap, background heap |
JS heaps after forced collections (the DevTools protocol). |
native host RSS, core server RSS |
Resident memory of the two processes this test’s browser started. |
Opening a file costs 250 ms on average before any check starts: the sites announce no file switch, so the page looks for one every 500 ms (POLL_MS in web/src/page/bridge.ts). That is why a cached reopen of a small file takes about 400 ms on the web against a few in VS Code.
Results, 2026-10-14¶
Ryzen 9 9900X3D, 24 threads, release core, Chromium 147, Harper only (BENCH_CONFIG). Medians of three, every ceiling met.
Document |
Open + cold check |
Reopen (cached) |
Type to findings |
Main thread held |
Page heap |
Core server RSS |
|---|---|---|---|---|---|---|
LaTeX 4 KB |
456 ms |
360 ms |
520 ms |
0 ms |
6.6 MiB |
174 MiB |
LaTeX 256 KB |
3.6 s |
306 ms |
747 ms |
0 ms |
10.3 MiB |
377 MiB |
LaTeX 1 MB (12 848 findings) |
12.9 s |
541 ms |
1.4 s |
21 ms |
20.2 MiB |
461 MiB |
Typst 4 KB |
502 ms |
360 ms |
513 ms |
0 ms |
6.6 MiB |
175 MiB |
Typst 256 KB |
3.4 s |
530 ms |
629 ms |
0 ms |
10.4 MiB |
316 MiB |
Typst 1 MB (12 914 findings) |
12.3 s |
329 ms |
958 ms |
26 ms |
20.6 MiB |
552 MiB |
One 256 KB line |
3.8 s |
212 ms |
767 ms |
0 ms |
10.2 MiB |
376 MiB |
Misspellings only, 64 KB (10 309 findings) |
1.0 s |
438 ms |
653 ms |
7 ms |
15.6 MiB |
212 MiB |
Malformed LaTeX |
570 ms |
290 ms |
522 ms |
0 ms |
6.6 MiB |
229 MiB |
Mixed languages, 64 KB |
978 ms |
469 ms |
564 ms |
0 ms |
9.3 MiB |
308 MiB |
Odd characters, 64 KB |
1.3 s |
118 ms |
542 ms |
0 ms |
7.7 MiB |
330 MiB |
A cold check is linear in the text: Harper at about 13 s a megabyte, as in VS Code (11.8 s for 1 MB of Markdown). The background holds 3 MiB throughout and the native host at most 24 MiB. The page’s main thread is never held for long: the longest task with 1 MB open was under 80 ms.
A progressive check of 1 MB of LaTeX in 16 KiB windows shows its first window after 1.4 s and its last after 17.4 s.
What costs a cached check of a long document is the parse: the core reads the whole text again each time it is sent, 685 ms a megabyte for LaTeX and 252 ms for Typst. The LaTeX figure is the grammar’s own (tree-sitter-latex, upstream HEAD as fast as the release used). An edit pays it once. Reopening the same text, or each window of a progressive check, does not: the core keeps its last extraction.
Found and fixed by these runs¶
Every check of a Typst document longer than about 400 KB re-linted all of it: 4.8 s for a cached check of 1 MB against 0.33 s now. The result cache held 4096 prose ranges, and a 1 MB Typst file is 10 026 of them, so each check evicted every answer before the next asked for it. The cache now holds 131 072 answers and at most 64 MiB (
performance.result_cache_entries).A progressive check parsed the whole document once per window: 61 s for 1 MB, now 17.4 s.
One lone surrogate (half of an emoji) anywhere in a page’s text cost the whole document its check: the core’s UTF-8 protocol cannot carry it, and the request was refused. The page now sends U+FFFD in its place, which keeps every offset.
A backslash before a multi-byte character in LaTeX or Forester (
\then U+FEFF, an accented letter, or that U+FFFD) panicked the extraction, and the document went unchecked. Every gap token now ends on a character boundary.
Memory at the edges¶
memory.bench.ts opens one hostile document per test, then goes back to a small file. Peaks are each process’s high-water mark (VmHWM) for that one document.
Document |
Cold check |
Core server peak |
Native host peak |
Page heap holding it |
Page heap kept after |
|---|---|---|---|---|---|
LaTeX at the size cap (2 M characters, 25 590 findings) |
25.0 s |
330 MiB |
27 MiB |
36 MiB |
0.6 MiB |
Misspellings only, 512 KB (82 683 findings) |
4.7 s |
260 MiB |
66 MiB |
75 MiB |
0.7 MiB |
One 1 MB line |
13.3 s |
298 MiB |
17 MiB |
20 MiB |
0.6 MiB |
A million blank lines |
799 ms |
165 MiB |
10 MiB |
19 MiB |
0.5 MiB |
100 000 unclosed |
649 ms |
210 MiB |
9 MiB |
8 MiB |
0.5 MiB |
Odd characters, 1 MB |
13.8 s |
298 MiB |
18 MiB |
22 MiB |
0.6 MiB |
Lone surrogates, NULs, lone CRs and BOMs, 256 KB (5822 findings) |
3.8 s |
284 MiB |
11 MiB |
13 MiB |
0.6 MiB |
The background kept under 0.1 MiB after each. CodeMirror draws only what is near the view, so the page’s DOM stayed under 4 400 nodes even with 82 683 findings. A document past the cap is not sent at all: the page says so once, and neither process grows. No process died and was restarted in any case.
Soak¶
soak.bench.ts keeps one page in use: typing a sentence with a mistake and taking it out again, switching between five files (one switch in five to a file never opened before), hovering a finding, opening and closing the palette, taking in a collaborator’s edit, and opening and closing SpeedFix every two minutes. The document returns to the same text after each step, so whatever grows grows with time alone. Every 30 s it samples the page’s heap, DOM nodes and event listeners, the background’s heap, and the native host’s and core server’s RSS, open files and threads. The growth per hour after the first quarter of the run (a least-squares slope) is gated.
A 20-minute soak on 2026-10-14 (838 steps, 1047 checks, 40 samples):
Series |
Growth per hour after warm-up |
Ceiling |
|---|---|---|
Page heap |
1.4 MiB |
8 MiB |
Background heap |
−0.1 MiB |
4 MiB |
Native host RSS |
0.3 MiB |
8 MiB |
Core server RSS |
47 MiB |
64 MiB |
DOM nodes |
−179 |
500 |
Event listeners |
0 |
50 |
Native host and server open files, server threads |
0 |
1–2 |
The page went from 6.0 to 9.7 MiB, most of it in the first minute, the background stayed at 3 MiB and the native host at 7.6 MiB. The core server climbs for longer: its caches (engine answers, Harper’s own) fill with every new text, up to their bounds. Twenty minutes does not reach them; checking_for_long_does_not_grow_the_server (above) runs a server through 100 000 checks of different text and gates on the last 25 000: 441 MiB after 75 000 checks, 446 MiB after 100 000 (debug build, 2026-10-14).
Against the sites’ own spellcheckers¶
Measured live on overleaf.com in Firefox 155 on 2026-10-14 with web/test/bench/live/load.js: from the page’s navigation start to each checker’s first marks, for the document the page opened with.
Document |
LangCheck |
Overleaf’s checker |
|---|---|---|
LaTeX 4 KB |
1.7–6.4 s, 45–67 findings |
2.3–8.5 s, 33–44 marks |
LaTeX 256 KB |
7.3 s, 3341 findings |
11.7 s, 50 marks |
LaTeX 1 MB |
7.5 s, 12 802 findings |
10.6 s, 41 marks |
The 4 KB ranges are two loads of the same project: the spread is between the loads, and in each LangCheck was first, by 0.6 s and 2.1 s. LangCheck’s figures for 256 KB and 1 MB are answers from the core’s cache: the text had been checked when it was put in the editor, before the reload (a cold check of 1 MB takes 13 s, above). Overleaf’s checker covers only what has been in view: it checks lines as they scroll into view and keeps what it found, so its count on load is the first screen’s (50 marks up to offset 4725 of the 256 KB document; 85 after scrolling to its end). LangCheck checks the whole document.
Typst’s own checker could not be compared: it marked nothing in the project used, with or without compile errors, so its spellcheck was most likely switched off there.