Benchmark
How we tested,
and every result.
The comparison on the technology page is a summary. This page has the setup, how each number was measured, all the results (including the ones where Leyr is not the best), the files to download and the steps to run it yourself.
Who ran it
The Leyr project, so this is not independent
We wrote the harness, chose the library and the settings, and ran the tests. That is a reason to check our work. Everything needed to rerun it is on this page. If you rerun it and get different numbers, send them to us and we will publish them next to ours with your setup.
Setup
Machine, software and limits
The machine is a desktop that also ran other work during the tests, so timings carry noise (see the caveats).
- Run by
- The Leyr project. This is not an independent test.
- Dates
- 2026-11-14, 2026-11-15
- Hardware
- AMD Ryzen 9 5950X 16-Core Processor, 32 threads, 47 GB RAM; music on local NVMe
- Operating system
- Arch Linux, kernel 7.3.4-arch1-2
- Container engine
- podman version 6.2.0, rootless, crun runtime
- Limits
- Every container: 4 CPU cores, 4 GB of memory, no swap (
--cpus=4 --memory=4g --memory-swap=4g), one container at a time. - Network
- Every container runs on an internal podman network with no route out (
podman network create --internal). Published ports still reach the host. Before each run the harness tries to reach three outside addresses from inside the container and stops if any answers; each result file records that check. - Leyr
- main 0504a876 plus the memory change (commit a105ba12; image built from the repository Dockerfile, image id c21eb7c848fb)
- Rerun
- On 2026-11-15 (evening) Leyr and gonic were rerun together on the same library and cores, with a build of Leyr from main before the change and one after it taken in between (the before-change runs are in results/memory-333-before). Navidrome, Jellyfin and Plex were not rerun: their rows are from earlier runs on the same library, and Jellyfin and Plex have private memory at idle only. The host was busy (load average 12 to 24) during the whole rerun, so timings are noisy. The newer Leyr build changes memory only: its request and stream timings match the earlier build within run-to-run noise.
- CPU pinning
- Rootless podman has no cpuset controller, so every container's processes were pinned to four fixed cores with
tasksetafter start, on top of the 4-CPU limit, away from the cores another job on the machine was pinned to. - Other load
- The machine was shared. The 1-minute load average before each run is in each library's
progress.loginside the results download; it ranged from about 5 to 63. Other software (compiles and tests by other jobs on the same machine) ran during the runs. - Clients
- The harness runs on the same host, in Python, over HTTP on 127.0.0.1.
| Server | Version | Image |
|---|---|---|
| Leyr | built from the commit below | localhost/bench-leyr |
| Navidrome | 0.64.2 (10114574) | docker.io/deluan/navidrome@sha256:38dc2727bfcfd5ede290f8ada114fc90368146f265ae4701ddddbcbe2a44ee52 |
| gonic | v0.23.0 | docker.io/sentriz/gonic@sha256:7b2298d905149ecb7e57cdd3bf7a40082758b9bda2e24454b41c17c99bf25e8f |
| Jellyfin | Jellyfin.Server 12.4.0.0 | docker.io/jellyfin/jellyfin@sha256:fea863d0cd9ec445985d9c75fc6ed64e38a1d2766cf0c250a33d553f4fc1b771 |
| Plex | 1.43.5.11079-b3b13e506 | docker.io/plexinc/pms-docker@sha256:6d778d41daae68f89115aa87a44ae14a96b23e71a2093da173b5a9aac4091336 |
Settings
What differs from the defaults, and why
The aim was the same job for every server and nothing leaving the machine. Each of these changes could favour or hurt a server, so they are listed.
| Server | Change | Why |
|---|---|---|
| All | Internal network, no route out; 4 CPUs, 4 GB; music mounted read-only; fresh volume per scan run | The same conditions for every server, and private music cannot leave the machine. |
| Leyr | SOULSEERR_BACKGROUND_JOBS=off, LEYR_REQUEST_QUEUE=off, LEYR_ROOMS_RUNTIME=off, LEYR_EXPORT_RUNTIME=off, LEYR_BACKGROUND_RUNTIME=off, LIBRARY_SCAN_MINUTES=0 | Turns off jobs that call outside services (downloads, Discord, MusicBrainz) or run on a timer. The first scan runs at boot regardless. |
| Navidrome | ND_SCANNER_SCHEDULE=0, ND_ENABLEEXTERNALSERVICES=false, ND_LASTFM_ENABLED=false, ND_ENABLEINSIGHTSCOLLECTOR=false | No timed scans and no outside lookups or telemetry. The scan is started through its API. |
| gonic | Scan interval, file watcher and scan-at-start off; playlist and podcast folders on volumes | No timed scans; gonic refuses to start without those folders. |
| Jellyfin | Realtime monitor off, internet providers off, and the metadata and image fetchers for artists, albums and tracks emptied | No outside lookups. Side effect: Jellyfin then extracts no album art, so its cover-art rows are n/a. With the providers on, the scan took 1,066 s when they could not connect. |
| Plex | Unclaimed (no account); ALLOWED_NETWORKS=0.0.0.0/0 so the API answers without a token; the library uses the Personal Media agent; loudness and sonic analysis, thumbnail generation, library auto-scan, DLNA and server publishing off | No account is needed or created. The Plex Music agent looks albums up on Plex's servers; the Personal Media agent reads tags and embedded art only. Plex also downloads its MP3 encoder from plex.tv the first time it converts a track, which cannot happen with no network, so every Plex conversion row (MP3, Opus, seek) is n/a. We did not open the network for it, to keep to the rule that nothing leaves the machine. |
Measurements
What was measured, and how
One harness (bench.py, Python standard library only) drives each server in turn. Every server gets two full runs and two extra scan-only runs on each library.
- Start-up
- Time from
podman startto the first good answer, in milliseconds: Leyr/api/health, Navidrome/ping, gonic a Subsonic reply, Jellyfin/Users/Public(its/healthanswers before it is ready), Plex/identityonce it stops reportingstartState. First boot is an empty volume. Restarts use the populated database, five per full run. - Scan
- Fresh volume, create the admin (or, for Plex, the library), start a full scan, and poll until the server says it is finished, the track count has stopped changing and the container used under 0.15 CPU-seconds in a 6-second window. The time ends at the start of that quiet window. This matters most for Jellyfin, which keeps working after its scan task ends. Plex starts scanning when its library is created, which the harness does as the scan step. Leyr starts scanning on its own at boot and finishes in a second or two, before the harness can poll, so for Leyr the scan time is the time between two lines of Leyr's own log ("native music library open" and "library scan finished"), and the harness still waits for the container to go quiet. For the others the time includes up to a second of polling. What each server does during a scan (art, analysis, search indexes) is not the same work.
- Memory and threads
- Resident memory (RSS) read from
/proc, added up over every process in the container, sampled every 0.5 s during the scan and every 0.25 s under load. Idle is one reading 300 s after the scan ends. Threads are the same sum. Private memory is theAnonymoustotal of each process'ssmaps_rollup, summed the same way and sampled at the same moments: heap, stacks and other pages the process owns that no file backs. RSS also counts the executable, libraries and mapped database files; the kernel can take those back whenever it needs memory, and how many are mapped at a given moment varies from run to run, so private memory is the steadier measure of what a server holds. The earlier Jellyfin and Plex runs recorded it at idle only. - API latency
- Six calls per server from the host over keep-alive HTTP, after 40 warm-up requests each: 200 requests one at a time, then 800 across 16 connections. Requests rotate through 20 albums and 10 search terms. Median and 95th percentile in milliseconds. The Python client adds a little time of its own, the same for every server, and tops out near 2,000 to 4,000 requests a second.
- Streams
- Time from sending the request to the first audio byte, on a new connection each time: the original FLAC; FLAC converted to MP3 at 320 kbit/s and to Opus at 128 kbit/s (20 different tracks each, so a cache cannot answer a repeat); and on the 63-minute FLAC the original, MP3 from the start, and MP3 and Opus starting at 25:00 (a different offset for each request, 1500 s plus 7 s per request). Repeating the same request ten times is reported separately, because Navidrome and Jellyfin answer repeats from a cache.
- Seek check
validate_seek.pydownloads the start of the converted stream at offset 0, 1500 s and 1507 s and decodes two seconds of each, comparing the loudness with the source file at the same positions. A server that ignores the offset fails; Jellyfin did, with the request form used, so its seek rows are n/a. Plex could not be checked: with no network it cannot convert audio, so its line in the seek file is empty (?).- Memory under load
- The same sampler during the 16-connection runs of all six calls.
| Row | Leyr, Navidrome, gonic (Subsonic) | Jellyfin | Plex |
|---|---|---|---|
| ping | ping | GET /System/Info/Public | GET /identity |
| getAlbumList2_newest50 | getAlbumList2 type=newest size=50 | GET /Items (albums, newest first, 50) | GET /library/sections/1/all?type=9&sort=addedAt:desc (50) |
| getArtists | getArtists | GET /Artists | GET /library/sections/1/all?type=8 |
| getAlbum | getAlbum | GET /Items?ParentId=<album> | GET /library/metadata/<album>/children |
| search3 | search3 (20 songs, 20 albums, 20 artists) | GET /Search/Hints?limit=60 | GET /hubs/search?limit=20 |
| getCoverArt_300 | getCoverArt size=300 | GET /Items/<id>/Images/Primary?maxWidth=300 | GET /photo/:/transcode?width=300&height=300 |
| flac_original | stream format=raw | /Audio/<id>/stream?static=true | the track's file URL (direct play) |
| flac_to_mp3_320, seek rows | stream format=mp3 maxBitRate=320 [timeOffset=1500] | /Audio/<id>/stream.mp3?audioBitRate=320000 [startTimeTicks] (ignored) | /music/:/transcode/universal/start.mp3?musicBitrate=320 [offset=1500] |
| flac_to_opus_128 | stream format=opus maxBitRate=128 | stream.opus | n/a: no conversion without network |
Libraries
Two libraries, the same tests
Every server ran on both libraries. The public one is the headline because anyone can rebuild it. The real one checks that the public results hold on real music.
Public library (generated)
374 albums, 3,098 tracks, 29.6 GB: 498 FLAC, 20 M4A, 2,580 MP3. Seed 20261113; generated with ffmpeg version n9.0.2 Copyright (c) 2000-2026 the FFmpeg developers in 17 minutes.
Made by make_library.py from a fixed seed and ffmpeg alone: sine tones and seeded noise, invented artists and album names, generated cover art. No recordings, nothing copyrighted. Tracks are 3 to 6 minutes, with tags (artist, album artist, album, title, track, disc, genre, year), compilations and featured artists, and an embedded cover in every file. One 63-minute FLAC ("Long Form Drone") is used for the seek tests. Checksums for every file are in the downloads.
Real library (private)
374 album folders from a private collection, copied to local disk: 3,129 audio files, 30.7 GB (2,587 MP3, 522 FLAC, 20 M4A) with embedded art, plus a 63-minute FLAC for the seek tests. It cannot be published, so its results cannot be checked by others. Only counts and the format mix are given here. build_subset.sh shows how a subset of any collection is copied.
Results
Public library, all results
Every row we measured, for every server. Where two numbers appear they are run 1 and run 2. The lowest value in a row is highlighted; a highlight inside the noise (about two times) is not a win.
Measured: Leyr 2026-11-15; Navidrome 2026-11-15; gonic 2026-11-15; Jellyfin 2026-11-15; Plex 2026-11-15. Leyr can be rerun alone, so its date can differ from the others.
Scan of the library
| What | |||||
|---|---|---|---|---|---|
| Scan time (s) | 1.4, 1.0, 1.0, 1.0 (median 1.0) | 6.8, 7.6, 6.3, 7.5 (median 7.1) | 14.4, 10.8, 10.4, 10.4 (median 10.6) | 283.1, 290.1, 259.1, 264.4 (median 273.8) | 109.3, 145.1, 103.0, 98.1 (median 106.1) |
| Peak memory during scan (MB) | 41, 42, 43, 43 | 50, 47, 55, 66 | 168, 142, 153, 175 | 502, 500, 507, 474 | 493, 491, 494, 494 |
| Private memory, peak during scan (MB) | 19, 20, 21, 21 | 27, 25, 33, 45 | 137, 114, 123, 145 | n/a | n/a |
| Tracks indexed | 3098 | 3098 | 3098 | 3098 | 3098 |
Memory and threads
| What | |||||
|---|---|---|---|---|---|
| Idle memory, 5 min after the scan (MB) | 26 ; 32 | 38 ; 43 | 55 ; 56 | 402 ; 361 | 401 ; 362 |
| Idle private memory (MB) | 14 ; 15 | 23 ; 20 | 24 ; 26 | 230 ; 210 | 245 ; 232 |
| Idle threads | 4 ; 4 | 11 ; 12 | 29 ; 29 | 20 ; 19 | 114 ; 111 |
| Memory under load, median (MB) | 50 ; 50 | 56 ; 60 | 75 ; 74 | 468 ; 434 | 442 ; 436 |
| Memory under load, peak (MB) | 58 ; 56 | 63 ; 64 | 120 ; 119 | 483 ; 448 | 453 ; 451 |
| Private memory under load, peak (MB) | 30 ; 25 | 42 ; 41 | 88 ; 89 | n/a | n/a |
Start-up
| What | |||||
|---|---|---|---|---|---|
| First boot, empty volume (s) | 0.4 ; 0.4 | 0.3 ; 0.3 | 1.1 ; 1.1 | 4.3 ; 3.9 | 28.3 ; 32.1 |
| Restart, median of 5 (ms) | 345 ; 313 | 283 ; 310 | 309 ; 288 | 2,282 ; 2,904 | 1,594 ; 1,861 |
| Restart, slowest of 5 (ms) | 375 ; 524 | 377 ; 356 | 373 ; 298 | 2,964 ; 3,615 | 1,808 ; 2,098 |
API latency, one client at a time
| What | |||||
|---|---|---|---|---|---|
| ping | 0.3/0.4 ; 0.3/0.5 | 0.3/0.3 ; 0.3/0.4 | 0.5/0.6 ; 0.4/0.5 | 3.0/4.3 ; 4.2/5.9 | 0.2/0.4 ; 0.2/0.4 |
| getAlbumList2_newest50 | 1.7/3.4 ; 1.2/1.6 | 5.4/7.6 ; 6.0/15.4 | 3.8/4.9 ; 2.7/3.6 | 17.0/20.2 ; 28.0/40.9 | 9.9/12.2 ; 11.0/15.3 |
| getArtists | 1.9/3.1 ; 1.2/1.6 | 6.6/8.9 ; 6.8/16.0 | 5.2/6.2 ; 3.4/4.7 | 21.6/26.1 ; 45.4/70.8 | 5.8/8.7 ; 5.7/7.6 |
| getAlbum | 1.1/2.4 ; 0.9/1.2 | 2.6/3.9 ; 2.5/6.5 | 2.1/2.8 ; 1.6/2.2 | 13.0/15.4 ; 18.0/23.0 | 3.3/4.3 ; 3.6/5.7 |
| search3 | 5.1/8.3 ; 4.1/6.8 | 7.5/10.9 ; 6.9/15.9 | 7.6/9.0 ; 4.9/6.4 | 21.1/26.7 ; 28.9/38.1 | 32.0/43.6 ; 35.0/45.6 |
| getCoverArt_300 | 0.7/1.1 ; 0.6/0.7 | 0.4/0.6 ; 0.3/1.0 | 0.8/1.0 ; 0.6/0.7 | n/a ; n/a | 0.4/0.7 ; 0.3/0.6 |
API latency, 16 clients at once
| What | |||||
|---|---|---|---|---|---|
| ping | 1.9/5.1 ; 1.7/4.8 | 2.1/6.5 ; 2.5/9.5 | 5.0/22.3 ; 2.6/10.0 | 12.9/20.5 ; 17.1/27.7 | 1.8/5.8 ; 2.0/6.1 |
| getAlbumList2_newest50 | 11.2/17.2 ; 10.3/15.5 | 32.9/49.2 ; 40.2/67.1 | 54.5/128.5 ; 36.5/83.3 | 77.9/121.7 ; 137.1/224.0 | 217.1/252.5 ; 235.4/268.7 |
| getArtists | 11.8/18.1 ; 9.5/12.8 | 29.2/51.8 ; 37.8/71.4 | 57.9/118.4 ; 40.6/82.8 | 116.7/185.6 ; 335.3/558.4 | 116.8/142.7 ; 141.2/181.2 |
| getAlbum | 8.8/12.6 ; 5.3/10.0 | 13.6/19.8 ; 15.3/24.3 | 40.1/109.6 ; 31.7/76.0 | 68.6/104.8 ; 164.7/275.4 | 77.6/106.0 ; 86.4/122.3 |
| search3 | 38.3/57.8 ; 26.4/33.5 | 40.8/59.5 ; 43.3/58.6 | 99.2/252.0 ; 43.2/102.5 | 110.5/169.7 ; 229.5/329.5 | 530.8/4,658.9 ; 865.4/6,892.8 |
| getCoverArt_300 | 2.3/4.9 ; 2.5/5.8 | 2.6/5.6 ; 2.0/6.4 | 6.6/26.5 ; 4.5/21.7 | n/a ; n/a | 2.6/5.8 ; 3.0/6.4 |
Streams: time to first audio byte
| What | |||||
|---|---|---|---|---|---|
| flac_original | 1.5/16.3 ; 1.1/15.2 | 0.9/1.5 ; 1.3/16.7 | 2.3/17.3 ; 1.8/16.6 | 11.5/76.2 ; 21.2/101.6 | 1.2/2.0 ; 2.3/6.3 |
| flac_to_mp3_320 | 59.9/209.6 ; 57.7/204.4 | 32.9/113.2 ; 31.2/113.2 | 143.3/293.0 ; 105.2/212.5 | 301.5/667.4 ; 421.8/707.4 | n/a ; n/a |
| flac_to_opus_128 | 60.4/66.2 ; 57.1/124.4 | 48.0/54.3 ; 44.4/48.9 | 718.0/948.8 ; 474.9/757.8 | 732.8/1,566.2 ; 833.8/1,775.4 | n/a ; n/a |
| long_flac_original | 1.3/3.0 ; 0.9/15.3 | 0.8/1.8 ; 0.7/15.6 | 25.2/39.7 ; 13.9/53.4 | 23.0/48.0 ; 27.4/52.3 | 1.5/2.4 ; 1.7/4.4 |
| long_flac_mp3_320_from_start_repeated | 57.8/65.3 (first 65) ; 59.5/72.1 (first 57) | 31.3/34.6 (first 30) ; 29.2/32.0 (first 29) | 27.1/543.9 (first 544) ; 28.1/521.9 (first 522) | 16.7/324.6 (first 325) ; 23.3/334.3 (first 334) | n/a ; n/a |
| long_flac_mp3_320_seek_1500s_plus | 61.4/77.9 ; 55.6/75.0 | 734.1/803.1 ; 889.6/1,027.2 | 722.5/859.3 ; 570.1/649.7 | n/a ; n/a | n/a ; n/a |
| long_flac_opus_128_seek_1500s_plus | 58.5/63.8 ; 55.9/59.8 | 951.0/1,253.9 ; 1,192.6/1,332.2 | 1,252.0/1,469.6 ; 914.6/1,031.1 | n/a ; n/a | n/a ; n/a |
| long_flac_mp3_320_seek_1500s_repeated | 59.7/66.8 (first 60) ; 53.5/57.1 (first 53) | 877.7/1,181.3 (first 818) ; 1,248.8/1,696.3 (first 1,344) | 11.5/43.4 (first 41) ; 4.8/40.0 (first 4) | n/a ; n/a | n/a ; n/a |
Where Leyr is not the best
Rows where another server had the lower median, or where Leyr was within 25% of the lowest (a tie, given the noise). Values are the median of the runs, in the row's unit.
- Peak memory during scan (MB) (scan of the library): Leyr 42.8, gonic 52.7. Leyr is within 25% of the best.
- Memory under load, median (MB) (memory and threads): Leyr 50.0, gonic 57.9. Leyr is within 25% of the best.
- Memory under load, peak (MB) (memory and threads): Leyr 56.9, gonic 63.6. Leyr is within 25% of the best.
- First boot, empty volume (s) (start-up): Leyr 0.4, gonic 0.3. Leyr is within 25% of the best.
- Restart, median of 5 (ms) (start-up): Leyr 329.3, gonic 296.3. Leyr is within 25% of the best.
- Restart, slowest of 5 (ms) (start-up): Leyr 449.7, Navidrome 335.6. Leyr is behind the best.
- ping (api latency, one client at a time): Leyr 0.3, Plex 0.2. Leyr is behind the best.
- getCoverArt_300 (api latency, one client at a time): Leyr 0.6, Plex 0.3. Leyr is behind the best.
- ping (api latency, 16 clients at once): Leyr 1.8, Plex 1.9. Leyr is within 25% of the best.
- getCoverArt_300 (api latency, 16 clients at once): Leyr 2.4, gonic 2.3. Leyr is within 25% of the best.
- flac_original (streams: time to first audio byte): Leyr 1.3, gonic 1.1. Leyr is within 25% of the best.
- flac_to_mp3_320 (streams: time to first audio byte): Leyr 58.8, gonic 32.0. Leyr is behind the best.
- flac_to_opus_128 (streams: time to first audio byte): Leyr 58.7, gonic 46.2. Leyr is behind the best.
- long_flac_original (streams: time to first audio byte): Leyr 1.1, gonic 0.8. Leyr is behind the best.
- long_flac_mp3_320_from_start_repeated (streams: time to first audio byte): Leyr 58.6, Jellyfin 20.0. Leyr is behind the best.
- long_flac_mp3_320_seek_1500s_repeated (streams: time to first audio byte): Leyr 56.6, Navidrome 8.1. Leyr is behind the best.
Results
Real library, all results
The same tests on the private library. These confirm whether the public results hold on real music; they cannot be reproduced by others.
Measured: Leyr 2026-11-15; Navidrome 2026-11-15; gonic 2026-11-15; Jellyfin 2026-11-15; Plex 2026-11-15. Leyr can be rerun alone, so its date can differ from the others.
Scan of the library
| What | |||||
|---|---|---|---|---|---|
| Scan time (s) | 3.5, 3.4, 3.0, 4.6 (median 3.5) | 10.1, 24.0, 9.9, 18.0 (median 14.0) | 25.5, 30.0, 26.3, 30.3 (median 28.1) | 177.9, 251.9, 198.7, 237.5 (median 218.1) | 183.3, 245.7, 205.5, 191.2 (median 198.4) |
| Peak memory during scan (MB) | 72, 82, 88, 59 | 71, 73, 72, 86 | 1,148, 1,131, 1,163, 1,122 | 611, 607, 505, 527 | 566, 553, 556, 557 |
| Private memory, peak during scan (MB) | n/a | n/a | n/a | n/a | n/a |
| Tracks indexed | 3122 | 3129 | 3122 | 3122 | 3122 |
Memory and threads
| What | |||||
|---|---|---|---|---|---|
| Idle memory, 5 min after the scan (MB) | 59 ; 45 | 55 ; 34 | 64 ; 63 | 486 ; 390 | 366 ; 338 |
| Idle private memory (MB) | 39 ; 25 | 33 ; 24 | 33 ; 32 | 246 ; 220 | 233 ; 232 |
| Idle threads | 6 ; 6 | 12 ; 12 | 29 ; 26 | 18 ; 18 | 113 ; 114 |
| Memory under load, median (MB) | 77 ; 73 | 71 ; 55 | 88 ; 84 | 551 ; 443 | 414 ; 297 |
| Memory under load, peak (MB) | 86 ; 82 | 77 ; 60 | 107 ; 115 | 564 ; 464 | 415 ; 374 |
| Private memory under load, peak (MB) | n/a | n/a | n/a | n/a | n/a |
Start-up
| What | |||||
|---|---|---|---|---|---|
| First boot, empty volume (s) | 0.4 ; 0.5 | 0.3 ; 0.3 | 1.0 ; 1.4 | 3.8 ; 5.1 | 46.9 ; 51.3 |
| Restart, median of 5 (ms) | 300 ; 380 | 313 ; 408 | 294 ; 386 | 2,225 ; 2,405 | 1,525 ; 1,805 |
| Restart, slowest of 5 (ms) | 309 ; 481 | 350 ; 436 | 3,316 ; 436 | 2,285 ; 8,708 | 2,524 ; 2,894 |
API latency, one client at a time
| What | |||||
|---|---|---|---|---|---|
| ping | 0.3/0.3 ; 0.3/0.5 | 0.2/0.3 ; 0.3/0.5 | 0.4/0.6 ; 0.7/1.2 | 2.8/3.8 ; 3.4/5.5 | 0.2/0.4 ; 0.3/0.5 |
| getAlbumList2_newest50 | 1.4/1.9 ; 2.2/4.3 | 4.7/6.5 ; 7.0/10.5 | 2.9/4.7 ; 5.2/13.3 | 15.8/18.6 ; 17.5/20.4 | 8.6/10.4 ; 11.2/15.1 |
| getArtists | 1.8/2.3 ; 3.2/5.4 | 7.5/8.9 ; 20.3/47.1 | 5.7/7.7 ; 9.8/22.4 | 20.5/24.1 ; 22.3/25.1 | 8.0/9.2 ; 11.1/14.5 |
| getAlbum | 1.0/1.8 ; 1.5/3.4 | 2.4/3.5 ; 4.6/7.5 | 1.6/2.5 ; 3.7/10.7 | 13.4/19.3 ; 14.7/21.9 | 3.1/4.3 ; 3.7/6.2 |
| search3 | 4.4/7.7 ; 6.3/10.6 | 5.4/7.7 ; 9.6/15.3 | 4.2/7.5 ; 8.3/21.0 | 20.3/23.5 ; 24.8/32.8 | 42.5/58.3 ; 53.5/75.2 |
| getCoverArt_300 | 0.5/0.6 ; 0.7/1.0 | 0.3/0.5 ; 0.5/1.1 | 0.6/1.0 ; 1.1/3.8 | n/a ; n/a | 0.3/0.3 ; 0.4/0.9 |
API latency, 16 clients at once
| What | |||||
|---|---|---|---|---|---|
| ping | 1.7/4.8 ; 2.3/6.3 | 1.8/6.1 ; 2.5/7.4 | 3.9/14.5 ; 5.6/23.5 | 11.7/17.3 ; 18.3/29.5 | 1.7/5.5 ; 2.0/6.3 |
| getAlbumList2_newest50 | 11.7/14.6 ; 18.7/28.8 | 26.9/40.5 ; 49.7/76.1 | 36.5/96.5 ; 106.9/319.3 | 79.9/114.8 ; 114.0/180.5 | 160.4/188.0 ; 188.5/221.9 |
| getArtists | 14.2/19.7 ; 16.5/24.7 | 38.5/66.7 ; 75.4/129.2 | 50.8/104.4 ; 236.8/417.2 | 107.6/190.2 ; 163.1/271.9 | 178.4/219.0 ; 236.7/287.8 |
| getAlbum | 6.9/10.1 ; 8.4/13.8 | 12.7/16.7 ; 24.5/38.7 | 31.6/89.2 ; 129.1/301.5 | 66.6/105.1 ; 77.4/125.7 | 56.0/85.5 ; 78.9/125.7 |
| search3 | 25.0/34.7 ; 32.9/42.8 | 32.5/49.0 ; 71.0/194.9 | 97.5/202.3 ; 91.9/220.2 | 99.7/149.5 ; 140.9/226.7 | 3,252.6/4,927.5 ; 4,109.9/6,611.5 |
| getCoverArt_300 | 2.2/4.6 ; 2.8/5.2 | 2.4/6.4 ; 4.7/11.0 | 8.1/28.4 ; 30.7/95.4 | n/a ; n/a | 2.9/6.2 ; 2.9/6.0 |
Streams: time to first audio byte
| What | |||||
|---|---|---|---|---|---|
| flac_original | 0.8/6.4 ; 0.9/14.3 | 0.8/1.4 ; 3.7/17.3 | 1.7/2.3 ; 4.3/34.8 | 8.5/57.9 ; 10.7/56.0 | 1.2/16.2 ; 2.9/21.6 |
| flac_to_mp3_320 | 55.7/110.8 ; 70.5/254.6 | 33.3/115.4 ; 61.0/220.4 | 124.2/217.1 ; 211.2/447.8 | 399.6/982.9 ; 679.6/1,123.1 | n/a ; n/a |
| flac_to_opus_128 | 57.6/75.6 ; 66.5/72.3 | 44.8/50.0 ; 78.2/90.2 | 472.8/683.2 ; 705.6/954.3 | 599.7/2,050.5 ; 777.7/3,182.8 | n/a ; n/a |
| long_flac_original | 1.0/2.8 ; 1.6/15.6 | 1.0/2.0 ; 1.7/6.6 | 8.6/39.4 ; 15.8/73.4 | 26.0/37.1 ; 58.6/95.2 | 1.1/2.3 ; 1.1/3.5 |
| long_flac_mp3_320_from_start_repeated | 56.8/60.1 (first 58) ; 61.6/66.4 (first 58) | 29.7/33.1 (first 28) ; 53.5/61.4 (first 54) | 18.1/437.7 (first 438) ; 24.2/587.3 (first 587) | 21.6/430.5 (first 430) ; 62.6/409.1 (first 409) | n/a ; n/a |
| long_flac_mp3_320_seek_1500s_plus | 58.9/83.1 ; 63.0/79.5 | 647.0/700.2 ; 1,050.1/1,372.3 | 459.0/602.6 ; 589.1/930.8 | n/a ; n/a | n/a ; n/a |
| long_flac_opus_128_seek_1500s_plus | 56.7/58.4 ; 60.7/66.3 | 863.5/1,034.3 ; 1,356.3/1,711.0 | 725.3/821.0 ; 807.9/883.8 | n/a ; n/a | n/a ; n/a |
| long_flac_mp3_320_seek_1500s_repeated | 54.7/56.1 (first 55) ; 62.9/66.7 (first 63) | 617.1/771.3 (first 686) ; 908.0/1,199.5 (first 1,027) | 9.7/38.5 (first 12) ; 7.7/56.7 (first 4) | n/a ; n/a | n/a ; n/a |
Where Leyr is not the best
Rows where another server had the lower median, or where Leyr was within 25% of the lowest (a tie, given the noise). Values are the median of the runs, in the row's unit.
- Peak memory during scan (MB) (scan of the library): Leyr 76.8, gonic 72.5. Leyr is within 25% of the best.
- Idle memory, 5 min after the scan (MB) (memory and threads): Leyr 52.1, gonic 44.7. Leyr is within 25% of the best.
- Idle private memory (MB) (memory and threads): Leyr 31.8, gonic 28.0. Leyr is within 25% of the best.
- Memory under load, median (MB) (memory and threads): Leyr 75.2, gonic 63.1. Leyr is within 25% of the best.
- Memory under load, peak (MB) (memory and threads): Leyr 84.1, gonic 68.5. Leyr is within 25% of the best.
- First boot, empty volume (s) (start-up): Leyr 0.5, gonic 0.3. Leyr is behind the best.
- Restart, median of 5 (ms) (start-up): Leyr 339.8, Navidrome 340.2. Leyr is within 25% of the best.
- Restart, slowest of 5 (ms) (start-up): Leyr 394.7, gonic 393.1. Leyr is within 25% of the best.
- ping (api latency, one client at a time): Leyr 0.3, Plex 0.2. Leyr is within 25% of the best.
- search3 (api latency, one client at a time): Leyr 5.4, Navidrome 6.2. Leyr is within 25% of the best.
- getCoverArt_300 (api latency, one client at a time): Leyr 0.6, Plex 0.3. Leyr is behind the best.
- ping (api latency, 16 clients at once): Leyr 2.0, Plex 1.9. Leyr is within 25% of the best.
- getCoverArt_300 (api latency, 16 clients at once): Leyr 2.5, Plex 2.9. Leyr is within 25% of the best.
- flac_to_mp3_320 (streams: time to first audio byte): Leyr 63.1, gonic 47.1. Leyr is behind the best.
- flac_to_opus_128 (streams: time to first audio byte): Leyr 62.0, gonic 61.5. Leyr is within 25% of the best.
- long_flac_original (streams: time to first audio byte): Leyr 1.3, Plex 1.1. Leyr is within 25% of the best.
- long_flac_mp3_320_from_start_repeated (streams: time to first audio byte): Leyr 59.2, Navidrome 21.2. Leyr is behind the best.
- long_flac_mp3_320_seek_1500s_repeated (streams: time to first audio byte): Leyr 58.8, Navidrome 8.7. Leyr is behind the best.
Caveats
What these numbers do not show
- Noise
- The machine is a shared desktop. The 1-minute load average at the start of runs ranged from about 7 to 44, and the same server's concurrent latency changed by up to four times between its two runs. Treat differences under about two times as noise.
- Size
- One library of about 3,100 tracks per set, one machine, two full runs per server. There are no confidence intervals. Large libraries and many users were not tested.
- Different APIs
- Jellyfin and Plex have no Subsonic API. Their rows use their closest native requests, which return different data, so they indicate rather than compare. Plex cannot convert audio with no network (see the settings table), so its MP3, Opus and seek cells are n/a. Jellyfin extracted no album art in this setup: its cover-art cells are n/a. Jellyfin did not seek with the request form used: those cells are n/a.
- Scan time
- It means "the server says it is done and has gone quiet", not "the same work was done". Leyr's time comes from its own log, the others from polling, which adds up to about a second.
- Transcoders
- Each image ships its own ffmpeg (or, for Plex, its own transcoder). Part of any conversion time is that difference.
- Page cache
- Music was read from local NVMe and the first run of a server found colder files than later runs. Scan times of the first run are slightly higher for that reason.
- The harness
- It is Python threads on the same host, so it adds a small, equal cost to every request and cannot go faster than a few thousand requests a second.
Run it yourself
Rebuild the library and test one server
You need Linux, rootless podman, ffmpeg, Python 3.10 or newer, curl and about 35 GB of free disk. The first command sets where the files are downloaded from.
1. Download the scripts
BASE=https://leyr.stream/technology/benchmark/files
mkdir leyr-bench && cd leyr-bench
for f in bench.py report.py make_library.py validate_seek.py; do curl -fsSLO "$BASE/$f"; done
curl -fsSLO "$BASE/public-library-checksums.sha256"
2. Generate the public library
python3 make_library.py ./library --jobs 4 --manifest ./library-checksums.sha256
cmp library-checksums.sha256 public-library-checksums.sha256 && echo "same bytes as ours"
It takes 15 to 25 minutes and about 30 GB. The plan (names, tags, lengths) is identical everywhere. The audio bytes depend on the ffmpeg build; if cmp reports a difference, the generated files still have the same names, tags and lengths. For a quick try, add --albums 10.
3. Test one server
export BENCH_MUSIC="$PWD/library" BENCH_LIBRARY=public
python3 bench.py navidrome --phase main --run 1 --idle 60 --out results
python3 report.py results
less results/report-tables.md
The first command pulls the pinned image, starts a container on an internal network with no route out, scans the library, waits, measures latency and streams, restarts it five times and removes everything it created. Use --idle 300 to match our runs. The other servers are gonic, jellyfin and plex. For Leyr, build an image from the Leyr source with podman build -t bench-leyr . (or set BENCH_LEYR_IMAGE) and run python3 bench.py leyr .... Run it with the machine otherwise idle if you can.
4. Check the seek test
export BENCH_LONG_FLAC_FILE="$(ls library/Test*/*Long*/*.flac)"
python3 validate_seek.py navidrome
A server that starts at the right place prints different loudness for the three offsets, matching the source.
Clean up
podman network rm bench-noegress
Downloads
Scripts, raw results and checksums
Everything is served from this site. Check a download with sha256sum.
| File | What it is | Size | sha256 |
|---|---|---|---|
| bench.py | the harness: one server per run | 42 kB | 8c67f21660ab739449d4ec3eb2bd893773117012aff252f2fb8b26800ca5c2dc |
| rss_breakdown.py | splits a server's resident memory by owner (used by bench.py) | 4 kB | 77cb93946660c07038f6fb19cb1a5f4980395b829aa9c73ba9cdeda87052742a |
| report.py | turns result files into tables | 9 kB | 0555f4faeb6a1e687bc9f0c6268b130f480d06d295327fc359345f7bfd61dd11 |
| make_library.py | generates the public test library | 12 kB | 94ea0b32166df6adffd00b2345a9777a31dd401ea87016f06086fe9740c75d09 |
| validate_seek.py | checks that each server really seeks | 2 kB | dd35f3732f3e5414e6cfd91a4fcc8fb845b0efd2688c12b41452ebf0336f1cec |
| build_subset.sh | copies a subset of a music collection (for your own library) | 1 kB | 51e6d0152109364ea8a0fe0c6b06ce4207335ee705ffa47bc41d827136757c95 |
| make_page.py | builds these web pages from the results | 15 kB | 55342e98a2551092e78b4726823aa073628c7466a5067cd22f86756bca21909b |
| run_all.sh | the run order used | 2 kB | 9be6f6fd45f01e641a6df1e187d84ac3c8bad1bb61398552a84296ea59b2d438 |
| results-public.tar.gz | raw result JSON for every run on the public library, with summary.csv, seek-validation.jsonl and report-tables.md | 24 kB | ae64e01d05a65694b3976395fc4629fd130c81843bac84b93152213e99d9897a |
| public-summary.csv | public library: every value by run | 11 kB | 1f12aa184bfd2f8045ed8590afc2d179d043e3b83c2609946aca8c7f6ee0e285 |
| public-seek-validation.jsonl | public library: the seek check | 2 kB | 7d27bce20d219a8ec3a42352d305c520b03e1f2e037a2999907e6aa329171087 |
| results-real.tar.gz | raw result JSON for every run on the real library, with summary.csv, seek-validation.jsonl and report-tables.md | 20 kB | fea167f45b990f194fe5787d6b27d2ba4f7725e5e1a803ff588db2d3a6bf1d87 |
| real-summary.csv | real library: every value by run | 10 kB | 3e65e1df4a56f5fa1bb7869dcc8bc02cfe39c7cf7fe1bb523d0773f522476e4b |
| real-seek-validation.jsonl | real library: the seek check | 2 kB | 63eb484df3da0b773168512b0ae6bd1c1859b7ef4a392281554a1509eaf9aa53 |
| results-archive-2026-11-13.tar.gz | the first published run (2026-11-13): four servers, the private subset, before the changes listed below | 22 kB | 16054f8698890e1107499428d00cc70adddf1280d79e5b7581b9c2d15ed07ab6 |
| archive-2026-11-13-tables.md | the 2026-11-13 results as a table | 5 kB | 101136d6829ed388fb4dddae52efe11c61086fbbe1b86b12ce7bcde84187c583 |
| public-library-checksums.sha256 | sha256 of every file of the generated library | 379 kB | d4260eb912eda983654021cc37283721d50504b4889a6863abc4bbd82bc30865 |
| public-library-plan.json | size and format counts of the generated library | 0 kB | 607ae7f2a75625eb1f1b3aeb3c36018205c4f45a99ebab0e2285fda1ced8c39f |
Change log
What changed between published runs
Earlier numbers stay available.
- 2026-11-15
- Second published run, on November 14 and 15. Five servers (Plex added), two libraries (the generated public one added; the private one rerun), every container on a network with no route out, Leyr built from the commit above (it includes the memory work and the media tokens), and Leyr's scan time taken from its own log because it finishes before the harness can poll. The public library is the headline. Leyr can be rerun alone; if its rows change, the date above each results table says when each server was measured. Later on November 15, Leyr, gonic and Navidrome were rerun together, one after another, with a Leyr build that keeps transparent huge pages out of the process and returns free memory once the server is quiet; private memory was added next to RSS for every server. Later that evening, Leyr and gonic were rerun together once more, with a Leyr build from before a change to its worker count and purge delay and one after it taken in between, one after another; the before-change runs are in
bench/results/memory-333-beforein the repository, and Leyr’s private memory under the 16-client load fell from 31–35 MB to 25–30 MB with unchanged request times, as far as a busy host lets us tell. - 2026-11-13
- First published run: Leyr, Navidrome, gonic and Jellyfin on the private subset only, with network access. Kept as results-archive-2026-11-13.tar.gz and its table. Its stream seek rows for Navidrome and Jellyfin repeated one offset ten times and were partly answered from a cache; the later runs use a different offset per request.