Recently Written · git

subplz-web

git clone https://github.com/equwal/subplz-web

Log | Files | Refs


README.md (27713 bytes)

1 # SubPlz Web
2 
3 Line an audiobook up with its ebook, sentence by sentence.
4 
5 ![subread.space: drop an audiobook and its ebook, get subtitles, videos and a read-along book](docs/screenshots/home.png)
6 
7 Drop in an audiobook and the ebook it was read from. You get back subtitles
8 timed to the narration, with the wording taken from your own book rather than
9 from a machine's guess at what it heard. Two things to do with that:
10 
11 - **HoshiReader whispersync** — the `.srt` is the timing file.
12 - **Subtitled video** — a YouTube-ready MP4 with a selectable caption track.
13 
14 Free in the browser, without limit; see [Pricing](#pricing).
15 
16 ---
17 
18 ## Quick start
19 
20 ```powershell
21 .\run.ps1
22 ```
23 
24 Opens <http://127.0.0.1:8420>. First run creates `.venv` and installs
25 everything, including the alignment backend — several minutes, because it
26 pulls torch.
27 
28 Already set up:
29 
30 ```powershell
31 .venv\Scripts\python.exe -m uvicorn backend.main:app --port 8420
32 ```
33 
34 Requires **ffmpeg on PATH** and **Python 3.11** — subplz pins `>=3.10,<3.12`,
35 so 3.12+ will not work.
36 
37 ---
38 
39 ## About "no Whisper"
40 
41 This app never runs `subplz gen`, the mode that transcribes a book from scratch.
42 Only `subplz sync` is reachable, and `gen` is not exposed anywhere in the API.
43 
44 That said, `subplz sync` is itself built on Whisper, and there is no way around
45 that. It works like this:
46 
47 1. A **tiny** Whisper model makes a rough transcript of the audio.
48 2. That transcript is aligned to *your* ebook text with Needleman–Wunsch.
49 3. The subtitle text that gets written is **your book's text**, not Whisper's.
50 
51 So Whisper is used as a timing device, not as a source of words. Nothing it
52 mis-hears reaches the `.srt`. If "no Whisper" meant "no model downloads and no
53 transcription step at all", subplz cannot do that — you would need a different
54 aligner (aeneas, Montreal Forced Aligner, WhisperX) and a different tool.
55 
56 ---
57 
58 ## Input formats
59 
60 **Audio** — `m4b`, `mp3`, `m4a`, `opus`, `flac`, `wav`, `mkv` and friends.
61 A single file, or **a folder of per-chapter files**: drop all 44 mp3s and they
62 are merged into one chaptered file, in natural order (`9.mp3` before `10.mp3`).
63 Files can arrive one drop at a time; the upload starts once both halves are in.
64 
65 **Book** — `epub`, `txt`, `srt`, `vtt`, `ass`, `fb2`, `fb2.zip`, and the
66 Kindle formats `mobi`, `azw3`, `azw` and `prc`. In the browser every one of
67 them is read in the tab (`frontend/engine/book.js`; the Kindle formats through
68 [foliate-js](https://github.com/johnfactotum/foliate-js), fetched by
69 `tools/fetch_vendor.py`). A server job converts the same formats on upload.
70 
71 Conversion on the server delegates rather than parsing ebook formats by hand
72 (`backend/convert.py`):
73 
74 | From | How |
75 |------|-----|
76 | `azw3` / KF8 | the `mobi` package unpacks it straight to epub |
77 | `mobi` (older) | `mobi` unpacks to HTML, ebooklib rebuilds the epub |
78 | `fb2`, `fb2.zip` | it is XML; lxml reads it, ebooklib writes the epub |
79 
80 `mobi` is the only added dependency; lxml, BeautifulSoup and ebooklib already
81 ship with subplz.
82 
83 **Conversion must preserve chapters**, and that is not a stylistic preference.
84 See [Will this text even match?](#will-this-text-even-match).
85 
86 ---
87 
88 ## Will this text even match?
89 
90 subplz fails late and unhelpfully when the text does not match the audio: it
91 transcribes the whole book, then reports "the generated transcript and the
92 provided text file are too different" and writes a `.subfail`. On CPU that is a
93 wasted hour. This app answers the question on upload, in seconds — and answers it
94 better than subplz can.
95 
96 ### Why a fixed threshold cannot work
97 
98 subplz compares chapters with `rapidfuzz.fuzz.ratio` against a constant
99 `SCORE_THRESHOLD = 40`. What two *unrelated* chapters score depends entirely on
100 how many characters the script has:
101 
102 | noise floor, unrelated chapters of one book | `fuzz.ratio` |
103 |---|---|
104 | Japanese | 23 – 26 |
105 | Russian | 39 |
106 | Spanish | 44 |
107 | English | 45 – 47 |
108 
109 For English and Spanish the floor is **above 40**, so unrelated chapters clear
110 the gate and it discriminates nothing. The constant suits Japanese, which is what
111 it was tuned on: thousands of distinct characters make coincidental similarity
112 rare, while twenty-six letters make it inevitable. Cleaning does not help — a
113 generic punctuation strip moved English only 45.3 → 42.4, because the cause is
114 alphabet size, not typography.
115 
116 ### What this app does instead — `backend/chapters.py`
117 
118 **Character 4-gram overlap, not edit distance.** Measured against the same audio:
119 
120 | metric | ru floor | en floor | ja floor | true match | runner-up | separation |
121 |---|---|---|---|---|---|---|
122 | `fuzz.ratio` | 39.2 | 45.1 | 26.1 | 86.3 | 40.8 | 2.1× |
123 | word Jaccard | 10.4 | 17.7 | 1.9 | 63.2 | 14.8 | 4.3× |
124 | **4-gram Jaccard** | **4.8** | **10.6** | **2.5** | 53.3 | 5.6 | **9.5×** |
125 
126 n-grams rather than words because many languages do not separate words with
127 spaces — a word-level metric collapses on Japanese, where a "word" is a whole run
128 of characters. Set intersection is also cheaper than edit distance.
129 
130 **A threshold calibrated per book, not per release.** Before judging anything,
131 the app samples ~150 unrelated chapter pairs *from the book in hand* to learn what
132 it scores by chance, then requires a match to clear that floor. One rule that
133 behaves correctly whether the floor is 2.5 or 45, with no constant to re-tune.
134 
135 **A runner-up test, which is what actually catches a wrong book.** A different
136 book in the same language scores *high* — it shares a vocabulary. What it cannot
137 do is make one chapter stand out. Measured, same audio:
138 
139 | book | best | runner-up | confidence | accept_at | verdict |
140 |---|---|---|---|---|---|
141 | the right one | 78 – 87 | ~30 | **2.7 – 2.9×** | 30.7 | accepted 4/4 |
142 | an unrelated Russian novel | 55 – 60 | 53 – 58 | **1.0×** | 92.1 | rejected 4/4 |
143 
144 Note the second row clears subplz's threshold of 40 comfortably and is still
145 rejected here, on both tests independently: its own noise floor is 57.6, so
146 `accept_at` rises to 92, and nothing stands out from the crowd.
147 
148 **Adaptive sampling.** Chapters are sampled from across the book, skipping
149 chapter 0 — publisher announcements and credits live there, so it is the least
150 representative chapter in the book. Sampling stops as soon as one chapter matches
151 confidently, so a good pair typically costs ~8s and a doubtful one ~20s.
152 
153 A poor verdict warns and relabels the button "Start anyway"; it never blocks,
154 because the check only samples and it is your book. Disable it entirely with
155 `SUBPLZ_WEB_MATCH_CHECK=false`.
156 
157 ### Measured and rejected
158 
159 - **Typographic normalisation** of quotes, dashes, soft hyphens, NBSP and
160   footnote markers: **+0.0**. Built, measured, deleted.
161 - **Un-gluing sentence boundaries** (`роса…Мне` → `роса… Мне`): multi-sentence
162   lines 13.6% → 12.4%, because pysbd does not treat `…` as a terminator for
163   Russian. Not worth the epub rewrite.
164 - **`token_set_ratio` / `token_sort_ratio`**: *worse* than `fuzz.ratio` on long
165   texts — nearly every common word appears in both chapters.
166 
167 ---
168 
169 ## Languages
170 
171 97 languages. The catch upstream does not document: subplz splits sentences with
172 `pysbd`, which supports **23** languages and raises `ValueError` on anything
173 else — **Portuguese and Finnish included**. subplz can use `stanza` instead when
174 given `--nlp`, which covers 74 more.
175 
176 This app resolves that per request. `backend/languages.json` records which
177 splitter each language needs, and the aligner adds `--nlp` automatically. You
178 never see the failure.
179 
180 | Language | Splitter | Notes |
181 |---|---|---|
182 | Spanish, Russian, Japanese | pysbd | fast path |
183 | Portuguese, Finnish | stanza | one-off model download on first use |
184 
185 subplz also defaults `--language` **and** `--lang` to Japanese. The aligner
186 always sets both explicitly, so a non-Japanese book cannot silently run as
187 Japanese.
188 
189 Regenerate the registry after upgrading pysbd or stanza:
190 
191 ```powershell
192 .venv\Scripts\python.exe -m backend.gen_languages
193 ```
194 
195 ---
196 
197 ## How it works
198 
199 ```
200 drop files ──► POST /api/uploads        stage, pair, convert, detect language
201                      │
202                      ▼                  (draft — correct the language here)
203                POST /api/jobs/{id}/start    entitlement check, enqueue
204                      │
205                      ▼
206                queue ──► runner ──► aligner ──► .srt ──► .mp4
207                                         │
208                                         ▼
209                                    storage + metadata.json
210 ```
211 
212 Upload and start are separate calls on purpose: a wrong language guess costs a
213 click instead of a re-upload and a wasted multi-hour run.
214 
215 ### Audio is prepared twice, deliberately
216 
217 The aligner is fed a **16 kHz mono** copy; the video keeps the original quality.
218 That is not tidiness — handing subplz 44.1 kHz stereo crashed ctranslate2 here
219 (integer divide by zero, part-way through a chapter), reproducibly. Doing the
220 conversion ourselves also repairs damaged input: the error-tolerant ffmpeg flags
221 drop corrupt frames instead of letting a single bad chapter abort the whole run.
222 
223 ### Files
224 
225 | Path | Role |
226 |---|---|
227 | `backend/api.py` | HTTP routes |
228 | `backend/aligner.py` | the alignment backend, behind an interface |
229 | `backend/runner.py` | stages inputs, drives the aligner, collects artifacts |
230 | `backend/convert.py` | fb2/mobi/azw3 → a chaptered epub |
231 | `backend/matching.py` | the preflight match score |
232 | `backend/render.py` | the YouTube MP4 |
233 | `backend/detect.py` | pairs the dropped files, detects the language |
234 | `backend/languages.py` | language registry and splitter routing |
235 | `backend/queue.py` | in-process or Redis dispatch |
236 | `backend/storage.py` | local disk or S3 |
237 | `backend/billing.py` | who may do what: free in the browser, a credit for a server job |
238 | `backend/payments.py` | Stripe: checkout, idempotent fulfilment, webhooks |
239 | `backend/accounts.py` | one email, one account; folding anonymous work in |
240 | `backend/auth.py` | emailed one-time sign-in links |
241 | `backend/pricing.py` | the plan catalogue, with the market it was set against |
242 | `frontend/` | vanilla HTML/CSS/JS, no build step |
243 | `tools/client.py` | CLI client, and a worked example of the API |
244 | `worker.py` | standalone worker for the Redis backend |
245 
246 ### Outputs
247 
248 | Artifact | What |
249 |---|---|
250 | `<name>.<lang>.srt` | the subtitles |
251 | `<name>.<lang>.mp4` | cover + audio + soft caption track (`mov_text`) |
252 | `metadata.json` | language, model, splitter, cue count, timing span |
253 | `subplz.log` | the full run log — the only way to debug a bad alignment |
254 
255 A job that runs in the browser tab can also make `<name>.read-along.epub`: the
256 epub with the narration inside it, as EPUB 3 Media Overlays
257 (`frontend/engine/epub.js`). Thorium, Storyteller and other EPUB 3 readers play
258 it and highlight each line. The cues do not say where in the pages their words
259 are, so the engine finds each cue's words again in the text of the pages, in
260 order, and puts a `<span id>` around them. An EPUB 2 book becomes EPUB 3 (a
261 navigation document is made from the NCX). The audio must be MP3 or AAC in
262 m4a/m4b, which are the types an EPUB 3 reader must play. `tests/engine/epub.test.mjs`
263 follows each overlay as a reader does, and gives the result to the W3C
264 `epubcheck` when `EPUBCHECK` points to its jar.
265 
266 The server keeps the files of each job together in `data/work/<job id>/`: the
267 audio and the book in `input/`, the subtitles in `out/`, and the run log. Nothing
268 deletes them: the operator deletes them by hand. A **failed** job can thus be
269 retried without a new upload.
270 
271 ---
272 
273 ## Swapping the alignment backend
274 
275 subplz is one implementation of `aligner.Aligner`, not a hard dependency.
276 Everything subplz-specific — its argument names, its Japanese defaults, the
277 shape of its progress output, where it writes the result — lives in
278 `SubPlzAligner`. To replace it:
279 
280 1. subclass `Aligner` (`build_command`, `progress_reader`, `locate_output`)
281 2. register it in `ALIGNERS`
282 3. set `SUBPLZ_WEB_ALIGNER` to its name
283 
284 The API, queue, storage and job runner do not change.
285 
286 ---
287 
288 ## Video
289 
290 A still cover image at 1 fps, the audio, and the subtitles as a **selectable
291 track** rather than burned in — so the file stays small, the encode stays fast,
292 and the viewer can turn captions off. The cover is the largest image in the
293 epub, or a plain dark card when there is not one.
294 
295 The H.264 encoder is **probed, not assumed**: `ffmpeg -encoders` lists encoders
296 that were compiled in, including hardware ones on machines with no such
297 hardware, so the app encodes one test frame with each candidate and takes the
298 first that actually works. On this machine that is `h264_amf`; a build with
299 `libx264` will prefer that. Set `SUBPLZ_WEB_VIDEO_ENCODER` to force one, or
300 `SUBPLZ_WEB_RENDER_VIDEO=false` to skip video entirely.
301 
302 ---
303 
304 ## CLI
305 
306 ```powershell
307 .venv\Scripts\python.exe tools\client.py submit book.m4b book.epub --wait
308 .venv\Scripts\python.exe tools\client.py submit .\chapters\ book.epub --wait
309 .venv\Scripts\python.exe tools\client.py list
310 .venv\Scripts\python.exe tools\client.py download <job_id> --dir out\
311 ```
312 
313 Use this rather than `curl` for non-ASCII filenames: curl on a non-UTF-8 console
314 mangles multipart filenames, and this client does not.
315 
316 ---
317 
318 ## Pricing
319 
320 All of the code is public, and anyone may host it. What this site sells is the
321 use of its operator's machines, and nothing else.
322 
323 **Free and paid are split by where the work is done, not by what comes out.**
324 
325 | Tier | Where it runs | You get | Costs |
326 |---|---|---|---|
327 | free | the visitor's browser (or the Android app) | each output: `.srt`, `.mkv`, `.mp4`, the read-along `.epub` | nothing, without limit |
328 | cloud | this server's hardware: Whisper tiny on a CPU, one book at a time | the same, from any device, with the tab closed: about 2 hours for a 10-hour book | one credit |
329 
330 | Plan | Price | Per book |
331 |---|---|---|
332 | 10 books | $4.99 | $0.50 |
333 | 100 books | $39.99 | $0.40 |
334 | 500 books | $174.99 | $0.35 |
335 
336 A book costs about $0.64 to convert on a rented GPU. Above about $5 a technical
337 buyer wraps a raw alignment API (ElevenLabs: $2.20 for a 10-hour book). Below
338 $3 the fixed card fee takes too much. There is no unlimited plan: use comes in
339 bursts, and one heavy user of such a plan costs more than the plan brings in.
340 An operator who wants a recurring plan adds one with `SUBPLZ_WEB_PLANS_JSON`.
341 Every number is an env var — see `backend/pricing.py`.
342 
343 A job in the browser costs the server nothing, so it is never counted and never
344 refused. A server job takes its credit at the start. A job that fails or is
345 cancelled gets the credit back.
346 
347 **Free credits.** They exist only while `SUBPLZ_WEB_CLOUD_ENABLED` is true.
348 
349 - A visitor without a verified email gets one free credit
350   (`SUBPLZ_WEB_FREE_CREDITS_ANONYMOUS`). A buyer whose email came only from
351   Stripe is in this group.
352 - An account whose email is verified gets two in all
353   (`SUBPLZ_WEB_FREE_CREDITS_VERIFIED`). The email is verified when its owner
354   opens a sign-in link that the server sent to it. A free credit that a device
355   used before counts toward the two, and a second device adds no free credits.
356 - A verified account also gets one free credit each day
357   (`SUBPLZ_WEB_DAILY_FREE_CREDIT`). It does not carry over to the next day.
358 - A verified account gets one more free credit for every two books that the
359   server converted for it (`SUBPLZ_WEB_BOOKS_PER_BONUS_CREDIT`). Books
360   converted in the browser do not count, because the server cannot check them.
361 
362 **Monthly plans.** A plan renews each month until the customer cancels it in
363 Stripe's customer portal ("Manage plan" on the page).
364 
365 | Plan | Price | Books |
366 |---|---|---|
367 | `month10` | $4.99 a month | 10 a month |
368 | `month30` | $9.99 a month | 30 a month |
369 | `unlimited` | $50.00 a month | no limit, one book at a time |
370 
371 The books of a month do not carry over: a new `current_period_end` from Stripe
372 sets `subscription_credits_used` back to 0. Retire a plan that was sold
373 (`pricing.RETIRED_PLANS`); do not delete it, or its subscribers get nothing.
374 
375 To let customers switch plans in the portal, make the Stripe prices once:
376 `runuser -u subplz -- .venv/bin/python tools/stripe_plans.py` on the server.
377 It makes one monthly price for each plan, with the plan id as lookup key, and
378 it is safe to run again. Then, in Stripe (Settings -> Billing -> Customer
379 portal), let customers switch plans between these three products. The lookup
380 key of the price names the plan, so a switch changes the plan at once, and the
381 books already used this month still count. A subscription whose price names
382 no plan of this site (the Stripe account also sells other products) is ignored.
383 
384 A server job spends the credit that ends first: the daily credit, then a book
385 of this month's plan, then a free credit, then a bought one. An unlimited plan
386 spends nothing. The account counts the free credits apart from the bought ones
387 (`free_credits_used`), so a refund of bought credits never pays out a free one.
388 
389 A switch above the drop zone says where the work is done. Each visit starts
390 with it on ("Convert in this browser (JavaScript)"), before a file is chosen.
391 Off: the files are uploaded, the job runs on the server, and the result waits
392 in the list of conversions, so the visitor can close the tab. The page asks
393 for the credit before the upload, not after it.
394 
395 The server keeps each upload and each result, for debugging. Nothing deletes
396 them automatically: the operator deletes them by hand, and on request.
397 `/terms.html` says so to the visitor, and shows `SUBPLZ_WEB_CONTACT_EMAIL`.
398 
399 During the beta (`SUBPLZ_WEB_BROWSER_COPY`, on by default), a job in the
400 browser also sends a copy of its audio and its book to
401 `POST /api/local/jobs/{id}/files`. The copy lands in `data/work/<job id>/input/`,
402 and the finished subtitles go next to it, so the files of every job are
403 together. The copy costs no credit. The page footer and the section "Input
404 data kept for debugging" of `/terms.html` say so to the visitor. Set the
405 variable to false after the beta, and change that policy and the page with it.
406 
407 `SUBPLZ_WEB_CLOUD_ENABLED` says that fast conversion is on offer. While it is
408 false the page shows no way to buy credits, whatever `SUBPLZ_WEB_BILLING_ENABLED`
409 says: credits that buy nothing must not be for sale.
410 
411 **An account is an email.** It gets attached either by following an emailed
412 one-time link (no passwords anywhere) or by paying, since Stripe collects one.
413 Whatever the device converted while anonymous is folded into the account, and
414 the cookie is re-pointed at it - including when the payment lands by webhook
415 with no browser attached (`Account.merged_into`).
416 
417 **Payments are Stripe Checkout**, with prices sent inline from
418 `backend/pricing.py`, so there is nothing to configure in the Stripe dashboard
419 beyond a webhook. Two rules: the browser returning from Stripe is never treated
420 as proof of payment (the session is re-read from Stripe), and fulfilment is
421 idempotent on the checkout session id, because Stripe reports one payment
422 several times.
423 
424 ### Turning payments on
425 
426 ```bash
427 # on the server, as root - each prompts for the value with input hidden
428 tools/set-secret.sh SUBPLZ_WEB_STRIPE_SECRET_KEY       # sk_live_... (or sk_test_...)
429 tools/set-secret.sh SUBPLZ_WEB_STRIPE_WEBHOOK_SECRET   # whsec_...
430 tools/set-secret.sh SUBPLZ_WEB_SMTP_PASSWORD           # for sign-in emails
431 ```
432 
433 In Stripe: Developers -> Webhooks -> add `https://<host>/api/billing/webhook`
434 with `checkout.session.completed`, `checkout.session.async_payment_succeeded`,
435 `customer.subscription.created`, `.updated` and `.deleted`. The monthly plans
436 need the three subscription events, and the customer portal (Settings ->
437 Billing -> Customer portal: let customers cancel at the end of the period and
438 update the card, then save). Then set `SUBPLZ_WEB_BILLING_ENABLED=true`,
439 `SUBPLZ_WEB_PUBLIC_BASE_URL`, `SUBPLZ_WEB_COOKIE_SECURE=true` and the `SMTP_*`
440 values in `.env` and restart.
441 
442 Sign-in email uses plain SMTP, so any provider works. With Resend: add the
443 domain in Resend, add the DNS records that it shows (they sit on a `send`
444 subdomain and `resend._domainkey`, so existing mail forwarding stays), create
445 an API key with sending access, and set:
446 
447 ```bash
448 SUBPLZ_WEB_SMTP_HOST=smtp.resend.com
449 SUBPLZ_WEB_SMTP_PORT=587
450 SUBPLZ_WEB_SMTP_USER=resend
451 tools/set-secret.sh SUBPLZ_WEB_SMTP_PASSWORD   # the Resend API key
452 ```
453 
454 Opening the emailed link verifies the address. A verified address gets the
455 free credits of an account (see *Free credits*).
456 
457 Not handled: refunds and disputes (do them in the Stripe dashboard and adjust
458 `purchased_credits` by hand), and tax (Stripe Tax is one checkout parameter away
459 once you are registered somewhere).
460 
461 #### Google Play purchases (Subrep)
462 
463 Google Play requires its own billing for digital goods. So the Google Play
464 build of Subrep sells the hour packs of `backend/captions.py` with Google Play
465 Billing. The Play product ids are the pack ids: `captions20`, `captions80` and
466 `captions200`. The GitHub build of Subrep still sells the packs with Stripe.
467 
468 The app sends each purchase to `POST /api/captions/play-purchase`, with its
469 device cookie. The server asks the Google Play Developer API for the purchase.
470 It adds the hours of the pack to the account once, and then it consumes the
471 purchase. The app never consumes. The table `play_purchases` keeps the SHA-256
472 of each purchase token, not the token.
473 
474 A buyer can get a refund from Google without you. Each hour, the housekeeping
475 pass reads the purchases that Google refunded or charged back in the last 30
476 days. It takes the hours of each one back from the account, once. A balance
477 never goes below zero, so the hours that the buyer used stay used.
478 
479 To turn it on:
480 
481 1. In Play Console, create the one-time products `captions20`, `captions80`
482    and `captions200`, each with one Buy option and a price.
483 2. Activate each product.
484 3. In Google Cloud, enable the Google Play Android Developer API.
485 4. Create a service account and a JSON key for it. Keep the key out of each
486    repository and each chat.
487 5. In Play Console, open Users and permissions and invite the email of the
488    service account.
489 6. For Subrep, grant View financial data and Manage orders and subscriptions.
490    The access can take some hours to start.
491 7. On the server, put the key at `/etc/subplz-web/play-service-account.json`,
492    owned by `subplz`, with mode 600. The user `subplz` must also be able to
493    open the folder `/etc/subplz-web`, for example with owner `subplz` and
494    mode 700.
495 8. Add `SUBPLZ_WEB_PLAY_SERVICE_ACCOUNT_FILE=/etc/subplz-web/play-service-account.json`
496    to `.env`.
497 9. Restart the server.
498 10. Make sure that `GET /api/captions` shows `"play_available": true`.
499 
500 The server reads the key file once. After you replace the file, restart the
501 server. If `subplz` cannot read the file, `GET /api/captions` shows
502 `"play_available": false`, the Play build shows no packs, and the endpoint
503 answers 503.
504 
505 ---
506 
507 ## Releases
508 
509 Every deploy is a tag (`vMAJOR.MINOR.PATCH`) with a GitHub release; `/healthz`
510 reports which one is running. Tag, push, then deploy that tag:
511 
512 ```bash
513 git tag -a v2.0.1 -m "what changed" && git push origin main --tags
514 gh release create v2.0.1 --notes "what changed"
515 ```
516 
517 ---
518 
519 ## Scaling out
520 
521 Everything environment-specific is an env var with a localhost default. See
522 `.env.example`.
523 
524 | Concern | localhost | public |
525 |---|---|---|
526 | Queue | thread pool in the API process | Redis + `worker.py` |
527 | Storage | `./data/artifacts` | S3, presigned download URLs |
528 | Database | SQLite | Postgres |
529 | Identity | cookie | cookie + emailed sign-in links (`SMTP_*`) |
530 | Billing | off, nothing locked | `SUBPLZ_WEB_BILLING_ENABLED=true` + Stripe keys |
531 
532 ```bash
533 SUBPLZ_WEB_QUEUE_BACKEND=redis \
534 SUBPLZ_WEB_REDIS_URL=redis://redis:6379/0 \
535 SUBPLZ_WEB_DATABASE_URL=postgresql+psycopg://user:pass@host/subplz \
536 SUBPLZ_WEB_STORAGE_BACKEND=s3 SUBPLZ_WEB_S3_BUCKET=subplz-artifacts \
537 SUBPLZ_WEB_DEVICE=cuda \
538 python worker.py
539 ```
540 
541 With an external queue the API copies staged uploads into shared storage before
542 enqueuing, and the worker pulls them down, so the API and workers do not need a
543 shared filesystem.
544 
545 Paid jobs and free jobs go to separate queues. `SUBPLZ_WEB_WORKER_QUEUES`
546 sets which a worker takes: `paid,free` (the default) or `paid`.
547 
548 **Burst mode** adds workers on the cheapest cloud (AWS spot or Hetzner) when
549 paid jobs wait, and removes them when they are idle. It is built with
550 Terraform and is off until the operator turns it on: see
551 [`infra/burst/README.md`](infra/burst/README.md).
552 
553 ### Before going public
554 
555 - Set the Stripe keys, the webhook and SMTP (see *Turning payments on*).
556 - Put a reverse proxy in front for TLS and upload limits.
557 - Watch the disk. The server keeps each upload and result until the operator
558   deletes it.
559 - Run workers on hardware that can take it. Alignment needs roughly 2–3 GB of
560   RAM; a 1 GB VPS will OOM. A 4h40m Russian audiobook took ~45 minutes on
561   `tiny`/CPU with 15 threads here, and would take many hours on one vCPU.
562 
563 ---
564 
565 ## Performance
566 
567 Alignment is dominated by the Whisper pass. `tiny` on CPU is the slow path; a
568 GPU with `--device cuda` is far faster. Chaptered files are processed chapter by
569 chapter, which is also what makes the progress bar meaningful — the runner
570 counts chapter completions rather than trusting the per-chapter bar, which
571 restarts at 0% for every chapter.
572 
573 Single files longer than about four hours can exhaust RAM, per upstream. Prefer
574 chaptered `m4b`, or a folder of per-chapter files.
575 
576 ## More projects
577 
578 - SubRead is also [for Android](https://github.com/equwal/subread-android/releases/latest),
579   [for YouTube](https://github.com/equwal/subread-extension/releases/latest)
580   and [for KOReader](https://github.com/equwal/subread.koplugin).
581 - [SubRead Overlay](https://github.com/equwal/subread-overlay/releases/latest): subtitle lines over any Android media player.
582 - [SubRead Dictionary](https://github.com/equwal/subread-dictionary/releases/latest): a pop-up dictionary for Android that reads Yomitan dictionaries.
583 - [SubRead Anki](https://github.com/equwal/subread-anki): one tap makes an Anki card from any Android app.
584 - [Subrep](https://github.com/equwal/subrep-android/releases/latest): live captions of the sound of your phone.
585 - [Book Simulator](https://booksimulator.com/): a reading room for Aozora Bunko and Project Gutenberg books.
586 - [honjimaku.com](https://honjimaku.com/): subtitles for Japanese audiobooks.
587 - [sbm Sync](https://sbmsync.com/): your bookmarks, the same on every device,
588   with [sbm](https://github.com/equwal/sbm) for dmenu,
589   [sbm for Android](https://github.com/equwal/sbm-android/releases/latest)
590   and the [sbm add-on](https://github.com/equwal/sbm-extension/releases/latest) for Firefox and Chrome.
591 - [Rebind](https://github.com/equwal/rebind/releases): remap the hardware buttons of e-ink readers and Android,
592   with [Ink Recents](https://github.com/equwal/ink-recents/releases/latest),
593   [Ink Dim](https://github.com/equwal/ink-dim/releases/latest)
594   and [Ink Update](https://github.com/equwal/ink-update/releases/latest).
595 - [dickt.store](https://dickt.store/): language-learning tools, flashcards and web toys.
596 - [hentaibun.online](https://hentaibun.online/): learn kanbun and kobun.
597 - [Recently Written](https://recentlywritten.com/): the blog, and a list of [all projects](https://recentlywritten.com/projects.html).
598 
599 ## Licence
600 
601 AGPL-3.0: see `LICENSE`. You may use, change and host this, and you must give
602 your users the source of what you host. `NOTICE` has the licences of the work
603 this is built on (SubPlz, whisper.cpp, Whisper).