README.md (9697 bytes)
1 # SubRead Overlay 2 3 Shows the lines of a subtitle file (`.srt`) over the app that plays your 4 audiobook or video. The line follows the position of the player. Tap a word to 5 select it, and send it to a dictionary. 6 7 It works with each Android player that publishes a media session: Voice, VLC, 8 mpv-android, YouTube, Smart AudioBook Player, podcast apps. No player needs a 9 change, and the app has no network permission. 10 11 No `.srt` for your audiobook? [SubRead](https://subread.space) makes one from 12 the audiobook and its ebook, in the browser or on 13 [Android](https://github.com/equwal/subread-android). 14 15 ## Screenshots 16 17 <p> 18 <img src="fastlane/metadata/android/en-US/images/phoneScreenshots/1.png" width="260" alt="The panel over Voice: the line of now, the move handle, pause, the timing row and close"> 19 <img src="fastlane/metadata/android/en-US/images/phoneScreenshots/2.png" width="260" alt="A tap on a word: the player pauses and SubRead Dictionary opens over the player"> 20 </p> 21 22 The pictures are from a Viwoods AiPaper Reader, with Voice as the player. 23 24 ## How to use it 25 26 1. Allow "Show over other apps". 27 2. Allow notification access. Android gives the position of other apps only to 28 an app with this access. The app uses it for the media position, the speed, 29 and play or pause. It does not read, keep or send a notification: the 30 listener has no `onNotificationPosted`. 31 3. Choose the `.srt` file. 32 4. Choose the dictionary for "Look up", or let Android ask each time. Each app 33 that has an entry in the text selection menu is in the list (Takoboto, 34 Aedict, AnkiDroid, a translator). 35 5. Press "Show the subtitles", then start the player. 36 37 Two settings change the panel: how much of the player shows through it, and 38 whether it shows the line before and the line after the line of now. 39 40 On the panel: 41 42 - `≡` moves the panel. 43 - A tap on a word pauses the player at once and selects the word. A drag 44 selects more words. When the finger lifts, the selection goes to the 45 dictionary. "Share" opens the share sheet of Android, for an app without an entry in 46 the text selection menu. "Copy" copies it. 47 - `▶` starts the player again after a lookup. `⏸` pauses it. 48 - `⋯` opens the timing row. `−0.5 s` and `+0.5 s` shift the subtitles. 49 `◀ line` and `line ▶` make the line before, or the next line, the line of 50 now. Use them when the media has an intro that the subtitle file does not 51 have. 52 - `✕` closes the panel. 53 54 The line stays on the panel until the next line starts, also in a silence, so 55 that there is time to look a word up. 56 57 ## How the timing works 58 59 A media session reports a position, the time of that report, and the speed. 60 `PlayClock` computes the position of now from these three. `Follower` sleeps 61 until the next line starts and wakes on each report of the player (pause, 62 seek, speed). It does not poll. 63 64 A player that reports no position: Android counts from zero when that player 65 starts to play, and the panel holds its line in a pause. Set the timing with 66 `◀ line` and `line ▶`. 67 68 The app does not use an accessibility service, and will not. Players that 69 hide their position (Netflix, some DRM players) are not supported. 70 71 ## Anki cards 72 73 With [SubRead Anki](https://github.com/equwal/subread-anki) installed, the 74 selection row has an "Anki" button. One tap makes a card in AnkiDroid: the 75 word, its reading and definition (from SubRead Dictionary), the line as the 76 sentence, a screenshot of the player, the word audio and the line read by the 77 voice of the device. The panel hides for a moment so that the screenshot shows 78 the player. SubRead Anki has its own optional capture service; this app stays 79 without one. 80 81 A book in many audio files: the player reports the position in the current 82 file, and the subtitle file has one clock for the whole book. Shift the 83 subtitles with the timing row at the start of each file. One `.m4b` for one 84 book has no such problem. 85 86 ## For reader apps 87 88 A reader app has no notification access, so it cannot see the position of 89 the player. This app answers for it, with a content provider at 90 `content://space.subread.overlay.player/state`. A query returns one row with 91 the column `state`: 92 93 ``` 94 playing=1;position=96153;speed=1.0;package=de.ph1b.audiobook 95 ``` 96 97 The position is in milliseconds, for the moment of the query. A problem is 98 `error=no_notification_access` or `error=no_player`. `call` with the method 99 `play`, `pause` or `seek` (the argument is the position in milliseconds) 100 controls the player. The panel does not need to be on the screen. 101 102 The [SubRead plugin for KOReader](https://github.com/equwal/subread.koplugin) 103 uses this to turn the pages with the audiobook. 104 105 ## For flash card apps 106 107 A query of `content://space.subread.overlay.player/line` returns one row: the 108 subtitle line of now, with the report of the player as it came, so that a 109 flash card app can put the line on a card and cut its sound. The columns: 110 111 - `state`: the same line as `/state`. 112 - `reported_position`, `reported_at`, `speed`, `playing`: the last report of 113 the player. `reported_at` is on the clock of `SystemClock.elapsedRealtime()`. 114 A pause reported a minute ago still maps each time of the media to the 115 moment it played. 116 - `offset`: the shift the user set. A time of the subtitle file is `offset` 117 milliseconds later than the same moment of the player. 118 - `index`, `start`, `end`, `text`, `before`, `after`: the line. `index` is -1 119 before the first line, and `text` is then null. The times are on the clock 120 of the subtitle file. 121 122 The query parameter `position` (milliseconds, on the clock of the player) 123 picks the line for that position in place of the position of now. The panel 124 does not need to be on the screen, but the user must have chosen the `.srt`. 125 126 [SubRead Anki](https://github.com/equwal/subread-anki) uses this. Choose it 127 as the dictionary in step 4: a tap on a word then makes a card with the line, 128 its sound and a picture of the screen. 129 130 ## For caption apps 131 132 An app that makes captions from live audio, for example speech recognition of 133 the sound of a video, can show its lines on the panel. The user then taps the 134 words and looks them up, the same as with a subtitle file. The app calls the 135 same content provider: 136 137 ```kotlin 138 val panel = Uri.parse("content://space.subread.overlay.player") 139 contentResolver.call(panel, "line", "It was a dark", bundleOf("partial" to true)) 140 contentResolver.call(panel, "line", "It was a dark night.", null) 141 contentResolver.call(panel, "end", null, null) 142 ``` 143 144 `line` shows the text. The extra `partial` is true while the sentence goes on: 145 the panel adds `…` to the line, and the next line replaces it. A final line 146 (no `partial`) stays as the line before, when the user shows three lines. `end` 147 gives the panel back to the subtitle file. Without `end`, live lines end ten 148 minutes after the last one. 149 150 The answer is in the bundle key `live`: `ok`, or the reason the panel cannot 151 show the line. `no_notification_access` and `no_overlay_permission`: the user 152 must allow steps 1 and 2. `panel_hidden`: the user must press "Show the 153 subtitles", or closed the panel with `✕`. The panel does not come back on its 154 own for a line, so that a close stays a close. 155 156 While a word is selected, the panel holds the line, so that the lookup has 157 time. The newest line comes when the selection goes. 158 159 From a shell, for a test: 160 161 ``` 162 adb shell content call --uri content://space.subread.overlay.player --method line --arg "It was a dark" --extra partial:b:true 163 ``` 164 165 ## Build 166 167 ``` 168 ./gradlew :core:test :app:assembleDebug 169 ./gradlew :app:connectedDebugAndroidTest # word selection and the follower, on a device 170 ``` 171 172 `:core` is plain Kotlin: the subtitle reader, the line for a position, the 173 clock. It has property tests. `:app` has the panel and the listener. 174 175 `-PplayStore=true` leaves the Ko-fi link out of the build for Google Play. 176 177 ## More projects 178 179 - [SubRead](https://subread.space/): read along with an audiobook, in the browser. 180 Also [for Android](https://github.com/equwal/subread-android/releases/latest), 181 [for YouTube](https://github.com/equwal/subread-extension/releases/latest) 182 and [for KOReader](https://github.com/equwal/subread.koplugin). 183 - [SubRead Dictionary](https://github.com/equwal/subread-dictionary/releases/latest): a pop-up dictionary for Android that reads Yomitan dictionaries. 184 - [SubRead Anki](https://github.com/equwal/subread-anki): one tap makes an Anki card from any Android app. 185 - [Subrep](https://github.com/equwal/subrep-android/releases/latest): live captions of the sound of your phone. 186 - [Book Simulator](https://booksimulator.com/): a reading room for Aozora Bunko and Project Gutenberg books. 187 - [honjimaku.com](https://honjimaku.com/): subtitles for Japanese audiobooks. 188 - [sbm Sync](https://sbmsync.com/): your bookmarks, the same on every device, 189 with [sbm](https://github.com/equwal/sbm) for dmenu, 190 [sbm for Android](https://github.com/equwal/sbm-android/releases/latest) 191 and the [sbm add-on](https://github.com/equwal/sbm-extension/releases/latest) for Firefox and Chrome. 192 - [Rebind](https://github.com/equwal/rebind/releases): remap the hardware buttons of e-ink readers and Android, 193 with [Ink Recents](https://github.com/equwal/ink-recents/releases/latest), 194 [Ink Dim](https://github.com/equwal/ink-dim/releases/latest) 195 and [Ink Update](https://github.com/equwal/ink-update/releases/latest). 196 - [dickt.store](https://dickt.store/): language-learning tools, flashcards and web toys. 197 - [hentaibun.online](https://hentaibun.online/): learn kanbun and kobun. 198 - [Recently Written](https://recentlywritten.com/): the blog, and a list of [all projects](https://recentlywritten.com/projects.html). 199 200 ## Licence 201 202 AGPL-3.0. See `LICENSE`.