How Loquela is building live and two-pass local voice typing
Inside Loquela's move from batch-only offline dictation to live local partials and an optional second pass for final accuracy.
Loquela's local voice typing has always favored a simple privacy boundary: download a speech model, record only when the user starts dictation, run recognition on the phone, and insert the resulting text into the active Android field.
Until v0.15, every local model followed one timing model. Loquela collected the utterance, waited for Stop, and then handed the complete recording to an offline recognizer. That is batch recognition. It works with every local model Loquela currently lists, but it cannot show genuine local text while someone is still speaking.
Version 0.15 changes that foundation. Loquela can now run Batch, Streaming, or Two-pass recognition plans behind the same keyboard experience without pretending that one model can do all three jobs.
Version 0.15.1 takes the next, intentionally more public step: beta testers can install the first genuine live local model and select all three behaviors themselves. The app calls them Batch, Sequential, and Both. “Streaming” and “two-pass” remain useful engineering terms, but the settings describe what the user actually experiences.
Three modes, three clear promises
Batch waits until Stop and then asks one offline model for the transcript. It remains the default and continues to support Loquela's established model catalog.
Sequential runs Nemotron while speech is arriving. Words appear in sequence as replaceable composing text, so the current hypothesis can improve without leaving duplicate fragments in the editor.
Both runs Sequential first and then sends the complete utterance to a separate batch model. The second result is authoritative only while the provisional text remains safe to replace. If the user moves the cursor, edits the text, sends the message, or leaves the field, Loquela must not apply a late correction over their work.
These are user choices, not silent fallbacks. If Nemotron is missing, Sequential and Both remain visibly selected and explain that the model must be installed. The app links directly to Voice Models rather than quietly dropping the user back to Batch.
Batch is not going away
Batch recognition gets to examine the complete utterance before producing its answer. That can be valuable when an early phrase is ambiguous or when the model benefits from hearing what came later in the sentence.
It also has a forgiving compute model. If a phone needs several seconds to process a longer utterance after Stop, the result may feel slow, but the recognizer is not accumulating an ever-growing live backlog while the user continues speaking.
Loquela therefore keeps its existing batch recognizer path and every compatible installed model. The new architecture adds strategies; it does not replace a proven path with a single fashionable recognizer.
What Streaming changes
A native streaming model receives small chunks while speech is still in progress. It retains cached recognition state between chunks and can revise its current hypothesis as more audio arrives.
On Android, those partial words belong in composing text. Composing text is provisional: the keyboard can replace it without committing a trail of duplicate fragments into the editor. Loquela already uses this contract for optional Remote realtime transcription. Version 0.15 brings the same capability to the isolated local recognition process.
The microphone remains on the keyboard side of the process boundary. One long-lived audio connection carries ordered samples to the recognizer, while callbacks return changed partials, endpoint events, the final transcript, or a content-free failure category. This avoids a separate IPC call for every audio frame and preserves the crash isolation already used by local models.
Streaming is not automatically less contextual than batch. It retains what has already been spoken in cached state. The difference is that it cannot hear future words before displaying its current guess. That is why partials must remain visibly and technically provisional.
What Two-pass adds
Two-pass combines the two timing models:
- A streaming package produces immediate local text while the user speaks.
- Loquela retains the utterance audio for the active session.
- After Stop, a separate batch package receives the complete utterance.
- The final result replaces or completes the still-safe composing text.
The second pass is not a general AI rewrite. Its job is still transcription: recover what was spoken using a model that can consider the complete utterance. Loquela does not insert a prose generator that silently changes tone, intent, or authorship.
Two-pass is also not a property stored on either model. A streaming package remains a streaming package, and a batch package remains a batch package. The user-facing mode creates a compatible pair and makes the cost visible.
That separation matters because two-pass can be expensive. Two model archives consume more storage. Keeping both recognizers resident can make final correction fast but raises peak memory. Loading the final model only after Stop lowers overlap but adds cold latency. A cheap batch model may fit comfortably and still fail to improve a much larger streaming model, while a stronger finalizer may be the pairing that costs the most.
The feature earns promotion only if the correction benefit is meaningful on real utterances and real phones.
There is a useful tension in choosing that second model. A small finalizer is easier to keep resident but may not improve a 600 million parameter streaming model enough to justify the second decode. A larger model such as Whisper Turbo is more likely to make valuable corrections, but pairing its archive and runtime memory with Nemotron is the expensive case. Beta measurements, not the existence of the architecture, decide whether a particular pairing deserves a recommendation.
Why Nemotron is the first serious streaming candidate
Loquela is evaluating NVIDIA's Nemotron 3.5 ASR Streaming 0.6B through sherpa-onnx's Android-compatible online recognizer.
The NVIDIA weights were trained in the NeMo ecosystem, but Loquela does not embed NVIDIA's Python runtime or require CUDA. Sherpa-onnx exports the model to ONNX and runs it through the same Android native runtime family Loquela already uses for local speech models.
Nemotron is a 600 million parameter cache-aware FastConformer-RNNT model. NVIDIA publishes five latency configurations from 80 ms through 1120 ms. Loquela begins with the 560 ms package because it is a plausible middle point between early partials and recognition quality, not because a desktop benchmark proves it is appropriate for a phone.
The model card distinguishes 19 transcription-ready locales, 13 broad-coverage locales, and 8 adaptation-ready locales that require additional fine-tuning. Loquela will expose only the 32 ready or broad-coverage locales unless later evidence supports more. English is the default because the published 560 ms results show a measurable English accuracy penalty for automatic language detection. Auto remains available when multilingual detection is the user's deliberate tradeoff.
The phone, not the architecture diagram, decides
A local streaming recognizer must stay ahead of incoming audio. Real-time factor equal to 1.0 is only break-even. It leaves no room for a warm phone, a busy foreground app, or normal scheduling bursts. Loquela's promotion target is sustained real-time factor at or below 0.5, combined with evidence that the audio backlog does not grow across repeated thermally warmed sessions.
The validation pass separates three timings:
- Cold preparation measures loading the model and native runtime.
- Warm speech-to-first-partial measures how quickly useful live text appears after the recognizer is ready.
- Stop-to-final measures how long the user waits after finishing the utterance.
Speed is only one gate. Physical low-, mid-, and flagship-tier phones must also show stable native memory across repeated sessions, recovery after process death, safe behavior in ordinary and unusual Android editors, and no text loss or duplication.
Battery is measured as an absolute product cost, not just a ratio against batch. The useful number is energy per typical utterance plus a projected daily cost at a stated dictation volume. Thermal throttling is a separate correctness concern because it can directly break the real-time and backlog guarantees.
If 560 ms misses a tier narrowly, Loquela can measure the 1120 ms package before abandoning that tier. It offers more lookahead and fewer recognizer invocations, but whether that improves phone throughput enough to justify the added partial latency must be measured rather than assumed.
The physical-phone failure that changed v0.15.1
The first Play-installed v0.15.0 test on a OnePlus device found a failure that ordinary JVM tests and a successful build could not reveal. Every local model reported that voice audio had not transferred completely. Android's logs showed that the reliable file-descriptor pipe had reported its remote side dead before the isolated recognizer received the expected sample count.
Loquela did not weaken the integrity check. Version 0.15.1 switches the transport to an ordinary Android pipe while preserving the separate expected-sample count and explicit finish signal. An Android instrumentation test now writes real float samples through the same pipe protocol, reads them to clean EOF, and verifies the exact sequence on physical hardware.
That distinction matters: a compile proves that APIs line up, and a byte-array unit test proves the framing logic, but neither proves Android file-descriptor behavior across the app's actual process boundary. The beta found a platform-level integration failure, and the regression test now targets that layer directly.
What v0.15.1 makes available
Nemotron's 560 ms package is listed for beta installation. Sequential and Both are always visible in voice settings, with a setup prompt when Nemotron is missing. A deleted or damaged selection remains selected but visibly needs repair; Loquela does not silently swap the user's recognition mode.
Moonshine v2 Tiny and Base also enter the beta catalog as much smaller English batch models. Tiny drops from roughly 108 MB for the older package to about 30 MB, while Base drops from roughly 251 MB to about 111 MB. The old packages remain usable for existing installations but are retired from new installs and point users toward their replacements.
Moonshine v2 exercises a different part of the foundation from Nemotron. Its merged-decoder .ort layout requires an explicit runtime profile, discovery, validation, recovery, and offline configuration support. Nemotron reuses familiar encoder, decoder, joiner, and token file roles, but those files must be opened by the new online recognizer strategy. File shape alone must never decide how a model runs.
What beta testers should try
The most useful Sequential tests are not only clean demo sentences. Try short commands, long dictation, names, technical terms, pauses, warm repeated sessions, and editors that aggressively manipulate composing text. Watch for growing delay, repeated or missing words, unsafe replacement after an edit, and behavior after switching apps.
For Both, compare the final result with the visible Sequential hypothesis. A second pass that merely changes punctuation or introduces churn has not earned its extra cost. The important signal is whether it reliably removes errors that would otherwise require manual correction.
Parakeet users who see the last word or sentence disappear can use the new decoder-tail beta test. It appends a controlled amount of silence after the captured utterance, giving the transducer additional trailing context without pretending that the earlier audio-transfer bug and decoder behavior are the same problem. The default remains zero until real comparisons identify a better value.
The Settings feedback shortcut is part of the same beta philosophy: feature requests, bug reports, and other observations should be one tap away from the general settings screen, not hidden in a product-specific submenu.
Privacy remains mode-specific and explicit
Batch, Streaming, and Two-pass local recognition keep audio on the device. Session audio is used for the active recognition plan rather than saved as a persistent recording. Two-pass retains it only long enough to perform the requested final decode.
Optional Remote transcription remains a separate network mode with its own account and privacy boundary. Diagnostic retention stores bounded, content-free failure information rather than dictated text or audio.
The goal is not merely to make text appear sooner. It is to make live private dictation possible without weakening final accuracy, hiding compute costs, or taking control away from the person typing.
See the v0.15.1 patch notes, Voice Models, and Privacy for the current product boundary.