Local speech recognition, with a traceable path.

An owned speech-to-text model family, runtime, and learning project. Shruti connects acoustic evidence to native-script text, keeps uncertainty reviewable, and studies how useful recognition can fit on local devices.

Read the roadmap ↓

Explore the projects →

Speech recognition is the first job.

Shruti is being developed as a local-first speech-to-text system with its own data contracts, training code, model changes, evaluation and runtime interfaces. Speech recognition comes first; text-to-speech, speaker diarization and meeting intelligence are separate later capabilities.

The family has four planned roles: Zero makes the complete recognition path understandable from scratch; Tiny targets streaming dictation and commands; Small targets multilingual local transcription; Core targets long-form and domain-adapted work. A tier earns its place through evaluation and runtime gates, not its parameter count.

Begin with the recording.

A recording is evidence before it becomes text. Its samples, timing and channel layout need an explicit interpretation, and its original audio hash stays with the transcript. Recording visibility, retention and training permission are separate decisions.

Zero starts with short WAV/PCM utterances in mono at 16 kHz. A PCM frame contains one sample per channel at one instant; sample rate and duration determine the frame count. Decoding, channel conversion and resampling must be tested before the recognizer can rely on comparable inputs.

One recording, several representations

A static map of the planned path. Each representation has a different job; this figure does not run recognition or plot an audio sample.

Representation	What it carries	What must be checked
Recording → samples	Timing, channels and the source audio hash	Repeatable decoding and comparable input preparation
Samples → acoustic features	Short windows and changing frequency energy	Owned STFT/log-Mel frontend against a reference
Features → candidate text	Encoder states, token scores and decoding	A reproducible trained checkpoint and unseen-audio evaluation
Candidate → reviewable transcript	Partial/committed state, native-script text and history	Explicit uncertainty, correction provenance and application view

Make the acoustic representation inspectable.

Overlapping windows expose how frequency content changes through a recording. Windowing, Fourier analysis, Mel filters and logarithmic energy form a sequence of representations that can be inspected independently of a trained recognizer.

The learning plan connects each concept to an experiment: generate tones, examine sampling and aliasing, implement a small DFT and FFT, then build STFT and log-Mel features. The frontend must agree with a trusted reference within a declared tolerance before its output becomes a training input.

Build a recognizer; then earn a streaming runtime.

Zero’s first architecture target combines convolutional subsampling, a BiGRU or small Conformer encoder, a CTC head and greedy decoding. It starts with an owned, resumable training loop and no open-weight initialization. Later tiers may evaluate declared, license-compatible initialization without hiding that lineage.

Streaming adds a different obligation: bounded chunks and left context must produce provisional text without confusing it with committed segments. Cache resets, partial revisions and offline-versus-streaming differences need their own tests. Transducer and autoregressive heads remain later candidates after the CTC baseline is reproducible.

A transcript that remembers its evidence.

ShrutiTranscript is the proposed interchange between the recognizer, runtime and applications. It keeps audio provenance, segment timing, language and script, transcript state, and model/checkpoint identity together. Uncertainty stays visible rather than disappearing into a fluent string.

Verbatim model text, normalized text, human corrections and the text an application used are distinct views. Native scripts are preserved; normalization never silently replaces the original words. Corrections add history events, and downstream requests refer back to committed segment IDs.

A partial format example from the specification. Empty text is a placeholder, not a recognition result.

{
  "shruti_transcript_version": "0.1",
  "audio": {
    "sample_rate_hz": 16000,
    "channels": 1
  },
  "segments": [
    {
      "start_ms": 0,
      "end_ms": 1200,
      "text_verbatim": "",
      "text_normalized": null,
      "state": "partial"
    }
  ],
  "privacy": {
    "local_only": true,
    "training_eligible": false
  }
}

Measure errors, latency, and memory separately.

Character Error Rate and Word Error Rate compare recognition against verified references through substitutions, deletions and insertions. Verbatim and normalized scores are separate. Hindi, English, code-switching, names, numbers, silence and difficult acoustic conditions need declared evaluation slices.

A useful result also identifies its checkpoint, dataset split, runtime, precision and device. First-partial latency, commit latency, partial revisions, package size and peak runtime memory answer different questions. Warm streaming behavior and initialization peaks must be measured rather than inferred from compressed weights or a universal accuracy percentage.

Four tiers, one disciplined roadmap.

The next concrete gate is a reproducible Zero path: deterministic fixtures, verified acoustic features, a small trained CTC baseline, unseen-audio evaluation, then local inference with a checkpoint and model card. Tiny, Small and Core build on that evidence; broader language and device support is earned per tier.

The wider research agenda stays connected to a working baseline:

Compact weights during execution, with actual resident memory measured.
Precision-aware training and calibration after ordinary floating-point/INT8 controls.
Bounded activations and streaming state, including reset and replay behavior.
Selective computation for uncertain speech without forcing a transcript commit.
Relevant Indic and code-mixed accuracy, including native scripts, names and numbers.

Recursive depth and low-bit execution are planned experiments, not measured Shruti improvements. Bolio is the separate cross-device voice-workspace lane; LibreOffice integrations, diarization, synthesis and full-duplex interaction have their own later gates. Document reasoning and actions stay outside the recognizer’s acoustic authority.

Explore the other projects →