lab

From experiments to an inference engineering career

Rewritten for Kelechi, 6 September 2026. Planning horizon: 7 September–29 November. Assumption: employment or paid engineering work is the immediate objective, with audio research remaining a long-term direction. The weekly budget below assumes 25 focused hours; scale the hours to your availability while preserving the priorities.

core mandate

Build your professional identity around efficient speech and multimodal inference, with careful evaluation on constrained hardware. The next twelve weeks should produce two defensible case studies, a small demonstration someone else can run, and a consistent pipeline of job or contract conversations.

01The direction

Build your professional identity around efficient speech and multimodal inference, with careful evaluation on constrained hardware.

Your workspace supports that direction: Whistle connects model architecture to latency and correctness; Kaminari connects inference optimization to serving; Dante exposes the difference between a faster component and a faster application; Verso can eventually demonstrate training and evaluation. These are complementary kinds of evidence.

Whistle

Connects model architecture to latency and bit-level correctness on constrained hardware.

Kaminari

Connects inference optimization and quantization to serving and real-request handling.

Dante

Exposes the difference between optimizing an isolated component and speeding up an end-to-end application.

Verso

Demonstrates training, evaluation discipline, and data hygiene beyond inference alone.

The next twelve weeks should produce two defensible case studies, a small demonstration someone else can run, and a consistent pipeline of job or contract conversations. Research gets a fixed allocation and a decision deadline.

critical correction

The original guide's claim that you have “no skill problem” is too certain. Repositories demonstrate activity and some substantial results; interviews will also test independent implementation, debugging, fundamentals, and communication. Make those abilities explicit in the plan.

02What the original guide needs to change

Original advice Evidence from this workspace Revised action
Lead with Whistle being “38% faster, bit-exact.” The README reports an older 34.8% latency reduction. Newer five-run JSONs report 84.052 s versus 54.310 s median: 35.4% lower latency, 1.55× speedup. Both newer timing files say codec_parity_checked: false. Reconcile the article, README, timing runs, and separate correctness evidence before choosing a headline.
Finish Kaminari by implementing SSE. /generate_stream already exists; the API also checks disconnects. Validate serving behavior and measure OCR quality under quantization.
Publish Whistle and Kaminari articles from scratch. Sciel already contains whistle-qwen3-tts.html and kaminari-engine-breakdown.html. Local files do not establish live publication. Review, reconcile, and verify public access to the existing articles.
Make Dante speculation the next major project. Dante's long-clip profile attributes roughly 15% of wall time to decode. Establish a workload-specific upper bound before implementing a new speculative runtime.
Run Verso “in the gaps.” Verso has a detailed data and training plan, including recorded concerns about labels, masking, leakage, and evaluation. Give it an explicit experiment slot or pause it. Training needs uninterrupted evaluation discipline.
Add several new products and an all-local voice capstone. There are already many experiments and a local checkout of an established speech-to-speech pipeline. Build one small integration using existing components when it serves the chosen role. Do not assume the whole stack fits in 6 GB.
Spend much of the guide on RL. The strongest near-term evidence is inference engineering. Keep RL as a bounded learning track until one controlled training result exists.
status of the findings

These are document and artifact findings, not freshly rerun GPU validations. The September 1.7B Whistle files have different expected frame counts, 1,279 versus 1,280, and neither records a parity check. Treat that comparison as exploratory until matched.

03Choose a role and the evidence it needs

Primary targets: speech/voice ML engineer, inference engineer, and applied ML engineer with deployment responsibilities.

Secondary targets: ML systems or research engineering roles whose requirements match the work you can defend.

Suggested introduction:

“I build and evaluate efficient speech and multimodal inference systems. My work focuses on latency, memory, streaming, and preserving output quality on constrained GPUs.”

Follow this with one verified result and a link. The 6 GB GPU explains the experimental constraints; your engineering decisions are the main story.

Inference performance and RL systems are recognizable hiring categories: Anthropic's current careers page lists both inference performance and RL engineering roles. Use these listings to study requirements, not as evidence that any specific position is accessible or a fit. Check location, seniority, and work authorization early. [Source: Anthropic careers].

evidence inventory rule

Keep an evidence inventory for each application: requirement → project or work example → exact artifact → limitation. Separate personal contributions from upstream functionality and agent-generated scaffolding. A local checkout of speech-to-speech is a useful integration starting point; its upstream production claims are not your personal work history.

04Two projects to finish first

Whistle: the flagship case study

The deliverable is a reproducible account of a specific optimization and its limits.

  1. Select baseline & configuration: Select one checkpoint, workload, decoding configuration, and baseline revision. Record hardware, software versions, warmup, output length, and timing boundaries.
  2. Reconcile timing & parity: Reconcile the September five-run results with the earlier exactness artifacts. Attach a correctness check for the release configuration; timing flags alone do not establish parity.
  3. Stress multiple inputs: Expand beyond one long passage: include short conversational text, a medium passage, a long passage, and an EOS/repetition stress case. Report observed failures as well as successful cases.
  4. Measure latency dimensions: Distinguish total generation time, RTF, and time to first playable audio. A faster full waveform does not establish a responsive streaming conversation.
  5. Align public claims: Update the existing article and README with the same claim. Include commands, raw results, audio samples, and the failed approaches that explain the final design.

Done means: another engineer can reproduce a bounded claim and understand why the optimization works. Stop adding features once that is true. Broader model or hardware coverage can be a later revision.

Kaminari: the serving and quality case study

The missing proof is the speed–quality tradeoff and behavior under real requests.

  1. Fixed evaluation benchmark: Assemble a small fixed evaluation set, such as 50–100 labeled pages covering clean print, receipts, and difficult scans. Keep it separate from examples used to tune prompts.
  2. Quantization vs reference: Compare the unquantized reference and int4 path on the same inputs. Report normalized OCR error; for structured receipts, also report field accuracy and schema validity.
  3. Isolate latency from batch throughput: Separate single-request latency from aggregate batch throughput. The README's 9.35× figure compares aggregate batch throughput with a single-request baseline; it is not a 9.35× latency improvement.
  4. Production edge cases: Exercise streaming completion, client cancellation, malformed input, queue pressure, and memory cleanup. Existing SSE code is the starting point for this verification.
  5. Interactive demonstration: Demonstrate one task: upload a receipt, see extracted fields and the original text, inspect mistakes, and see the measured latency.

Done means: a user can try the task and a reviewer can inspect the quality/performance tradeoff. A reproducible local demo and recording are sufficient initially; always-on GPU hosting is not a prerequisite.

05A twelve-week execution plan

Job search begins in week one and continues through every phase. The dates are work deadlines, not promises of an offer.

Period Main deliverable Completion gate
Week 1
Sep 7–13		 Evidence inventory, focused CV, Whistle claim reconciliation One defensible headline; accessible links; first applications submitted
Week 2
Sep 14–20		 Whistle release package and revised existing article Reproduction instructions, matched correctness evidence, raw timings, short demo
Weeks 3–4
Sep 21–Oct 4		 Kaminari evaluation and narrow demo Quality table plus latency/memory results; serving failure cases checked
Weeks 5–6
Oct 5–18		 One research experiment A short report with a go/no-go decision; no second research branch
Weeks 7–8
Oct 19–Nov 1		 One external-use artifact A runnable integration, a focused upstream PR submitted, or a scoped pilot proposal based on a real need
Weeks 9–12
Nov 2–29		 Interview preparation and iteration from actual feedback Improve the weakest observed part of the application/interview pipeline
slip rules

If Whistle slips, reduce optional benchmark breadth before extending the deadline indefinitely. If Kaminari quality is weak, publish the limitation and narrow the supported workload. If interviews arrive, reduce research time first.

Weekly time budget (25 focused hours)

12 hours: Active deliverable  ·  7 hours: Applications and conversations  ·  4 hours: Interview practice  ·  2 hours: Reading or research design.

Do not run Whistle cleanup, Kaminari development, Verso training, and Dante implementation as four simultaneous priorities.

06Research: choose the question before the implementation

Recommended default after the two case studies: one controlled Verso SFT experiment, because it would add training evidence to an inference-heavy portfolio. Choose Dante instead if the positions or collaborators you are pursuing specifically value speculative inference.

Option A: Verso SFT (Default)

Verify recorded label/masking/split concerns against current code rather than assuming old notes still describe current bugs. Establish a zero-shot baseline; overfit a tiny training set as a pipeline check; then run one decoder LoRA configuration.

Split by song, report WER/CER by language, and evaluate repetition and formatting separately. Use validation for decisions and reserve the final test evaluation. Mixture audio is the starting point in the newer plan; vocal separation is an ablation. Success is a credible comparison with your own matched baseline, not beating a published number from another protocol.

Option B: Dante CTC speculation

First measure the phase breakdown on the intended workload. If decode is 15% of runtime, even eliminating it entirely caps speedup at 1 / 0.85 ≈ 1.18×; doubling decode speed gives only 1 / (0.85 + 0.15/2) ≈ 1.08×, before drafting and verification overhead. That bound applies to the measured workload, not every ASR request.

An oracle experiment should estimate acceptance opportunities and verifier cost. Include draft-model memory, tokenization/alignment, cache rollback, and actual end-to-end timing before investing in a full implementation. A negative finding is a valid report. Alternatively, study long-form resampling/repetition failures by varying one factor at a time.

two-week sprint boundary

Limit the chosen research sprint to two weeks. Continue only when the result justifies the next specific experiment. Keep ACE-Distill, SAE/music search, new TTS runtimes, and additional products as backlog unless one becomes the explicitly chosen research track.

07Learning that supports the work

Practice explaining prefill versus decode, bandwidth versus compute limits, KV memory, CUDA graph constraints, quantization error, and trustworthy GPU timing. Derive a memory budget and an Amdahl bound without assistance. Trace one optimization from Python through the relevant kernels.

Kernel surgery in decode-lab

Use decode-lab for a contained implementation exercise: write a small RMSNorm or GEMV variant, establish numerical tolerance, measure it, and explain why it wins or loses at the shapes you actually use. A learning exercise may be valuable without an immediate production speedup; give it a fixed time budget.

Weekly engineering habits

Each week, do one independent coding/debugging session and one spoken walkthrough of a project. Be able to explain what you accepted, changed, or rejected in generated code. Reserve interview time for ordinary software skills too: testing, API design, concurrency, data handling, and debugging.

reinforcement learning discipline

For RL, learn policy gradients, advantages, KL, sampling, and reward failure modes; then run one tiny controlled exercise. GRPO avoids a separate critic, but rollout and activation memory remain significant. TRL documents trainer/rollout colocation and related memory considerations; fitting a quantized model for inference does not prove that its GRPO training configuration fits. [Source: TRL GRPO documentation].

Audio RL comes after a trustworthy SFT baseline and independent evaluation. Noisy reward improvements should never substitute for better held-out transcription or listening quality.

09The first seven days

  1. Verify public links: Verify the existing public project and article links; list which claims have raw evidence.
  2. Choose Whistle release config: Choose Whistle's release configuration and reconcile its timing and parity records.
  3. Draft focused CV: Write a one-page CV around relevant work and the two strongest projects.
  4. Build target list & apply: Create the 20-team target list and submit the first five suitable applications.
  5. Record technical walkthrough: Record a short Whistle walkthrough explaining the bottleneck, optimization, result, and limitation.
  6. Lock Kaminari evaluation specs: Specify Kaminari's evaluation set and metrics before writing more serving features.
  7. Weekly review & triage: Review the week: identify the next missing piece of evidence and schedule it. Leave all other project ideas in one backlog.

10Evidence reviewed

  • Original Sciel guide: Prior revision of the Sciel career guide (August 2026).
  • Whistle: Whistle technical article · Repository README (whistle/README.md) · Five-run official timings (official_current_p50_5runs.json) · Five-run V7 timings (v7_current_p50_5runs.json)
  • Whistle 1.7B: Official 1.7B timings (official_1.7b_3runs.json) · V7 1.7B timings (v7_1.7b_3runs.json)
  • Kaminari: Kaminari engine breakdown · Repository README (kaminari/README.md) · API implementation (src/kaminari/api.py)
  • ASR & Training: Dante results (dante/results.md) · Verso project plan (verso/project.md) · Hummingbird scope (hummingbird/project.md)
  • Tooling & Baselines: decode-lab (decode-lab/README.md) · Local speech-to-speech checkout
review limitation

This review does not establish current employment status, public deployment status, full code correctness, or completion of experiments absent from the inspected artifacts. Refresh the evidence inventory as results change.