ridervoice
Use when speaking text in the owner's own voice. Rider Voice — the owner's voice spoken by a model trained on his voice (F5-TTS v1 Base fine-tuned). Use ALWAYS when the owner says /ridervoice, 'speak
SKILL.mdiclone/skills/ridervoice/SKILL.md
GitHubPorted from the owner's skills library (port 1.0.0, upstream version 1.5; the source hash and what it needs are in
manifest.jsonnext to this file). Paths written{skill_dir}/…are this skill's own folder,{skills_root}/<name>/…a sibling icloneOS skill,{agent_home}/…the agent's home. Runs on macos only. Needsffmpegon PATH,python3on PATH,$RIDERVOICE_HOME,$F5TTS_VENV,$RIDERVOICE_DIR,$RIDERVOICE_SUBTITLES_DIR,$RIDERVOICE_PIPELINE_DIR,~/Movies. Not ported (the owner's own records — notes, logs, examples, project data — never ship):references/CHANGELOG.md,references/LEARNED.md,references/pronunciation.md,references/revoice-video.md,references/training-record.md,tools/revoice.sh. In this copy measured voice-profile numbers of a real person are written<n>; folders of the owner's machine are environment variables the host sets:RIDERVOICE_DIR(the Rider Voice folder: engine, docs and samples),RIDERVOICE_SUBTITLES_DIR(the subtitles of the videos to re-voice),RIDERVOICE_PIPELINE_DIR(the raw-video pipeline); the owner's own folder and project names are writtensample project(in the case and joiner they had); a value labelled as the owner's own prosody is written<n>or left out; pointers to a file that is not ported are left out. Kept by the owner's decision: "Rider Voice" (the owner's product name and public persona, kept by his decision).
Rider Voice — the owner's voice
What it is. A model that speaks like the owner in English: his timbre, pitch, intonation, rhythm and pauses, without hesitations, studio-clean. Trained on his real voice (28.7 clean min, 224 sentences) and approved by the owner on the owner's own verdict: "velocity, entonation, pronunciation, space between phrases is all very similar".
Where it is.
- Model + references + profile:
$RIDERVOICE_HOME/(model/rider_v1_ck2000.safetensors,model/vocab.txt,refs.json,refs/*.wav,profile_his_voice.json). Outside iCloud on purpose. - Full engine, docs and samples:
$RIDERVOICE_DIR/(local git). Heavy training work:~/Movies/sample_project/. - Environment:
$F5TTS_VENV(F5-TTS 1.1.22, mlx-whisper, resemblyzer, jiwer). GPU = Apple MPS.
1. Speak (the everyday case)
{skill_dir}/tools/say.sh "The text to say. Short sentences work best." out.mp3
{skill_dir}/tools/say.sh @script.txt out.wav --speed 1.12 --seed 23
Approved defaults: --speed 1.12 (= his real rhythm, <n> words/s), --pause 0.4 (pause between sentences), --seed 23, --verify 3 (each sentence is heard by whisper and re-generated with another seed while
a word is missing or extra), mastered to −16 LUFS / peak −1.5 dBTP, 48 kHz. --no-master gives the raw 24 kHz output.
Text rules (the model reads characters):
- English. Short sentences (7–25 words); sentences under 7 words are merged with their neighbour automatically.
- Numbers spelled out: "number seven", "two thousand twenty six", "ten percent". Never digits.
- Names as they are written and as he says them: iCLONE OS, iNFT, Virtuals Protocol, Claude Code, GitHub, OpenSea, KYC.
Rare/foreign words may need phonetic spelling;
--verifycatches most of them. - Only letters, space and
. , ? ! '(;and:become commas). No emojis, markdown, parentheses. - Punctuation is prosody: comma = short pause; full stop = his pause;
?rises.
2. Measure (never deliver without numbers)
{skill_dir}/tools/check.sh out.wav "The text to say. Short sentences work best."
Gates (from SAMPLE_PROJECT.md): WER ≤ 0.10 · 0 fillers · 0 repetitions · similarity ≥ 0.90 (the finals give 0.97–0.98)
· pitch ±2.5 st · rhythm 0.85–1.25× · median pause <n> s · noise floor ≤ −55 dBFS · peak ≤ −1 dBFS in the
delivered file. One FAIL = re-generate (another seed, or split the sentence) before delivering. For a large batch
(8+ texts), the adversarial critic (Opus agent with engine/SAMPLE_PROJECT.md) reads the measurements and approves only with numbers.
3. Deliver
- Files with a readable name (
s01 - Title.mp3), WAV + MP3 when it is for the owner to listen on his phone (SendUserFile delivers straight to the iPhone via Remote Control). - Always state the measured numbers (WER, similarity, rhythm) in one line; "sounds good" is not a result.
3b. Revoice a finished video (the video's whole voice becomes the model's)
When the owner asks "do the video's voice with the model", "replace my voice in the video", "revoice": the picture (burned-in subtitles and
panel, vignettes) is not touched; the speech is regenerated group by group from the subtitles, fitted into each window
(speed first, time-stretch only as a last resort), verified word by word and by forced alignment, and the
body sound is remixed with the original recipe (music, keys, ceiling). Full step-by-step, gates and traps in
references/revoice-video.md.
{skill_dir}/tools/revoice.sh <in_folder> <out_folder> [keys] # sample_project series (+ d1)
$F5TTS_VENV/bin/python "$RIDERVOICE_DIR/engine/revoice_video.py" <key> in.mp4 out.mp4
$F5TTS_VENV/bin/python "$RIDERVOICE_DIR/engine/verify_revoice.py" in.mp4 out.mp4 cues.srt --voice <key>_voice_new.wav
The owner's video is replaced only when verify_revoice says PASS (duration, identical video stream, vignettes, −18…−15 LUFS,
WER ≤ 0.10, every cue within ≤ 0.5 s by forced alignment, voice in the mix at lag 0). The originals go to a DATA/ subfolder.
A video that fails on 1–2 words is fixed with MODE=patch SEED=<new> revoice.sh … <key> (regenerates only the groups with
errors, reuses the rest); never regenerate the whole video for one error.
3c. Raw video (screen recording with his voice, no subtitles): the voice first, the picture follows it
When the owner hands over a raw recording and asks "prepare the full video, transcribe it and fix the voice with my model" (sample_project): word-by-word transcription → clean US English script (script critic) → voice sentence by sentence at his rhythm → picture re-timed to the voice (anchors on the 1st/last word of each sentence + event anchors: a click lands on the word that says it) → subtitles from the forced alignment of the new voice, positioned by screen → robot, music, keys, vignettes → gates → delivery critic.
P=$RIDERVOICE_PIPELINE_DIR
$P/run_a1.sh 3 # everything: voice → plan → subtitles/layout → robot → mix → assembly → 14 gates
$F5TTS_VENV/bin/python $P/voice_a1.py $P/text/script_a1.json ~/Movies/sample_project/voice --cands 3 --only 12 --speeds 1.06,1.12,1.18
$F5TTS_VENV/bin/python $P/tone_ref.py $P/text/script_a1.json <recording.mov> $P/qa/tone_ref_a1.json
$F5TTS_VENV/bin/python $P/sample_project.py $P/text/script_a1.json ~/Movies/sample_project/voice $P/qa/tone_ref_a1.json # the whole sequence, in his tone
The tone is his, and the whole sequence is chosen (owner: "aggressive, off tone, the last sentence does not finish"):
tone_ref.py measures on his recording, sentence by sentence, the relative pitch, the relative energy and the final contour; sample_project.py applies the hard
gates per take (words · name by TWO recognisers in context AND in the sentence cut on its own · sim ≥ 0.86 · intonation ≥ 1.3 st ·
<n> words/s · silences ≤ 0.22 s within the sentence / ≤ 0.65 s at punctuation · the LAST WORD of each on-screen sentence: ≤ +0.3 st
on '.', ≤ +4.5 st on ','; pitch ±1.8 st and volume ≥ −3.5 dB between neighbouring sentences; the sign-off falling) and chooses the sequence by
Viterbi (cost against his tone + jump cost between neighbours). Never ! for the model (it shouts and rises at the end).
NO take passes every hard gate on a sentence = generate 24 more takes of that sentence (--only g --refs … --speeds … --append), never
assemble with the "least bad" one: the verifier has the same gates and fails the assembly (20 min lost per round, sample_project r13–r15).
Spelling for the model: iNFT → eye N F T, iclone os → eye clone oh ess (written as-is it almost always fails at the end of the sentence).
One reference can pull a sentence 3 st higher in every take: voice_a1.py --cands … --refs 0,1,2,5 swaps them.
The numbers of the first delivery live in the owner's own records, which this port does not ship.
4. Improve the model (when there is new voice data of his)
Full pipeline in $RIDERVOICE_DIR/engine/ (see references/pipeline.md):
clean (df_clean.py + voice_chain.sh) → segment.py (cuts hesitations, 2 listens) → make_dataset.py batches →
text editors (Sonnet, SAMPLE_PROJECT.md) → make_dataset.py build → train.sh → auto_eval.sh →
critic (SAMPLE_PROJECT.md) chooses the checkpoint → export slim to $RIDERVOICE_HOME/model/ → samples → approval.
Overfitting shows up after ~2000
updates with 29 min of data (repetitions, rising pitch): always evaluate with 2 seeds.
5. Self-improvement (mandatory, on every use)
At the end of every job with this skill, without the owner asking:
references/CHANGELOG.md— date, what was spoken/trained, the numbers, what failed and how it was solved.references/LEARNED.md— each new lesson in three lines (Rule / Why (measured) / How to apply); a correction from the owner goes in on the spot with the prefixowner:and his words.- A lesson that appears twice is promoted to the right section of this SKILL.md or into the engine (a new gate
goes into the script, not only into the text) and
versiongoes up 0.1. - Words the model says wrong go into
references/pronunciation.mdwith the spelling that worked.
6. References
references/pipeline.md— the engine step by step, paths, commands, traps already paid forreferences/revoice-video.md— replacing the whole voice of a finished video, fitted to the subtitles: method, commands, gatesreferences/pronunciation.md— hard words and the spelling the model reads wellreferences/LEARNED.md·references/CHANGELOG.md— the skill's living memory