# FC Hybrid Rig — Production Build Spec (Cal → Hermes, 6/6)

## Goal
Turn the proven hybrid POC into the real FC animation system: **warp (real-time geometry) + inpaint content-keys (offline) + a morph engine driven by `[PERFORM: emotion intensity]`, integrated into the floating window.** Replaces the brittle per-frame inpaint-composite approach.

## What Cal proved locally (the POC — reproduce this on production assets first)
On Malin's anchor (`malin_anchor.jpg`), Mac-side, no 5090 needed:
1. **Parametric warp = identity-locked geometry** (corners/cheeks/brows/lids) — her real pixels, nothing generated.
2. **Inpaint adds the content warp can't** — Nano Banana opened her mouth into a real teeth-smile while preserving identity 100% (freckles/eyes/hair/skin/framing).
3. **Hybrid in motion** — a warp-assisted cross-dissolve morphs neutral → inpainted smile-key cleanly, no ghosting (Nano Banana holds composition so the keys align). Clips: `talk_hold.mp4` (brow holds while mouth talks), `smile.mp4` (pure warp), `hybrid_smile.mp4` (the hybrid morph).

## The architecture (4 parts)
1. **Content-key library (offline, generated ONCE):** from the REAL `neutral_warm.png`, inpaint a small set of identity-locked content keys, ALL aligned to the same crop/pose:
   - start: `neutral`, `smile_teeth`, `talk_open` (mouth open mid-speech, teeth)
   - then: `surprised`, `soft_smile`, etc. as needed
   - Tool: **local 5090 inpaint** (Malin LoRA + face-lock — PREFERRED, keeps us independent per Jun's north-star) OR **Nano Banana** (proven, identity-preserving — fallback if local drifts). Keys are made once offline, so either is fine.
   - CRITICAL: alignment across keys (same framing/head pose) or the morph ghosts. Cal's POC confirmed Nano Banana holds it; verify each key vs neutral (eyes/nose/hairline must not move; 50/50 blend should stay crisp, mean abs-diff ~5-6 = good).
2. **Warp engine (real-time geometry):** your existing `build_warp_v1_prototype.py` — the displacement warp for cheap in-betweens (brows, cheeks, lids, head-pose axis, blink), with last night's global ×~1.5 exaggeration + the head-pose axis (judging=chin up+back, skeptical=chin turn, focused=lean in, regretful=head down).
3. **Morph engine (`[PERFORM:]`-driven):** blend warp + content-key. Cal's proven recipe:
   `out(t) = addWeighted( warp(neutral, geom·PK·t), 1−w, content_key, w )`, `w = ease_in_out_cubic(t)`, `PK ≈ 1.6`. Entry ~22f / hold ~20f / exit ~18f @30fps. `[PERFORM: smile 0.8]` → morph toward `smile_teeth` at intensity 0.8 (scales geom + blend). Intensity ladder is free (scale t). Per-expression timing follows last night's spec (soft_smile slow-onset + synced eye-squint; judging chin-up; etc.).
4. **Lip-sync (INDEPENDENT param):** drive mouth-open by audio amplitude/phonemes as a SEPARATE channel — warp the mouth-open OR morph toward `talk_open` — running WHILE an expression key holds. Cal's `talk_hold.mp4` proved the composability (brow holds while mouth talks). This answers Jun's "how long can an expression sustain with dialogue" — it just holds while the mouth runs independently.

## FC integration
Wire the morph engine's output into the floating window (`malin_v1_floating_window.py` / `fc_state.json`), reacting to `[PERFORM:]` + talking/idle states in real time. The morph engine REPLACES the auto-cycling-into-drifty-img2img-renders that caused the freckle/blur bug — these keys are aligned, so they crossfade clean.

## FIRST bounded step (reproduce the POC on production assets — then report)
1. Inpaint 2 content keys from the REAL `neutral_warm`: `smile_teeth` + `talk_open`, aligned + verified.
2. Reproduce Cal's morph: neutral → `smile_teeth` (warp-assisted dissolve) = a short clip.
3. Reply to jkhybridmotion@gmail.com with the clip + the 2 keys (contact sheet) + any blockers.
THEN we wire the morph engine + `[PERFORM:]` + lip-sync + FC integration, iterating together. Breadth-first: prove the morph carries to production assets before building the whole runtime.

## Collaboration + priority
Cal = architecture + recipe + review/iterate (designed + proved the POC). Hermes = build on 5090 against real assets + integrate + test. Report via email; Cal reviews artifacts + tunes the recipe. Priority per Jun (6/6): **start NOW — the FC hybrid takes the front seat**; sequence the long-form-video-consistency R&D around it as you see fit.

## Proven recipe details (replicate exactly)
- Nano Banana instruction that worked (identity held): "Edit ONLY her mouth/smile. Open into a natural warm smile with upper teeth showing slightly. Keep everything else 100% identical: face, freckles, eyes, hair, skin, lighting, framing. Don't alter her identity or any feature other than the smile."
- Morph math + timing as above. Global exaggeration ×~1.5 on the warp geometry.
- Cal's POC code (for reference / porting): `specs/malin-harness/warp_poc.py` (modes: still / smile_clip / talk_hold_clip / hybrid_smile_clip) on the Mac. You have the better-positioned engine (real assets + MediaPipe landmarks) — take the RECIPE, not necessarily the code.

-- Cal