ALDERSHOT: How It Works, Moment to Moment — A "How It Works" Dossier
- Aldershot should behave like two products in one skin: a frictionless one-file cutter for the user who already knows what they want, and a library that quietly compounds into a proactive collaborator as footage accumulates — with suggestions presented as a calm "possible videos" shelf, never as match scores or a "second brain." The core feel never changes; only the number of doors that open changes.
- The contextual engine is a layered pyramid — words → moments → chapters → videos → threads → library — built once at ingest (transcript with word timings, diarized speakers, chapters, moment cards, visual descriptions, embeddings, the Read) and queried later via retrieval + hierarchical/map-reduce summaries, because even 2026's million-token windows cannot (and should not, on cost) hold an entire 500-video library at once.
- The game-like layer should be "shown, not scored": quests for recording tasks, an inventory-like Footage library, fog-of-war reveal of long sources, and an overnight "worked while you slept" loop — grounded in flow theory and calm-technology principles, and deliberately avoiding streaks, badges, and loss-aversion mechanics that research ties to the overjustification effect and Duolingo-style dark patterns.
1Key Findings
- Two modes, one feel. The one-file user needs zero library noise; the 500-video user needs the library to surface combinations they'd never find by hand. The right design keeps the home composer identical and lets capability accrete — new doors appear, nothing is taken away, and nothing nags.
- Context is a pyramid, not a window. Everything expensive (transcription, diarization, moment extraction, embeddings, the Read) runs once per video at ingest and is stored. Everything per-request is cheap retrieval + summary reduction. This is the only architecture that stays affordable on Cloudflare economics as the library grows.
- Long-context models did not kill retrieval. As of mid-2026 the frontier is ~1M tokens — with Meta's Llama 4 Scout the open-weights outlier, which Meta's April 2026 announcement describes as increasing "the supported context length from 128K in Llama 3 to an industry leading 10 million tokens"[1] — but "effective" context is always less than advertised, cost scales with tokens, and an 8-hour stream alone is roughly 80–90k tokens of transcript. A library is orders of magnitude past any window. Retrieval over moment cards + map-reduce over long sources + prompt caching of the stable "library brief" is the durable pattern.
- **Proactive discovery should copy Google/Apple Photos Memories, Spotify Daylist, and Readwise resurfacing in spirit (delight, calm, control) while rejecting their weakest parts (notification-driven nostalgia jump-scares, opaque scoring).** Present a "possible video" as a premise sentence + real frames + runtime it could fill + what's missing + one button — never "87% match."
- The game layer must be flow-shaped, not casino-shaped. Flow needs clear goals, immediate feedback, and challenge-skill balance; the overjustification effect and documented Duolingo dark patterns show that points/streaks/loss-aversion actively corrode intrinsic motivation for a craft the user already loves. Progress shown as reveal and completion, not as scores and streaks, fits the "expensive-feeling" aesthetic.
2Details
1. Two contexts: the one-file user and the growing library
The one-file, "I know what to do" path (fast, no friction). The home composer is a sentence plus footage. When a user drops a single file and types "cut the best 90 seconds on the housing rant," Aldershot should:
- Classify intent immediately (extract) and start the Read on that file only.
- Show no library suggestions, no "this connects to…", no ideas shelf. The working row shows this one job. The library exists but is silent.
- Treat the Read as a within-file operation: threads inside the video, not across the library.
The principle: the library never taxes the one-file user. Cross-video machinery is strictly additive and only wakes up when there is more than one video that actually shares meaning. A brand-new account and a 1-video account look and feel the same as the fast path forever; the library UI (Moments, Threads, Sources, Ask) is present but shows "just this video" until there's a reason to show more.
How capability accretes (1 → 50 → 500 videos). The feel stays constant; what changes is how many doors are open:
- 1 video: Editor X, Clipper X, Footage for that one source. No cross-links possible, so none shown.
- 2–3 videos: the first cross-video link can form. This is the moment to introduce the concept gently and once — e.g., a single quiet line in Footage ("This clip and one from Stream 03 are about the same thing") rather than a notification. First contact should feel like a discovery, not a feature tour.
- ~10–50 videos: enough recurring topics and moments exist that "possible videos" become real. The ideas shelf (a drawer, not a popup) starts holding a small number of proposals. A weekly digest becomes worthwhile.
- 500 videos: the library is now the product's superpower. The same shelf holds richer proposals (then-vs-now, anniversaries, series arcs). Crucially the home screen is unchanged — the depth lives one click away in Footage and the shelf, so the surface never gets busier.
This mirrors how Google Photos Memories scales: the same gallery surface silently gains "themed" and "on this day" collections as the library grows, built by background analysis "often long after the initial upload occurred."[2] The lesson to copy is the restraint of the surface; the lesson to reject is Memories' reliance on push notifications that pull people back in (a design its own admirers, writing in Fast Company, call one that "maybe worked too well, sucking me into my phone").[3]
2. How the library "understands" footage — plain-English mechanics and limits
What gets extracted and stored once, at ingest (per video). Think of ingest as reading a book carefully one time and writing thorough margin notes, so you never have to re-read the whole book to answer a question. Per video, Aldershot computes and stores:
- Transcript with word-level timings — every word with a start/end timecode, so any quote maps back to an exact frame.
- Speakers (diarization) — "who spoke when," produced by a standard pipeline: voice-activity detection → speaker-embedding extraction → clustering[4] (industry/academic pipelines use exactly this VAD → embedding → clustering shape, e.g., pyannote with ECAPA-TDNN speaker embeddings). Optionally fused with on-screen active-speaker detection (e.g., TalkNet) for multi-person footage.
- Chapters / topics — the transcript segmented into coherent sections by topic boundary (semantic chunking rather than arbitrary time slices).
- Moment cards — the atomic unit of the library: a clip with a representative frame, a quote, its source, and a timecode. This is what search, threads, and suggestions all point at.
- Visual descriptions — short captions of what's on screen at sampled frames (a 1-frame-per-second sampling rate is common in multimodal video pipelines), plus scene/cut detection (e.g., PySceneDetect on HSV color changes) so "moments" respect visual boundaries, not just speech.
- Embeddings — numeric "meaning fingerprints" for each moment/chapter, stored in Vectorize, enabling search-by-meaning and cross-video linking.
- The Read — the human-readable summary of what the footage is, its threads, and where it lands.
Everything above is the expensive, once-per-video work. It's the right place to spend compute because it's amortized over every future request.
How cross-video links form. Links are not a mystical "knowledge graph" (a framing Chris has rejected) — they're concrete matches between stored moment cards:
- Recurring topics: moments whose embeddings cluster together across videos ("the housing rant appears in 6 streams").
- People: speaker embeddings (and optionally faces) that recur across sources.
- Running jokes / callbacks: near-duplicate phrasings or repeated quotes across dates.
- Evolving opinions / then-vs-now: the same topic cluster, ordered by source date, where the language or stance shifts.
- Series continuity: explicit source metadata (title patterns, dates) plus topic overlap.
Present these as plain sentences with real frames ("You've talked about X in these 6 clips"), never as "fits X · 87% match."
How an LLM works over far more footage than fits its window. The honest constraint: a whole library never fits in a context window, and even when it could, you wouldn't pay for it. Four techniques, layered:
- Retrieval (RAG) over moment cards + embeddings. For any request, fetch only the few dozen most relevant moments instead of the whole library. This is the workhorse. Research consistently finds that retrieving a focused set (in one time-sensitive-QA study, summarizing the top-5 passages struck the best balance) often beats stuffing everything in, because retrievers "exclude irrelevant passages and reduce noise."[5] It also avoids the "lost in the middle" failure — documented by Liu et al. in "Lost in the Middle: How Language Models Use Long Contexts" (TACL 2024), which found that "performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle[6] of long contexts."
- Hierarchical summaries. Store a summary at each layer (chapter summary → video summary → thread summary → library brief). A question about the whole library reads the small top-layer summaries first, then drills down only where needed. (Per Microsoft's FastTrack team, cited in a Galileo AI write-up, simple flat chunking "fails to maintain contextual relationships in 68% of complex technical documents"[7] — hierarchy is what fixes that.)
- Map-reduce over an 8-hour transcript. Split the giant transcript into chunks, summarize each chunk independently ("map"), then combine those summaries ("reduce"). This is the standard pattern for documents longer than the window; full-document attention scales at O(n²) and is "impractical for production documents commonly ranging from 40,000 to 160,000 tokens."[8]
- Long-context when it's genuinely one big thing. For a single source that fits, "send the whole document"[9] is now often the right call rather than retrieve — but a library is never one big thing.
Realistic 2025–2026 context sizes and costs, and what runs once vs. per request.
- Window sizes: ~1M tokens is the 2026 baseline for frontier closed models (GPT-5.x, Gemini 3.x Pro, and the Claude Opus/Sonnet families all converged on 1M), with Meta's Llama 4 Scout an open-weights outlier at 10M. But comparison benchmarks note "effective context falls short of advertised on every model ever benchmarked,"[10] so treat 1M as a ceiling, not a working budget.
- What a video costs in tokens: per OpenAI's tokenizer guidance, 1 token ≈ 4 characters of English and "100 tokens ≈ 75 words." An 8-hour stream at natural speaking rate is very roughly 80,000–90,000 words of transcript ≈ ~110k–120k tokens — already about a "short novel" and near the practical limit of the older 128k tier. Ten such streams blow past any single window.
- Long-context surcharges are real: Google's Gemini API pricing doubles the input rate above its 200K-token threshold (e.g., Gemini 2.5 Pro moving from $1.25 to $2.50 per million input tokens), and GPT-5.5 has been reported to charge ~2× input pricing for prompts over 272K tokens[11] — a direct argument for retrieval over "stuff it all in." (Notably, Claude Opus/Sonnet 4.6 were reported to charge standard rates across the full 1M window with no long-context surcharge.)[11]
- Prompt caching is the key cost lever: the stable prefix — system prompt, the library brief, tool definitions — can be cached. Per Anthropic's docs, cache reads are billed at 0.1× the base input price (a 90% discount), with cache writes at 1.25× base for the 5-minute TTL; OpenAI's automatic caching bills cached prefixes (over a 1,024-token[12] stable-prefix threshold) at 50% of the normal input rate. For Aldershot this means the "who you are / what your library is" preamble is written once and re-read cheaply on every request.
- Once vs. per request: transcription, diarization, moment extraction, embeddings, and the Read are once per video. Retrieval, summary-reduction, and the actual cut are per request, kept small by design.
The layer sketch (words → library): `` LIBRARY one "library brief" — themes, people, threads across everything └─ THREADS moment-clusters grouped by meaning (a topic, a person, a joke) └─ VIDEOS one Read + summary per source └─ CHAPTERS topic-bounded sections, each summarized └─ MOMENTS clip = frame + quote + source + timecode ← the atom └─ WORDS transcript with per-word timings `` Reads flow up (words compose into moments, moments into chapters, etc.); requests flow down (start at the smallest summary that can answer, drill only as needed).
3. "Potential video" discovery
Signals that should make a combination worth proposing. Rank them by how legible the resulting premise is to the user:
- Recurring topic across N videos (strongest, most explainable): "You've made the same argument about X in 6 clips."
- Enough strong moments to fill a runtime: there are ≥ k kept/high-quality moments on a theme totaling ≥ target minutes.
- A natural arc across videos (setup → turn → payoff): moments that, ordered, form a beginning/middle/end.
- Contrast or disagreement: two clips that take opposite positions — inherently a good video spine.
- Then-vs-now evolution: same topic, stance shifts over time (uses source dates).
- Anniversaries / "this time last year": date-based, like Photos/Facebook "On This Day" — but see the guardrail below.
- Series continuity: next installment in an established pattern.
- Timeliness: a topic in the library matches something the user is uploading a lot of right now.
- A short clip performing well: (owner-provided signal) suggests a supercut or follow-up on that theme.
What a potential-video card contains — no match score. Borrow the structure of YouTube Studio's "idea card" (name, one-line summary, supporting detail) but strip the analytics/prediction framing Chris rejected — YouTube's card explicitly leans on "data-driven insights into how well an idea might perform… Audience interest indicators… and prediction signals,"[13] which is exactly the "match score" register to avoid. Each Aldershot card shows:
- The premise in one sentence ("Your case against open-plan offices, built from 5 rants").
- The moments behind it, as real frames (the green-accented clips), so the evidence is visible and clickable — mirroring how Aldershot's Ask field "returns the clips behind its answer" and how Perplexity/Grok keep sources in their own visible cards.
- The runtime it could fill ("~4:30 of strong material").
- What's missing ("no intro; two clips have low audio").
- One button to start it → scaffolds a project.
Presentation forms, ranked by fit with the calm aesthetic:
- The ideas shelf / "possible videos" drawer (best fit). A quiet, browsable place that's pulled, not pushed. Like Readwise's Daily Review, it's there when you go to it; it never chases you. Amber Case's calm-technology principles — "technology should require the smallest possible amount of attention" and should "move easily from the periphery of our attention, to the center, and back" — argue for exactly this.
- A Home card in the existing "Since you were away" queue. Since each item there is already "a decision with a real action," a possible-video is a natural fit — one card, one glance, dismissable.
- A weekly digest. Low-frequency, batched, respectful — the anti-notification. Readwise's weekly "Themed Connections" (a Saturday batch "exploring a specific theme or idea") is the model.
- In-context, while you work ("this moment pairs with 3 others you've kept"). Highest value because it's contextual — Horvitz's utility-directed alerting model balances "the value of information awareness and cost of interruption,"[14] and relevance to the current activity is exactly what raises value — but it must be a subtle inline affordance, never a modal.
- Notifications (use sparingly, if at all). See guardrails; default off or digest-only.
Cadence, thresholds, and learning from dismissals.
- Confidence threshold before surfacing: require a minimum (e.g., a real recurring cluster of several strong moments and enough runtime) so the shelf is never padded with weak ideas. Better to show 2 great possibilities than 20 mediocre ones — sameness and low quality are the fastest ways to train users to ignore the shelf.
- Cadence: think Spotify Daylist / Readwise rhythm — refreshed on a predictable schedule (e.g., after each new ingest completes, plus a weekly sweep), not continuously pinging. (Readwise's own resurfacing is a "decaying algorithm based on a recall probability half-life," and "each time a highlight is shown, its probability of resurfacing is significantly decreased" — a good template for not repeating the same idea.)
- Learning from dismissals: copy Google Photos' explicit, gentle controls — the "show less" option that "means that a particular person won't show up nearly as often as they otherwise would have… it isn't a blanket block"[15] — plus implicit learning (a dismissed premise, or a whole thread the user never opens, gets downweighted). Photos moved from all-or-nothing blocking to a "show less" middle ground precisely because human preferences are fickle; Aldershot should start there.
- Frequency tuning: Readwise's "Frequency Tuning" lets users up/down-weight sources so a highlight-heavy book doesn't dominate; Aldershot should let a user turn a thread's suggestion frequency up or down per thread.
4. Missing-piece detection and capture prompts
Because the system already knows what a strong video of a given type needs (the Rules and the Read encode this), it can name the gap concretely:
- "These 5 clips make the argument, but there's no intro — record a 20-second setup in your voice to tie them together." → becomes a small recording task (gold accent = your voice).
- "You have the setup and the payoff but no example of X — one more would complete it." → either a search prompt ("find it in Footage") or a go-shoot-this prompt.
- "Two clips are from a noisy source; re-record the line or accept the seam." → a fix task.
Turn each gap into a seam the user fills with a "+": their voice, a reaction, or a clip from Footage — with best matches drawn from what was left out. This is where the "what's missing" line on a possible-video card becomes an action, not a critique.
Combining clips from different sources so it feels like one piece.
- Bridging with voice seams: a short gold-accent voice line over the cut point is the single most powerful unifier — it re-establishes a single narrator and covers the jump.
- Matching audio levels: loudness-normalize every source to a common target so cuts don't jump in volume (an automatic, once-per-source measurement stored at ingest).
- Aspect ratios: reframe to a common canvas with subject tracking (the Clipper's 9:16 reframing — the same "keep moving subjects centered with object tracking"[16] capability seen in tools like OpusClip's ClipAnything — generalizes here); flag clips that can't be reframed cleanly as a "missing piece."
- Different speakers: use diarization + visual descriptions to keep attribution clear (lower-thirds, or a voice seam that names who's next), so a multi-source cut reads as a deliberate conversation rather than a jumble.
- The "one piece" test: consistent loudness + consistent framing + a single narrating voice at the seams is what turns a supercut from a playlist into a film.
5. Guided, animated, game-like process — ranked ideas
The governing rule, from the research. Flow — the state this editor wants its users in — requires, per conditions drawn from Csikszentmihalyi (and summarized on PositivePsychology.com), that "perceived challenges and action opportunities should stretch skills and abilities, though not excessively,"[17] that "goals should be relatively short term (proximal), specific, and detailed,"[17] and that "immediate and actionable feedback must be available to promote continuing progress."[17] The challenge–skill "flow channel" sits on the diagonal where the two are balanced: too little challenge breeds boredom, too much breeds anxiety.
Gamification research is blunt that bolting on extrinsic rewards (points, badges, streaks) can trigger the overjustification effect — the finding, tracing to Deci's 1971 experiment and confirmed in Deci, Koestner & Ryan's 1999 meta-analysis, that[18] introducing external rewards for an already intrinsically motivated behavior reduces that intrinsic motivation. Duolingo is the cautionary tale: it appears on Harry Brignull's deceptive.design as a documented dark-pattern case (notably its pay-to-repair streak mechanic and "overly pushy reminders"), and its own critics describe "streak anxiety" and users "chasing XP instead of challenges." Editing is a craft users are already intrinsically motivated to do well; the game layer's job is to make the work legible and satisfying, not to bribe. So: progress shown, not scored.
Ranked mechanics (best fit → flag as gimmick):
Tier 1 — Strong fit (ship these):
- **Quests/missions per video as structure, not scoring.** The React flow becomes "5 blocks to record, one at a time" — a clear, finite goal with immediate feedback as each block lands. This is pure flow design (clear proximal goals + immediate feedback), not gamification. No XP, no badge — just "3 of 5 recorded" and a satisfying completion.
- Footage as an inventory. The library is an inventory of moments; presenting it as a well-organized "what you have to work with" (green clips, your voice in gold, other sources in cyan) borrows the legibility of a game inventory without any points. Fits the existing accent-colour system exactly.
- Fog-of-war reveal of a long source as it's Read. An 8-hour stream reveals its chapters as they're processed — the map fills in. This turns waiting into watching (see §6) and is honest: it shows real progress through real material. Highest "juice-for-taste" ratio.
- Overnight "worked while you slept" loop. The existing overnight queue, surfaced in the morning as "Since you were away," is already this. Framing it as quiet, completed work (not "your crew leveled up!") fits the calm aesthetic and the idle-game loop's genuine appeal (progress accrues while absent) without the manipulation.
- Completion states and satisfying finishes. A cut that "lands," a project bar that fills, a clean end-of-render moment — Things 3 is the precedent. MacStories' review praised how its common actions "give a sense of depth, of meaningful tactile engagement not found in… most other modern apps,"[19] and how "pulling down… forms a perfect marriage with haptic feedback," making "every interaction with the app feel that much more real."[19] That is premium, not childish — the tactile bar to clear.
Tier 2 — Good fit with care:
- Clips as cards (deck-builder influence, handled tastefully). Balatro/Slay the Spire make combination and synergy the pleasure — players "must build their deck 'on the fly,' trying to develop potential combinations and synergies between cards." For Aldershot, moment-cards that visibly combine into a video echo this satisfying "these fit together" feeling. Risk: cards can read as gimmicky/cute if over-animated. Keep them as clean frames, let the combination be the delight.
- Crafting / recipes. "These 5 moments + a voice intro = a supercut" as a visible recipe is a legible way to show what a possible-video needs. Good if it stays a plain diagram; bad if it turns into RPG crafting flourish.
- Unlocks as the library grows ("with 10 more clips on this topic, a supercut becomes possible"). Genuinely useful as a capability signal — it tells the user what's within reach. Frame as "what's possible now / soon," never as a locked treasure chest with a progress bar to grind.
- A map of the library that fills in over time. A spatial view of threads/topics that grows is a beautiful ambient progress display (calm-tech "peripheral awareness"). Risk: can become a vanity "knowledge graph" — the exact framing Chris rejected. Only ship if it's genuinely navigational, not decorative.
Tier 3 — Flag as gimmick / avoid:
- Streaks, XP, points, leaderboards, badges. Directly implicated in the overjustification effect and Duolingo's dark-pattern critiques. Wrong audience, wrong craft, wrong aesthetic. Duolingo's own designer of that saturated, energetic palette notes "muted, sophisticated colors would undermine the playful, accessible tone"[20] — which is precisely the tone Aldershot is not going for.
- A named "crew" of little worker characters. "Your crew worked while you slept" as literal characters veers into Duolingo-owl territory (the mascot that turns a notification into "anthropomorphic stakes").[20] Keep the loop, drop the mascots.
- Sound/haptics everywhere. Great on a few completion moments (Things 3 model); a gimmick if every action buzzes. Reserve for meaningful finishes.
A note on tasteful restraint as a positive model. Teenage Engineering is the reference point for "premium but calm": their approach makes "every control visible and tactile"[21] under strict formal constraints, and — tellingly for Aldershot — uses color as an assigned system of meaning rather than decoration ("orange means recording across every TE product… a color system with assigned meaning").[22] Aldershot's own accent language (gold = your voice, green = clips, cyan = another source, violet = generated, red = record) is exactly this kind of disciplined, meaning-bearing system, and the motion/game layer should stay equally restrained.
6. Animation and motion that communicates the AI's work
The principle: an AI wait is not a download wait. As Smashing Magazine puts it, when an agent pauses "it's not just downloading something; it's thinking… weighing options,"[23] and a plain spinner leaves users "confused and anxious." The calm-default pattern is progress steps that are collapsed by default and expand on demand — a trace kept "collapsed by default to keep the interface calm"[24] that "expands on demand so users who want provenance… can inspect what happened."[24] Aldershot's advantage is that its "steps" are visual footage, not log lines.
Concrete motion concepts:
- The Read forming: text assembles as chapters resolve — the fog-of-war reveal. The Read literally writes itself as the map fills, so the summary and the progress are the same animation.
- Rules taking effect: when a rule ("cut the hedging everywhere") toggles on, the affected blocks visibly leave the cut — a physical removal, so the rule's effect is seen, not described. (Editor X already promises "each showing its live effect"; motion is how that effect becomes legible.)
- Blocks moving in/out of the cut: squash-and-stretch-adjacent physicality — a block that's cut collapses and the timeline closes the gap; a restored block pushes its neighbours apart. Disney's first principle (weight communicated through deformation) is what makes this read as real rather than abstract.
- Threads connecting across videos: when a cross-video link forms, a quiet line draws between two real frames — a single, one-time reveal (not a persistent web), matching "this clip pairs with one from Stream 03."
- Progress on 8-hour sources: a source bar that fills chapter-by-chapter with real thumbnails appearing, so a long Read is a watchable timelapse of the machine reading. Waiting becomes watching.
- Alternate cuts / "one more thought": the A/B proposal cross-fades between the two versions on the same frames, so the difference is felt instantly.
- Provenance stays visible: like Grok, which "separates the sources it retrieves into their own card to keep them separate from its logic,"[25] Aldershot keeps the moments behind any suggestion one glance away — the anti-black-box move that builds trust.
7. Nitty-gritty walkthroughs
(a) First-timer, one video, knows what they want. Chris drops a 52-minute podcast and types "pull the 60-second bit where he loses it about landlords." Intent → extract. The working row shows one job; the source bar fills as the Read runs (fog-of-war, ~seconds to a minute for a file this size). The Read names the threads inside this video only. Editor X opens on the extracted candidate; Chris reads the transcript, drags the in/out by editing text, toggles a "cut the ums" rule and watches three ums vanish from the cut. Done. No library suggestions appear. The whole thing feels like a fast, calm tool.
(b) Second and third uploads — first connections. Chris uploads two more podcasts. During the third ingest, once embeddings land, the first cross-video link forms: in Footage, a single quiet line appears under a moment — "This and one clip from Ep. 2 are about the same thing." No notification, no popup. If he opens it, he sees the two real frames side by side with the shared topic named. This is first contact: a discovery, introduced once.
(c) Library of 50+, ideas shelf. Chris opens Footage; the possible-videos drawer holds three cards. Top one: "Your case against open-plan offices — 5 clips, ~4:30, no intro." Real frames, green accents. He's not pushed here; he came looking. He dismisses one ("show less on this topic") and the shelf learns.
(d) Accepting a possible video → scaffolded project. He clicks "Start" on the open-plan card. A new project opens with the composition bar pre-filled: 5 green clips in a suggested order, two seams marked "+", and a gold recording task: "20-sec intro to tie these together." One clip is flagged: "low audio — re-record or accept." The project is a scaffold with its gaps named, not a finished thing.
(e) React recording quest. Chris picks React on a new source. It becomes "5 blocks to record, one at a time." Block 1 shows the source moment to react to; the record button glows red; he records; the block fills and the counter reads "1 of 5." Immediate feedback, one clear goal at a time — textbook flow. At 5 of 5 there's a clean completion animation (Things-3-style pop + a single haptic on mobile). No XP.
(f) Drop an 8-hour stream, come back next morning. Evening: Chris drops a raw 8-hour stream into the overnight queue. Overnight, map-reduce chews the ~110k-token transcript into chapter summaries; moments and embeddings land; the Read is written. Morning: "Since you were away" shows completed work — the Read, the chapters as a filled-in map, and (if thresholds are met) one possible-video card drawn from the new material. It reads as worked while you slept, quietly, no mascots.
(g) Clipper batch-processing a week of streams. Chris selects a week of streams for the Clipper. Each is diarized/chaptered once; the Clipper scores 9:16 shorts across all of them, each short showing "the reasons it scored" (plain reasons, not a percentage). Because ingest already ran, batch clipping is cheap retrieval + reframe, not a re-scan. Output: a tray of candidate shorts with real frames and reasons, ready to accept or refine.
8. Risks and guardrails
- Notification fatigue. Default to pull (shelf, digest), not push. If notifications exist at all, gate them behind Horvitz-style relevance/timing logic (balancing information value against interruption cost) and make digest-only the default. The whole ethos is calm-tech: minimal attention, periphery-first.
- Creepiness & privacy with personal footage. The Facebook/Google "On This Day" disasters — Eric Meyer's blog account of being shown his deceased daughter's face "surrounded by illustrations of dancing people and balloons"[26] is the canonical warning — show what goes wrong when a system surfaces "the ideal user, the happy, upbeat, good-life user"[26] and ignores the rest. Aldershot's personal vlog archive carries the same risk. Guardrails: honour "show less/hide" per person and per thread; never auto-surface anniversary/"this time last year" content about a person the user has down-weighted; make date-based memories opt-in, not default. Nothing auto-deleted, nothing auto-shared.
- Bad or repetitive suggestions. Enforce the confidence threshold; cap the shelf's size; suppress near-duplicate premises. Two strong ideas beat twenty weak ones.
- Suggestions converging into sameness. Actively diversify the shelf across signal types (don't show five recurring-topic cards); down-weight a premise shape the user keeps dismissing. (The Spotify Daylist critique — that such algorithms "don't just react to your music taste, they shape it,"[27] constructing tastes "for us over time"[27] — is the caution against a self-narrowing loop.)
- Compute cost of re-scanning a growing library. Don't re-scan. Ingest is once-per-video and immutable; cross-video linking is incremental (new video links against existing embeddings, not a full recompute). The library brief is cached (per provider docs, ~90% off cached input on Anthropic, ~50% on OpenAI). Re-embed only when models change, as a deliberate batch, not on every request.
- Trust and control. Let users pause suggestions entirely, mute a thread, tune frequency per thread (Readwise model), and always see the moments behind any proposal (the anti-black-box provenance card). Undo everywhere, per the existing principles.
- Keeping the one-file user clean. The hard rule: cross-video machinery stays invisible until there's genuine cross-video meaning and the user has more than one relevant source. The fast path must never regress into a busier surface as the account ages.
3Recommendations
Stage 1 — Ship the invisible foundation (do first, regardless of the game layer). Build the ingest pyramid (transcript+timings → moments → chapters → video summary → embeddings → Read) and lock in the "once per video" boundary. Add prompt-caching of a stable library brief. Benchmark that would change the plan: if per-video ingest cost or latency is too high on Cloudflare economics, cut visual-description sampling rate (e.g., from 1 fps down) before cutting transcript/diarization, which are load-bearing.
Stage 2 — Turn on cross-video linking silently. Let links form at the 2–3 video mark and appear only as quiet inline lines in Footage. No shelf yet. Threshold to advance: only build the ideas shelf once real accounts routinely produce several strong recurring-topic clusters — i.e., when there's genuinely something worth a drawer.
**Stage 3 — Ship the possible-videos drawer as a pull surface, with cards that show premise + real frames + runtime + what's missing + one button, and per-thread frequency tuning + "show less." Do not ship push notifications in this stage. Benchmark:** track shelf open-rate and accept-rate; if accept-rate is low, raise the confidence threshold and shrink the shelf rather than adding more cards.
Stage 4 — Add the flow-shaped guided layer: React-as-quest ("5 blocks, one at a time"), fog-of-war Reads, and Things-3-grade completion animations. Explicitly forbid streaks, XP, points, leaderboards, badges, and worker-mascots in the design spec so they can't creep in later.
Stage 5 — Only then consider a library map, and only if it's navigational rather than a decorative "knowledge graph."
Cross-cutting benchmarks that should change course:
- If users mute/pause suggestions at high rates → the shelf is too loud or too weak; move to digest-only and raise thresholds.
- If a personal-archive user ever reports a painful resurfacing → immediately make all date/person-based memories opt-in and audit the down-weighting logic.
- If ingest cost per video rises materially → attack sampling rate and re-embed cadence, never the once-vs-per-request boundary.
4Open questions for Chris
- Where does the ideas shelf live? Inside Footage only, or also a Home entry point? The safer default is Footage-only until it's earning its place.
- Do notifications ever exist? The research pushes hard toward pull-only/digest. Is there any case (a truly time-sensitive combination) worth a single, mutable notification — and can that be opt-in?
- How aggressive should "then vs. now" be with the personal archive? This is the highest-creepiness signal; it may deserve to be off by default even when the topic signals are strong.
- What's the minimum viable "strong moment"? The whole discovery system hinges on a quality bar for moments; that bar (and who sets it — the user's kept clips vs. the system's scoring) determines everything downstream.
- Should combining across sources ever auto-normalize audio/aspect, or always ask? Auto-fixing is smoother but risks the "automated process" feel Chris is wary of; a named "I matched levels across 4 sources" line may thread the needle.
5Caveats
- Model/price specifics are a fast-moving 2026 snapshot. Exact window sizes, per-token prices, surcharge thresholds, and caching discounts shift constantly; several figures here come from comparison blogs rather than primary provider docs and should be re-verified against provider pricing pages before any cost model is finalized. The architecture (once-vs-per-request, retrieval + hierarchy + caching) is stable; the numbers are not.
- Token estimates for an 8-hour stream are back-of-envelope (word rate × 8h × OpenAI's ~100 tokens ≈ 75 words guidance); real transcripts vary widely with speaking pace, overlap, and multi-speaker content.
- The proactive-discovery precedents (Photos, Daylist, Readwise) are consumer-scale and engagement-optimized; Aldershot's calm, pull-first stance deliberately diverges from their push mechanics, so their engagement results won't transfer — only their curation and control patterns.
- Flow and gamification research is about motivation in general; its application to a professional editing craft is a reasoned inference, not a measured result for this specific product. Treat the "no streaks/points" stance as a strong design bet grounded in the overjustification literature, not a proven outcome for creators.
- Out of scope as instructed: nothing here touches research, fact-checking, or verification features.
- Source quality varies. The Horvitz attention work, Amber Case's calm-technology principles, the Csikszentmihalyi flow conditions, the "Lost in the Middle" / map-reduce / RAG papers, and the Duolingo dark-pattern critiques (deceptive.design) are the most solidly grounded and should anchor any external write-up; design-blog and comparison-site citations are secondary corroboration.
6Sources
- The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation
- Google Photos memories and why they keep reappearing - TechCybo
- Google Photos just got much better—here's how (exclusive)
- Joint Training of Speaker Embedding Extractor, Speech and Overlap Detection for Diarization Computing on IT4I supercomputer was supported by the Czech Ministry of Education, Youth and Sports through the e-INFRA CZ (ID:90254).
- MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering
- Lost in the Middle: How Language Models Use Long Contexts - ACL Anthology
- Grounding Long Documents: Summarization and Hierarchical RAG for LLMs
- Master LLM Summarization Strategies and their Implementations | Galileo.ai
- RAG Summarization 2026: Patterns, Code + Long-Context Tradeoffs
- AI Model Context Window Comparison 2026: Advertised vs. Real - elvex
- Context Window Sizes: Every Major LLM Compared for 2026 - Adaptive Recall
- Prompt Caching in 2026: Anthropic, OpenAI, Azure ...
- Explore Inspiration tab on YouTube - YouTube Help
- Models of Attention in Computing and Communication: From Principles to Applications - Microsoft Research
- Google Photos is giving you better control over who shows up in your Memories — here’s how it works
- OpusClip: #1 AI video clipping tool to create viral shorts
- Mihály Csíkszentmihályi: The Father of Flow
- Self-Determination Theory: Deci & Ryan's 6 Mini-Theories
- Things 3: Beauty and Delight in a Task Manager
- Duolingo: Gamification as Design Language
- Teenage Engineering Marketing Strategy
- Teenage Engineering Rabbit R1: Worth Buying in 2026?
- Practical Interface Patterns For AI Transparency (Part 2) — Smashing Magazine
- Progress Steps · Trust AI UX Pattern | AI UX Playground
- AI UX Patterns | Stream of Thought | ShapeofAI.com
- www.malaymail.com
- Spotify daylist: algorithms don’t just react to your music taste, they shape it