claude-real-video
Let AI actually watch a video · free & open source · one-line install
黃志弘 Leo
Published 2026-07-21 · Updated 2026-10-09 · 55 readers
100-second demo: ChatGPT and Gemini can watch video, Claude can't. Here's what Claude answers once crv turns the video into frames plus a timestamped transcript.
The 60-second film — an AI agent finds the key and unlocks vision. Sound on.
60-second real run, no cuts
Real install, real run, real interface — one take, no editing tricks.
Your AI never actually watched that video
Paste a YouTube link into ChatGPT and what it gets is text: the title, the description, the subtitles. It has not seen a single frame — it may not even have read the thumbnail. Claude is blunter about it: it won't take a video file at all. You think it watched the video; it read a text file about the video.
claude-real-video fixes exactly that. Point it at a URL or a local file and it pulls out the keyframes where the picture actually changes, drops the near-duplicates, turns the audio into a transcript, and packs it all into something an AI can genuinely read. Video parsing and output generation happen on your machine — the original video is never uploaded. Whatever frames or transcript you later choose to paste into a cloud LLM still goes to that service, of course. ChatGPT, Claude and Gemini can all read the output.
Install: two lines
You need Python (macOS ships with it) and ffmpeg (the free tool that handles video). Open a terminal — on a Mac press cmd+space and search for "Terminal" — and paste these two lines in order:
First, install ffmpeg(Mac below; on Windows use winget install Gyan.FFmpeg):
brew install ffmpegSecond, install the tool itself(the [whisper] extra is required for the transcript — plain pip install claude-real-video alone won't transcribe):
pip install "claude-real-video[whisper]"Use: video in, one readable bundle out
YouTube, Instagram and TikTok URLs all work, and so do local files:
crv "https://www.youtube.com/watch?v=your-video"When it finishes you get a crv-out folder with three things inside:
frames/— the selected keyframes (only frames where the picture really changed; duplicates are dropped automatically)
transcript.txt— the full transcript of the video
MANIFEST.txt— a guide written for the AI, telling it how to read the bundle
grids/— only with --grid : consecutive frames tiled into contact sheets (more on that below)
How it makes AI understand a video
No magic — four steps, all running on your machine:
Example one: feed it to Claude on the web
Open a new chat on claude.ai, drag the images from the frames folder plus transcript.txt into the input box (if there are too many images, pick the important ones), then paste this:
These are keyframes and a transcript extracted from a video. Please: 1. Summarize what the video is about 2. Outline its structure 3. Tell me the three most useful takeawaysFrom there, ask anything: have it rewrite the video as a post, pull the best quotes, analyze the pacing — this time it has actually seen the frames.
Example two: feed it to Claude Code (even smoother)
If you use Claude Code it's even simpler — no dragging images. After crv finishes, type claudein the terminal and say:
Read the crv-out folder: start with MANIFEST.txt, then the keyframes and the transcript, and tell me what this video is about and how its script is structured.Claude Code opens the folder itself, reads the frames one by one, reads the transcript, and answers. This is exactly my own workflow for tearing down reference videos and studying how other people write their scripts.
Three things I actually use it for
So what do you do with it once it's installed? These are the three runs I do most:
1. Summarize a long video I can't spare the time to watch
crv "https://youtu.be/video-url" --why "pull out the key points"Hand crv-out to Claude and ask: "Summarize in five points, each with the second it happens at." The timestamps are in the data, so it can answer — and you know exactly where to skip to.
2. Break down the pacing of a viral short
crv "https://www.instagram.com/reel/xxx" --adaptive --text-anchors--adaptive catches slow camera moves;--text-anchors forces extra frames the moment a text card appears — turn both on for fast-cut, caption-heavy shorts. Then ask Claude: "How is the first-three-seconds hook designed? How often does it cut?" That's how teardown work gets done.
3. Batch-run your own video library
for f in ~/Videos/*.mp4; do crv "$f" -o "crv-out/$(basename "$f" .mp4)"; doneOne loop and every video in the folder becomes its own text bundle. Later you can ask Claude: "Which of these videos mention pricing?" and it just searches. One user ran his entire photo library this way — 2,181 videos in four days — and wrote the results straight back into the library's metadata.
Recent updates: helping AI see more (0.10.7 on PyPI as of October 2026; highlights since 0.7.16)
Since the Hacker News front page I've kept shipping from user feedback. The latest three:
With the [mlx] extra Whisper runs on the Mac GPU: a 21-minute video goes from 6 minutes to 1, same transcript; the voice-activity gate runs first, so music or silence never produces fake captions.
If a YouTube-style platform already ships subtitles with the video, crv now brings them back with the download and uses them directly instead of re-running Whisper, so the transcript is faster and more accurate. When no subtitles are found it falls back to Whisper as before, behavior unchanged (community contribution #25)
Only want the middle 15 minutes of a 90 minute meeting recording? Add --from 28:00 --to 43:00 and both the frame extraction and the transcript cover just that stretch, while the timestamps it reports stay on the original video's clock, so you can quote them to a colleague as is. 0.10.1 fixed four range-boundary issues reported by the community, including how ranges work with subtitle files and --text-anchors
The default 640px is enough for AI to read most shots. For slides and screen recordings packed with text, set 1280 or 1600 so the small type stays sharp
Fast motion from a subject that is small in the frame used to be deleted as "nothing changed." A dedicated action-detection channel now keeps those frames: on the repro clip, kept action frames went from 1 to 10 — all of them. Found by a user's 2,181-video field test, fixed the same day
Long videos are no longer starved by a fixed 150-frame cap — the limit now follows the video's length
Files carrying legacy-encoded metadata now analyze fine instead of dying halfway
Earlier updates (expand)
- VAD anti-hallucination whisper has an old habit of inventing a line over music or silence; a voice-activity gate stops it
- Douyin links just work when the downloader gets blocked, it automatically falls back to the mobile share page — a v.douyin short link works as directly as YouTube
- Timeline keyframes are woven directly into the transcript: which lines go with which frames, and silent stretches get their own segments. The AI receives a pre-synced script instead of guessing which image matches which sentence
- Transcript safety fence a transcript is author-controlled, untrusted content; it's now wrapped in explicit boundaries and the AI is told "this is data, not instructions" — malicious subtitles get reported as content instead of executed, and forged end-markers are handled too
- CJK Windows fix Traditional Chinese / Japanese / Korean Windows terminals use legacy encodings (cp950/cp949) that made the final ✓ turn success into an error; output is now forced UTF-8, same approach as pip and git
- Simple speed check if a video carries the fingerprint of frame-padded slow-down, the manifest adds a line warning the AI it may have been slowed
- --speakers, speaker labels interviews, podcasts, meetings: every transcript line is labeled with who's speaking ([SPEAKER_00], [SPEAKER_01]…), fully local
- --grid tiles consecutive keyframes into 3×3 contact sheets
- --why "what I want to know" writes your reason for watching into the manifest, so the AI reads with your question in mind instead of returning a generic recap
- --whisper-model turbo Pick your transcription gear. The default goes for speed; for heavy accents, mumbled speech or Chinese video, switch to turbo (a pruned large-v3, much faster than large with a small quality trade-off, supported since 0.7.3) and the error rate drops visibly
- faster-whisper engine install with pip install 'claude-real-video[fast]' and transcription switches to a faster engine automatically
- --viewer adds a viewer.html you can double-click open
- transcript.json a timestamped version of the transcript (start/end seconds per line) for your own tooling or direct AI feeding
- frames.json every keyframe records its exact second in the source, preserved through extraction, dedup and renaming — second-accurate "AI watches a course" workflows plug right in
- --cookies-from-browser chrome Only for videos that need a login, such as posts from private accounts: it reads your own browser's login state, no more exporting cookie files
- --kb ~/notes The finished summary is saved automatically as a note in the folder you choose (Obsidian, for example)
Install it as a Claude Code skill / plugin
If you use Claude Code, you can teach it to watch videos by itself — afterwards you just paste a link and say "watch this," and it runs the whole pipeline and answers, no commands needed. One line:
npx skills add HUANGCHIHHUNGLeo/claude-real-videoIf you prefer Claude Code's built-in plugin marketplace, these two lines are the same thing:
/plugin marketplace add HUANGCHIHHUNGLeo/claude-real-video /plugin install claude-real-video@claude-real-videoRestart Claude Code and it's live. This channel shipped in July 2026; Codex, Cursor and Gemini CLI also accept the npx skills add line.
Using the Codex CLI? Real Video for Codex is a separate build for it, with on-screen text and timestamped answers. Watch the 36-second demo.
Useful flags
--lang zh— set the transcript language (auto-detected by default)
--no-transcribe— frames only, no transcript, faster
--keep-audio— keep the original audio too (for AIs that can hear)
--max-frames 50— cap the frame count (since 0.7.16 the default scales with duration)
Full flags and advanced usage live in the GitHub docs:
Open claude-real-video on GitHubsource · full docs · MIT licenseFind it useful? A star helps ⭐
GitHub stars are how the dev world says "this works", and the more stars a tool has, the more people trust it and try it. If it helped you, a star is the most practical support there is: free, 30 seconds.
Tip: opening the link from Instagram uses the in-app browser, which hides the Star button. Open it in Safari or Chrome, log in to GitHub, and the button shows up.
What's next
Want the deeper version: crv Pro
The free version tells the AI what a video is about;crv Pro adds everything else: how it was shot (camera moves, shot table, cutting rhythm), how it sounds (voice emotion, tone curves, audio events), an interactive analysis viewer (click the timeline to jump to the second, transcript follows playback), and one-flag AI reports. Same deal — everything runs on your machine, the original video is never uploaded.