agentic-video Gallery

Quote what a lecturer says at one moment

What was saidLectureTool missed
0:27CC-BY-3.0 · Berkman Center (presenter: William Fisher)

Proof frames

Click a frame to seek

Tool calls

3 calls · agentic-video 0.3.1
transcribeimac · 56 ms

Arguments

{
  "source": "$CLIP"
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "requested_model_size": "tiny",
    "requested_model_cached": false,
    "transcription_status": "started",
    "retry_after_seconds": 15,
    "transcription_message": "The tiny transcription started in the background. About 0% done (estimate): 0 s elapsed of roughly 10 s. Retry in about 15 seconds.",
    "elapsed_sec": 0,
    "estimated_total_sec": 10,
    "estimated_percent": 0
  }
}
transcribeimac · 2 ms

Arguments

{
  "source": "$CLIP"
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "requested_model_size": "tiny",
    "requested_model_cached": false,
    "transcription_status": "in_progress",
    "retry_after_seconds": 15,
    "transcription_message": "A transcription for this video is already running. About 30% done (estimate): 3 s elapsed of roughly 10 s. Retry in about 15 seconds.",
    "elapsed_sec": 3,
    "estimated_total_sec": 10,
    "estimated_percent": 30
  }
}
transcribeimac · 2 ms

Arguments

{
  "source": "$CLIP"
}

Result

{
  "ok": true,
  "data": {
    "source_path": "~/hill/gallery/repo/cards/quote-lecture/clip.mp4",
    "language": "en",
    "duration_sec": 27.027,
    "segment_count": 9,
    "segments": [
      {
        "id": 1,
        "start_sec": 0.02,
        "end_sec": 3.02,
        "text": "to ensure that you have the most current version.",
        "start_timecode": "00:00:00.020",
        "end_timecode": "00:00:03.020",
        "confidence": 0.979
      },
      {
        "id": 2,
        "start_sec": 3.02,
        "end_sec": 7.02,
        "text": "So, now let's turn to the topic for today,",
        "start_timecode": "00:00:03.020",
        "end_timecode": "00:00:07.020",
        "confidence": 0.979
      },
      {
        "id": 3,
        "start_sec": 7.02,
        "end_sec": 10.02,
        "text": "which is indicated is copyright theory.",
        "start_timecode": "00:00:07.020",
        "end_timecode": "00:00:10.020",
        "confidence": 0.979
      },
      {
        "id": 4,
        "start_sec": 10.02,
        "end_sec": 13.02,
        "text": "As I've just mentioned by theory, I mean nothing more than",
        "start_timecode": "00:00:10.020",
        "end_timecode": "00:00:13.020",
        "confidence": 0.979
      },
      {
        "id": 5,
        "start_sec": 13.02,
        "end_sec": 17.02,
        "text": "arguments concerning when and why copyrights should be created",
        "start_timecode": "00:00:13.020",
        "end_timecode": "00:00:17.020",
        "confidence": 0.979
      },
      {
        "id": 6,
        "start_sec": 17.02,
        "end_sec": 21.02,
        "text": "and what the scope or limits of those rights should be.",
        "start_timecode": "00:00:17.020",
        "end_timecode": "00:00:21.020",
        "confidence": 0.979
      },
      {
        "id": 7,
        "start_sec": 21.02,
        "end_sec": 24.02,
        "text": "Before zeroing in on the specific theories",
        "start_timecode": "00:00:21.020",
        "end_timecode": "00:00:24.020",
        "confidence": 0.979
      },
      {
        "id": 8,
        "start_sec": 24.02,
        "end_sec": 26.02,
        "text": "that will occupy us during this lecture,",
        "start_timecode": "00:00:24.020",
        "end_timecode": "00:00:26.020",
        "confidence": 0.979
      },
      {
        "id": 9,
        "start_sec": 26.02,
        "end_sec": 28.02,
        "text": "I should...",
        "start_timecode": "00:00:26.020",
        "end_timecode": "00:00:28.020",
        "confidence": 0.62
      }
    ],
    "has_speech": true,
    "notice": "Note: Non-speech audio (music, sound effects, alarms) is NOT transcribed.",
    "confidence": 0.939,
    "uncertainty": "Whisper confidence is a no_speech_prob proxy; transcript content and alignment remain model-dependent."
  }
}

Question and answer

Question
What exactly is said starting about 4 seconds into this clip?
Answer
The transcript has the speaker saying "So, now let's turn to the topic for today," from 3.0 seconds.
Tool returned
So, now let's turn to the topic for today,
Truth
So now let's turn to the topic for today, which as I've indicated,
Check failedmissing phrases: which

How it was checked

Truth from
captions
Fixed at

Taken from the human captions published with the source: the caption line at 66.97 to 70.98 s in the source, which is 4.00 to 8.01 s in the clip. The clip is the source from 62.97 to 90.01 s, cut with no edit. The line was picked by a seeded rule (seed from the card id) from 16 eligible caption lines of 8 to 25 words, 2 to 10 s long, starting at least 4 s in, ending 0.5 s before the window does, with at least two words of 5 or more letters, and with no sound tag, music mark, dialogue dash or speaker label; the required phrases are the first 3 of those words in the line. No picture or sound was changed, and no tool was asked for the answer.

The check is the published caption file, and the 27-second clip was cut from the source with no edit, so a word that transcribe hears differently counts as a miss.

Try it yourself

Use agentic-video to transcribe my video, then quote the exact words said starting about 4 seconds in. Use my own file.
  1. Get the .mcpb file and open it in Claude Desktop.
  2. Or: pip install agentic-video

Clip and licence

Licence
CC-BY-3.0
Author
Berkman Center (presenter: William Fisher)
Source
commons.wikimedia.org
This file is licensed under the Creative Commons Attribution 3.0 Unported license.