Find when a word shows on screen in lecture footage

Read the screenLectureVerified

0:30CC-BY-3.0 · Susan Dean, Barbara Illowsky

Proof frames

Click a frame to seek

Tool calls

14 calls · agentic-video 0.3.1

count_occurrencesimac · 1636 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 15 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 28 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 42 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 2 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 56 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 73 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 86 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 100 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 111 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 124 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 138 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 150 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "OFFLINE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "complete",
    "count": 1,
    "spans": [
      {
        "index": 1,
        "start_sec": 10.2,
        "end_sec": 13.2,
        "duration_sec": 3,
        "score": 1,
        "representative_sec": 11.8,
        "frame_badge": "#1",
        "weak": false,
        "region": {
          "x": 0.256,
          "y": 0.464,
          "w": 0.488,
          "h": 0.139
        },
        "travel": "stationary"
      }
    ],
    "spans_returned": 1,
    "spans_omitted": 0,
    "window": {
      "start_sec": 0,
      "end_sec": 30
    },
    "fps_used": 5,
    "frames_sampled": 150,
    "frames_analyzed": 150,
    "checks": {
      "text": "OFFLINE",
      "description": null,
      "color": null,
      "shape": null
    },
    "contact_sheet": {
      "file_path": "<cache>/40f85fe2af417a8976ba/contact_sheets/occurrences_aefbde0df8065b929372.jpg",
      "sample_count": 1,
      "rows": 1,
      "cols": 1,
      "frame_width": 288,
      "frame_height": 192,
      "labels": [
        "#1 11.8s"
      ],
      "tool_file": {
        "_tag": "ToolFile",
        "name": "occurrences_aefbde0df8065b929372.jpg",
        "mimeType": "image/jpeg",
        "encoding": "base64",
        "data": "<22912 base64 chars dropped>",
        "byteLength": 17184
      }
    },
    "note": "Spans are sample-based (+/- 1/fps); verify each representative_sec with inspect_window. contact_sheet is one image with a tile per listed span, in span order: a zoomed frame of the span's best match, boxed, labeled with the span number and time. Look at each tile and drop any span whose tile is not the asked-for thing; spans marked * (`weak`) are the likeliest false ones. The answer is the number of tiles that match, plus any omitted spans and respecting any merged gaps noted above.",
    "verify": "Rough count: spans can be false, merge across scene cuts, split one appearance, or miss crossings. Before answering, look at each tile of the contact_sheet image (one per span, in span order, a zoomed frame of its best match, boxed) and drop any span whose tile is not the asked-for thing; spans with weak=true (tile label *) are the likeliest false ones. The answer is the number of tiles that match, plus any spans the sheet does not show; if you also count by eye and differ, trust your own."
  }
}

frame_atimac · 583 ms

Arguments

{
  "source": "$CLIP",
  "time_sec": 11.6
}

Result

{
  "ok": true,
  "data": {
    "frame_time_sec": 11.6,
    "width": 320,
    "height": 240,
    "source_width": 320,
    "source_height": 240,
    "crop": null,
    "scale": 1,
    "file_path": "<cache>/40f85fe2af417a8976ba/frame_at/frame_at_120e292c31b96513.png",
    "tool_file": {
      "_tag": "ToolFile",
      "name": "frame_at_120e292c31b96513.png",
      "mimeType": "image/png",
      "encoding": "base64",
      "data": "<73792 base64 chars dropped>",
      "byteLength": 55342
    }
  }
}

Question and answer

Question: When does the word OFFLINE show up on the screen in this video?
Answer: The tool finds the word OFFLINE on screen from 10.2 to 13.2 seconds.

Check passedIoU 0.94, minimum 0.5

How it was checked

Truth from: constructed
Fixed at: 2026-10-30 20:10

Planted by this recipe before any tool ran: a Pillow-drawn card reading 'OFFLINE' (white letters on a dark box) was overlaid with ffmpeg on the first 30 s of 'Elementary Statistics video lecture: Probability Topics' (licence CC-BY-3.0) from 10.1 s to 13.1 s, the picture otherwise unchanged. The truth is that interval, 10.1 to 13.1 s, checked by an IoU of at least 0.5; frame_at asks for the middle of it, 11.6 s.

A script drew the word OFFLINE on a card over this lecture footage for 3 seconds, at a time it chose; that planted 3-second span is the truth. The answer is the longest span the tool listed, and frame_at was asked for the middle of the planted span, not of the tool's span. Other spans, if any, are not shown on the card.

Try it yourself

Use agentic-video on my video. Find when the exact word OFFLINE is on the screen: run count_occurrences on the whole video with text OFFLINE, then run frame_at at the middle of the longest span it lists. Tell me the start and end seconds of that span. Use my own file.

Get the .mcpb file and open it in Claude Desktop.
Or: pip install agentic-video

Clip and licence

Licence: CC-BY-3.0
Author: Susan Dean, Barbara Illowsky
Source: commons.wikimedia.org

This file is licensed under the Creative Commons Attribution 3.0 Unported license.