Find when a word shows on screen in film footage

Read the screenFilmVerified

0:30CC-BY-3.0 · Blender Foundation (Durian Open Movie project)

Proof frames

Click a frame to seek

Tool calls

7 calls · agentic-video 0.3.1

count_occurrencesimac · 1258 ms

Arguments

{
  "source": "$CLIP",
  "text": "STOP",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "STOP",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 40 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "STOP",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 68 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "STOP",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 100 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "STOP",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 134 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "STOP",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "complete",
    "count": 1,
    "spans": [
      {
        "index": 1,
        "start_sec": 6,
        "end_sec": 9,
        "duration_sec": 3,
        "score": 1,
        "representative_sec": 7.6,
        "frame_badge": "#1",
        "weak": false,
        "region": {
          "x": 0.421,
          "y": 0.474,
          "w": 0.158,
          "h": 0.097
        },
        "travel": "stationary"
      }
    ],
    "spans_returned": 1,
    "spans_omitted": 0,
    "window": {
      "start_sec": 0,
      "end_sec": 30
    },
    "fps_used": 5,
    "frames_sampled": 150,
    "frames_analyzed": 129,
    "checks": {
      "text": "STOP",
      "description": null,
      "color": null,
      "shape": null
    },
    "contact_sheet": {
      "file_path": "<cache>/9f6dc495db1af31e887b/contact_sheets/occurrences_2a7e9e2be1add759b565.jpg",
      "sample_count": 1,
      "rows": 1,
      "cols": 1,
      "frame_width": 288,
      "frame_height": 192,
      "labels": [
        "#1 7.6s"
      ],
      "tool_file": {
        "_tag": "ToolFile",
        "name": "occurrences_2a7e9e2be1add759b565.jpg",
        "mimeType": "image/jpeg",
        "encoding": "base64",
        "data": "<13132 base64 chars dropped>",
        "byteLength": 9848
      }
    },
    "note": "Spans are sample-based (+/- 1/fps); verify each representative_sec with inspect_window. contact_sheet is one image with a tile per listed span, in span order: a zoomed frame of the span's best match, boxed, labeled with the span number and time. Look at each tile and drop any span whose tile is not the asked-for thing; spans marked * (`weak`) are the likeliest false ones. The answer is the number of tiles that match, plus any omitted spans and respecting any merged gaps noted above.",
    "verify": "Rough count: spans can be false, merge across scene cuts, split one appearance, or miss crossings. Before answering, look at each tile of the contact_sheet image (one per span, in span order, a zoomed frame of its best match, boxed) and drop any span whose tile is not the asked-for thing; spans with weak=true (tile label *) are the likeliest false ones. The answer is the number of tiles that match, plus any spans the sheet does not show; if you also count by eye and differ, trust your own."
  }
}

frame_atimac · 306 ms

Arguments

{
  "source": "$CLIP",
  "time_sec": 7.4
}

Result

{
  "ok": true,
  "data": {
    "frame_time_sec": 7.375,
    "width": 640,
    "height": 360,
    "source_width": 640,
    "source_height": 360,
    "crop": null,
    "scale": 1,
    "file_path": "<cache>/9f6dc495db1af31e887b/frame_at/frame_at_3fa4364671abc751.png",
    "tool_file": {
      "_tag": "ToolFile",
      "name": "frame_at_3fa4364671abc751.png",
      "mimeType": "image/png",
      "encoding": "base64",
      "data": "<88728 base64 chars dropped>",
      "byteLength": 66545
    }
  }
}

Question and answer

Question: When does the word STOP show up on the screen in this video?
Answer: The tool finds the word STOP on screen from 6 to 9 seconds.

Check passedIoU 0.94, minimum 0.5

How it was checked

Truth from: constructed
Fixed at: 2026-10-30 19:39

Planted by this recipe before any tool ran: a Pillow-drawn card reading 'STOP' (white letters on a dark box) was overlaid with ffmpeg on the first 30 s of 'Sintel trailer (animated, 1080p original, 640 px)' (licence CC-BY-3.0) from 5.9 s to 8.9 s, the picture otherwise unchanged. The truth is that interval, 5.9 to 8.9 s, checked by an IoU of at least 0.5; frame_at asks for the middle of it, 7.4 s.

A script drew the word STOP on a card over this film footage for 3 seconds, at a time it chose; that planted 3-second span is the truth. The answer is the longest span the tool listed, and frame_at was asked for the middle of the planted span, not of the tool's span. Other spans, if any, are not shown on the card.

Try it yourself

Use agentic-video on my video. Find when the exact word STOP is on the screen: run count_occurrences on the whole video with text STOP, then run frame_at at the middle of the longest span it lists. Tell me the start and end seconds of that span. Use my own file.

Get the .mcpb file and open it in Claude Desktop.
Or: pip install agentic-video

Clip and licence

Licence: CC-BY-3.0
Author: Blender Foundation (Durian Open Movie project)
Source: commons.wikimedia.org

This file is licensed under the Creative Commons Attribution 3.0 Unported license.