Find when a word shows on screen in a screen recording

Read the screenScreen recordingVerified

0:28CC0 · NS2000000000

Proof frames

Click a frame to seek

Tool calls

11 calls · agentic-video 0.3.1

count_occurrencesimac · 443 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 15 of 140 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 25 of 140 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 68 of 140 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 79 of 140 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 88 of 140 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 129 of 140 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 137 of 140 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 140 of 140 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "DONE",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "complete",
    "count": 1,
    "spans": [
      {
        "index": 1,
        "start_sec": 17.4,
        "end_sec": 20.4,
        "duration_sec": 3,
        "score": 1,
        "representative_sec": 19,
        "frame_badge": "#1",
        "weak": false,
        "region": {
          "x": 0.416,
          "y": 0.477,
          "w": 0.169,
          "h": 0.105
        },
        "travel": "stationary"
      }
    ],
    "spans_returned": 1,
    "spans_omitted": 0,
    "window": {
      "start_sec": 0,
      "end_sec": 28
    },
    "fps_used": 5,
    "frames_sampled": 140,
    "frames_analyzed": 76,
    "checks": {
      "text": "DONE",
      "description": null,
      "color": null,
      "shape": null
    },
    "contact_sheet": {
      "file_path": "<cache>/2925942ae928213bec6d/contact_sheets/occurrences_3647babbb7129c43dd72.jpg",
      "sample_count": 1,
      "rows": 1,
      "cols": 1,
      "frame_width": 288,
      "frame_height": 192,
      "labels": [
        "#1 19.0s"
      ],
      "tool_file": {
        "_tag": "ToolFile",
        "name": "occurrences_3647babbb7129c43dd72.jpg",
        "mimeType": "image/jpeg",
        "encoding": "base64",
        "data": "<14636 base64 chars dropped>",
        "byteLength": 10976
      }
    },
    "note": "Spans are sample-based (+/- 1/fps); verify each representative_sec with inspect_window. contact_sheet is one image with a tile per listed span, in span order: a zoomed frame of the span's best match, boxed, labeled with the span number and time. Look at each tile and drop any span whose tile is not the asked-for thing; spans marked * (`weak`) are the likeliest false ones. The answer is the number of tiles that match, plus any omitted spans and respecting any merged gaps noted above.",
    "verify": "Rough count: spans can be false, merge across scene cuts, split one appearance, or miss crossings. Before answering, look at each tile of the contact_sheet image (one per span, in span order, a zoomed frame of its best match, boxed) and drop any span whose tile is not the asked-for thing; spans with weak=true (tile label *) are the likeliest false ones. The answer is the number of tiles that match, plus any spans the sheet does not show; if you also count by eye and differ, trust your own."
  }
}

frame_atimac · 517 ms

Arguments

{
  "source": "$CLIP",
  "time_sec": 18.8
}

Result

{
  "ok": true,
  "data": {
    "frame_time_sec": 18.8,
    "width": 640,
    "height": 306,
    "source_width": 640,
    "source_height": 306,
    "crop": null,
    "scale": 1,
    "file_path": "<cache>/2925942ae928213bec6d/frame_at/frame_at_3faedd9652477b1e.png",
    "tool_file": {
      "_tag": "ToolFile",
      "name": "frame_at_3faedd9652477b1e.png",
      "mimeType": "image/png",
      "encoding": "base64",
      "data": "<56840 base64 chars dropped>",
      "byteLength": 42629
    }
  }
}

Question and answer

Question: When does the word DONE show up on the screen in this video?
Answer: The tool finds the word DONE on screen from 17.4 to 20.4 seconds.

Check passedIoU 0.94, minimum 0.5

How it was checked

Truth from: constructed
Fixed at: 2026-10-30 19:37

Planted by this recipe before any tool ran: a Pillow-drawn card reading 'DONE' (white letters on a dark box) was overlaid with ffmpeg on the first 28 s of 'Browser on a Wikipedia sandbox page with the cursor moving' (licence CC0) from 17.3 s to 20.3 s, the picture otherwise unchanged. The truth is that interval, 17.3 to 20.3 s, checked by an IoU of at least 0.5; frame_at asks for the middle of it, 18.8 s.

A script drew the word DONE on a card over this a screen recording for 3 seconds, at a time it chose; that planted 3-second span is the truth. The answer is the longest span the tool listed, and frame_at was asked for the middle of the planted span, not of the tool's span. Other spans, if any, are not shown on the card.

Try it yourself

Use agentic-video on my video. Find when the exact word DONE is on the screen: run count_occurrences on the whole video with text DONE, then run frame_at at the middle of the longest span it lists. Tell me the start and end seconds of that span. Use my own file.

Get the .mcpb file and open it in Claude Desktop.
Or: pip install agentic-video

Clip and licence

Licence: CC0
Author: NS2000000000
Source: commons.wikimedia.org

This file is made available under the Creative Commons CC0 1.0 Universal Public Domain Dedication.