Find when a word shows on screen in tutorial footage

Read the screenTutorialVerified

0:30CC0 · Dev Jadiya

Proof frames

Click a frame to seek

Tool calls

21 calls · agentic-video 0.3.1

count_occurrencesimac · 1612 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 28 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 2 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 45 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 53 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 66 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 70 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 74 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 87 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 102 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 106 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 114 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 125 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 131 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 143 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 145 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 148 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 150 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 150 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "in_progress",
    "retry_after_seconds": 5,
    "message": "Analyzed 150 of 150 sampled frames; repeat this call to poll."
  }
}

count_occurrencesimac · 1 ms

Arguments

{
  "source": "$CLIP",
  "text": "SAVED",
  "max_spans": 5
}

Result

{
  "ok": true,
  "data": {
    "status": "complete",
    "count": 1,
    "spans": [
      {
        "index": 1,
        "start_sec": 13.4,
        "end_sec": 16.4,
        "duration_sec": 3,
        "score": 1,
        "representative_sec": 15,
        "frame_badge": "#1",
        "weak": false,
        "region": {
          "x": 0.403,
          "y": 0.478,
          "w": 0.194,
          "h": 0.089
        },
        "travel": "stationary"
      }
    ],
    "spans_returned": 1,
    "spans_omitted": 0,
    "window": {
      "start_sec": 0,
      "end_sec": 30
    },
    "fps_used": 5,
    "frames_sampled": 150,
    "frames_analyzed": 94,
    "checks": {
      "text": "SAVED",
      "description": null,
      "color": null,
      "shape": null
    },
    "contact_sheet": {
      "file_path": "<cache>/6121d3006a003c66fecd/contact_sheets/occurrences_9a640a7392436e370730.jpg",
      "sample_count": 1,
      "rows": 1,
      "cols": 1,
      "frame_width": 288,
      "frame_height": 192,
      "labels": [
        "#1 15.0s"
      ],
      "tool_file": {
        "_tag": "ToolFile",
        "name": "occurrences_9a640a7392436e370730.jpg",
        "mimeType": "image/jpeg",
        "encoding": "base64",
        "data": "<17648 base64 chars dropped>",
        "byteLength": 13235
      }
    },
    "note": "Spans are sample-based (+/- 1/fps); verify each representative_sec with inspect_window. contact_sheet is one image with a tile per listed span, in span order: a zoomed frame of the span's best match, boxed, labeled with the span number and time. Look at each tile and drop any span whose tile is not the asked-for thing; spans marked * (`weak`) are the likeliest false ones. The answer is the number of tiles that match, plus any omitted spans and respecting any merged gaps noted above.",
    "verify": "Rough count: spans can be false, merge across scene cuts, split one appearance, or miss crossings. Before answering, look at each tile of the contact_sheet image (one per span, in span order, a zoomed frame of its best match, boxed) and drop any span whose tile is not the asked-for thing; spans with weak=true (tile label *) are the likeliest false ones. The answer is the number of tiles that match, plus any spans the sheet does not show; if you also count by eye and differ, trust your own."
  }
}

frame_atimac · 674 ms

Arguments

{
  "source": "$CLIP",
  "time_sec": 14.8
}

Result

{
  "ok": true,
  "data": {
    "frame_time_sec": 14.8,
    "width": 640,
    "height": 360,
    "source_width": 640,
    "source_height": 360,
    "crop": null,
    "scale": 1,
    "file_path": "<cache>/6121d3006a003c66fecd/frame_at/frame_at_9cc35b117a454584.png",
    "tool_file": {
      "_tag": "ToolFile",
      "name": "frame_at_9cc35b117a454584.png",
      "mimeType": "image/png",
      "encoding": "base64",
      "data": "<116964 base64 chars dropped>",
      "byteLength": 87721
    }
  }
}

Question and answer

Question: When does the word SAVED show up on the screen in this video?
Answer: The tool finds the word SAVED on screen from 13.4 to 16.4 seconds.

Check passedIoU 0.94, minimum 0.5

How it was checked

Truth from: constructed
Fixed at: 2026-10-30 20:13

Planted by this recipe before any tool ran: a Pillow-drawn card reading 'SAVED' (white letters on a dark box) was overlaid with ffmpeg on the first 30 s of 'Screencast: uploading an image to Wikimedia Commons (2025)' (licence CC0) from 13.3 s to 16.3 s, the picture otherwise unchanged. The truth is that interval, 13.3 to 16.3 s, checked by an IoU of at least 0.5; frame_at asks for the middle of it, 14.8 s.

A script drew the word SAVED on a card over this tutorial footage for 3 seconds, at a time it chose; that planted 3-second span is the truth. The answer is the longest span the tool listed, and frame_at was asked for the middle of the planted span, not of the tool's span. Other spans, if any, are not shown on the card.

Try it yourself

Use agentic-video on my video. Find when the exact word SAVED is on the screen: run count_occurrences on the whole video with text SAVED, then run frame_at at the middle of the longest span it lists. Tell me the start and end seconds of that span. Use my own file.

Get the .mcpb file and open it in Claude Desktop.
Or: pip install agentic-video

Clip and licence

Licence: CC0
Author: Dev Jadiya
Source: commons.wikimedia.org

This file is made available under the Creative Commons CC0 1.0 Universal Public Domain Dedication