agentic-video Gallery

Find when a magenta square first appears in a cooking clip

Find a momentCookingVerified
0:30CC-BY-SA-4.0 · مصطفى ملو (Wikimedia Commons user)

Proof frames

Click a frame to seek

Tool calls

1 calls · agentic-video 0.3.1
watch_formacbook · 5149 ms

Arguments

{
  "source": "$CLIP",
  "conditions": [
    "Is a bright magenta square visible?"
  ],
  "interval_sec": 2,
  "consecutive": 2
}

Result

{
  "ok": true,
  "data": {
    "source_path": "~/Code/agentic-video-gallery/cards/watch-cooking/clip.mp4",
    "start_sec": 0,
    "scanned_to_sec": 28,
    "interval_sec": 2,
    "cut": 0.5,
    "consecutive": 2,
    "cooldown_sec": 30,
    "model": "clef",
    "conditions": [
      {
        "index": 0,
        "condition": "Is a bright magenta square visible?",
        "wakes": 1,
        "dropped_by_cooldown": 4,
        "max_chance": 0.9928,
        "max_chance_sec": 14
      }
    ],
    "wakes": [
      {
        "time_sec": 10,
        "condition_index": 0,
        "condition": "Is a bright magenta square visible?",
        "first_seen_sec": 8,
        "chance": 0.9921
      }
    ],
    "samples": 15,
    "complete": true,
    "next_start_sec": null,
    "input_tokens": 5520
  }
}

Question and answer

Question
When does a bright magenta square first appear in this video?
Answer
The tool first sees the magenta square at 8 seconds.
Check passedanswer 8.0, expected 6.1 ±3.0

How it was checked

Truth from
constructed
Fixed at

Planted by this recipe: ffmpeg overlaid a 180 px bright magenta square, drawn by Pillow, on the right of the first 30 s of 'Filling and laying dough for a nomad flatbread on hot stones' from 6.1 s to 18.1 s. The truth is that planted start, the first moment the square is on screen; the tolerance 3 s is one watch_for interval (2 s) plus 1 s.

A script drew the square over 'Filling and laying dough for a nomad flatbread on hot stones' from a time it chose (it stays 12 seconds), so the answer is that planted time within 3 seconds, not a real object in the footage.

Try it yourself

Using agentic-video, find when a bright magenta square first appears in my video. Run watch_for on my file with the question 'Is a bright magenta square visible?', one frame every 2 seconds and 2 yes answers in a row to wake. Tell me the first time it wakes.
  1. Get the .mcpb file and open it in Claude Desktop.
  2. Or: pip install agentic-video

Clip and licence

Licence
CC-BY-SA-4.0
Author
مصطفى ملو (Wikimedia Commons user)
Source
commons.wikimedia.org
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.