agentic-video Gallery

Find the exact start of a spliced-in shot

Find a momentTutorialVerified
0:24CC0 · Dev Jadiya

Proof frames

Click a frame to seek

Tool calls

2 calls · agentic-video 0.3.1
refine_temporal_beamimac · 453 ms

Arguments

{
  "source": "$CLIP",
  "windows": [
    {
      "start_sec": 7.667,
      "end_sec": 18.667
    }
  ],
  "fanout": 4,
  "samples_per_child": 3
}

Result

{
  "ok": true,
  "data": {
    "parents": [
      {
        "parent_index": 1,
        "input_start_sec": 7.667,
        "input_end_sec": 18.667,
        "start_sec": 7.667,
        "end_sec": 18.667,
        "children": [
          {
            "child_index": 1,
            "start_sec": 7.667,
            "end_sec": 10.417,
            "sample_timestamps": [
              8.125333333333334,
              9.042,
              9.958666666666666
            ],
            "contact_sheet_path": "/private<cache>/45a5ac7e352b48fb4f66/contact_sheets/sheet_3samples_3cols_18c5f8.jpg",
            "sample_count": 3,
            "tool_file": {
              "_tag": "ToolFile",
              "name": "sheet_3samples_3cols_18c5f8.jpg",
              "mimeType": "image/jpeg",
              "encoding": "base64",
              "data": "<206720 base64 chars dropped>",
              "byteLength": 155040
            }
          },
          {
            "child_index": 2,
            "start_sec": 10.417,
            "end_sec": 13.167000000000002,
            "sample_timestamps": [
              10.875333333333334,
              11.792000000000002,
              12.708666666666668
            ],
            "contact_sheet_path": "/private<cache>/45a5ac7e352b48fb4f66/contact_sheets/sheet_3samples_3cols_8fff48.jpg",
            "sample_count": 3,
            "tool_file": {
              "_tag": "ToolFile",
              "name": "sheet_3samples_3cols_8fff48.jpg",
              "mimeType": "image/jpeg",
              "encoding": "base64",
              "data": "<141428 base64 chars dropped>",
              "byteLength": 106070
            }
          },
          {
            "child_index": 3,
            "start_sec": 13.167000000000002,
            "end_sec": 15.917000000000002,
            "sample_timestamps": [
              13.625333333333336,
              14.542000000000002,
              15.458666666666668
            ],
            "contact_sheet_path": "/private<cache>/45a5ac7e352b48fb4f66/contact_sheets/sheet_3samples_3cols_a1b496.jpg",
            "sample_count": 3,
            "tool_file": {
              "_tag": "ToolFile",
              "name": "sheet_3samples_3cols_a1b496.jpg",
              "mimeType": "image/jpeg",
              "encoding": "base64",
              "data": "<147412 base64 chars dropped>",
              "byteLength": 110558
            }
          },
          {
            "child_index": 4,
            "start_sec": 15.917000000000002,
            "end_sec": 18.667,
            "sample_timestamps": [
              16.375333333333334,
              17.292,
              18.20866666666667
            ],
            "contact_sheet_path": "/private<cache>/45a5ac7e352b48fb4f66/contact_sheets/sheet_3samples_3cols_c15a85.jpg",
            "sample_count": 3,
            "tool_file": {
              "_tag": "ToolFile",
              "name": "sheet_3samples_3cols_c15a85.jpg",
              "mimeType": "image/jpeg",
              "encoding": "base64",
              "data": "<185012 base64 chars dropped>",
              "byteLength": 138759
            }
          }
        ]
      }
    ],
    "total_samples": 12,
    "estimated_vision_tokens": 3096
  }
}
detect_cutsimac · 229 ms

Arguments

{
  "source": "$CLIP",
  "start_sec": 7.667,
  "end_sec": 18.667
}

Result

{
  "ok": true,
  "data": {
    "cuts": [
      {
        "timestamp_sec": 11.666674,
        "score": 1,
        "kind": "cut"
      },
      {
        "timestamp_sec": 14.666674,
        "score": 1,
        "kind": "cut"
      }
    ],
    "total": 2
  }
}

Question and answer

Question
Exactly when does the new shot start in this clip?
Answer
The first cut in the window, into the new shot, is at 11.67 seconds.
Check passedanswer 11.666674, expected 11.666667 ±0.5

How it was checked

Truth from
constructed
Fixed at

Planted by this recipe before any tool ran: ffmpeg cut 24 s of the source from 62.625 s, and from 11.667 s for 3 s the picture is a red still with the words INSERTED SHOT (the source has no sound track, so the clip has none). Before the cut, the whole 24 s bed was checked with an ffmpeg scene filter (score above 0.15): no cut in it, so the first cut is the planted one. The truth is the planted start 11.667 s, with tolerance 0.5 s. Both tools get the coarse window 7.667 s to 18.667 s, fixed before the run as 4 s before and 7 s after the planted start.

A script put a 3-second red still with the words INSERTED SHOT over the picture at a set time, so the answer is that planted time, and refine_temporal_beam only zooms in and does not say where the change is, so the time comes from detect_cuts.

Try it yourself

Use agentic-video to find when a new shot starts in my video. Run refine_temporal_beam on my video over the window from 7.667 to 18.667 seconds, with fanout 4 and 3 samples per child. Then run detect_cuts on that same window and tell me the time of the first cut it lists. Use my own file.
  1. Get the .mcpb file and open it in Claude Desktop.
  2. Or: pip install agentic-video

Clip and licence

Licence
CC0
Author
Dev Jadiya
Source
commons.wikimedia.org
This file is made available under the Creative Commons CC0 1.0 Universal Public Domain Dedication