agentic-video Gallery

Find the exact start of a spliced-in shot

Find a momentScreen recordingVerified
0:24PD · Fl.schmitt

Proof frames

Click a frame to seek

Tool calls

2 calls · agentic-video 0.3.1
refine_temporal_beamimac · 486 ms

Arguments

{
  "source": "$CLIP",
  "windows": [
    {
      "start_sec": 2.8,
      "end_sec": 13.8
    }
  ],
  "fanout": 4,
  "samples_per_child": 3
}

Result

{
  "ok": true,
  "data": {
    "parents": [
      {
        "parent_index": 1,
        "input_start_sec": 2.8,
        "input_end_sec": 13.8,
        "start_sec": 2.8,
        "end_sec": 13.8,
        "children": [
          {
            "child_index": 1,
            "start_sec": 2.8,
            "end_sec": 5.55,
            "sample_timestamps": [
              3.2583333333333333,
              4.175,
              5.091666666666667
            ],
            "contact_sheet_path": "/private<cache>/1168f973d4780eff619c/contact_sheets/sheet_3samples_3cols_54c0c8.jpg",
            "sample_count": 3,
            "tool_file": {
              "_tag": "ToolFile",
              "name": "sheet_3samples_3cols_54c0c8.jpg",
              "mimeType": "image/jpeg",
              "encoding": "base64",
              "data": "<278956 base64 chars dropped>",
              "byteLength": 209217
            }
          },
          {
            "child_index": 2,
            "start_sec": 5.55,
            "end_sec": 8.3,
            "sample_timestamps": [
              6.008333333333333,
              6.925000000000001,
              7.841666666666667
            ],
            "contact_sheet_path": "/private<cache>/1168f973d4780eff619c/contact_sheets/sheet_3samples_3cols_1efd98.jpg",
            "sample_count": 3,
            "tool_file": {
              "_tag": "ToolFile",
              "name": "sheet_3samples_3cols_1efd98.jpg",
              "mimeType": "image/jpeg",
              "encoding": "base64",
              "data": "<180376 base64 chars dropped>",
              "byteLength": 135281
            }
          },
          {
            "child_index": 3,
            "start_sec": 8.3,
            "end_sec": 11.05,
            "sample_timestamps": [
              8.758333333333335,
              9.675,
              10.591666666666667
            ],
            "contact_sheet_path": "/private<cache>/1168f973d4780eff619c/contact_sheets/sheet_3samples_3cols_285a8f.jpg",
            "sample_count": 3,
            "tool_file": {
              "_tag": "ToolFile",
              "name": "sheet_3samples_3cols_285a8f.jpg",
              "mimeType": "image/jpeg",
              "encoding": "base64",
              "data": "<184384 base64 chars dropped>",
              "byteLength": 138288
            }
          },
          {
            "child_index": 4,
            "start_sec": 11.05,
            "end_sec": 13.8,
            "sample_timestamps": [
              11.508333333333335,
              12.425,
              13.341666666666667
            ],
            "contact_sheet_path": "/private<cache>/1168f973d4780eff619c/contact_sheets/sheet_3samples_3cols_3a0fdd.jpg",
            "sample_count": 3,
            "tool_file": {
              "_tag": "ToolFile",
              "name": "sheet_3samples_3cols_3a0fdd.jpg",
              "mimeType": "image/jpeg",
              "encoding": "base64",
              "data": "<289072 base64 chars dropped>",
              "byteLength": 216802
            }
          }
        ]
      }
    ],
    "total_samples": 12,
    "estimated_vision_tokens": 3096
  }
}
detect_cutsimac · 208 ms

Arguments

{
  "source": "$CLIP",
  "start_sec": 2.8,
  "end_sec": 13.8
}

Result

{
  "ok": true,
  "data": {
    "cuts": [
      {
        "timestamp_sec": 6.8,
        "score": 1,
        "kind": "cut"
      },
      {
        "timestamp_sec": 9.8,
        "score": 1,
        "kind": "cut"
      }
    ],
    "total": 2
  }
}

Question and answer

Question
Exactly when does the new shot start in this clip?
Answer
The first cut in the window, into the new shot, is at 6.80 seconds.
Check passedanswer 6.8, expected 6.8 ±0.5

How it was checked

Truth from
constructed
Fixed at

Planted by this recipe before any tool ran: ffmpeg cut 24 s of the source from 8.627 s, and from 6.800 s for 3 s the picture is a red still with the words INSERTED SHOT (the source has no sound track, so the clip has none). Before the cut, the coarse window of the bed was checked with an ffmpeg scene filter (score above 0.25): no cut in it, because the stretches of this source are busier than the whole-bed rule allows, so the first cut is the planted one. The truth is the planted start 6.800 s, with tolerance 0.5 s. Both tools get the coarse window 2.800 s to 13.800 s, fixed before the run as 4 s before and 7 s after the planted start.

A script put a 3-second red still with the words INSERTED SHOT over the picture at a set time, so the answer is that planted time, and refine_temporal_beam only zooms in and does not say where the change is, so the time comes from detect_cuts.

Try it yourself

Use agentic-video to find when a new shot starts in my video. Run refine_temporal_beam on my video over the window from 2.8 to 13.8 seconds, with fanout 4 and 3 samples per child. Then run detect_cuts on that same window and tell me the time of the first cut it lists. Use my own file.
  1. Get the .mcpb file and open it in Claude Desktop.
  2. Or: pip install agentic-video

Clip and licence

Licence
PD
Author
Fl.schmitt
Source
commons.wikimedia.org
This work has been released into the public domain by its author, Fl.schmitt.