agentic-video Gallery

Tour a 3-section film montage and find each section start

OrientFilmTool missed
0:24CC-BY-2.5 · Orange Open Movie Team (Blender Foundation)

Proof frames

Click a frame to seek

Tool calls

2 calls · agentic-video 0.3.1
overview_chunksimac · 776 ms

Arguments

{
  "source": "$CLIP",
  "chunk_sec": 60,
  "samples_per_chunk": 8
}

Result

{
  "ok": true,
  "data": {
    "chunks": [
      {
        "file_path": "/private<cache>/8dd3814d1fef0b60cfd1/contact_sheets/sheet_8samples_4cols_64844.jpg",
        "sample_count": 8,
        "rows": 2,
        "cols": 4,
        "start_sec": 1.5225,
        "end_sec": 22.8375,
        "timestamps": [
          1.5225,
          4.5675,
          7.6125,
          10.657499999999999,
          13.7025,
          16.7475,
          19.7925,
          22.8375
        ],
        "frame_width": 768,
        "frame_height": 432,
        "index": 1,
        "range_start_sec": 0,
        "range_end_sec": 24.36,
        "tool_file": {
          "_tag": "ToolFile",
          "name": "sheet_8samples_4cols_64844.jpg",
          "mimeType": "image/jpeg",
          "encoding": "base64",
          "data": "<385972 base64 chars dropped>",
          "byteLength": 289478
        }
      }
    ],
    "total": 1,
    "start_sec": 0,
    "end_sec": 24.36,
    "chunk_sec": 60
  }
}
detect_cutsimac · 491 ms

Arguments

{
  "source": "$CLIP"
}

Result

{
  "ok": true,
  "data": {
    "cuts": [
      {
        "timestamp_sec": 4.04,
        "score": 0.168508,
        "kind": "cut"
      },
      {
        "timestamp_sec": 7.76,
        "score": 0.58179,
        "kind": "cut"
      },
      {
        "timestamp_sec": 15.92,
        "score": 0.505621,
        "kind": "cut"
      },
      {
        "timestamp_sec": 22.15,
        "score": 0.145055,
        "kind": "transition"
      }
    ],
    "total": 4
  }
}

Question and answer

Question
Give me a quick tour of this video. Where do the sections start?
Answer
detect_cuts lists 4 changes, at 4.04, 7.76, 15.92, 22.15 seconds.
Tool returned
[4.04,7.76,15.92,22.15]
Truth
[7.72,15.88]
Check failedprecision 0.50 (min 1), recall 1.00 (min 1), tolerance 0.5s

How it was checked

Truth from
constructed
Fixed at

Planted by this recipe before any tool ran: ffmpeg joined 3 sections with hard cuts and no sound, in this order: elephants-dream, melies-voyage-impossible, sintel-trailer. The sections start at 0.68 s, 6.92 s, 34.44 s in their sources and are 7.72, 8.16, 8.48 s long (no scene change above 0.3 inside any section; sources skipped for having no such section: battleship-potemkin, tears-of-steel). The truth is the planted join times 7.72 s, 15.88 s, the start of each section after the first, with tolerance 0.5 s. The calls are overview_chunks over the whole montage as one sheet, then detect_cuts with its default threshold, both fixed before the run.

A script joined 3 sections, each from a different clip of the same kind, with hard cuts and no sound, so the answer is the planted joins and not a real film's structure. The first section's start (0 s) is not counted.

Try it yourself

Using agentic-video, give me a quick tour of my video. Run overview_chunks on the whole video to see its sections, then run detect_cuts on the whole video and tell me the time of each section start, in seconds. Use my own file.
  1. Get the .mcpb file and open it in Claude Desktop.
  2. Or: pip install agentic-video

Clip and licence

Licence
CC-BY-2.5
Author
Orange Open Movie Team (Blender Foundation)
Source
commons.wikimedia.org
This file is licensed under the Creative Commons Attribution 2.5 Generic license.

Also uses (insert 1)

Licence
PD
Author
Georges Méliès
Source
commons.wikimedia.org
This work is in the public domain in the United States because it was published (or registered with the U.S. Copyright Office) before January 1, 1931.

Also uses (insert 2)

Licence
CC-BY-3.0
Author
Blender Foundation (Durian Open Movie project)
Source
commons.wikimedia.org
This file is licensed under the Creative Commons Attribution 3.0 Unported license.