Find each shot change in a 4-shot nature clip
0:23CC-BY-SA-4.0 · Chris Light
Proof frames
Tool calls
detect_cuts
Arguments
{
"source": "$CLIP"
}
Result
{
"ok": true,
"data": {
"cuts": [
{
"timestamp_sec": 4.96,
"score": 0.534942,
"kind": "cut"
},
{
"timestamp_sec": 18.36,
"score": 0.362486,
"kind": "cut"
}
],
"total": 2
}
}
inspect_windows
Arguments
{
"source": "$CLIP",
"windows": [
{
"start_sec": 3.92,
"end_sec": 5.92
},
{
"start_sec": 10.76,
"end_sec": 12.76
},
{
"start_sec": 17.36,
"end_sec": 19.36
}
],
"fps": 2
}
Result
{
"ok": true,
"data": {
"windows": [
{
"index": 1,
"input_indices": [
0
],
"start_sec": 3.92,
"end_sec": 5.92,
"fps": 2,
"frames": [
{
"index": 1,
"timestamp_sec": 3.92,
"timecode": "00:00:03.920",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_55a7708a131df38e5702/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 3.92,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 2,
"timestamp_sec": 4.42,
"timecode": "00:00:04.420",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_55a7708a131df38e5702/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 4.42,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 3,
"timestamp_sec": 4.92,
"timecode": "00:00:04.920",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_55a7708a131df38e5702/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 4.92,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 4,
"timestamp_sec": 5.42,
"timecode": "00:00:05.420",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_55a7708a131df38e5702/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 5.42,
"decoded_timestamp": null,
"timestamp_delta": null
}
],
"contact_sheet_path": "/private<cache>/2f9efb66ecfb697e4275/contact_sheets/sheet_4s…",
"estimated_vision_tokens": 1032,
"tool_file": {
"_tag": "ToolFile",
"name": "sheet_4samples_4cols_90aacc.jpg",
"mimeType": "image/jpeg",
"encoding": "base64",
"data": "<253480 base64 chars dropped>",
"byteLength": 190108
}
},
{
"index": 2,
"input_indices": [
1
],
"start_sec": 10.76,
"end_sec": 12.76,
"fps": 2,
"frames": [
{
"index": 1,
"timestamp_sec": 10.76,
"timecode": "00:00:10.760",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_a0da66dc7b7738b059a6/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 10.76,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 2,
"timestamp_sec": 11.26,
"timecode": "00:00:11.260",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_a0da66dc7b7738b059a6/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 11.26,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 3,
"timestamp_sec": 11.76,
"timecode": "00:00:11.760",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_a0da66dc7b7738b059a6/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 11.76,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 4,
"timestamp_sec": 12.26,
"timecode": "00:00:12.260",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_a0da66dc7b7738b059a6/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 12.26,
"decoded_timestamp": null,
"timestamp_delta": null
}
],
"contact_sheet_path": "/private<cache>/2f9efb66ecfb697e4275/contact_sheets/sheet_4s…",
"estimated_vision_tokens": 1032,
"tool_file": {
"_tag": "ToolFile",
"name": "sheet_4samples_4cols_2c6540.jpg",
"mimeType": "image/jpeg",
"encoding": "base64",
"data": "<324056 base64 chars dropped>",
"byteLength": 243040
}
},
{
"index": 3,
"input_indices": [
2
],
"start_sec": 17.36,
"end_sec": 19.36,
"fps": 2,
"frames": [
{
"index": 1,
"timestamp_sec": 17.36,
"timecode": "00:00:17.360",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_f7a2dbe37108862d60c7/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 17.36,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 2,
"timestamp_sec": 17.86,
"timecode": "00:00:17.860",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_f7a2dbe37108862d60c7/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 17.86,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 3,
"timestamp_sec": 18.36,
"timecode": "00:00:18.360",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_f7a2dbe37108862d60c7/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 18.36,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 4,
"timestamp_sec": 18.86,
"timecode": "00:00:18.860",
"file_path": "<cache>/2f9efb66ecfb697e4275/dense_f7a2dbe37108862d60c7/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 18.86,
"decoded_timestamp": null,
"timestamp_delta": null
}
],
"contact_sheet_path": "/private<cache>/2f9efb66ecfb697e4275/contact_sheets/sheet_4s…",
"estimated_vision_tokens": 1032,
"tool_file": {
"_tag": "ToolFile",
"name": "sheet_4samples_4cols_d95743.jpg",
"mimeType": "image/jpeg",
"encoding": "base64",
"data": "<396092 base64 chars dropped>",
"byteLength": 297068
}
}
],
"total": 3,
"total_frames": 12,
"estimated_vision_tokens": 3096
}
}
Question and answer
- Question
- Where does the shot change? Show me each change.
- Answer
- detect_cuts lists 2 shot changes, at 4.96, 18.36 seconds.
- Tool returned
- [4.96,18.36]
- Truth
- [4.92,11.76,18.36]
Check failedprecision 1.00 (min 1), recall 0.67 (min 1), tolerance 0.5s
How it was checked
- Truth from
- constructed
- Fixed at
Planted by this recipe before any tool ran: ffmpeg joined 4 single shots with hard cuts only (no fades, no sound, each scaled to 640x360): shot 1 is 4.92 s of ocean-shores-waves-seagulls from 39.52 s; shot 2 is 6.84 s of yakushima-deer-sounds from 2.28 s; shot 3 is 6.6 s of yakushima-deer-sounds from 5.84 s; shot 4 is 4.64 s of nairobi-giraffe-eating from 14.04 s. The planted shot changes are at 4.92 s, 11.76 s, 18.36 s, and the truth is those times with tolerance 0.5 s. The calls are detect_cuts with its default threshold, then inspect_windows over 1 s either side of each planted time at 2 fps, a window fixed here before the run.
A script joined 4 single shots from 3 public nature clips with three hard cuts and no fades or sound, at times it chose. The windows are centred on those planted changes, not on the tool's output, and the card does not show any other change the tool might list.
Try it yourself
Using agentic-video on my video (the file I attach or name): run detect_cuts on the whole video and list the time of every shot change it finds. Then run inspect_windows over one second either side of each change and show me the frames there. Give me each change time.
- Get the .mcpb file and open it in Claude Desktop.
- Or:
pip install agentic-video
Clip and licence
- Licence
- CC-BY-SA-4.0
- Author
- Chris Light
- Source
- commons.wikimedia.org
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.
Also uses (insert 1)
- Licence
- CC-BY-SA-3.0
- Author
- Geraldshields11
- Source
- commons.wikimedia.org
This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.
Also uses (insert 2)
- Licence
- CC-BY-SA-4.0
- Author
- Addshore
- Source
- commons.wikimedia.org
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.