Find each shot change in a 4-shot music clip
0:23CC0 · B.G. DeSylva Productions Inc
Proof frames
Tool calls
detect_cuts
Arguments
{
"source": "$CLIP"
}
Result
{
"ok": true,
"data": {
"cuts": [
{
"timestamp_sec": 4.52,
"score": 0.395189,
"kind": "cut"
},
{
"timestamp_sec": 17.96,
"score": 0.789981,
"kind": "cut"
}
],
"total": 2
}
}
inspect_windows
Arguments
{
"source": "$CLIP",
"windows": [
{
"start_sec": 3.48,
"end_sec": 5.48
},
{
"start_sec": 9.72,
"end_sec": 11.72
},
{
"start_sec": 16.92,
"end_sec": 18.92
}
],
"fps": 2
}
Result
{
"ok": true,
"data": {
"windows": [
{
"index": 1,
"input_indices": [
0
],
"start_sec": 3.48,
"end_sec": 5.48,
"fps": 2,
"frames": [
{
"index": 1,
"timestamp_sec": 3.48,
"timecode": "00:00:03.480",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_8a91ec5693e5a616acfc/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 3.48,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 2,
"timestamp_sec": 3.98,
"timecode": "00:00:03.980",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_8a91ec5693e5a616acfc/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 3.98,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 3,
"timestamp_sec": 4.48,
"timecode": "00:00:04.480",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_8a91ec5693e5a616acfc/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 4.48,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 4,
"timestamp_sec": 4.98,
"timecode": "00:00:04.980",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_8a91ec5693e5a616acfc/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 4.98,
"decoded_timestamp": null,
"timestamp_delta": null
}
],
"contact_sheet_path": "/private<cache>/ed5b46142a60d9bd3b95/contact_sheets/sheet_4s…",
"estimated_vision_tokens": 1032,
"tool_file": {
"_tag": "ToolFile",
"name": "sheet_4samples_4cols_17999f.jpg",
"mimeType": "image/jpeg",
"encoding": "base64",
"data": "<168924 base64 chars dropped>",
"byteLength": 126692
}
},
{
"index": 2,
"input_indices": [
1
],
"start_sec": 9.72,
"end_sec": 11.72,
"fps": 2,
"frames": [
{
"index": 1,
"timestamp_sec": 9.72,
"timecode": "00:00:09.720",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_865a4bdda40d2f5e8dac/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 9.72,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 2,
"timestamp_sec": 10.22,
"timecode": "00:00:10.220",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_865a4bdda40d2f5e8dac/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 10.22,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 3,
"timestamp_sec": 10.72,
"timecode": "00:00:10.720",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_865a4bdda40d2f5e8dac/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 10.72,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 4,
"timestamp_sec": 11.22,
"timecode": "00:00:11.220",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_865a4bdda40d2f5e8dac/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 11.22,
"decoded_timestamp": null,
"timestamp_delta": null
}
],
"contact_sheet_path": "/private<cache>/ed5b46142a60d9bd3b95/contact_sheets/sheet_4s…",
"estimated_vision_tokens": 1032,
"tool_file": {
"_tag": "ToolFile",
"name": "sheet_4samples_4cols_89d949.jpg",
"mimeType": "image/jpeg",
"encoding": "base64",
"data": "<171524 base64 chars dropped>",
"byteLength": 128641
}
},
{
"index": 3,
"input_indices": [
2
],
"start_sec": 16.92,
"end_sec": 18.92,
"fps": 2,
"frames": [
{
"index": 1,
"timestamp_sec": 16.92,
"timecode": "00:00:16.920",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_d958e4bd9ad1d9063123/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 16.92,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 2,
"timestamp_sec": 17.42,
"timecode": "00:00:17.420",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_d958e4bd9ad1d9063123/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 17.42,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 3,
"timestamp_sec": 17.92,
"timecode": "00:00:17.920",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_d958e4bd9ad1d9063123/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 17.92,
"decoded_timestamp": null,
"timestamp_delta": null
},
{
"index": 4,
"timestamp_sec": 18.42,
"timecode": "00:00:18.420",
"file_path": "<cache>/ed5b46142a60d9bd3b95/dense_d958e4bd9ad1d9063123/fram…",
"width": 640,
"height": 360,
"requested_timestamp": 18.42,
"decoded_timestamp": null,
"timestamp_delta": null
}
],
"contact_sheet_path": "/private<cache>/ed5b46142a60d9bd3b95/contact_sheets/sheet_4s…",
"estimated_vision_tokens": 1032,
"tool_file": {
"_tag": "ToolFile",
"name": "sheet_4samples_4cols_e8b23b.jpg",
"mimeType": "image/jpeg",
"encoding": "base64",
"data": "<215880 base64 chars dropped>",
"byteLength": 161910
}
}
],
"total": 3,
"total_frames": 12,
"estimated_vision_tokens": 3096
}
}
Question and answer
- Question
- Where does the shot change? Show me each change.
- Answer
- detect_cuts lists 2 shot changes, at 4.52, 17.96 seconds.
- Tool returned
- [4.52,17.96]
- Truth
- [4.48,10.72,17.92]
Check failedprecision 1.00 (min 1), recall 0.67 (min 1), tolerance 0.5s
How it was checked
- Truth from
- constructed
- Fixed at
Planted by this recipe before any tool ran: ffmpeg joined 4 single shots with hard cuts only (no fades, no sound, each scaled to 640x360): shot 1 is 4.48 s of andy-russell-drum-solo-1945 from 14.08 s; shot 2 is 6.24 s of brian-silas-piano-bhopal from 36.12 s; shot 3 is 7.2 s of brian-silas-piano-bhopal from 42.64 s; shot 4 is 5.08 s of street-musicians-new-arbat from 47.24 s. The planted shot changes are at 4.48 s, 10.72 s, 17.92 s, and the truth is those times with tolerance 0.5 s. The calls are detect_cuts with its default threshold, then inspect_windows over 1 s either side of each planted time at 2 fps, a window fixed here before the run.
A script joined 4 single shots from 3 public music clips with three hard cuts and no fades or sound, at times it chose. The windows are centred on those planted changes, not on the tool's output, and the card does not show any other change the tool might list.
Try it yourself
Using agentic-video on my video (the file I attach or name): run detect_cuts on the whole video and list the time of every shot change it finds. Then run inspect_windows over one second either side of each change and show me the frames there. Give me each change time.
- Get the .mcpb file and open it in Claude Desktop.
- Or:
pip install agentic-video
Clip and licence
- Licence
- CC0
- Author
- B.G. DeSylva Productions Inc
- Source
- commons.wikimedia.org
This file is made available under the Creative Commons CC0 1.0 Universal Public Domain Dedication.
Also uses (insert 1)
- Licence
- CC-BY-SA-4.0
- Author
- Suyash Dwivedi
- Source
- commons.wikimedia.org
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.
Also uses (insert 2)
- Licence
- CC-BY-4.0
- Author
- Predatorsamsung
- Source
- commons.wikimedia.org
This file is licensed under the Creative Commons Attribution 4.0 International license.