Find when a scoreboard-like box in one region changes
0:20CC0 · B.G. DeSylva Productions Inc
Proof frames
Tool calls
track_region_changes
Arguments
{
"source": "$CLIP",
"start_sec": 0,
"end_sec": 20.02,
"region": {
"x": 0.215625,
"y": 0.715962441314554,
"width": 0.275,
"height": 0.15023474178403756
},
"sample_fps": 4,
"max_candidates": 5
}
Result
{
"ok": true,
"data": {
"region": {
"x": 0.215625,
"y": 0.715962441314554,
"width": 0.275,
"height": 0.15023474178403756
},
"start_sec": 0,
"end_sec": 20.02,
"sample_fps": 4,
"sample_count": 80,
"temporal_resolution_sec": 0.25,
"scan_segments": [
{
"start_sec": 0,
"end_sec": 20.02,
"sample_count": 80
}
],
"threshold": 0.02,
"baseline_median": 0.000627,
"baseline_mad": 0.000346,
"steady_change": {
"detected": false,
"active_fraction": 0.329114,
"activity_floor": 0.001,
"note": "Most sample pairs are still; changes, if any, are the listed…"
},
"samples": [
{
"from_timestamp_sec": 0,
"timestamp_sec": 0.25,
"from_timestamp_provenance": {
"requested_timestamp": 0,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 0.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.002037,
"mean_abs_diff": 0.002082,
"changed_fraction": 0.001953
},
{
"from_timestamp_sec": 0.25,
"timestamp_sec": 0.5,
"from_timestamp_provenance": {
"requested_timestamp": 0.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 0.5,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.001048,
"mean_abs_diff": 0.001087,
"changed_fraction": 0.000977
},
{
"from_timestamp_sec": 0.5,
"timestamp_sec": 0.75,
"from_timestamp_provenance": {
"requested_timestamp": 0.5,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 0.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.001671,
"mean_abs_diff": 0.001913,
"changed_fraction": 0.001221
},
{
"from_timestamp_sec": 0.75,
"timestamp_sec": 1,
"from_timestamp_provenance": {
"requested_timestamp": 0.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 1,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.003324,
"mean_abs_diff": 0.003799,
"changed_fraction": 0.002441
},
{
"from_timestamp_sec": 1,
"timestamp_sec": 1.25,
"from_timestamp_provenance": {
"requested_timestamp": 1,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 1.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.001159,
"mean_abs_diff": 0.00152,
"changed_fraction": 0.000488
},
"…trimmed 74 more"
],
"candidates": [
{
"from_timestamp_sec": 8.5,
"timestamp_sec": 8.75,
"from_timestamp_provenance": {
"requested_timestamp": 8.5,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 8.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.556775,
"mean_abs_diff": 0.430515,
"changed_fraction": 0.79126,
"rank": 1
},
{
"from_timestamp_sec": 2,
"timestamp_sec": 2.25,
"from_timestamp_provenance": {
"requested_timestamp": 2,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 2.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.556304,
"mean_abs_diff": 0.429263,
"changed_fraction": 0.792236,
"rank": 2
},
{
"from_timestamp_sec": 5.25,
"timestamp_sec": 5.5,
"from_timestamp_provenance": {
"requested_timestamp": 5.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 5.5,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.5556,
"mean_abs_diff": 0.428575,
"changed_fraction": 0.791504,
"rank": 3
}
]
}
}
Question and answer
- Question
- When does the scoreboard change?
- Answer
- The tool reports changes in the box at 2.25, 5.5, 8.75 seconds.
Check passedprecision 1.00 (min 1), recall 1.00 (min 1), tolerance 0.5s
How it was checked
- Truth from
- constructed
- Fixed at
Planted by this recipe before any tool ran: ffmpeg scaled the source's first 20 s to 640x426 px and overlaid a 176x64 px scoreboard-like box at x=138, y=305 px (Pillow pictures: dark slate body, amber body in the planted intervals, white bars in both). The box changes at 2.25, 5.43, 8.83 s, the state switching at every time. The tolerance 0.5 s covers the tool's 0.25 s sample step.
A script drew the box over a fixed spot and changed its colour and brightness at 3 times it chose, so the answer is those planted times, not a real scoreboard; the tool takes the region as fractions of the frame, so the pixel box was converted.
Try it yourself
Use agentic-video on my video. A box in the picture is 176 by 64 pixels, with its top-left corner at x 138 px and y 305 px, in a 640 by 426 px frame. Convert that area to fractions of the frame and call track_region_changes on the first 20.02 seconds with sample_fps 4 and max_candidates 5. Tell me the times, in seconds, when the box changes colour.
- Get the .mcpb file and open it in Claude Desktop.
- Or:
pip install agentic-video
Clip and licence
- Licence
- CC0
- Author
- B.G. DeSylva Productions Inc
- Source
- commons.wikimedia.org
This file is made available under the Creative Commons CC0 1.0 Universal Public Domain Dedication.