Find when a scoreboard-like box in one region changes
0:20CC-BY-SA-4.0 · Sea Cow
Proof frames
Tool calls
track_region_changes
Arguments
{
"source": "$CLIP",
"start_sec": 0,
"end_sec": 20.02,
"region": {
"x": 0.1890625,
"y": 0.6333333333333333,
"width": 0.275,
"height": 0.17777777777777778
},
"sample_fps": 4,
"max_candidates": 5
}
Result
{
"ok": true,
"data": {
"region": {
"x": 0.1890625,
"y": 0.6333333333333333,
"width": 0.275,
"height": 0.17777777777777778
},
"start_sec": 0,
"end_sec": 20.02,
"sample_fps": 4,
"sample_count": 80,
"temporal_resolution_sec": 0.25,
"scan_segments": [
{
"start_sec": 0,
"end_sec": 20.02,
"sample_count": 80
}
],
"threshold": 0.02,
"baseline_median": 0.000946,
"baseline_mad": 0.000343,
"steady_change": {
"detected": false,
"active_fraction": 0.455696,
"activity_floor": 0.001,
"note": "Most sample pairs are still; changes, if any, are the listed…"
},
"samples": [
{
"from_timestamp_sec": 0,
"timestamp_sec": 0.25,
"from_timestamp_provenance": {
"requested_timestamp": 0,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 0.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.000588,
"mean_abs_diff": 0.000773,
"changed_fraction": 0.000244
},
{
"from_timestamp_sec": 0.25,
"timestamp_sec": 0.5,
"from_timestamp_provenance": {
"requested_timestamp": 0.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 0.5,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.000946,
"mean_abs_diff": 0.00093,
"changed_fraction": 0.000977
},
{
"from_timestamp_sec": 0.5,
"timestamp_sec": 0.75,
"from_timestamp_provenance": {
"requested_timestamp": 0.5,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 0.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.001018,
"mean_abs_diff": 0.00104,
"changed_fraction": 0.000977
},
{
"from_timestamp_sec": 0.75,
"timestamp_sec": 1,
"from_timestamp_provenance": {
"requested_timestamp": 0.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 1,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.00086,
"mean_abs_diff": 0.00106,
"changed_fraction": 0.000488
},
{
"from_timestamp_sec": 1,
"timestamp_sec": 1.25,
"from_timestamp_provenance": {
"requested_timestamp": 1,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 1.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.000955,
"mean_abs_diff": 0.001075,
"changed_fraction": 0.000732
},
"…trimmed 74 more"
],
"candidates": [
{
"from_timestamp_sec": 2.75,
"timestamp_sec": 3,
"from_timestamp_provenance": {
"requested_timestamp": 2.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 3,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.563519,
"mean_abs_diff": 0.435499,
"changed_fraction": 0.80127,
"rank": 1
},
{
"from_timestamp_sec": 9,
"timestamp_sec": 9.25,
"from_timestamp_provenance": {
"requested_timestamp": 9,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 9.25,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.561827,
"mean_abs_diff": 0.432502,
"changed_fraction": 0.802002,
"rank": 2
},
{
"from_timestamp_sec": 15.75,
"timestamp_sec": 16,
"from_timestamp_provenance": {
"requested_timestamp": 15.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 16,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.561487,
"mean_abs_diff": 0.432242,
"changed_fraction": 0.801514,
"rank": 3
},
{
"from_timestamp_sec": 5.75,
"timestamp_sec": 6,
"from_timestamp_provenance": {
"requested_timestamp": 5.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 6,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.560574,
"mean_abs_diff": 0.431494,
"changed_fraction": 0.800293,
"rank": 4
},
{
"from_timestamp_sec": 12.5,
"timestamp_sec": 12.75,
"from_timestamp_provenance": {
"requested_timestamp": 12.5,
"decoded_timestamp": null,
"timestamp_delta": null
},
"timestamp_provenance": {
"requested_timestamp": 12.75,
"decoded_timestamp": null,
"timestamp_delta": null
},
"change_score": 0.559718,
"mean_abs_diff": 0.429651,
"changed_fraction": 0.80127,
"rank": 5
}
]
}
}
Question and answer
- Question
- When does the scoreboard change?
- Answer
- The tool reports changes in the box at 3, 6, 9.25, 12.75, 16 seconds.
Check passedprecision 1.00 (min 1), recall 1.00 (min 1), tolerance 0.5s
How it was checked
- Truth from
- constructed
- Fixed at
Planted by this recipe before any tool ran: ffmpeg scaled the source's first 20 s to 640x360 px and overlaid a 176x64 px scoreboard-like box at x=121, y=228 px (Pillow pictures: dark slate body, amber body in the planted intervals, white bars in both). The box changes at 2.88, 5.9, 9.31, 12.66, 16.03 s, the state switching at every time. The tolerance 0.5 s covers the tool's 0.25 s sample step.
A script drew the box over a fixed spot and changed its colour and brightness at 5 times it chose, so the answer is those planted times, not a real scoreboard; the tool takes the region as fractions of the frame, so the pixel box was converted.
Try it yourself
Use agentic-video on my video. A box in the picture is 176 by 64 pixels, with its top-left corner at x 121 px and y 228 px, in a 640 by 360 px frame. Convert that area to fractions of the frame and call track_region_changes on the first 20.02 seconds with sample_fps 4 and max_candidates 5. Tell me the times, in seconds, when the box changes colour.
- Get the .mcpb file and open it in Claude Desktop.
- Or:
pip install agentic-video
Clip and licence
- Licence
- CC-BY-SA-4.0
- Author
- Sea Cow
- Source
- commons.wikimedia.org
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.