Name the sound in one 5-second window of game footage
0:30CC-BY-SA-3.0 · Wildfire Games
Proof frames
Tool calls
label_sounds
Arguments
{
"source": "$CLIP",
"start_sec": 10,
"end_sec": 15,
"window_sec": 5,
"labels": [
"siren",
"church bells",
"car horn",
"dog barking",
"clapping",
"train",
"helicopter",
"glass breaking",
"thunderstorm",
"rooster crowing"
]
}
Result
{
"ok": true,
"data": {
"source_path": "~/Code/agentic-video-gallery/cards/label-gaming/clip.mp4",
"start_sec": 10,
"end_sec": 15,
"window_sec": 5,
"model": "clef-omni",
"labels": [
"siren",
"church bells",
"car horn",
"dog barking",
"clapping",
"train",
"helicopter",
"glass breaking",
"thunderstorm",
"rooster crowing"
],
"windows": [
{
"start_sec": 10,
"end_sec": 15,
"top_label": "siren",
"top_chance": 0.7834,
"none_chance": 0.0197,
"chances": {
"siren": 0.7834,
"church bells": 0.0168,
"car horn": 0.0301,
"dog barking": 0.0209,
"clapping": 0.0163,
"train": 0.024,
"helicopter": 0.0332,
"glass breaking": 0.019,
"thunderstorm": 0.0181,
"rooster crowing": 0.0185
}
}
],
"input_tokens": 385,
"complete": true,
"next_start_sec": null,
"note": "Audio was sent to Cloudflare Workers AI. Chances are the model's own and sum to 1 with none_chance; top_label is null when 'none of these' is more likely than every label. Sounds outside `labels` are forced onto the nearest label or none, so give every sound that may be present."
}
}
Question and answer
- Question
- What sound is heard from 10 to 15 seconds?
- Answer
- The tool names the sound in the 5-second window from 10 to 15 seconds as siren.
Check passedrequired labels present, no forbidden labels
How it was checked
- Truth from
- constructed
- Fixed at
Planted by this recipe: ffmpeg cut the first 30 s of the source and mixed a synthetic siren (pitch sweep 550 to 1250 Hz, 3.1 s long, starting at 10.8 s) over the source's own audio at gain 0.2. The siren lies inside the 5 s window from 10 s to 15 s, the window that holds its start. The truth is the label siren for that window; the call covers that same window.
A script mixed a synthetic siren over this game clip's own music at a time it chose, and the answer is the top label of that one 5-second window, so the card does not show every sound in the clip.
Try it yourself
Use agentic-video to name the sound in my video from 10 to 15 seconds. Call label_sounds on that range with 5-second windows and this list of sounds: siren, church bells, car horn, dog barking, clapping, train, helicopter, glass breaking, thunderstorm, rooster crowing. Tell me the sound it names. Use my own file.
- Get the .mcpb file and open it in Claude Desktop.
- Or:
pip install agentic-video
Clip and licence
- Licence
- CC-BY-SA-3.0
- Author
- Wildfire Games
- Source
- commons.wikimedia.org
This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.