agentic-video Gallery

Name the sound in one 5-second window of a generated test clip

SoundSyntheticVerified
0:30generated · generated

Proof frames

Click a frame to seek

Tool calls

1 calls · agentic-video 0.3.1
label_soundsmacbook · 2248 ms

Arguments

{
  "source": "$CLIP",
  "start_sec": 0,
  "end_sec": 5,
  "window_sec": 5,
  "labels": [
    "siren",
    "church bells",
    "car horn",
    "dog barking",
    "clapping",
    "train",
    "helicopter",
    "glass breaking",
    "thunderstorm",
    "rooster crowing"
  ]
}

Result

{
  "ok": true,
  "data": {
    "source_path": "~/Code/agentic-video-gallery/cards/label-synthetic/clip.mp4",
    "start_sec": 0,
    "end_sec": 5,
    "window_sec": 5,
    "model": "clef-omni",
    "labels": [
      "siren",
      "church bells",
      "car horn",
      "dog barking",
      "clapping",
      "train",
      "helicopter",
      "glass breaking",
      "thunderstorm",
      "rooster crowing"
    ],
    "windows": [
      {
        "start_sec": 0,
        "end_sec": 5,
        "top_label": "siren",
        "top_chance": 0.7629,
        "none_chance": 0.035,
        "chances": {
          "siren": 0.7629,
          "church bells": 0.0197,
          "car horn": 0.0407,
          "dog barking": 0.0168,
          "clapping": 0.0136,
          "train": 0.0223,
          "helicopter": 0.031,
          "glass breaking": 0.0155,
          "thunderstorm": 0.0234,
          "rooster crowing": 0.0191
        }
      }
    ],
    "input_tokens": 385,
    "complete": true,
    "next_start_sec": null,
    "note": "Audio was sent to Cloudflare Workers AI. Chances are the model's own and sum to 1 with none_chance; top_label is null when 'none of these' is more likely than every label. Sounds outside `labels` are forced onto the nearest label or none, so give every sound that may be present."
  }
}

Question and answer

Question
What sound is heard from 0 to 5 seconds?
Answer
The tool names the sound in the 5-second window from 0 to 5 seconds as siren.
Check passedrequired labels present, no forbidden labels

How it was checked

Truth from
constructed
Fixed at

Planted by this recipe: ffmpeg cut the first 30 s of the source and mixed a synthetic siren (pitch sweep 550 to 1250 Hz, 3.1 s long, starting at 0.8 s) over the source's own audio at gain 0.2. The siren lies inside the 5 s window from 0 s to 5 s, the window that holds its start. The truth is the label siren for that window; the call covers that same window.

A script mixed a synthetic siren over this generated test clip's own sound at a time it chose, and the answer is the top label of that one 5-second window, so the card does not show every sound in the clip.

Try it yourself

Use agentic-video to name the sound in my video from 0 to 5 seconds. Call label_sounds on that range with 5-second windows and this list of sounds: siren, church bells, car horn, dog barking, clapping, train, helicopter, glass breaking, thunderstorm, rooster crowing. Tell me the sound it names. Use my own file.
  1. Get the .mcpb file and open it in Claude Desktop.
  2. Or: pip install agentic-video

Clip and licence

Licence
generated

Generated for the gallery; no third-party material.