Vision-Language Model (VLM) for Semantic Video Detection and Search

Real-Time Semantic Detection with VLM

Simply describe what you want to detect — whether it’s “a running human” or “a woman wearing a black dress.” The AI-powered Meta-Detector VLM processes the query and identifies matching scenes as they appear in the video feed, offering seamless and precise scene detection. In search mode, the user enters a text query, and the system translates it into the metadata representation used by the neural network to find the closest matches.

Real-time scene recognition

Detect specific objects and events as they happen and receive alarms when a text query matches the scene description in live video.

Flexible query structure

Combine objects, attributes (size, color, etc.), environment, and positioning for precise detection.

No additional training required

The neural network understands diverse scenarios out-of-the-box, helping users work with both live alerts and retrospective search without creating custom models for each task.

VLM Bridges Human Intent and Machine Execution

Instead of configuring complex rules, you simply describe what you need, and the system does the rest. This makes video analytics more accessible, more flexible, and easier to adapt to changing operational needs.

In live mode, Meta-Detector VLM works as an online tool: the user enters a text query and receives an alarm when that query matches the frame description in real time.

For retrospective search, users can enable metadata storage on the Meta-Detector side. This makes the frame descriptions available for archive search later, without requiring any detectors to be configured in advance. Search is performed across all recorded metadata generated by the neural network during video recording.

Examples of Detection

• Detect “a package left at the entrance”.
• Identify “a person climbing a fence near the loading dock”.
• Recognize “a white car at a crosswalk from a top view”.
• Search archived footage for “person wearing pink”.

Meta-Search in Recorded Video

Once metadata storage is enabled, users can open the search tab, select Meta-Search VLM, and work with it in the same intuitive way as in detector settings. They enter a text query, and the system returns frames and video fragments whose stored metadata best matches that request.

Users can also set similarity in percent to refine search results. This makes it possible to broaden or narrow the match depending on how specific the query is and how many relevant results are needed.

Meta-search does not generate events by itself. Its role is to use previously stored metadata for retrospective retrieval. If dedicated detectors are already configured and generating events in real time, then search can be performed through those events instead.

The Value of VLM in Axxon One VMS

AI-driven Vision-Language Model capabilities transform traditional surveillance systems into more intelligent, more flexible video management platforms. By combining Meta-Detector VLM for live semantic detection with Meta-Search VLM for archive retrieval, Axxon One enables:

• Real-time scene understanding for live video streams.
• Faster response times to security threats.
• Improved accuracy in identifying complex scenes.
• Less time spent on manual review and configuration.

By allowing users to interact with both live and recorded video through human-readable descriptions, Axxon One VLM provides an effective approach to real-time scene understanding and AI-powered archive search. This extends video analytics from event detection toward semantic video understanding inside a full VMS workflow.