AI Detection#

1. Document Information#

Item	Details
Manual Version	V1.5.0
Last Updated	2026-03-02
Software Version	AIBox V1.4.0 and above
Scope	AIBox Full Edition (macOS / Linux / Docker deployment)

Target Audience#

Role	Recommended Chapters
Regular User	3 – 6
System Admin	3 – 8
Operations	3, 7, 8, 9

Conventions#

Symbol	Meaning
【Button】	Clickable button in the UI, e.g. 【Save】, 【Create】
"Menu"	Left sidebar navigation menu item, e.g. "Task Management"
Field	Form input field name
⚠️	Important notice
💡	Tips and tricks
🔒	Requires admin privileges

---

2. Product Overview#

2.1 Introduction#

AIBox (Smart Video AI Detection Platform) is an intelligent video analysis platform that integrates video stream ingestion, AI-powered detection, real-time alerting, and streaming output. The system supports RTSP cameras and video streams, leverages multiple AI algorithms for real-time video content detection, and delivers results via Webhook callbacks, video overlay output, and more.

2.2 Main Modules#

Module	Description
Dashboard	System status overview including sources, tasks, detection stats, and resource usage
Video Wall	Multi-view real-time video monitoring with customizable grid layouts
Input Sources	Add and manage RTSP cameras and video stream URLs
Output Streams	Configure FFmpeg stream output (RTSP/RTMP/RTP)
Model Management	Upload and manage YOLO detection model files
Task Management	Create, start, stop, and monitor AI detection tasks
Detection Results	View, filter, and export detection results and screenshots
Callback Management	Configure Webhook callback endpoints with JWT authentication
LLM Analysis	Use LLM vision models for intelligent image analysis
LLM Providers	Configure LLM service providers (OpenAI / Ollama, etc.)
User Management	Manage system user accounts and roles
System Config	Global system parameter settings

2.3 Supported Detection Algorithms#

Algorithm	Description
YOLO Object Detection	General object detection (pedestrians, vehicles, animals, 80+ categories)
Intrusion Detection	Detect target intrusion within designated zones
License Plate Recognition	Detect license plates and recognize numbers via OCR
Line Crossing Counter	Count targets crossing a specified line
Open Vocabulary Detection (YOLO-World)	Custom object detection based on text keywords
Text-Prompted Detection (GroundingDINO)	Object detection based on natural language descriptions
HuggingFace Models	Load models from HuggingFace Hub for classification/detection

2.4 Use Case Examples#

• Campus Security: Connect surveillance cameras to detect unauthorized intrusion and crowd gathering
• Traffic Management: License plate recognition, traffic flow counting, illegal parking detection
• Industrial Safety: Detect workers without hard hats, smoke/flame recognition
• Drone Monitoring: Automatic aerial target detection and tracking
• Smart City Management: Street vendor detection, garbage pile detection, and more

---

3. System Requirements#

3.1 Browser Support#

Browser	Minimum Version	Recommended
Google Chrome	90+	✅ Recommended
Microsoft Edge	90+	✅ Recommended
Mozilla Firefox	85+	✓ Supported
Safari	15+	✓ Supported

⚠️ Video preview relies on HTTP-FLV playback (mpegts.js). Chrome or Edge is recommended for the best experience.

3.2 Network Requirements#

• The client must have network connectivity to the platform server
• Video preview requires access to the ZLMediaKit streaming server (default HTTP port 8080)
• RTSP cameras must be on a network accessible from the platform server

3.3 Resolution Recommendations#

• Minimum screen resolution: 1280×720
• Best experience: 1920×1080 or higher

3.4 Accounts & Permissions#

The system supports two roles:

Role	Permissions
Regular User	View dashboard, video wall, sources, streams, tasks, results, callback logs, LLM analysis
Admin	All of the above + create/edit/delete operations + user management + system config + LLM providers

💡 Default admin account: username admin, password set during deployment. Contact your operations team for initial login credentials.

---

4. Login & Logout#

4.1 Access URL#

Enter the platform URL in your browser:

For example: http://ai-box.your-domain.com or http://aibox.your-domain.com

4.2 Login Steps#

1. Open your browser and navigate to the platform URL to reach the login page

2. Enter your username in the Username field

3. Enter your password in the Password field (click the eye icon to reveal the password)

4. Click the 【Login】 button

5. Upon successful login, you will be redirected to the Dashboard

💡 You can switch the interface language in the top-right corner of the login page: 简体中文 / 繁體中文 / English

4.3 Token-Based Auto Login#

You can use a URL parameter to pass a JWT token for automatic login, ideal for third-party system embedding:

The token is automatically saved to the browser's local storage and removed from the URL.

4.4 Password Reset#

Regular users cannot reset their own passwords. Please contact the system admin. Admins can edit user profiles and set new passwords in "User Management".

4.5 Logout#

Click the 【Logout】 button in the top-right corner to sign out. The page will automatically redirect to the login page.

⚠️ Login tokens have an expiration time. After prolonged inactivity, you will be automatically redirected to the login page.

---

5. Interface Overview#

5.1 Overall Layout#

The system interface consists of three parts:

5.2 Sidebar Navigation#

The left sidebar displays all feature module menus. Click the collapse/expand button to toggle the sidebar.

Icon	Menu	Description	Access
📊	Dashboard	System overview	All users
🖥️	Video Wall	Multi-view live monitoring	All users (can be hidden in system config)
📹	Input Sources	RTSP source management	All users (editing requires admin)
🔗	Output Streams	Stream output management	All users (editing requires admin)
📦	Models	AI model management	All users (upload requires admin)
📋	Tasks	Detection task management	All users (creation requires admin)
🖼️	Results	Detection results viewer	All users
🔗	Callbacks	Webhook callback config	All users (editing requires admin)
🔍	LLM Analysis	LLM vision analysis	All users
🔗	LLM Providers	LLM provider config	🔒 Admin
👤	Users	User account management	🔒 Admin
⚙️	System Config	Global system settings	🔒 Admin

5.3 Top Bar#

From left to right:

• Collapse Button: Toggle sidebar visibility
• Welcome Message: Displays the current logged-in username
• Theme Toggle: Light mode ☀️ / Dark mode 🌙
• Language Selector: Switch interface language (简体中文 / 繁體中文 / English)
• Logout Button: Sign out

5.4 Common Operations#

Operation	Description
Add	Click the 【Add】/【Create】 button at the top, fill in the dialog form, and submit
Edit	Click the ✏️ edit button in the table row's action column
Delete	Click the 🗑️ delete button; a confirmation dialog will appear
Batch Delete	Select multiple rows and click the 【Batch Delete】 button
Filter	Use the filter controls at the top (dropdowns, search box) to filter data
Pagination	Switch page size (10/20/50/100) and navigate pages at the table bottom
Export	Click the 【Export】 button to download data as an Excel file

---

6. Core Features#

6.1 Dashboard#

Purpose: System status overview with key metrics and statistical charts.

Access: Automatically displayed after login, or click "Dashboard" in the sidebar.

Page Content:

Stat Cards (top row):

Card	Description
Online Sources	Number of online/total input sources
Running Tasks	Number of running/total detection tasks
Total Detections	Cumulative detection result count
CPU Usage	Current server CPU utilization

Charts (2×2 grid):

• Task Status Distribution: Pie chart showing task counts by status (pending/running/stopped/failed/completed)
• Detection Category Stats: Bar chart showing detection count distribution by category
• Detection Trend: Line chart showing detection count over time
• Algorithm Distribution: Pie chart showing algorithm type usage across tasks

System Status (bottom cards):

• CPU / Memory / Disk usage (ring charts)
• Server IP address
• License status and remaining days

💡 Dashboard data refreshes automatically every 10 seconds.

---

6.2 Video Wall#

Purpose: Multi-view real-time video monitoring with customizable grid layouts.

Access: Click "Video Wall" in the sidebar.

Steps:

1. Select Grid Layout: Choose a layout from the "Grid Preset" in the bottom control bar (1×1 / 2×2 / 3×3 / 4×4 / Custom)
2. Assign Video Streams: Click the dropdown menu in the top-right corner of each grid cell and select a source or output stream
3. Save Layout: Click 【Save Layout】 and enter a layout name to save the current configuration
4. Switch Layout: Select a saved layout from the "Layout Management" dropdown

Notes:

• Custom grids support up to 6×6 (36 views)
• Layout configurations are stored in the browser's local storage; they need to be reconfigured after clearing cache or switching browsers
• Video preview depends on ZLMediaKit; ensure the streaming service is running

---

6.3 Input Sources#

Purpose: Manage RTSP cameras and video stream URLs connected to the system.

Access: Click "Input Sources" in the sidebar.

Add Source 🔒#

1. Click 【Add Source】 at the top of the page

2. Fill in the dialog:

   • Source Name: A recognizable name for the camera (required)
   • RTSP URL: The camera's RTSP stream URL (required), e.g. rtsp://user:pass@IP:port/path
   • Description: Notes (optional)

3. Click 【Save】

Test Connection#

Click the 【Play】 button in the table row to open a live video preview window. Click 【Test Connection】 to check if the RTSP stream is available; on success, resolution, frame rate, and codec info will be displayed.

Edit / Delete 🔒#

• Click the ✏️ edit button to modify source information
• Click the 🗑️ delete button to remove a source (a confirmation prompt will appear)

⚠️ If a source is referenced by a detection task, deleting it may affect the related task.

---

6.4 Output Streams#

Purpose: Configure streaming output URLs for detection result video (supports RTSP / RTMP / RTP).

Access: Click "Output Streams" in the sidebar.

Add Output Stream 🔒#

1. Click 【Add Output Stream】
2. Fill in:
   • Stream Name: Identifier for the output stream (required)
   • Stream Type: Protocol type — RTSP / RTMP / RTP (required)
   • Stream URL: Target push URL (required), e.g.:
     • RTSP: rtsp://server-ip:554/live/stream1
     • RTMP: rtmp://server-ip:1935/live/stream1
   • Status: Enable/disable toggle
   • Description: Notes
3. Click 【Save】

Preview Output Stream#

Click the 【Play】 button in the table to preview the push stream via HTTP-FLV in a popup window.

💡 Output streams push detection-overlaid video to a streaming server. You can bind a detection task to an output stream in "Task Management".

---

6.5 Model Management#

Purpose: Upload and manage YOLO series detection model files.

Access: Click "Models" in the sidebar.

Upload Model 🔒#

1. Click 【Upload Model】
2. Fill in:
   • Model Name: Identifier for the model (required)
   • Version: Model version number, e.g. v8n, v8s, v11n
   • Description: Model usage description
   • Model File: Click the upload area to select a .pt model file
3. Click 【Submit】 and wait for the upload to complete

⚠️ Model files are typically large (tens to hundreds of MB). Please be patient during upload and do not close the page.

💡 The system includes pre-installed YOLOv8n and YOLOv11n models that can be used directly for detection tasks. Upload custom-trained models here if needed.

---

6.6 Task Management#

Purpose: Create, configure, and start/stop AI detection tasks. This is the most critical module of the system.

Access: Click "Tasks" in the sidebar.

Create Detection Task 🔒#

1. Click 【Create】
2. Configure in the dialog:

Basic Information:

Field	Description
Task Name	Task identifier (required)
Input Source	Select from added sources (required)

Algorithm Configuration (dynamically displays options based on the selected algorithm type):

Algorithm Type	Configurable Options
YOLO Object Detection	Model selection, confidence threshold
Intrusion Detection	Model, confidence threshold, detection zone drawing, person-only mode
License Plate Recognition	Model, confidence threshold, OCR toggle, OCR language, save plate images
Line Crossing Counter	Model, confidence threshold, line drawing, direction, target types
Open Vocabulary Detection	Detection keywords (comma-separated), confidence threshold
Text-Prompted Detection	Detection keywords, confidence threshold
HuggingFace Model	Model repo name, task type, Top-K, confidence threshold

Advanced Configuration:

Field	Description
Frame Skip	Detect every N frames; higher = better performance but less real-time (0 = every frame)
Output Stream	Toggle + select stream to overlay detection results and push
Overlay Output	Whether to overlay detection boxes and labels on the output stream
Callback	Toggle + select callback config to trigger Webhook on detection
Event Detection	Only record results when targets are detected (reduces unnecessary data)
Video Recording	Automatically record video clips when events are detected
Clip Duration	Duration of recorded video clips (1–30 seconds)

3. Click 【OK】 to save the task

Draw Detection Zone / Counting Line#

For "Intrusion Detection" and "Line Crossing Counter" algorithms, you need to draw detection zones or counting lines:

1. Click 【Draw Detection Zone】 or 【Draw Counting Line】 in the algorithm config area
2. A drawing panel opens with a live camera snapshot as the background
3. Drawing operations:
   • Left-click to add vertices
   • Zone drawing: At least 3 points to form a polygon; right-click or double-click to finish
   • Line drawing: 2 points to form a line segment
4. Click 【Finish Drawing】 to save

Start / Stop Task#

• Click ▶️ to start a detection task
• Click ⏸ to stop a running task

Auto-Start#

Each task has an "Auto-Start" toggle (switch in the table). When enabled, the task will automatically resume when the system restarts.

View Task Logs#

Click the 📋 log button to view detailed task execution logs in a popup window, useful for troubleshooting.

Update Detection Keywords#

For "Open Vocabulary Detection" and "Text-Prompted Detection" tasks, click the 🪄 button to update detection keywords in real-time without restarting the task.

View Output Stream#

For running tasks with an output stream configured, click 🖥️ to preview the live video with detection results overlaid.

💡 A frame skip value of 3–5 is recommended for a good balance between detection accuracy and system resource usage.

---

6.7 Detection Results#

Purpose: View, filter, and export AI detection result records.

Access: Click "Results" in the sidebar, or click the 📄 view results button in Task Management.

Filter Results#

Filter controls are available at the top of the page:

• Filter by Task: Select a specific task from the dropdown
• Filter by Model: Select a specific model from the dropdown
• Search by Category: Enter detection category keywords (e.g. "person", "car")
• Click 【Reset】 to clear all filters

View Result Details#

• Thumbnail: Each record displays a detection screenshot thumbnail; click to view the full-size image
• Detected Objects: Shows detected target categories and confidence (up to 3 displayed, excess shown as "+N")
• Additional Info: Extended information (e.g. line crossing counts, license plate numbers)
• Video Clips: If video recording is enabled, click the play button to watch event videos

Export Results#

Click the 【Export】 button at the top to export the current filtered results as an Excel (.xlsx) file.

Delete Results 🔒#

• Single delete: Click the delete button in the row's action column
• Batch delete: Select multiple records and click 【Delete(N)】 at the top

💡 The detection results page auto-refreshes every 30 seconds to display the latest data.

---

6.8 Callback Management#

Purpose: Configure Webhook callback endpoints to automatically push detection results to third-party systems.

Access: Click "Callbacks" in the sidebar.

Add Callback Configuration 🔒#

1. Click 【Add】
2. Fill in the basic configuration:
   • Name: Callback configuration name (required)
   • URL: HTTP endpoint URL to receive callbacks (required)
   • Method: HTTP request method (POST / GET / PUT / PATCH / DELETE)
   • Status: Enable/disable toggle
3. For JWT authentication, enable "Enable JWT" and configure:
   • Enter Token Directly: Fill in the JWT Token field with an existing token
   • Auto Login to Obtain: Provide the JWT login URL, username, password, and token path; the system will automatically obtain and refresh the token
4. Click 【Confirm】 to save

Test Callback#

Click the 【Test】 button in the table to send a test message to the configured URL and verify connectivity.

View Callback Logs#

Click the 【Logs】 button to view the push history for a callback configuration, including:

• Push time, URL, request method
• Push status (success/failure)
• HTTP status code
• Request body and response body (click 【Details】 to view full content)

💡 When creating a detection task, you can associate a callback configuration. Results will be automatically POSTed to the callback URL when detected.

---

6.9 LLM Analysis#

Purpose: Use LLM vision models (e.g. GPT-4o) to intelligently analyze uploaded images and perform object detection.

Access: Click "LLM Analysis" in the sidebar.

Steps:

1. Enter a detection prompt in the Prompt field (e.g. "Detect all vehicles in the image", "Identify safety hazards")
2. Select a configured LLM provider from the Provider dropdown
3. Click 【Select Image】 to upload an image for analysis
4. Click 【Send】 to start the analysis
5. When complete:
   • Detection Results: Detected target bounding boxes and labels are automatically drawn on the image
   • Model Response: Displays the LLM's raw text response and token usage

⚠️ You must configure at least one LLM provider in "LLM Providers" before using LLM Analysis.

---

7. Admin Features#

🔒 The following features are only visible to and operable by admins.

7.1 User Management#

Purpose: Manage system user accounts, including creating, editing, and deleting users.

Access: Click "Users" in the sidebar.

Add User#

1. Click 【Add User】
2. Fill in user information:
   • Username: Login username (required, must be unique)
   • Email: User email (must be a valid email format)
   • Password: Login password (required, 8+ characters recommended)
   • Role: Regular User / Admin
3. Click 【Save】

Edit User#

Click the edit button on a user row to modify username, email, and role. Leave the password field empty to keep the current password.

Delete User#

Click the delete button and confirm to delete the user account.

⚠️ Do not delete all admin accounts, or you will be unable to access the admin panel.

---

7.2 LLM Provider Management#

Purpose: Configure LLM (Large Language Model) service provider connection information.

Access: Click "LLM Providers" in the sidebar.

Add Provider#

1. Click 【Add Provider】
2. Fill in the configuration:

Field	Description
Provider Type	OpenAI / Ollama
Provider Name	Custom name
Base URL	API endpoint URL (OpenAI default: https://api.openai.com/v1)
API Key	API key (not required for Ollama)
Organization	Organization ID (optional)
Project	Project ID (optional)
Model	Model name (e.g. gpt-4o, gpt-4o-mini; supports manual input of custom model names)
Description	Notes

3. Click 【Submit】

Test Provider#

Click 【Test】 to open a test dialog:

• Enter a prompt and upload an image
• Click 【Send】 to verify model availability
• View the model response and token consumption

💡 To use OpenAI API-compatible third-party services (e.g. Azure OpenAI, Qwen, etc.), select the OpenAI type and modify the Base URL.

---

7.3 System Configuration#

Purpose: Global system parameter settings.

Access: Click "System Config" in the sidebar.

Configurable Options:

Setting	Description	Default
System Name	Platform name displayed on the login page and sidebar	Smart Video AI Detection Platform
Hash Comparison Window	Time window for adjacent frame deduplication (seconds)	2.0
Hash Difference Threshold	Threshold for determining different frames	0.3
Send Email on Startup	Whether to send notification emails when the system starts	Off
Detection Alert Sound	Whether to play a sound when a detection alert triggers	On
Show Video Wall Menu	Whether to show the Video Wall entry in the sidebar	On
ZLMediaKit Server Address	Streaming server HTTP address	http://zlm.your-domain.com:8080
ZLMediaKit API Secret	Streaming server management API secret	Preset value

Click 【Save】 at the top of the page to apply changes.

⚠️ After changing the ZLMediaKit address, all video preview features will use the new server address. Ensure it is correct and accessible.

---

8. FAQ#

Q1: Can't log in?#

Possible Cause	Solution
Wrong username or password	Verify input, mind case sensitivity. Contact admin to reset password
Backend service not running	Contact ops to verify backend service status
Network unreachable	Check if you can ping the server IP
Token expired	Clear browser cache and log in again

Q2: Page won't load / loads slowly?#

Possible Cause	Solution
Frontend service not running	Contact ops to verify Nginx / frontend service status
Browser cache issue	Press Ctrl + Shift + R (Mac: Cmd + Shift + R) to force refresh
High network latency	Check network connection quality
First model load	First task start requires loading the AI model, which may take 30–60 seconds

Q3: Video preview is black / won't play?#

Possible Cause	Solution
ZLMediaKit not running	Contact ops to verify streaming service status
ZLMediaKit address misconfigured	Admin should check the ZLMediaKit address in "System Config"
RTSP source offline	Click 【Test Connection】 in "Input Sources" to check stream availability
Unsupported browser	Use Chrome or Edge

Q4: Detection task fails to start?#

Possible Cause	Solution
RTSP source offline	Confirm camera is online and check RTSP URL
Model file missing	Verify the selected model has been uploaded and the file is intact
Insufficient resources	Check CPU / memory usage; stop unnecessary tasks
Algorithm misconfiguration	Check algorithm parameters (detection zone, keywords, etc.)

Q5: No detection results?#

Possible Cause	Solution
Task not running	Confirm task status is "Running"
Confidence threshold too high	Lower the confidence threshold (recommended: 0.3–0.5)
Keywords don't match	Check keyword settings for open vocabulary / text-prompted algorithms
Frame skip too high	Reduce frame skip to increase detection frequency
Event detection enabled	Event detection mode only records when target changes are detected

Q6: Export / upload fails?#

Possible Cause	Solution
Too much data	Use filters to narrow the export range and retry
Model file too large	Ensure network stability; don't close the page during upload
Insufficient disk space	Contact ops to free up server disk space

Q7: Insufficient permissions?#

Regular users cannot perform create, edit, or delete operations — the corresponding buttons are not displayed on the page. Contact your system admin to upgrade your role to admin if you need these permissions.

Q8: Callback not received?#

Possible Cause	Solution
Callback not enabled	Check if the callback configuration "Status" is set to enabled
Task not linked to callback	Edit the task, enable "Callback" and select a callback config
Callback URL unreachable	Use the 【Test】 button to verify connectivity
JWT Token expired	Check error messages in callback logs and update JWT configuration

---

9. Appendix#

Appendix A: Glossary#

Term	Definition
RTSP	Real Time Streaming Protocol — used for camera video stream transmission
RTMP	Real Time Messaging Protocol — commonly used for live streaming
YOLO	You Only Look Once — an efficient real-time object detection algorithm
GroundingDINO	Text-prompted open-set object detection model
YOLO-World	Open vocabulary YOLO object detection model
ZLMediaKit	High-performance streaming media server used for video stream proxying and HTTP-FLV forwarding
HTTP-FLV	FLV video stream over HTTP, used for low-latency browser video playback
OCR	Optical Character Recognition
LLM	Large Language Model
JWT	JSON Web Token — an identity authentication token standard
Webhook	A callback mechanism that automatically sends HTTP requests to a specified URL when events occur
Confidence	The model's certainty level for a detection result, range 0–1, higher means more certain
Frame Skip	Execute detection every N frames to reduce CPU/GPU load
Hash Dedup	Filter duplicate frame detection results through image hash comparison

Appendix B: URL Parameters#

The system supports URL query parameters to control interface behavior:

Parameter	Values	Effect
secret	JWT string	Auto-login (token stored locally, then removed from URL)
redirect	Path (e.g. /tasks)	Redirect to specified page after login
theme	light / dark / system	Override current theme setting
show_content_only	1 / true / yes / on	Hide sidebar and top bar (Kiosk/embed mode)

Example:

Appendix C: Shortcuts#

Action	Description
Ctrl + Shift + R	Force refresh page (clear cache)
Right-click / Double-click	Finish zone drawing
Click thumbnail	View full-size detection result image
Toggle auto-start switch	Switch directly in the task list, no need to open edit dialog

Appendix D: Contact#

If you encounter issues not covered in this manual, please contact technical support:

• Technical Support Email: support@geotwin.cn
• Operations Contact: Please consult your system administrator

---

This manual was last updated on March 2, 2026. Please refer to the actual system functionality if you have any questions.