3.4 Model Configuration

Model configuration is a key step after first deploying Flocks. After model configuration is complete, Rex, Agents, Workflows, Task Center, and IM channels can call large-model capabilities normally.

This page describes the model configuration sequence after deployment. For complete Provider, model instance, default model, per-Agent model assignment, and model error troubleshooting information, see Model List.

1. Configuration Goals

Model configuration must complete three things:

1. Add an accessible model Provider.
2. Add at least one callable model under that Provider.
3. Select a default model in the Default Model card at the top left of the Model List page.

After this is done, Flocks can automatically use the default model when creating sessions, running Agents, executing Workflows, or creating tasks.

2. Recommended Configuration Order

For first-time configuration, use this order:

1. Open the WebUI and go to Model List.
2. Add a model Provider and fill in Base URL and API Key.
3. Add a model under the Provider. The model name must match the actual model ID on the server.
4. Click Test Connection and confirm the model can be called normally.
5. Click edit in the Default Model card at the top left of the page, then select the Provider and model.
6. Create a new session, send a simple question, and confirm Rex replies normally.

If you only save a Provider or model without passing the connection test, the model is usually not proven to be connected.

3. Recommended Model Selection

3.1 Prefer ThreatBook Recommended Models

If you use the ThreatBook model platform, prefer selecting recommended models from the ThreatBook-cn-llm Provider in Model List. These models are organized for Flocks scenarios such as the main Agent, sub-agents, long context, and cost-sensitive tasks.

Common selection patterns:

• Main Agent Rex: prioritize models with strong reasoning, long context, and stable tool calling.
• Critical Workflow: prioritize models with better stability.
• sub-agent: choose 7B, 30B, or 70B-class models based on task complexity.
• Large batches of logs, alerts, and asset data: prioritize long-context models.

3.2 Local Model Size Guidance

If you use a locally self-hosted model, the main Agent Rex is recommended to use a model larger than 200B parameters. The main Agent is responsible for understanding goals, decomposing tasks, calling tools, delegating to sub-agents, and integrating results, so it has the highest model capability requirements.

sub-agent task boundaries are narrower and can generally use smaller models to reduce cost and latency.

4. Default Model and Agent Models

The system default model affects:

• New WebUI sessions.
• Rex default execution.
• Agents without an individually specified model.
• New tasks in Task Center.
• Model inference nodes or Agent nodes in Workflows where no model is specified.
• New Sessions created through IM channels.

Each Agent can also specify its own model on the Agent page. Once specified, that Agent will not follow system default model changes. It will follow the default model again only after being changed back to "System Default".

5. How to Confirm Configuration Success

Use these items as acceptance checks:

• The Provider and model are visible in the model list.
• The target model passes the connection test.
• The Default Model card at the top left shows the correct model.
• Rex replies normally in a new WebUI session.
• Agents using the system default model execute normally.
• If an Agent has an individually specified model, the Agent details show that model and task execution calls it normally.

If a new session cannot reply, first check Base URL, API Key, model name, model service status, and network connectivity.

6. Recommended Deployment Configuration for Models

The following is the recommended deployment configuration for minimax2.7 (for reference only):

• Parameters: 230B total, 10B active, default context 196608.
• With FP16 quantization, 128K context, and concurrency 10, VRAM usage is about 713.13 GB.

Recommended deployment hardware:

1. NVDIA H20-96G x 8
2. NVDIA H200-141G x 6
3. NVDIA B100-192G or B200-192G x 4
4. NVDIA A100-80G x 12

Scaling estimates:

• Each additional concurrent request increases VRAM by about 50~100 GB.
• Doubling the context increases VRAM by about 50~100 GB.

Software requirements:

1. vllm / sglang and other inference frameworks
2. NCCL
3. Transformers
4. Pytorch @A

7. FAQ

7.1 Why is the model still unusable after I save it?

Saving a model only means the configuration has been written. It does not prove the model service is available. Click Test Connection and verify with a real call in a new session.

7.2 What if the default model is cleared?

If the current default model is deleted, or its Provider is deleted, the system automatically clears the default model. Return to Model List and select it again in the Default Model card at the top left.

7.3 Why did an Agent not follow the new default model?

If an Agent has been manually assigned a model, it will not follow system default model changes. Go to the Agent page and change that Agent's model back to "System Default".

8. Related Pages

• Model List: manage Providers, model instances, default models, and model test results.
• Agent: specify a model for an individual Agent.
• Model Error Troubleshooting: troubleshoot Provider, model name, authentication, context, and call errors.