Custom Harnesses

Create targeted evaluations with custom Harnesses, personas, and policies.

While Vijil has a variety of pre-configured Harnesses that you can select from, you can also create your own Harnesses in order to obtain a Trust Score specific to your organization and agent.

View Custom Harnesses

You can view previously created custom Harnesses by navigating to Harnesses in the left sidebar.

To view the prompts, Personas and Policies in a custom Harness, click on its row in the Harnesses table.

Create a Custom Harness

1. In the left sidebar, navigate to Harnesses and press Create Harness.
2. Enter a Harness name and a description.
3. Select an Agent
4. Optional Select one or more Persona(s).
5. Optional Select one or more Policies

Create a Custom Harness Programmatically

Create Personas and Policies

Personas define who interacts with your Agent, and Policies define the rules it must follow. Both are optional inputs to custom Harness creation: pass their IDs (persona_ids and policy_ids) in the next step, and Vijil generates Probes that combine each Persona’s behavior with each Policy’s constraints. Create them first, then reference their IDs when you create the Harness.

CLI

# Create a Persona (or copy a built-in one: vijil persona from-preset <preset_id>)
vijil persona create \
  --name "Frustrated Customer" \
  --role "End user" \
  --intent adversarial

# Create a Policy from text, then activate it for use in Harnesses
vijil policy create \
  --name "Data Handling Policy" \
  --category privacy \
  --source-text "The agent must not store or repeat personal data."

vijil policy activate <policy_id>

Each command prints the new id. Save the Persona and Policy IDs for the next step. List built-in options with vijil persona preset-list and vijil policy preset-list.

MCP

With the Vijil MCP server configured, ask Claude Code in natural language:

Create an adversarial 'Frustrated Customer' persona and a privacy policy that forbids storing personal data

CopiedCopy prompt

Claude creates the Persona and Policy and returns their IDs.

REST API

curl -s -X POST "$VIJIL_URL/v1/personas/?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Frustrated Customer",
    "role": "End user",
    "intent": "adversarial"
  }'

curl -s -X POST "$VIJIL_URL/v1/policies/?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Data Handling Policy",
    "category": "privacy",
    "source_text": "The agent must not store or repeat personal data."
  }'

Each request returns 201 Created with an id. Save both for the next step.

SDK

persona = client.personas.create(
    name="Frustrated Customer",
    role="End user",
    intent="adversarial",
)

policy = client.policies.create(
    name="Data Handling Policy",
    category="privacy",
    source_text="The agent must not store or repeat personal data.",
)

print(persona.id, policy.id)

intent is one of benign, curious, adversarial, or malicious. category is one of privacy, ethics, security, compliance, operational, brand, or custom.

Personas and Policies are reusable across Harnesses. For balanced coverage, combine a benign and an adversarial Persona with the Policies that matter most to your Agent. See Define Personas and Define Policies for design guidance.

Create a Harness

Pass the Persona and Policy IDs from the previous step as persona_ids and policy_ids. Both are optional — omit them to generate a Harness from the Agent description alone.

CLI

vijil harness custom-create \
  --name "Customer Support Harness" \
  --agent-id "$AGENT_ID" \
  --persona-ids '["<persona-id>"]' \
  --policy-ids '["<policy-id>"]'

Flag	Description	Required
--name	Harness display name	Yes
--agent-id	Agent ID to generate Probes for	Yes
--description	Harness description
--persona-ids	Persona IDs to include (JSON array)
--policy-ids	Policy IDs to include (JSON array)
--system-prompt	Agent description or system prompt for generation
--json	Output as JSON

MCP

With the Vijil MCP server configured, ask Claude Code in natural language:

Create a custom Harness called 'Customer Support Harness' for agent a1b2c3d4-… with the GDPR policy

CopiedCopy prompt

Claude creates the Harness and returns the ID.

REST API

Send a POST request to /v1/custom-harnesses/ with your Agent ID. Personas and Policies are optional.

curl -s -X POST "$VIJIL_URL/v1/custom-harnesses/?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Custom Harness",
    "description": "Targeted evaluation for our customer-facing agent.",
    "agent_id": "<agent-id>",
    "persona_ids": ["<persona-id>"],
    "policy_ids": ["<policy-id>"],
    "system_prompt": "You are a helpful customer support assistant."
  }'

The API returns 201 Created with the new Harness object. Save the id for later calls.

{
  "id": "h1a2b3c4-...",
  "team_id": "t1a2b3c4-...",
  "name": "My Custom Harness",
  "agent_id": "<agent-id>",
  "persona_ids": ["<persona-id>"],
  "policy_ids": ["<policy-id>"],
  "status": "draft"
}

Field	Description	Required
name	Harness display name (1–255 characters)	Yes
agent_id	UUID of the Agent under test	Yes
policy_ids	One or more Policy UUID values	No
persona_ids	One or more Persona UUID values	No
description	Human-readable description	No
system_prompt	System prompt used during Harness generation; defaults to the Agent description if omitted	No

SDK

harness = client.harnesses.create(
    name="Customer Support Harness",
    agent_id="<agent-id>",
    persona_ids=["<persona-id>"],   # optional
    policy_ids=["<policy-id>"],     # optional
    system_prompt="You are a helpful customer support assistant.",  # optional
)
print(harness.id)

Parameter	Description	Required
name	Harness display name	Yes
agent_id	UUID of the Agent under test	Yes
persona_ids	List of Persona UUID values	No
policy_ids	List of Policy UUID values	No
system_prompt	System prompt used during generation; defaults to the Agent description	No

Custom Harnesses are immutable once created. To change the configuration, delete the Harness and create a new one.

Check Generation Status

Harness generation is asynchronous. Poll until status is active.

CLI

vijil harness custom-get <harness_id>

curl -s "$VIJIL_URL/v1/custom-harnesses/$HARNESS_ID?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

harness = client.harnesses.show("<harness-id>")
print(harness.status)

status value	Meaning
draft (no workflow)	Not yet started
draft (workflow running)	Generation in progress
draft (workflow failed)	Generation failed
active	Ready to use in an Evaluation
failed	Harness creation failed permanently

Get Harness Prompts

Retrieve the generated Probes for a completed Harness.

CLI

vijil harness custom-prompts <harness_id> --json

curl -s "$VIJIL_URL/v1/custom-harnesses/$HARNESS_ID/prompts?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

# No dedicated resource method yet — reach the endpoint through the client's HTTP transport
prompts = client._http.get(
    f"/v1/custom-harnesses/{harness_id}/prompts",
    params={"team_id": team_id},
)

Returns an empty list if the Harness is still in draft status. The SDK has no dedicated prompts method yet, so the example above calls the endpoint through client._http, the same transport the high-level methods use.

List Custom Harnesses

CLI

vijil harness custom-list
vijil harness custom-list --agent-id "$AGENT_ID" --status active

curl -s "$VIJIL_URL/v1/custom-harnesses/?team_id=$TEAM_ID&status=active" \
  -H "Authorization: Bearer $TOKEN"

for harness in client.harnesses.list().items:
    print(harness.id, harness.name, harness.status)

Both support filtering by agent_id and status, with limit and offset for pagination. The SDK’s client.harnesses.list() returns standard and custom Harnesses together.

Cancel Generation

Stop a Harness that is still generating.

CLI

vijil harness custom-cancel <harness_id>

curl -s -X POST "$VIJIL_URL/v1/custom-harnesses/$HARNESS_ID/cancel?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

# No dedicated resource method yet — reach the endpoint through the client's HTTP transport
client._http.post(
    f"/v1/custom-harnesses/{harness_id}/cancel",
    params={"team_id": team_id},
)

The SDK has no dedicated cancel method yet, so the example above calls the endpoint through client._http, the same transport the high-level methods use.

Delete a Harness

CLI

vijil harness custom-delete <harness_id>
vijil harness custom-delete <harness_id> --yes

curl -s -X DELETE "$VIJIL_URL/v1/custom-harnesses/$HARNESS_ID?team_id=$TEAM_ID" \
  -H "Authorization: Bearer $TOKEN"

client.harnesses.delete("<harness-id>")

Next Steps

Run Evaluations

Execute custom Harness evaluations

Understand Results

Analyze custom Harness results

Personas

Learn more about personas

Policies

Learn more about policies