Articles in this section

Monitor AI agent and guardrail activity with execution logs

Teena Killa
Updated

This article explains how to view and use logs related to AI agents and Guardrails in your environment.  When you use AI agents and guardrails, you can track exactly what happened during a request using Execution logs. Logs help you:

  • Trace a request from start to finish
  • Understand why an agent responded a certain way
  • See whether guardrails allowed or blocked content
  • Identify errors and retries
  • Monitor token usage and AI credit consumption

The log data captured by log level is available in your standard logging tools on your Run history page and can be searched using a trace key, timestamp, or agent name.

Execution logs, audit logs, and error records

Execution logs, audit logs, and error records serve different purposes. Knowing which one to check helps you find the right evidence for troubleshooting, security review, or compliance reporting.

  • Execution logs record what happened while an AI agent or Guardrail ran, including the model used, Tools invoked, and Guardrail decisions.
  • Guardrail evaluation details record policy results, violations, severity, and enforcement actions for a specific guardrail check. See "Logs for Guardrails" below.
  • Audit logs record user or administrator activity and configuration changes, such as who created, edited, or deployed an AI agent, Tool, or Guardrail.
  • Error records support troubleshooting and retry workflows for failed or interrupted requests. See Investigate & troubleshoot errors.

Best practices

  • Keep basic logging enabled at all times so you always have visibility into activity, usage, and policy outcomes.
  • Turn on detailed logging only when you need deeper diagnostics. This approach gives you clear, actionable insight into how your agents and guardrails behave while keeping production environments efficient and controlled.
  • With structured logging in place, you can confidently monitor, debug, and optimise your agents and guardrails using the same tools and standards you already rely on.

Logs for AI agents

When you trigger an agent, you can see the following:

  • Invocation details: Agent name, trace ID, timestamp, and triggering flow or source.
  • LLM generation details: Model used, request duration, input and output token counts, and credit usage.
  • Tool calls: Tool name, input and output summary, success or failure status, retry attempts, and cost (if applicable).
  • Errors and recovery: Error type, retry attempts, and outcome.
  • Detailed logging only (optional): Full prompt content, full model output, and internal reasoning traces.

You can view three logging levels, depending on how much detail you need.

Basic logging (default)

This level gives you everything required for monitoring and production support. You can see:

  • When an agent is invoked, it captures the agent name, timestamp, trace key, model used, and service tier
  • Token and credit usage - input, output, total tokens, credits consumed, model parameters
  • Tool usage - which tool ran, status, success/failure, cost
  • Errors - error type and occurrence details

Use this level to:

  • Monitor production activity
  • Track usage and cost
  • Investigate failures
  • Confirm that requests completed successfully

Detailed logging (opt-in)

Enable this only when you need deep troubleshooting or behavioural analysis. In addition to basic logs, you can see:

  • Agent reasoning trace - how the agent reached its decision
  • Tool execution trace - tool inputs/outputs, retries, validation results, fallback behavior
  • Model output - full generated and processed responses
  • Raw LLM request and response

This level may include sensitive content and should only be enabled in controlled environments.

Debug mode

When debug mode is enabled, it generates detailed execution logs for all configured agent hooks (e.g., pre-map, mapping, and post-map hooks).

Logs for guardrails

Guardrails logs help you verify policy enforcement and understand decision outcomes. When Guardrails are evaluated, you can see:

  • Evaluation details: Guardrails policy name, trigger conditions, and input context.
  • Decision results: Pass or fail result and decision timestamp
  • Violations: Rule violated, severity level, and action taken.
  • Enforcement actions: Corrective action executed and outcome status.
  • Detailed logging only (optional): Rule evaluation reasoning and additional context metadata.

You control the level of detail you see.

Basic logging (default)

You can see the following:

  • Which guardrails policy was evaluated
  • Whether the request passed or failed
  • Any violations detected. Severity level
  • Enforcement action taken. Outcome

Use this level to:

  • Confirm compliance behaviour
  • Understand why content was blocked or modified
  • Monitor policy effectiveness

Detailed logging (opt-in)

This level includes everything in basic logging plus full evaluation detail. It should only be enabled under strict governance controls. When enabled, you can also see:

  • Detailed rule evaluation reasoning
  • Guardrail (PII)
  • Guardrail (Moderation)
  • Guardrails (AI agent/custom)
  • Additional evaluation context
  • Full raw request sent to guardrails
  • Full raw response returned by guardrails

Enable this only when investigating unexpected policy behaviour.

Debug mode

When debug mode is enabled, detailed execution logs are generated for all configured guardrail hooks (e.g., pre-map, mapping, and post-map hooks).

Review execution and Guardrail data for compliance reporting

To gather evidence for an internal or external compliance review, you:

  1. Select the AI agent execution to review.
  2. Identify the model and Tools used in that execution.
  3. Review the applicable Guardrail evaluation for that execution.
  4. Confirm whether the Guardrail passed or failed.
  5. Review the enforcement action taken.
  6. Capture the evidence needed for the compliance review (execution details, Guardrail evaluation, and enforcement outcome).

Retention and edition availability

Execution and Guardrail log data follows the same retention and edition rules as standard execution logs. You can configure the retention period (7, 15, 30, 60, or 180 days); the default is 30 days. Standard and Professional editions support Basic (default), Detailed, and Debug logging, though Debug requires an add-on. Enterprise editions include Debug logging by default, with no add-on required. For more information, see Introduction to execution logs.

You can export execution and Guardrail log data for a specific run as a downloadable diagnostics file from the run's Actions menu in Run history. This is a per-run manual download, not a bulk CSV export or a dedicated reporting API.

Separation of duties

AI agents and Guardrails are built and run as import steps within a flow, so building or editing an agent and configuring a Guardrail follow the same permission model as any other import or export step in integrator.io: they require Manage access to that flow (Manage all at the account level, or Manage scoped to that specific integration). Deploying or enabling the agent falls under this same Manage-level requirement, since enabling a flow requires the same access as editing it. Monitor-level users can view these components but cannot build, edit, or enable them.

Reviewing execution logs follows the same flow-level split: any user with Manage or Monitor access to a flow can view its execution logs. Monitor-level users can view logs but cannot purge stored run logs and payloads; purging is restricted to Admins, the account Owner, and users with Manage access (account-wide or for that specific integration).

Viewing sensitive prompts, full model output, or other payload-level detail is gated in two steps rather than by a distinct role. First, an Owner or Admin must enable Store payload consent for the environment in Execution log preferences, a decision that is itself written to the account audit log. Until that consent is granted, no one, including Manage-level users, can select Detailed or Debug logging on a flow. Once consent is granted, any user with Manage access to that specific flow can switch its log mode to Detailed or Debug and see the resulting payloads, headers, and (in Debug mode) reasoning traces. There is no separate, narrower role that restricts this beyond ordinary flow-level Manage access once the account-level consent gate is open.

Trace runtime activity and configuration changes

Execution logs and audit logs answer different questions, and a complete investigation may require both:

  • Execution logs show what happened during a specific run: the model used, Tools invoked, and Guardrail outcomes.
  • Audit logs show who changed the agent, Tool, model, prompt, connection, or Guardrail configuration, and when.

Example: If you're investigating a failed or blocked agent execution, start with the execution log for that run to see which Guardrail or Tool caused the failure. Then check the Audit Log history for that agent (or its Tools or Guardrails) around the same timestamp to determine whether a recent configuration change such as an edited Guardrail rule or a modified Tool contributed to the outcome.

Related articles