Gateway

Gateway API reference

The central orchestrator for all LLM calls.

Constructor

Gateway(
    limits: ExecutionLimits | None = None,
    loop_detection: LoopDetectionConfig | None = None,
    tool_access: ToolAccessController | None = None,
    checkpoint_enabled: bool = False,
    on_before_call: Callable | None = None,
    on_after_call: Callable | None = None,
)

Parameter	Type	Description
limits	ExecutionLimits	Budget/step/time limits
loop_detection	LoopDetectionConfig	Loop detection config
tool_access	ToolAccessController	Tool access rules
checkpoint_enabled	bool	Enable checkpointing
on_before_call	Callable	Hook before each call
on_after_call	Callable	Hook after each call

Methods

configure_provider

gateway.configure_provider(
    provider: str | LLMProvider,
    api_key: str | None = None,
    **kwargs
) -> Gateway

Configure an LLM provider.

call

await gateway.call(
    agent_id: str,
    provider: str | LLMProvider,
    model: str,
    messages: list[LLMMessage],
    tools: list[dict] | None = None,
    temperature: float | None = None,
    max_tokens: int | None = None,
    state: SharedState | dict | None = None,
    workflow_id: str | None = None,
    step: int | None = None,
) -> LLMResponse

Make an LLM call through the gateway.

call_with_tools

await gateway.call_with_tools(
    agent_id: str,
    provider: str | LLMProvider,
    model: str,
    messages: list[LLMMessage],
    tools: list[dict],
    tool_executor: Callable,
    max_iterations: int = 10,
    state: dict | None = None,
) -> LLMResponse

Make an LLM call with automatic tool execution loop.

get_metrics

gateway.get_metrics() -> dict

Returns:

{
    "total_cost": 0.05,
    "total_steps": 10,
    "total_tokens": 5000,
    "input_tokens": 3000,
    "output_tokens": 2000,
    "elapsed_seconds": 30.5,
    "remaining": {...},
    "call_count": 10,
}

get_call_history

gateway.get_call_history(
    agent_id: str | None = None,
    limit: int | None = None,
) -> list[CallRecord]

reset

gateway.reset() -> None

Reset all state, metrics, and history.

Properties

metrics

gateway.metrics -> ExecutionMetrics

Current execution metrics.

CallRecord

Record of a single LLM call.

Properties

Property	Type	Description
agent_id	str	Agent that made the call
provider	LLMProvider	Provider used
model	str	Model used
request	LLMRequest	The request
response	LLMResponse	The response
error	`str	None`	Error message if failed
timestamp	datetime	When the call was made
success	bool	Whether call succeeded
cost	float	Cost of the call
latency_ms	float	Latency in milliseconds