OpenAI Python SDK through FastFence

This client sends text chat to FastFence's /v1/chat/completions. FastFence applies its policy before calling the configured model provider. The client credential is a FastFence agent token, not your upstream provider key.

Download the complete examples into your installation's examples/ directory. Run commands from the installation directory; uv run supplies Python 3.12 and the FastFence package for each example, without activating a virtual environment.

Run with Ollama

Follow installation: start Ollama, run uv tool run --python 3.12 [email protected] init --anonymization, then uv tool run --python 3.12 [email protected] serve. Initialization installs Laya and prepares the default Qwen3:4b model. Set FASTFENCE_AGENT_TOKEN to your provisioned agent credential.

uv run --python 3.12 --no-project --with fastfence==1.0.1 --with openai==2.21.0 python examples/openai_client.py 'Hi'

The script prints the protected model response. Set FASTFENCE_MODEL if your allowlisted model differs. Set FASTFENCE_URL if the gateway listens elsewhere; this remains the gateway URL, never the upstream URL.

Complete client

"""Use the OpenAI Python SDK against FastFence, with no direct provider bypass."""

import argparse
import os

from openai import APIStatusError, OpenAI


def complete(client: OpenAI, model: str, prompt: str) -> str:
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=256,
        temperature=0,
        stream=False,
    )
    return response.choices[0].message.content or ""


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("prompt", nargs="?", default="Hi")
    args = parser.parse_args()
    with OpenAI(
        base_url=os.getenv("FASTFENCE_URL", "http://127.0.0.1:8000").rstrip("/")
        + "/v1",
        api_key=os.environ["FASTFENCE_AGENT_TOKEN"],
        timeout=90,
        max_retries=0,
    ) as client:
        try:
            print(
                complete(
                    client,
                    os.getenv("FASTFENCE_MODEL", "qwen3:4b"),
                    args.prompt,
                )
            )
        except APIStatusError as error:
            # No automatic retry or fallback to an unprotected provider.
            print(
                f"FastFence returned HTTP {error.status_code}; inspect Activity for the decision."
            )
            raise SystemExit(1) from None


if __name__ == "__main__":
    main()

Download openai_client.py · View source

Verify blocking

In Policies, add a model-input literal rule for forbidden, test and activate it. Then run:

uv run --python 3.12 --no-project --with fastfence==1.0.1 --with openai==2.21.0 python examples/openai_client.py 'forbidden'

Expected: nonzero exit and an HTTP denial. Activity shows the input rule and upstream_executed: false. The client disables automatic retries and never falls back to a direct provider.

Use a different model backend

Keep this client unchanged and follow OpenAI-compatible upstream configuration on the gateway. The gateway's provider key remains server-side. Laya's semantic model is configured separately.

Supported here: non-streaming, text-only chat at temperature zero. Streaming, tool-call generation and multimodal messages are not implemented by this compatibility adapter. Use REST if you need the complete security verdict in the response.