OpenAI Python SDK through FastFence

This client sends text chat to FastFence's /v1/chat/completions. FastFence applies its policy before calling the configured model provider. The client credential is a FastFence agent token, not your upstream provider key.

Download the complete examples into your installation's examples/ directory and use the activated FastFence virtual environment. Run commands from the installation directory.

Run with Ollama

Follow installation, start Ollama, pull qwen3:4b for Laya and qwen3:0.6b for completion, then run fastfence serve. Set FASTFENCE_AGENT_TOKEN to your provisioned agent credential.

python -m pip install openai
python examples/openai_client.py 'Hi'

The script prints the protected model response. Set FASTFENCE_MODEL if your allowlisted model differs. Set FASTFENCE_URL if the gateway listens elsewhere; this remains the gateway URL, never the upstream URL.

Complete client

"""Use the OpenAI Python SDK against FastFence, with no direct provider bypass."""

import argparse
import os

from openai import APIStatusError, OpenAI


def complete(client: OpenAI, model: str, prompt: str) -> str:
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=256,
        temperature=0,
        stream=False,
    )
    return response.choices[0].message.content or ""


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("prompt", nargs="?", default="Hi")
    args = parser.parse_args()
    with OpenAI(
        base_url=os.getenv("FASTFENCE_URL", "http://127.0.0.1:8000").rstrip("/")
        + "/v1",
        api_key=os.environ["FASTFENCE_AGENT_TOKEN"],
        timeout=90,
        max_retries=0,
    ) as client:
        try:
            print(
                complete(
                    client,
                    os.getenv("FASTFENCE_MODEL", "qwen3:0.6b"),
                    args.prompt,
                )
            )
        except APIStatusError as error:
            # No automatic retry or fallback to an unprotected provider.
            print(
                f"FastFence returned HTTP {error.status_code}; inspect Activity for the decision."
            )
            raise SystemExit(1) from None


if __name__ == "__main__":
    main()

Download openai_client.py · View source

Verify blocking

In Policies, add a model-input literal rule for forbidden, test and activate it. Then run:

python examples/openai_client.py 'forbidden'

Expected: nonzero exit and an HTTP denial. Activity shows the input rule and upstream_executed: false. The client disables automatic retries and never falls back to a direct provider.

Use a different model backend

Keep this client unchanged and follow OpenAI-compatible upstream configuration on the gateway. The gateway's provider key remains server-side. Laya's semantic model is configured separately.

Supported here: non-streaming, text-only chat at temperature zero. Streaming, tool-call generation and multimodal messages are not implemented by this compatibility adapter. Use REST if you need the complete security verdict in the response.