teff.provider

teff.provider

Providers — wire protocols, presets, and the registry.

A :class:Provider is a named endpoint: its type selects the wire protocol (openai_compatible / anthropic_compatible / ollama) and base_url/chat_path/auth keys point at the endpoint. Providers are declared in a workflow's providers: block, registered once in a :class:ProviderRegistry, or picked from the built-in presets in :data:DEFAULT_PROVIDERS.

Modules:

Name	Description
builtin	Built-in provider presets — subclasses of :class:Provider.
concurrency	Global per-provider concurrency guards.
providers	Backward-compatible alias for the :mod:teff.provider package.
registry	The :class:ProviderRegistry and the built-in preset catalogue.
resolve	Provider resolution helpers shared by the harness and graph layers.

Classes:

Name	Description
Provider	A named model endpoint: wire protocol + endpoint data.
ProviderRegistry	A named collection of providers.

Functions:

Name	Description
provider_concurrency	Return the current global concurrency limit for provider (if any).
resolve_provider	Resolve a provider key from an explicit value or a default name.
resolve_provider_entry	Resolve the effective :class:Provider for provider_key.
set_provider_concurrency	Globally cap concurrent model calls for provider.
to_provider_registry	Normalize providers into a :class:ProviderRegistry.
validate_provider_refs	Enforce that every provider reference is declared in providers.

Provider

A named model endpoint: wire protocol + endpoint data.

name is the registry key used by provider= references. type is the protocol discriminator — openai_compatible / anthropic_compatible / ollama — and decides the request body, streaming chunk parsing, and response extraction held by :class:~teff.harness.Harness.

Built-in presets subclass this and set name (and the other fields) once; a custom provider is a plain instance. Fields may be overridden at construction:

Provider(name="my-vllm", type="openai_compatible", base_url="http://vllm:8000/v1")

type is deliberately a distinct concept from name: the name is just a key and never carries protocol meaning.

Methods:

Name	Description
from_mapping	Build from a config dict, keeping only known fields.
to_dict	All provider fields as a plain dict (for YAML serialisation).

Source code in teff/provider/builtin/base.py

class Provider:
    """A named model endpoint: wire protocol + endpoint data.

    ``name`` is the registry key used by ``provider=`` references.
    ``type`` is the protocol discriminator — ``openai_compatible`` /
    ``anthropic_compatible`` / ``ollama`` — and decides the request body,
    streaming chunk parsing, and response extraction held by
    :class:`~teff.harness.Harness`.

    Built-in presets subclass this and set ``name`` (and the other fields)
    once; a custom provider is a plain instance.  Fields may be overridden
    at construction:

        Provider(name="my-vllm", type="openai_compatible", base_url="http://vllm:8000/v1")

    ``type`` is deliberately a distinct concept from ``name``: the name is
    just a key and never carries protocol meaning.
    """

    name: str = ""
    type: str = "openai_compatible"
    base_url: str = ""
    chat_path: str = "/chat/completions"
    api_key_env: str = ""
    auth_header: str = "Authorization"
    auth_prefix: str = "Bearer "
    timeout: float = 120.0

    def __init__(self, **overrides):
        unknown = set(overrides) - set(PROVIDER_FIELDS)
        if unknown:
            raise TypeError(f"unknown Provider field(s): {', '.join(sorted(unknown))}")
        for field in PROVIDER_FIELDS:
            if field in overrides:
                setattr(self, field, overrides[field])

    @classmethod
    def from_mapping(cls, cfg: dict) -> "Provider":
        """Build from a config dict, keeping only known fields."""
        return cls(**{f: cfg[f] for f in PROVIDER_FIELDS if f in cfg})

    def to_dict(self) -> dict:
        """All provider fields as a plain dict (for YAML serialisation)."""
        return {f: getattr(self, f) for f in PROVIDER_FIELDS}

    def __repr__(self) -> str:
        shown = ", ".join(
            f"{f}={getattr(self, f)!r}" for f in PROVIDER_FIELDS if getattr(self, f)
        )
        return f"{type(self).__name__}({shown})"

from_mapping classmethod

from_mapping(cfg)

Build from a config dict, keeping only known fields.

Source code in teff/provider/builtin/base.py

@classmethod
def from_mapping(cls, cfg: dict) -> "Provider":
    """Build from a config dict, keeping only known fields."""
    return cls(**{f: cfg[f] for f in PROVIDER_FIELDS if f in cfg})

to_dict

to_dict()

All provider fields as a plain dict (for YAML serialisation).

Source code in teff/provider/builtin/base.py

def to_dict(self) -> dict:
    """All provider fields as a plain dict (for YAML serialisation)."""
    return {f: getattr(self, f) for f in PROVIDER_FIELDS}

ProviderRegistry

A named collection of providers.

Register providers once and reference them by name anywhere a provider key is expected. The registry starts empty and is authoritative: an unregistered name raises :class:ConfigError rather than silently loading a built-in preset — every provider a graph uses must be declared here.

Because the canonical input is a dict {name: Provider}, the same value can also be built from one and consumed with :func:dict-style lookups, so graph.run(state, providers={...}) keeps working.

Example::

from teff import Provider, ProviderRegistry

reg = ProviderRegistry()
reg.register(Provider(name="vllm", base_url="http://vllm:8000/v1"))
reg.register(AnthropicCompatibleProvider(name="claude-proxy", base_url="http://proxy"))

graph = Graph(
    {"llm": LLM(model="m", provider="claude-proxy")}, [], "llm",
    providers=reg,
    default_provider="claude-proxy",
)
await graph.run({})

Registering a name that is already registered (including by a previous call) raises :class:ConfigError — names are unique.

Methods:

Name	Description
from_presets	Build a registry holding the named built-in presets.
items	Registered (name, provider) pairs (custom entries only).
register	Add provider under provider.name; returns self for chaining.
resolve	Resolve name to a registered :class:Provider.

Source code in teff/provider/registry.py

class ProviderRegistry:
    """A named collection of providers.

    Register providers once and reference them by name anywhere a provider
    key is expected.  The registry starts empty and is authoritative: an
    unregistered name raises :class:`ConfigError` rather than silently
    loading a built-in preset — every provider a graph uses must be
    declared here.

    Because the canonical input is a dict ``{name: Provider}``, the same
    value can also be built from one and consumed with :func:`dict`-style
    lookups, so ``graph.run(state, providers={...})`` keeps working.

    Example::

        from teff import Provider, ProviderRegistry

        reg = ProviderRegistry()
        reg.register(Provider(name="vllm", base_url="http://vllm:8000/v1"))
        reg.register(AnthropicCompatibleProvider(name="claude-proxy", base_url="http://proxy"))

        graph = Graph(
            {"llm": LLM(model="m", provider="claude-proxy")}, [], "llm",
            providers=reg,
            default_provider="claude-proxy",
        )
        await graph.run({})

    Registering a name that is already registered (including by a previous
    call) raises :class:`ConfigError` — names are unique.
    """

    def __init__(self, providers: "dict[str, Provider] | None" = None):
        self._entries: dict[str, Provider] = {}
        if providers:
            for name, provider in providers.items():
                if not provider.name:
                    provider.name = name
                elif provider.name != name:
                    raise ConfigError(
                        f"provider name mismatch: registered as {name!r} "
                        f"but Provider.name is {provider.name!r}"
                    )
                self.register(provider)

    def register(self, provider: Provider) -> "ProviderRegistry":
        """Add *provider* under ``provider.name``; returns self for chaining.

        Raises:
            ConfigError: If ``provider.name`` is empty or already registered.
        """
        if not provider.name:
            raise ConfigError("cannot register a Provider without a name")
        if provider.name in self._entries:
            raise ConfigError(f"provider {provider.name!r} is already registered")
        self._entries[provider.name] = provider
        return self

    def resolve(self, name: str) -> Provider:
        """Resolve *name* to a registered :class:`Provider`.

        Only explicitly registered providers are usable.  Unknown names
        raise :class:`ConfigError`.
        """
        if name in self._entries:
            return self._entries[name]
        raise ConfigError(
            f"unknown provider: {name!r} — declare it by registering it in "
            "the ProviderRegistry, a `providers=`/`providers:` block, or "
            f"naming a built-in preset ({', '.join(BUILTINS)})"
        )

    @classmethod
    def from_presets(cls, *names: str) -> "ProviderRegistry":
        """Build a registry holding the named built-in presets.

        Convenience for declaring built-ins explicitly::

            reg = ProviderRegistry.from_presets("openai", "ollama")

        An unknown preset name raises :class:`ConfigError`.
        """
        reg = cls()
        for name in names:
            preset = BUILTINS.get(name)
            if preset is None:
                raise ConfigError(
                    f"unknown preset: {name!r} — pick from {', '.join(BUILTINS)}"
                )
            reg.register(preset())
        return reg

    def items(self) -> list[tuple[str, Provider]]:
        """Registered ``(name, provider)`` pairs (custom entries only)."""
        return list(self._entries.items())

    def __contains__(self, name: object) -> bool:
        return name in self._entries

    def __getitem__(self, name: str) -> Provider:
        return self._entries[name]

    def __iter__(self):
        return iter(self._entries)

    def __len__(self) -> int:
        return len(self._entries)

from_presets classmethod

from_presets(*names)

Build a registry holding the named built-in presets.

Convenience for declaring built-ins explicitly::

reg = ProviderRegistry.from_presets("openai", "ollama")

An unknown preset name raises :class:ConfigError.

Source code in teff/provider/registry.py

@classmethod
def from_presets(cls, *names: str) -> "ProviderRegistry":
    """Build a registry holding the named built-in presets.

    Convenience for declaring built-ins explicitly::

        reg = ProviderRegistry.from_presets("openai", "ollama")

    An unknown preset name raises :class:`ConfigError`.
    """
    reg = cls()
    for name in names:
        preset = BUILTINS.get(name)
        if preset is None:
            raise ConfigError(
                f"unknown preset: {name!r} — pick from {', '.join(BUILTINS)}"
            )
        reg.register(preset())
    return reg

items

items()

Registered (name, provider) pairs (custom entries only).

Source code in teff/provider/registry.py

def items(self) -> list[tuple[str, Provider]]:
    """Registered ``(name, provider)`` pairs (custom entries only)."""
    return list(self._entries.items())

register

register(provider)

Add provider under provider.name; returns self for chaining.

Raises:

Type	Description
ConfigError	If provider.name is empty or already registered.

Source code in teff/provider/registry.py

def register(self, provider: Provider) -> "ProviderRegistry":
    """Add *provider* under ``provider.name``; returns self for chaining.

    Raises:
        ConfigError: If ``provider.name`` is empty or already registered.
    """
    if not provider.name:
        raise ConfigError("cannot register a Provider without a name")
    if provider.name in self._entries:
        raise ConfigError(f"provider {provider.name!r} is already registered")
    self._entries[provider.name] = provider
    return self

resolve

resolve(name)

Resolve name to a registered :class:Provider.

Only explicitly registered providers are usable. Unknown names raise :class:ConfigError.

Source code in teff/provider/registry.py

def resolve(self, name: str) -> Provider:
    """Resolve *name* to a registered :class:`Provider`.

    Only explicitly registered providers are usable.  Unknown names
    raise :class:`ConfigError`.
    """
    if name in self._entries:
        return self._entries[name]
    raise ConfigError(
        f"unknown provider: {name!r} — declare it by registering it in "
        "the ProviderRegistry, a `providers=`/`providers:` block, or "
        f"naming a built-in preset ({', '.join(BUILTINS)})"
    )

provider_concurrency

provider_concurrency(provider)

Return the current global concurrency limit for provider (if any).

Returns the active semaphore's capacity (explicit or auto-grown via max_parallel), or None when the provider has no semaphore.

Source code in teff/provider/concurrency.py

def provider_concurrency(provider: str) -> int | None:
    """Return the current global concurrency limit for *provider* (if any).

    Returns the active semaphore's capacity (explicit or auto-grown via
    ``max_parallel``), or ``None`` when the provider has no semaphore.
    """
    return _PROVIDER_LIMITS.get(provider.lower())

resolve_provider

resolve_provider(provider=None, default_provider=None)

Resolve a provider key from an explicit value or a default name.

The explicit provider (node-level) wins; otherwise default_provider (the graph-level default) is used. Model-name auto-detection was removed — a provider must be stated explicitly.

Raises:

Type	Description
ConfigError	When neither a provider nor a default is configured.

Source code in teff/provider/resolve.py

def resolve_provider(
    provider: str | None = None, default_provider: str | None = None
) -> str:
    """Resolve a provider key from an explicit value or a default name.

    The explicit *provider* (node-level) wins; otherwise *default_provider*
    (the graph-level default) is used.  Model-name auto-detection was removed
    — a provider must be stated explicitly.

    Raises:
        ConfigError: When neither a provider nor a default is configured.
    """
    p = provider or default_provider
    if not p:
        raise ConfigError(
            "no provider configured: set `provider=` on the node, pass "
            "`default_provider=` to the graph, or declare a top-level "
            "`default_provider:` in the workflow"
        )
    return p.lower()

resolve_provider_entry

resolve_provider_entry(provider_key, providers=None)

Resolve the effective :class:Provider for provider_key.

When providers is a :class:ProviderRegistry or dict it is authoritative — provider_key must be declared in it. With None (a bare, standalone Harness) a built-in preset is used. Unknown names raise a :class:ConfigError — there is no silent fallback to the OpenAI shape, so typos surface early instead of silently routing to the wrong wire protocol.

Raises:

Type	Description
ConfigError	When provider_key is neither declared in providers nor (with providers=None) a built-in preset name.

Source code in teff/provider/resolve.py

def resolve_provider_entry(
    provider_key: str,
    providers: "dict[str, Provider] | ProviderRegistry | None" = None,
) -> Provider:
    """Resolve the effective :class:`Provider` for *provider_key*.

    When *providers* is a :class:`ProviderRegistry` or dict it is
    authoritative — *provider_key* must be declared in it.  With ``None``
    (a bare, standalone ``Harness``) a built-in preset is used.  Unknown
    names raise a :class:`ConfigError` — there is no silent fallback to the
    OpenAI shape, so typos surface early instead of silently routing to the
    wrong wire protocol.

    Raises:
        ConfigError: When *provider_key* is neither declared in *providers*
            nor (with ``providers=None``) a built-in preset name.
    """
    if isinstance(providers, ProviderRegistry):
        return providers.resolve(provider_key)
    if providers and provider_key in providers:
        return providers[provider_key]
    if providers is None:
        preset = BUILTINS.get(provider_key)
        if preset is not None:
            return preset()
    raise ConfigError(
        f"unknown provider: {provider_key!r} — declare it in the `providers=` "
        f"map / `providers:` block, or name a built-in preset "
        f"({', '.join(BUILTINS)})"
    )

set_provider_concurrency

set_provider_concurrency(provider, limit)

Globally cap concurrent model calls for provider.

Overrides any per-harness max_parallel for that provider. Pass limit <= 0 to remove the cap.

Source code in teff/provider/concurrency.py

def set_provider_concurrency(provider: str, limit: int) -> None:
    """Globally cap concurrent model calls for *provider*.

    Overrides any per-harness ``max_parallel`` for that provider.
    Pass ``limit <= 0`` to remove the cap.
    """
    provider = provider.lower()
    if limit <= 0:
        _EXPLICIT_LIMITS.pop(provider, None)
        _PROVIDER_SEMAPHORES.pop(provider, None)
        _PROVIDER_LIMITS.pop(provider, None)
        _PROVIDER_ACTIVE.pop(provider, None)
    else:
        _EXPLICIT_LIMITS[provider] = limit
        _PROVIDER_SEMAPHORES[provider] = asyncio.Semaphore(limit)
        _PROVIDER_LIMITS[provider] = limit
        _PROVIDER_ACTIVE[provider] = 0

to_provider_registry

to_provider_registry(providers)

Normalize providers into a :class:ProviderRegistry.

Accepts an existing :class:ProviderRegistry, a {name: Provider} dict, or None (empty registry). There is no string shorthand — every provider must be an explicit instance, so graph.providers truthfully reflects what is configured.

Source code in teff/provider/resolve.py

def to_provider_registry(
    providers: "ProviderRegistry | dict[str, Provider] | None",
) -> ProviderRegistry:
    """Normalize *providers* into a :class:`ProviderRegistry`.

    Accepts an existing :class:`ProviderRegistry`, a ``{name: Provider}``
    dict, or ``None`` (empty registry).  There is no string shorthand —
    every provider must be an explicit instance, so ``graph.providers``
    truthfully reflects what is configured.
    """
    if isinstance(providers, ProviderRegistry):
        return providers
    if providers is None:
        return ProviderRegistry()
    return ProviderRegistry(providers)

validate_provider_refs

validate_provider_refs(providers, default_provider=None, nodes=None)

Enforce that every provider reference is declared in providers.

default_provider and each node's config.provider must name a provider registered in providers — there is no implicit built-in fallback. Raises :class:ConfigError on the first undeclared name.

Source code in teff/provider/resolve.py

def validate_provider_refs(
    providers: ProviderRegistry,
    default_provider: str | None = None,
    nodes: "dict[str, Provider] | dict | None" = None,
) -> None:
    """Enforce that every provider reference is declared in *providers*.

    *default_provider* and each node's ``config.provider`` must name a
    provider registered in *providers* — there is no implicit built-in
    fallback.  Raises :class:`ConfigError` on the first undeclared name.
    """
    valid = set(providers)
    if default_provider and default_provider not in valid:
        raise ConfigError(
            f"default_provider {default_provider!r} is not declared in "
            "`providers=` / `providers:`"
        )
    for nid, node in (nodes or {}).items():
        cfg = getattr(node, "config", None) or {}
        prov = cfg.get("provider")
        if prov and prov not in valid:
            raise ConfigError(
                f"node {nid!r}: provider {prov!r} is not declared in "
                "`providers=` / `providers:`"
            )