teff.provider.concurrency

teff.provider.concurrency

Global per-provider concurrency guards.

Shared across :class:~teff.harness.Harness instances so parallel branches (each with its own harness) throttle model traffic together instead of blowing past provider rate limits.

Functions:

Name	Description
provider_concurrency	Return the current global concurrency limit for provider (if any).
set_provider_concurrency	Globally cap concurrent model calls for provider.

provider_concurrency

provider_concurrency(provider)

Return the current global concurrency limit for provider (if any).

Returns the active semaphore's capacity (explicit or auto-grown via max_parallel), or None when the provider has no semaphore.

Source code in teff/provider/concurrency.py

def provider_concurrency(provider: str) -> int | None:
    """Return the current global concurrency limit for *provider* (if any).

    Returns the active semaphore's capacity (explicit or auto-grown via
    ``max_parallel``), or ``None`` when the provider has no semaphore.
    """
    return _PROVIDER_LIMITS.get(provider.lower())

set_provider_concurrency

set_provider_concurrency(provider, limit)

Globally cap concurrent model calls for provider.

Overrides any per-harness max_parallel for that provider. Pass limit <= 0 to remove the cap.

Source code in teff/provider/concurrency.py

def set_provider_concurrency(provider: str, limit: int) -> None:
    """Globally cap concurrent model calls for *provider*.

    Overrides any per-harness ``max_parallel`` for that provider.
    Pass ``limit <= 0`` to remove the cap.
    """
    provider = provider.lower()
    if limit <= 0:
        _EXPLICIT_LIMITS.pop(provider, None)
        _PROVIDER_SEMAPHORES.pop(provider, None)
        _PROVIDER_LIMITS.pop(provider, None)
        _PROVIDER_ACTIVE.pop(provider, None)
    else:
        _EXPLICIT_LIMITS[provider] = limit
        _PROVIDER_SEMAPHORES[provider] = asyncio.Semaphore(limit)
        _PROVIDER_LIMITS[provider] = limit
        _PROVIDER_ACTIVE[provider] = 0