Connecting to the PCE

Proxy Settings

If you need to use a proxy to communicate with the PCE, HTTP/S proxies can be configured using the set_proxies function:

>>> pce.set_proxies(
...     http_proxy='http://my.proxyserver.com:8080',
...     https_proxy='http://my.proxyserver.com:8080'
... )

If not set in the session, the requests library will pull proxy settings from environment variables, see the requests proxy documentation for details.

Note

Proxy values set with set_proxies will apply to the session, and will be overwritten by proxy values set in the executing shell environment. If you need to override environment proxy settings, you can specify the proxies parameter directly as a keyword argument:

>>> pce.ip_lists.get(proxies={'http': 'http://proxy.server:8080', 'https': 'http://proxy.server:8080'})

TLS Certificates

If you’re using the illumio library with an on-prem PCE, you may be using self-signed or internal ceriticate chains for your instance.

Requests through the PolicyComputeEngine can leverage the requests library verify and cert parameters to specify CA certificates and cert/key pairs respectively.

See the requests documentation for details.

Disable TLS verification:

>>> pce.labels.get(verify=False)

Verify using custom CA bundle:

>>> pce.labels.get(verify='/path/to/ca/bundle')

Verify using a local client-side cert pair:

>>> pce.labels.get(verify=True, cert='/path/to/keypair.pem')
>>> pce.labels.get(verify=True, cert=('/path/to/client.crt', '/path/to/client.key'))

Retries and rate limiting (HTTP 429)

The PCE enforces API rate limits. When you exceed them it responds with HTTP 429 (Too Many Requests). The client retries automatically, but if the request rate stays too high the retries are eventually exhausted and you’ll see an error like:

illumio.exceptions.IllumioApiException: HTTPSConnectionPool(host='...', port=443):
Max retries exceeded with url: /api/v2/orgs/.../labels?key=app&value=AID011
(Caused by ResponseError('too many 429 error responses'))

Default retry behavior

Every PolicyComputeEngine mounts a requests retry adapter with these defaults:

  • total = retry_count (default 5)

  • backoff_factor = 2 — waits of ~1, 2, 4, 8, 16 seconds between retries

  • status_forcelist = [429, 500, 502, 503, 504]

  • the PCE’s Retry-After header is honored

Tuning what’s exposed

The number of retries and the per-request timeout are constructor arguments; the timeout can also be changed later:

pce = PolicyComputeEngine('my.pce.com', port=443, org_id=1,
                          retry_count=10,       # Retry(total=10)
                          request_timeout=60)   # seconds
pce.set_timeout(90)                             # change the timeout afterwards

backoff_factor and status_forcelist are not exposed through the public API and there is no set_retries method. To change them, mount your own adapter on the session:

from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

pce = PolicyComputeEngine('my.pce.com', port=443, org_id=1)
pce.set_credentials(key, secret)

retry = Retry(
    total=10,
    backoff_factor=3,                 # 3, 6, 12, 24, ... seconds
    status_forcelist=[429, 500, 502, 503, 504],
    respect_retry_after_header=True,  # obey the PCE's Retry-After
    # backoff_max=120,                # cap the wait (urllib3 >= 2)
)
adapter = HTTPAdapter(max_retries=retry)
pce._session.mount("https://", adapter)
pce._session.mount("http://", adapter)

Prefer fixing the request rate

More retries only wait longer between the same volume of requests — they treat the symptom. A 429 storm usually comes from calling the API in a tight loop, for example resolving labels one at a time. Fetch the collection once and resolve in memory instead:

# one request instead of thousands
labels = pce.labels.get_all()
label_map = {(l.key, l.value): l for l in labels}

app_label = label_map.get(('app', 'AID011'))   # no API call

For large collections, the async interface (get_async / get_all) has the PCE build the result as a single job rather than many paged GETs. If you issue requests concurrently, reduce the concurrency and add a small delay between batches.