Category: python

How to speed up http calls in python? With Examples

Blocking HTTP requests

The most popular and easy to use blocking http client is requests. The requests docs are simple and straight forward...for humans.

The biggest performance gain you can acquire (provided you are making requests to a single host) is using an http session. This creates a persistent connection, meaning that additional requests will use the existing session. More info on that in this blog post on python and fast http clients.

Example Blocking HTTP persistent connection vs new connections

Here is some example code for getting quotes from quotes.rest

import requests
import time

def get_sites(sites):
    data = []

    session = requests.Session()

    for site in sites:
        response = session.get(site)
        data.append(response.json())

    return data

if __name__ == '__main__':
    categories = ["inspire", "management", "sports", "life", "funny", "love", "art", "students"]

    sites = [
        f'https://quotes.rest/qod?category={category}' for category in categories
    ]

    start_time = time.time()
    data = get_sites(sites)
    duration = time.time() - start_time
    print(f"Downloaded {len(sites)} sites in {duration} seconds")

Don't overuse this API as they have rate limits and will eventually give you a 429 http status code as a response - Too Many Requests

So when I run this code:

Downloaded 8 sites in 3.7488651275634766 seconds

That is pretty fast, but what would be the case if I used requests.get() instead of using the session?

In that case the result was:

Downloaded 8 sites in 10.602024793624878 seconds

So in the first example resusing the existing HTTP connection was 2.8 times faster.

Threaded HTTP Requests

There is a library that uses requests apparently called requests_futures that uses threads - preemptive multithreading.

Example Threaded Request Futures

from concurrent.futures import as_completed
from requests_futures import sessions
import time

def get_sites(sites):
    data = []

    with sessions.FuturesSession() as session:
        futures = [session.get(site) for site in sites]
        for future in as_completed(futures):
            resp = future.result()
            data.append(resp.json())

    return data

if __name__ == '__main__':
    categories = ["inspire", "management", "sports", "life", "funny", "love", "art", "students"]

    sites = [
        f'https://quotes.rest/qod?category={category}' for category in categories
    ]

    start_time = time.time()
    data = get_sites(sites)
    duration = time.time() - start_time
    print(f"Downloaded {len(sites)} sites in {duration} seconds")

When running this code it was faster:

Downloaded 8 sites in 1.4970569610595703 seconds

Interestingly if I set the max workers to 8 sessions.FuturesSession(max_workers=8), it slows it down dramatically;

Downloaded 8 sites in 5.838595867156982 seconds

Anyway the threaded requests is 7 times faster than non-persistent blocking http and 2.5 times fast than persistent blocking http.

Asynchronous HTTP Requests

The next thing to look at is co-operative multitasking, which still uses a single thread (and single process) but will give control of execution back to the event loop once's it is done - it won't block.

Python has a few aync http libraries: aiohttpand httpx

Example Async Aiohttp

from aiohttp import ClientSession
import asyncio
import time

async def get_sites(sites):
    tasks = [asyncio.create_task(fetch_site(s)) for s in sites] 
    return await asyncio.gather(*tasks)  

async def fetch_site(url):
    async with ClientSession() as session:
        async with session.get(url) as resp:  
            data = await resp.json()
    return data

if __name__ == '__main__':
    categories = ["inspire", "management", "sports", "life", "funny", "love", "art", "students"]

    sites = [
        f'https://quotes.rest/qod?category={category}' for category in categories
    ]

    start_time = time.time()
    data = asyncio.run(get_sites(sites))
    duration = time.time() - start_time
    print(f"Downloaded {len(sites)} sites in {duration} seconds")

The result of this code was:

Downloaded 8 sites in 1.271439790725708 seconds

That is the fastest response we have had yet. More than 8 times faster than the non-persistent blocking HTTP connection, almost 3 times faster than the persistent blocking HTTP connection.
Also 17% Faster than the threaded blocking HTTP requests.

Thoughts on Aiohttp

The problem with aiohttp as rogouelynn mentions in a blog post is everything needs to be async

In real life scenarios you often need to do some syncronous stuff first, like authenticating and receiving a token.

You can't just do:

session = ClientSession()
response = session.get('https://iol.co.za') 

>>> response
<aiohttp.client._RequestContextManager at 0x102985c80>

As you only get back a context manager.

Potencially an easier to use library is httpx because syncronous requests are as native and easy to do as asynchronous requests.

r = httpx.get('https://httpbin.org/get')

Putting it all together

How to speed up http calls in python...well go through the steps until you get the speed you need.

  1. Use simple blocking persistent HTTP connections with requests.Session()
  2. Use an Asynchronous http client like asyncio or httpx.

In the steps above I skip over the threading part as you will find that when you scale up threading can become unreliable and it is usually the case where asyn is better or matches threading performance.

python-http-client-speed-comparison

Building a rest API client or SDK as a python package

The best way to learn ways of building your API wrapper / SDK is to build it yourself from scratch is to look at ones that already exist. Here is a list of python api wrappers you can checkout and view. Most if not all of them are already python packages.

Here is a good post on how to test and build an API client / wrapper. In the post the author uses vcrpy as a way to replay stored responses, however they do actually happen the first time. To prevent your tests from ever hitting a real API you should look at Responses.

I also found a few generic wrappers for the requests library that are supposed to make your life easier, but it looked like many were just abstraction for abstraction sake and the closer your are to requests the better.

I did however find uplink which is a python api boilerplate module based on retrofit - which is the goto http client for Android development. I like it and I think it will make my life a bit easier when creating the api wrapper. It is however still in beta. Tapioca is another one, but I didn't rate it. It arised from this fellow not liking how API wrappers are done and the added abstraction.

As a side note, Braintree doesn't even make the rest API available, it forces clients to use a client or wrapper built by them mainly to ensure security.

I found uplink hard to work with...it seems as though the further you are from requests the harder your life becomes.

Allowing for a User Provided Session

One of the key things needed when making an API wrapper/client available is letting the user provide their own session. A reason for this is that user could have added additional auth to the API they are connecting to or may be using a different method of auth altogether.

Also, the client may be using an API gateway like kong so the client may need an auth token or some other permission. So it is better to just allow the user to provide their own session - which can then be modified with the session id's.

So in the class constructor - __init__() method, ensure a session can be set and default it to none. The below example is adapted from the hvac client. eg:


class MyClient(object):
    """My Api Client"""

    def __init__(self, base_uri=DEFAULT_BASE_URI, cert=None, verify=True, timeout=30, proxies=None, session=None):
        """Create a new request adapter instance.

        :param base_uri: Base URL for the Vault instance being addressed.
        :type base_uri: str
        :param cert: Certificates for use in requests sent to the Vault instance. This should be a tuple with the
            certificate and then key.
        :type cert: tuple
        :param verify: Either a boolean to indicate whether TLS verification should be performed when sending requests to Vault,
            or a string pointing at the CA bundle to use for verification. See http://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification.
        :type verify: Union[bool,str]
        :param timeout: The timeout value for requests sent to Vault.
        :type timeout: int
        :param proxies: Proxies to use when preforming requests.
            See: http://docs.python-requests.org/en/master/user/advanced/#proxies
        :type proxies: dict
        :param session: Optional session object to use when performing request.
        :type session: request.Session
        """
        if not session:
            session = requests.Session()

        self.base_uri = base_uri
        self.token = token
        self.namespace = namespace
        self.session = session
        self.allow_redirects = allow_redirects

        self._kwargs = {
            'cert': cert,
            'verify': verify,
            'timeout': timeout,
            'proxies': proxies,
        }

As you can see in the example above it is not just the session. Other things like SSL verification, the certificate to use for SSL verification and proxies can be passed to the constrator to allow for versatility.

Should the client raise it's own Errors

If a request on your managed node API returns an error 4xx or 5xx, should your API client raise an exception or just let the response be handled by the caller?

Say you do catch exceptions, should those exceptions be custom created by the client package or just be part of pythons standard lib, such as ValueError.

From what I have seen, many python api clients do raise their own errors stemming from a root exception. According to Brett Slatkin of Effective Python:

Having a root exception lets consumers of your API catch exceptions you raise on purpose

They should help you find bugs, not hide them.

Some examples:

Although you can choose to use the standard libraries excpetions if you want, an example of this would be the DEPRECATED heroku python client wrapper or the new heroku python client wrapper

How to store a session based token that gets Revoked

Different API's authenticate in different ways. Some will provide you a token that only lasts a certain length of time, thereafter your requests stop being authenticated.

How should we handle this? Get a new token each time - probably not a good idea, store the token in a key value store or store it as a global variable.

Also what mechanism should we use to ensure a new token is requested, when the old one expires?

How to automatically reauth for a specific error code or catch errors

In order to catch these error codes and raise our own, it is best to do it in a single place. Pretty much however the client is making requests - perhaps with the requests library request method. We would have to extend from the request class and do our clients error checking after we get a response - in our custom requests class.

One way of doing that is how hvac used in the adapter class.

The problem is that we are allowing users of the library to provide their own requests.Session() object, that won't be using our custom request method.

The idea here is to have your client class define it's own request method and call that method instead of using requests. In other words MyApiClient.get() which behind the scenes uses self.session.get, but adds error checking.

 

Building a Python HTTP client that automatically refreshes access tokens

How often is it that when calling API's you do an initial round of auth to receive a token that expires after a certain amount of time?
It is quite common these days. What isn't common is a generic way to ensure that this authentication process is automatically restarted when the existing token expires.

The first thing to remember is that we should use some persistent storage mechanism to keep the token.

Where are we going to store this access token value then? We need to store it as it needs to be accessed between multiple requests.

The database is not a terrible place, I've seen it done before. Or you can use django's cache API.

Finding an Example

I wish there was a client I could view and see how other people have implemented this in python.

Here is an apt comment from the php community:

...for other APIs I have found the decorator pattern around the HTTP client (especially a PSR-18 client, because its interface is so simple) is a great place to put the renewal logic. That's the point where the expiry response is known, the full original request is known so can be tried again, and the renewed authentication token can be saved in the current client and saved to a persistent storage.

So I understand, here you have a stack of middleware (kind of) handlers that each wrap the request and pass back the response. On an expired token response you try renewing the token, then bounce the same request back down the chain again? The retry count allows you to give up after a number of bounces.

You could inject a closure in this middleware here to handle the persisting of the new token to storage.

JudgeJ comment on guzzle

A Javascript implementation example, the gist:

  1. Store the token
  2. Wrap every request in a decorator that reauths when a `401` is received
  3. Ensure the original request is redone

I managed to find a python implementation for python-fitbit

Storing the Token in Cache

You can store the auth token in cache like so:


from django.core.cache import cache 

@property
def session(self):
    session = requests.Session()
    session.verify = settings.VERIFY_SSL

    # Check there is a token saved
    auth_header = cache.get('auth_header')
    if not auth_header:
        auth_header = self.get_authorization(session)
        cache.set('auth_header', auth_header)

    session.headers.update(auth_header)

    return session