Categories
python python performance task queues

Python Queue Broker Comparison: Kafka vs SQS vs Redis for Background Task Processing

This is an AI updated revision of the original post A stock update endpoint takes 5-14 seconds, blocking customer requests and locking up workers. The obvious fix is a background task queue – but which broker and Python client should you use? This post benchmarks Apache Kafka, Amazon SQS, and Redis across reliability, throughput, and […]

Categories
python

Python: Using get with default or using the or keyword with the default

Say you have some dictionary arriving and you want to ensure the data from the dictionary has an acceptable default for a field.. Should you use a get with a default like this: my_key = my_dict.get(‘key’, False) or using python’s or keyword: my_key = my_dict.get(‘key’) or False Some Scenarios Where the key does not exist […]

Categories
Containers python python performance task queues

Python queue Comparisons: Kafka, AWS SQS and Redis

An interesting question has come up for a real world task. What queue and python queue client should be used? In this post, a few queue options will be tried in the attempt to answer which one should be used. The Problem The real world problem is currently a single api is used to serve […]

Categories
python python performance

Python Simple Asyncio [Part 1]

Is asyncio simple? Some ideas: https://charlesleifer.com/blog/asyncio/ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/ Came across a well written and simple post about Async programming in Python. A good place to learn about python is always the documentation – here is the documentation for asyncio There is also this Python and Async simplified post. Many of the examples of asyncio can […]

Categories
python

Python Implicit Namespace Packages

A very confusing topic is python imports and paths. One area that makes it more confusing was the introductions of implicit namespace packages. https://peps.python.org/pep-0420/#differences-between-namespace-packages-and-regular-packages These are packages that are found even without an __init__.py. A folder can be created on the python path and that folder automatically becomes a python package. Not sure why or […]