Python Multi-Thread Coding

Python Multi Thread Coding is something you should consider when you have some code that needs to run multiple times. For example – doing a version check across multiple routers. You can either run the check in sequence (series), or run it in parallel. In this post we will have a look at the ways to run this in parallel – which is easier that it looks at first glance !

For our first example, lets look at reading the OS version from a set of routers. The function we want to run is something simple, and we will assume an ssh_to_device function has already been built:

devices = ["router1", "router2","router3"]
for device in devices:
result = read_device_version(device)

def read_device_version(device):
cmd = "show version"
result = ssh_to_device(device, cmd)
return result

This will call the ssh_to_device function and run the check on each device, in sequence. Thats fine when its 3 routers. What about when you start to scale (think AWS – 10,000’s of routers). This will be slow, and anything we can do to decrease the time taken will be worth the code update. Time for threads ! ThreadPoolExecutor is exactly what we need.

Here is the updated code using ThreadPoolExecutor:

from concurrent.futures import ThreadPoolExecutor, as_completed
threads = []
result = []
with ThreadPoolExecutor() as executor:
for device in devices:
threads.append(
executor.submit(
read_device_version,
device
)
)
for thread in as_completed(threads):
result.append(thread.result())

So this routine will iterate through the devices list and use the submit function to create each threads. We will append that data to the threads list value. Each thread runs the read_device_version_data, and then adds that data back in to the result list variable. We then take advantage of the as_completed call which waits for each thread to return. Simple !

The arguments you have access to with the thread value are listed HERE – and for the result argument this also gives you some room to work as the result will be whatever we are returning from the function call (e.g. string, dictionary, list of dictionaries etc.).

This will save seconds per call when you run multiple at the same time – and lets say it needs to be called 10,000 times (so on 10,000 routers) – that’s:
20,000s = 333.3 mins = 5.55 hrs
That’s a LOT of time saved on a task, and well worth the time adding in the extra code to support the thread function.

What about thread CPU overload ?

One concern from threading is the possibility or exhausting the CPU with threads from the application calls. Again think AWS – if un-managed we could call 20,000 threads which would cause major issues for the host and probably take it offline. This is handled by using the max_workers argument being added in to the ThreadPoolExecutor call. For example if we want to limit the max threads running to be 7 – simply add:

with ThreadPoolExecutor(max_workers=7) as executor:

You could also do something like a CPU count based limit (max_workers=os.cpu_count()). Note per the docs if you leave this blank, for Python 3.5+ it uses cpu_count *5. For python 3.8+ it uses min(32, os.cpu_count()+4), so just decide what is best for your host and needs.

Working with partial and map

Another way to code this is to use the partial function, which allows us to build a set of functions ready to be run, each with a standard set of arguments already applied.

So if we use the above example and add in more arguments we want to have per call:

def read_device_version(device, user, password, ssh_key, ssh_port):
cmd = "show version"
result = ssh_to_device(device, user, password, ssh_key, ssh_port, cmd)
return result

Now we build a partial routine that has all the arguments that are static (user, password, ssh_key, ssh_port) and then applies the devices list to a map function to run the threads. The Map function only takes 1 argument, so this allows us to add many arguments first before the map call.

from concurrent.futures import ThreadPoolExecutor
from functools import partial

result = []
with ThreadPoolExecutor() as executor:
threads = partial(read_device_version, user, password, ssh_key, ssh_port, result)
executor.map(threads, devices, timeout=30)

Now the map function takes care of calling the threads with the complete argument set, and we will get the results stored in the result argument once each thread is done. This will give you more tools as a Python coder to using parallelization and threading in your code, which is something you need to think about when ever you think scale !

Using WITH … AS for creating variables

One note on the ‘with [function] as [variable]:’ method of making these calls. Using ‘with’ here automates the closing of the variable being called. In some cases you may open an SSH connection per call, or open a file to read per call. By using ‘with’, there is no need to close() the variable and what was opened with it – that’s done automatically for you. Just a cleaner way to manage closing variables once finished.