Integrate Legacy APIs with Django, Celery & Tenacity

Good old medical services

How do you envision premedical IT services?

They should be reliable. After all, they support the medical sector, right? And they are reliable. Since I've started working for Medishout, I've never seen anything that fails spectacularly within the project. So what causes the problem? APIs do. The thing is peri-medical IT services are just... strange.

In Medishout, we have a few 3rd party integrations. Most of them were painful to do. The reason is that the services that we had to integrate with were just monolithic (or just old).

There were plenty of issues. Most of them only degraded the mood of the programmer but one, which is APIs outages, could affect the users as well. And if you work in peri-medical IT industries, you don't want to burden the users with your errors. This kind of error can cost someone their health, or even life.

The first thing that can be thought of is "If someone's API fails, he should just fix it" right? In most cases, they will, but it can take months onto years. Startups do not have that kind of time to wait.

In this article, I'll describe a case study of one of the integrations. You will see how to integrate external API in an overly complicated way, and why should we be bothered to try so hard.

Problem overview

At some point in Medishout evolution, we noticed that in some cases our users had to manually pass service requests to an external platform. Those requests are for administrators, to let them know that something is broken, and they have to repair it.

To make their lives easier, we decided to integrate with this external API, in such a way, that once a thing is requested to be fixed, we save it on our dashboard and after that, we pass it to a 3rd party.

During the development process, we noticed a few problems.

we had to perform more than one API request to pass the request as a whole
3rd party responses were slow enough to aggravate the user
documentation and the overall look at the 3rd party convinced us, that making a few API requests is insufficient to make sure that the request is delivered successfully

Celery: healthy, tasty, and useful

Most likely, the audience of this article is familiar with Celery, an amazing library to perform long-lasting tasks in the background. If not, I recommend this amazing article written by Vitor Freitas (the whole blog is brilliant tho).

Now you might be thinking: "Hey Matt, why do you bring Celery in for such a simple task?".

Well, there are a few reasons:

Sometimes creating an entity in a third-party service requires more than one simple POST request, as someone might assume. That means, for example, that after I create an entity in a 3rd party database through API, I have to modify it with PUT request(s).
Our requests may fail. Therefore if we don't move the whole process to the background, we have insufficient time to retry or perform complicated logic on it (recall that on the other side of the screen, there is a user that waits for the server's response).

Let's get our hands dirty

Using service in practice

After we’ve acquired well-separated logic responsible for API requests, we can create tasks that will use it.

Alright, after we execute these tasks in some convenient place (perhaps in one of APIView methods), we move the process to the background, thanks to Celery. The user will not have to wait for a response. He only receives confirmation that everything will be processed. So everything is just fine. Except it's not.

‍

Handling APIs outage using Tenacity

In the whole process, we want to add some robustness to communication with 3rd party. Moving the process to the background tasks ensures us that a user will not have to wait for a response all day, but it won't save us from things that might go wrong during creating and updating entity processes.

One reasonable thing we can do is to keep trying.

Wrapping it up

The last thing to do is to use tenactious_requests in Celery tasks. Nothing easier:

And that's it. We ended up with background tasks that will do their best to pass user requests to a 3rd party. Using a service pattern makes it easy to test, and try-except structure provides the developer with a convenient way to handle cases when something goes wrong.

Final notes tl;dr.

In this article we have seen how to create a piece of software that will handle complicated communication processes with a third party. By using Celery, we moved the whole process to the background, so a regular user does not have to wait until the process finishes. With Tenacity, we added robustness to communication with third-party API.

In the shown flow, we do not track the state of a 3rd party request (whether it's updated or just created). I decided not to put it in this article, but if you, dear reader, consider implementing such a thing in your system, check Django Finite State Machine. It will make your life a lot easier.