Your First REST API Call In Python

This post originally appeared on the Packet Pushers’ Ignition site on June 10, 2020.

Introduction

In many automation scripts, you’ll be retrieving information via some sort of interface and then doing something with the data. The interface is often an API–application programmatic interface. For folks new to APIs, they might seem daunting, but they need not be. Interacting with a device via an API is as straightforward as any other device conversation you might have.

For example, databases often use SQL. Structured query language is well-documented and easy to understand once you get the hang of it. Mail servers use SMTP. Back in the day, engineers might telnet to port 25 on a mail server and send mail by hand using simple mail transfer protocol commands. You know. For fun. Web servers use HTTP, the language of which is as well-understood as perhaps any protocol can be.

In each of these examples, there’s an understood format for the respective conversation. The format will specify many things, but most critically for us, these protocols will describe…

1. How to ask a question.
2. What to expect back in response.

With APIs, that’s much of what you need to understand–how to ask the right question and then interpret the answer. APIs from many vendors are well-documented, making it easy to know what question to ask to retrieve the information you’re looking for. The format of the answer will also be documented, so that you can code your script to parse the answer appropriately.

Good API documentation is not universal. In fact, that might be a decision point you’ll need to make on using or eschewing an API–whether or not it’s documented well.

In Python, there are a number of available libraries that make it easy to interact with an API. We’re going to focus on RESTful APIs because they are so common. As RESTful APIs using HTTP-style calls are the most typical, we’ll use the Python requests http library in our example.

For this article, we’re not going to cover sending data via an API. Naturally, you’ll want to send data in your automation scripts at times. That’s how you’ll make things happen in your applications or with your infrastructure. However, we want to keep the risk factor low (it’s your first REST API call!), so we’ll keep our API work here read-only.

Authentication

The first step to accessing an API is authenticating to it, which should be logical. You can’t perform an SNMPv2c query without the correct community string or configure a network device without a username, password, and possibly the right RADIUS or TACACS profile.

Authentication to an API often comes in the form of one or more tokens. That is, rather than a username and password, you might need to interact with the API provider to generate a token, and then present that token as part of your API call. With the token, you’ll be authenticated and your call authorized. Without the token, the API server will ignore you or (more likely) return an error.

In some cases, the authentication token scheme can be convoluted. For instance, I find Twitter’s implementation of OAuth to be confounding, requiring as it does an API key (consumer key), API secret (consumer secret), access token, and access token secret to successfully make a call against a Twitter API. On the other hand, the Tweepy Python library means I don’t have to understand how the OAuth scheme works. I give Tweepy the four tokens, and Tweepy handles the OAuth machinations and API call formatting for me.

In other cases, the authentication token scheme is more straightforward. I find Slack’s token generation process for apps mostly intuitive. For a rudimentary Slackbot I wrote in Python, I pass the Slackbot app’s token to the Slack Python library. With that token, I’m allowed to make whatever API calls I authorized my Slackbot app to make when I created the application and installed it into my Slack space.

Other APIs are much simpler. For instance, NetBox’s API authentication scheme requires that you generate an authorization token for your account, and then present that token as a string in an HTTP header accompanying your request. Unlike Twitter’s OAuth dance of token doom, there is nothing convoluted about this.

For example, here’s a non-Python API call to NetBox using the curl utility against a non-production NetBox instance I run in a lab.

curl -X GET "http://172.22.45.1/api/ipam/aggregates/" -H "accept: application/json" -H "Authorization: Token bc19a8a967173c8f37e0f5d7ba32da9cba8dea8e"

Notice the -H “Authorization: Token…” bit. With that bit, NetBox sends me back a blob of JSON data I care about.

{"count":9, ... }

Without that authorization token, NetBox still sends me JSON. But instead of data I care about, NetBox tells me that I have disappointed it deeply.

{"detail":"Authentication credentials were not provided."}

If I send an incorrect authorization token, NetBox expresses its disappointment differently.

{"detail":"Invalid token"}

So far, we’ve been talking about APIs somewhat abstractly. We haven’t translated any of our concepts into Python code, so let’s get going with that. In the following sections, we’ll excerpt a few lines at a time of a short Python script and talk through what’s happening.

URL Formatting

One of the things we need to know with API calls is how to ask the question. I’m going to continue using NetBox as our example. To know how to ask the NetBox API questions, you can refer to the NetBox API documentation. Once you’ve installed NetBox, the API documentation link is in the footer of the web UI.

As our example, let’s ask the NetBox API about all of the regions we’ve defined in the Data Center Infrastructure Manager (DCIM) feature. You could ask anything you like, but in my NetBox installation, there are just 2 DCIM regions defined, making for a tidy example.

The NetBox API is RESTful (representational state transfer) and accessed using HTTP. REST and HTTP are not synonymous, although you’ll notice that REST APIs often use HTTP to transfer the representation of state.

Did you get that? The whole “representation of state” thing? That’s a technical way to describe what’s going on with a REST API call.

“Hey, API. I want to know this specific thing about the state of the device you’re sitting on. Please transfer that state to me.”

“Sure, but only because you’re authorized! Here’s a representation of that specific state.”

Using that background, the way we need to ask the question is via an HTTP URL. In Python, I like to build the URL via a set of variables. It makes the code easily re-usable, say if you wanted to put the API call into a function later. Let’s examine the code below.

Line 1 imports the environment variable NETBOX_APITOKEN, and assigns it to the variable apiToken. Importing tokens and other authorization strings from local environment variables is an entry-level way to keep your secrets out of your code. Sure, I could have embedded the token right in the script, but that’s a terrible practice. I’m not going to explain how to import environment variables into Python here.

Line 2 creates a Python dictionary containing a single entry. The key Authorization has a literal string value of Token followed by a space, plus the value of the token I imported from the NETBOX_APITOKEN environment variable. Why a dictionary and not a string variable? Because the Python requests library requires a dictionary for all of the HTTP headers we need to include with the request, and NetBox requires that the authentication token is passed as an HTTP header.

Line 3 is intentional whitespace. Whitespace and comments are important in code. In Python, whitespace is actually a formatting requirement at times.

Line 4 builds the first part of the URL. 172.22.45.1 is the IP address of a NetBox server running in my lab. /api prefaces all API calls to NetBox.

Line 5 builds the second part of the URL. I obtained /dcim/regions/ from the NetBox API documentation built into the server. Access the API documentation via the API link in the footer of the web UI. Pay attention to trailing slashes. In the case of NetBox, the API call will fail if you call /dcim/regions instead of /dcim/regions/.

Line 6 combines the URL components into one big URL that is the API call we’re going to make over HTTP. Although NetBox chooses to pass the authentication token as an HTTP header, many other APIs are set up to include authentication as part of the URL itself. If that were the case here, I would have had an additional variable where I defined the authentication portion of the URL string.

Reading API Documentation

You might want more detail about how I knew that /dcim/regions/ was the right thing, and that’s worth pointing out. Why? Because NetBox uses Swagger for API documentation, and Swagger will come up rather often in your API explorations. In fact, many equipment vendors talk about their Swagger API documentation as if it’s table stakes for any API implementation.

Here is an example of Swagger-generated API documentation. On my NetBox server, I scrolled down to the /dcim/regions/ section of the API docs page and expanded it to capture this image.

The screenshot truncates the rest of the expanded section, but what you see is enough for you to get the idea. There’s the highest level URL you can ask, along with qualifying fields you can populate if you want to narrow the result set.

For instance, I’ve defined our API call URL as 172.22.45.1/api/dcim/regions/. But if I wanted to know information about a DCIM region named “Lab”, my API call would be 172.22.45.1/api/dcim/regions/?name=Lab.

Asking The Question

Now that we’ve constructed the question, we’re ready to ask the question.

Line 1 uses the Python requests library to make the API call, and save the response in the nbApiRawAnswer variable.

Effectively, we’re saying…

“Hey, requests! I need to you to get something for me. Specifically, get the URL stored in nbApiQuestion. Oh, and I need you to include all the HTTP headers stored in the nbApiHeaders dictionary in your get operation. Whatever the server sends back to you, save it in nbApiRawAnswer. Okay? Thanks!”

Perhaps I embellished a little with the social niceties, but I like to imagine that APIs respond well to politeness. I know I do.

Understanding The Answer

Assuming I asked a valid question and included a valid authorization token, NetBox will respond to my API call with an answer. NetBox API responses are sent as JSON. Data formatted as JSON appears as a hierarchy of key-value pairs delimited by curly braces, square brackets, and commas.

The trick for you is to comprehend this JSON output. What is it that you’re looking at? How do you make sense of it?

We’ll go back to Python in a moment, but first, look at this raw JSON blob retrieved via a macOS host using curl.

curl -X GET -H "Authorization: Token bc19a8a967173c8f37e0f5d7ba32da9cba8dea8e" "http://172.22.45.1/api/dcim/regions/"
{"count":2,"next":null,"previous":null,"results":[{"id":1,"name":"Lab","slug":"lab","parent":null,"site_count":4},{"id":2,"name":"Production","slug":"production","parent":null,"site_count":1}]}

You can kind of make sense of the response, but it’s a little hard to read as a human, right? It is, so let’s add some whitespace and newlines to render this JSON blob hierarchically. In the programming world, this is known as prettyprinting. I put my API call into the Firefox browser (yes, you can do that if you’re already logged into your NetBox server), and Firefox rendered the “pretty” result as you see below.

In Python, a way to prettyprint the answer is using the json library that is core to Python.

Line 1 tells Python to print the JSON portion of nbApiRawAnswer using the json.dumps formatter, using an indent level of 4. The result is identical to the Firefox prettyprint output above.

You can read through the output and make pretty good sense of it now. (See what I did there?)

1. The question was about DCIM regions.
2. The structured answer we got back tells us there are 2 regions.
3. The first one is called “Lab” and the second one is called “Production”.
4. The Lab region has 4 sites. The Production region has 1 site.

Of course, those observations are all our own inference. What if we want more detail about the answer? For that, we can return to the documentation Swagger generated for NetBox. In the /dcim/regions/ section of the doc page I screenshotted above, Swagger also documents the expected answer. If we expand it out fully, we see this information about the JSON structure.

In our case, we have no parent objects tied to our DCIM regions, so the JSON blob we got back isn’t as complex as the Swagger documentation indicates it could have been. But the larger point is that with the API documentation, you have a reference to guide your interpretation of the answer you will receive to your API call.

Narrowing The Scope

A crucial point to understand is that you’re getting back the entire JSON answer the API writer designed as a response to your call, whether you want the entire blob or not. That is, APIs aren’t necessarily efficient. Maybe you only wanted to know one very specific detail. Too bad. Here’s all this other stuff, too. Deal with it.

With SNMP, you poll a specific OID and retrieve that specific leaf object from the MIB tree. You typically get back a number. A string. Each leaf object is a narrowly scoped, specific item. By contrast, an API call returns more often a branch of a tree with all of the corresponding twigs and leaves.

It’s up to you to dig through the API documentation and determine how to ask a question that gives you just what you want. You might not be able to ask about specific leaf objects if there’s no API call that lets you do that, but strive to be as efficient as you can be.

For example, if I know that the ID of the Lab region is 1, and all I want to know about is the Lab region, the NetBox API documentation tells me there is a /dcim/regions/{id}/ API call I can make. This scopes the API question more narrowly, and returns a smaller JSON blob with less extraneous data–leaves–for my Python script to have sitting around in memory just so that I can access one single leaf object I might care about.

The Entire Script

For completeness, I need to show you the entire script.

The only lines I hadn’t yet mentioned are the import statements at the top of the script. Imports are how you’ll gain the additional functionality that the json, os, and requests libraries offer.

I love Python libraries. Well-written libraries are like gaining Python super-powers. Why write your own JSON blob parser when someone’s done that work for you and made it freely available in a library? I import Python libraries everywhere that I can.

For More Information

Once you’ve got the data back from the API, you need to do something with it. For that, you should read the following article. This will help you understand how to reference the nested Python objects you tend to get back in JSON structures.