Introducing Contentstack's New Analytics API

As Contentstack has grown from a leading standalone headless CMS to a multiproduct platform that can be deployed for almost any digital experience use case, the amount of activity processing through our systems at any given moment has also grown tremendously, and so has the potential for us to offer new tools and products that will give Contentstack customers an edge over their competitors through powerful insights and analytics.

It’s my pleasure to be the first to announce that this month we are rolling out one of my favorite new features here at Contentstack, a new Analytics API. Today’s most competitive organizations are using analytics and business intelligence tools in the hopes of gaining an edge on competitors. The new Analytics API sets the foundation for Contentstack developers and business operations specialists to integrate Contentstack data into your existing BI tools, extract large amounts of data from our servers, and build new tools to empower your digital teams–an example of the latter is below.

Although in this initial phase the new API will be somewhat limited, it lays the groundwork for our customers to gain new BI superpowers. We will continue to expand the depth and breadth of Contentstack data access, include more products and data in the API, and take direction from your feedback about what you’d like to see next.

Understanding the Scope: Defining 'Data' in Our Context

Before we get too deep into the philosophy behind- and capabilities of this new API, let’s first define what we mean by data since anything on the web could technically be called the same. For the purposes of our Analytics API, data primarily refers simply to the quantitative data about your Contentstack utilization.

If you’re already familiar with Product Analytics and Mission Control in your Organization’s Admin panel, it’s these data we are exposing via the Analytics API in this first phase. This new API will give your team programmatic access to the data currently only available within Product Analytics and Mission Control UI,and liberate it for you to analyze in whatever way is useful to your business.

What you can do today: API Capabilities & How they work

Today, you’ll be able to generate and download JSON data showing subscription utilization, device usage, product usage, top URLs, status codes, cache usage, and more across most Contentstack products. Details of each request and sample payloads are viewable on our docs site.

The new API is asynchronous, meaning that you’ll send a query over in an initial request, allowing your data job to run on our servers without any risk of timing out for larger queries. On your second request, you’ll send the UID returned from your first request to access and download paginated JSON results.

If this new feature sounds like it’s geared toward developers, it is! But we also have some exciting announcements coming up related to our Product Analytics and Mission Control apps that you will not want to miss, so stay tuned to our blog and social media channels for more.

Analytics API in action: A Step-by-Step Demo

In the meantime, let’s jump into a quick demonstration showing the power of this new Analytics API. In this example, I’ll be leveraging Contentstack Automate, OpenAI’s GPT API, and our new Analytics API to automatically generate an executive summary of my Contentstack utilization for the current month compared to the previous month.

Creating utilization reports with the Analytics API and Automate

First, create a new automation in Automate. You could use any trigger you want to start the flow, but for this example we’ll be using the Scheduler trigger, which works similar to a Linux cron job. I have my report setup to run weekly (even though it’s a monthly report) because I like to see how my current month’s utilization is progressing throughout the current month compared to the last month. You could run this report at month’s end for a full month-over-month (MoM) comparison or change your API query parameters to run it weekly or even daily to suit your needs.

Making your first action

Next, create your first action. Select Codeblock and use the following code to your first request to the API. You’ll note the use of input.apiKey, input.authToken, and input.orgUid.

These values are defined in the Input options for the Codeblock step. You’ll want to create these 3 input fields and populate them with your own information. You can generate an Auth Token by following the documentation.

Building and sending a fetch URL to query the Analytics API

The code above builds and sends my fetch URL, setting the date range of the query. Before you save the Codeblock, you should test it to make sure you’ve properly set my input variables. If successful, you should see a result like this:

Working with Asynchronous Responses from the Analytics API

Now, since the API is asynchronous, depending on the size and complexity of your query, you may have to wait several seconds or longer before the job completes and your data is accessible on our servers. For this demo, I’m using a Wait action of 3 seconds to ensure the job has finished, but in a production use case, you would probably want to loop your calls to the job API to ensure your query job has completed, particularly when running queries with larger result sets or for mission critical integrations.

Now we’re ready to retrieve the data. Create another Codeblock action with the following code to retrieve the data from your initial query.

Retrieving current month utilization data from the Analytics API

Similar to the initial request, you’ll need to set input.apiKey, input.authToken, and input.orgUid variables in the Input section of the Codeblock action. But for the second request, you’ll also need to reference the jobId returned from the first request. This is easy with Automate. Simply select the jobId from the previous response using the dropdown list of options when you click into the Input Value field.

When you test your Codeblock, you should see a result like this below. You’ll notice that a few of my result values are 0, but that’s just because this Contentstack Organization isn’t used for publishing. You will see different data depending upon your utilization of Contentstack.

Retrieving the previous month’s utilization data

Now that we’ve successfully generated our current month’s data, we can move on to generating the previous month’s data. To do this, we’re simply going to create 3 new steps similar to the ones we just created. The differences in these next 3 steps are that you will use a slightly different code block to specify the start and end dates of the prior month, and you’ll reference the jobId value from this Codeblock in the second trip to the Analytics API for this dataset. Your Codeblock should resemble the following code:

Retrieving previous month utilization data from the Analytics API

From here you’ll create another Wait action and then your request to the data retrieval endpoint, like you did for the current month’s data. When you test and save your data retrieval step, you should see different data from the sample above. Here is the result for my previous month’s data, which you can see is higher than the current month if only because this sample represents a full month’s worth of data. As a reminder, I’m running my automation weekly so I can track growing progress against the previous month, but your use case may vary.

Quick review of automation steps

Now we’re at the really fun part, but before we proceed, let’s review the steps in our automation so far:

1. Trigger the Automation using a Scheduler Trigger.
2. Send a query request to the Analytics API to generate the current month’s data.
3. Wait 3 seconds.
4. Send a request to the Analytics API with the jobId in Step 2 to retrieve the current month’s data.
5. Send a query request to the Analytics API to generate the previous month’s data.
6. Wait 3 seconds.
7. Send a request to the Analytics API with the jobId in Step 5 to retrieve the previous month’s data.

Putting our utilization data to work

At this point, we now have JSON data payloads for the current month and previous month showing utilization across all of the Contentstack products we’re using. You could apply the principles above and leverage other endpoints of the Contentstack Analytics API to build your own custom reports and dashboards.

Comparing our monthly utilization data using AI

In our next step, we’re going to send both payloads to OpenAI’s GPT API with a prompt to compare both months’ data and generate an executive summary of the differences. If you’re new to OpenAI or ChatGPT, you’ll need to sign up and generate an API key to connect your account to Automate. Once you’ve done that, the next step is to write a prompt describing the report your want and pass in the two different payloads, like this:

This prompt generates pretty consistent results, but you’ll want to tinker with the prompt until you get the report results that work for you. My prompt and data generated this result below when I tested the automation step.

From here you can send this payload as an email or–what I like to do–clean it up and Slack it to myself! The results check out because it’s not even halfway through the current month, and I’ve been doing a lot less experimenting in my secret lab and much more… writing! My results take on a variation of this:

It’s fun to watch how OpenAI’s GPT API interprets my data and the broad range of results it generates. On that note, if you are using this approach when accuracy and specificity matter, please don’t just leave it up to AI–you really do want a human being at least sanity checking any AI generated text. Feel free to experiment with prompts or skip the AI step altogether and process your data programmatically to generate 100% consistent results.

Wrapping things up

Stepping back for a moment, the example above is really just a proof of concept to show you some of the potential of the new Analytics API, particularly when you combine it with Contentstack Automate. Generating an AI-powered report like this may create a useful artifact for you, but you could also use the same basic approach to alert you when values go above or below a desired threshold. Or generate a summary of your URL requests or SDK usage. We’ve shown basic insights into these data for some time, but now you’ll be able to analyze them pursuant to your own specific business needs.

We’re excited to make this new resource available, and we’re looking forward to continuing to explore its potential with you. As you take advantage of the new Analytics API, please tell us about your use cases and let us know what additional data or query functionality you’d like to see. We’ll be adding more products and data points to the API in the months ahead.