API monitoring saves the day.
As a developer, once your API is live on production that is where the real work begins, you and your team need to ask different questions such as how do we ensure that the API is healthy and it remains so?
As the API is serving millions of users around the world, how do you anticipate problems or critical issues that can impair the functioning of the application?
While a simple API change can impact hundreds of internal API calls, creating a cascade of issues that can range from the back-end access of data to the end-user who is interacting with the application. How will your team anticipate, isolate, and resolve application issues before they become so problematic for end-users and partners?
In as much as there’s no straight way to identify and solve all these questions, teams can constantly monitor and observe their API while in production to gain a useful and comprehensive insight into the API usage, fail states and points, etc.
Therefore, the process of constantly monitoring and observing your API either manually or automatically is called API Monitoring and Observability.
In the next section, we will explore API monitoring to understand the importance of monitoring your API to gain useful insight.
Overview of API Monitoring
API Monitoring is the process of tracking the performance of an API (Application Programming Interface) in real-time to ensure that it is functioning properly and meeting the needs of its users. This process involves monitoring the API’s performance metrics such as response time, availability, and error rates. It also includes tracking the API’s usage and understanding how it is being used by its users.
API Monitoring helps to identify any potential issues that may arise, allowing developers to quickly address them before they become major problems.
Additionally, API Monitoring can be used to identify trends in API usage and to better understand the needs of its users.
It is also a critical part of ensuring that your applications are running smoothly and efficiently. It helps you identify any issues that may be occurring with your APIs and helps you take the necessary steps to fix them.
Furthermore, API monitoring can be done manually or with the help of a third-party service. Manual API monitoring involves manually checking the performance of your APIs on a regular basis. This can be time-consuming and tedious, but it can be useful if you need to identify any issues quickly.
Relating this concept to the API you deployed, you can easily integrate some tools or write up some code to report bugs, downtime, and performance issues to your email, slack, or any medium so you don’t have to manually check your production API for insights.
Now that you understand what API monitoring is about, in the next section, we will explore API Observability, its importance, and possibly the difference between API observability and API monitoring.
Overview of API Observability
API observability is a form of monitoring called white-box monitoring, which involves the process of observing, identifying, and predicting possible occurrences in a system. It is a crucial element to effectively carry out APIOps cycles.
To clearly understand the distinction between API monitoring and observability, you need to answer a few questions about your API.
Is your API providing the functionality they were designed to deliver?
Is your API providing the optimum performance for the users?
Is your API secured against threats and attacks?
How are users consuming your API?
To answer these questions, you will explore different methods to observe your API in production first before drawing a conclusion to answer those questions. We call that API Observability.
According to Mosif, traditional monitoring focuses on the “known unknown” metrics meaning that monitoring focuses on the metrics that you already know but you don’t know the values or result yet. For example Request Per Second or Errors Per Second, etc. While the metric value may be unknown beforehand, you already know what to measure.
However, that is not the case in Observability because API observability focuses on the “unknown unknowns” metric meaning that you need to observe as much as possible the internal workings of a system from its outputs by inferring state and behavior so that you are able to answer any arbitrary question around your API behavior, not just a few predefined metrics that are directly measurable.
In addition, you can use API Observability to understand more deeply how your API-driven applications work and properly rectify issues that might impact the availability, performance, and security of the entire API.
We have explored the importance of API observability to developers and engineering teams, in the next section, we will explore an API monitoring and observability tool that handles and automates these processes for you and your team.
Selecting an API Monitoring Tool
There are a variety of tools available to handle your API monitoring needs. Therefore, selecting an API monitoring tool that best fits your team’s needs is a tough choice but selecting an API Monitoring solution that can provide actionable data is essential, not only to increase your ROI but to get genuinely useful performance data.
When choosing an API monitoring solution, it’s important to keep these functionalities in mind:
Easy to understand & Intuitive: You need to choose a tool that is easy to understand and set up by your team. You won’t be able to utilize the full potential of any tool if you don’t understand it or if it’s difficult to use. Therefore, finding an API monitoring tool that is easy to use and intuitive will reduce your ramp-up time as well as increase the probability of your reliance on the tool.
Customizable, Collaboration, and Shareable: You need to choose a product that you can customize the dashboard and see different types of API analytics and metrics. Also, a tool that allows team collaboration, generating, and sharing reports.
Adaptable: Make sure to choose a tool that is adaptable, flexible, and easy to customize, so it can easily fit into your stack and your team’s preferences.
Alerts: Alerts are very crucial and most times the essence of monitoring, make sure to choose a tool that allows you to create alerts and events based on the result of monitoring your API.
Sequencing and Assertions: Make sure you choose an API monitoring tool that allows you to add assertions and validate the data the API returns. This is an essential functionality for any API monitoring tool.
If you apply the listed functionalities when choosing an API monitoring solution, you will be good to go choosing the right tool.
In the next section, we will work you through an API monitoring tool that checks the above criteria and more for your API Monitoring and Observability needs.
What is Treblle?
Treblle is a lightweight SDK that helps Engineering and Product teams build, ship & maintain REST-based APIs faster. It makes it easy to understand what’s going on with your APIs and the apps that use them.
Treblle passes all the criteria we set when choosing your API monitoring and observability tools. It’s easy to use, customizable, and allows developers to see the different types of metrics on their API.
Installing Treblle into any API project is very easy as there are different SDKs to include in any project irrespective of the programming language of choice.
Features of Treblle
Below are some of the features of Trebble and why it’s the best choice for the engineering team when it comes to API monitoring and observability.
API Monitoring and Observability
With Treblle monitoring, you can miss any data or insight into your API as you can get over 40+ different data points for every single request and understand everything from your API performance to user behavior.
With actionable data at a glance, you can different metadata such as HTTP methods, response codes, authentication, and API performance, understand who your users are, where are they coming from, and what devices they use and at the same time see the actual request data while keeping the sensitive information secure with our SDK masking feature.
There’s no limit to what you can do with Treblle monitoring, you can automatically detect endpoints, authentication details, and versions, track code-based errors as they happen and replay requests with one click, and Filter and search by more than 40 different data points for requests.
Auto-generated API Docs
API documentation is a crucial part of creating easy-to-use and marketable APIs. Therefore, it’s important to generate documentation that explains how and when to use your API with all the responses, error codes, and request examples.
Moreover, creating API docs is always tedious and difficult for busy engineers and even a dedicated technical writer like me but Treblle is to the rescue.
According to the 19th Developer Economics Survey by Slashdata, it shows that over 90% of developers use third-party APIs, and according to an article by Exate and research from FinancesOnline, it shows that investing in documentation management results in a 404% ROI over five years.
Now that you know how important documentations are to your API as a product. Treblle’s API Documentation generator removes the stress involved in creating API documentation and automates the process.
It fast-tracks your documentation by auto-detecting your API structure including endpoints, endpoint groups, and versions, and creating your documentation based on real-time changes over time on each of your endpoints.
Most importantly, you can customize your API docs to your look and feel, add or remove endpoints and versions, etc.
Benchmark your API
Treblle grades your API into 3 different categories namely performance, security & quality on every single request, and provides you with suggestions on how to improve allowing you to easily benchmark your API and fix any issues that might arise during the process of monitoring and observing your API.
Most importantly, you can compare the performance of your API with your peers and see how your API is performing compared to industry standards and best practices.
API Analytics and Observability
This feature allows you to go beyond just performance-related API metrics and understand your product, users, and find patterns that drive usage-based decisions.
You can build custom dashboards to suit your team and organizational goals. Also, with this feature, you can set alerts and completely observe how your API is performing in real-time.
There are more features and benefits of your Treblle as your API monitoring and observability tools as a team but in the next section, we will explore how to integrate Treblle into your Node.js application and start monitoring.
API monitoring and observability in an Express API with Treblle
To integrate Treblle, the best approach is to download and configure the SDK for your specific programming language, since we will be running Node.js, we will use the Express SDK for this purpose.
However, before we begin make sure you have Node.js installed and you know how to develop APIs with Express.js. You can read the ultimate guide to Express.js to get started.
Setting up Express Project
Setting a project is very simple and requires you to run the following commands to create a project, initialize package.json and install the Express package. Run the following command in sequence and in the directory where you want to create your project.
mkdir express-treblle-democd express-treblle-demo
npm init -y
npm install expressNext, open the newly created folder in your favorite text editor (using VSCode here) and create your first index.js file to set up Express.js.
Setting up Express Server
Express is one of the easiest Node.js frameworks for creating APIs. All you need to do is to import the library and start creating routes and endpoints.
Open your index.js file and paste in the following code snippets:
const express = require('express');
const app = express();
const port = 3002;
app.use(express.json());
app.use(express.urlencoded({ extended: false }));
// Integrate Treblle starts here
// ..................code here
// Integrate Treblle end here
// Data Source, could be replaced with a real database
// ............code here
// Data source ends here
// Endpoint starts here
// ............code here
// Endpoint ends here
// App listens to incoming requests here
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`);
});First line, we required the Express library into our project and the next two lines, we initialized the Express library and created our port variable.
Each of the comment blocks indicates where we will include code snippets as we progress with the article.
Lastly, we use express middleware to configure Bodyparser to receive params data as JSON. You can learn more about it from the ultimate guide to Express.js.
Creating your first Express Endpoint
Creating endpoints in Express.js is easy so in this demo, we are going to create dummy Todo app endpoints for our Todo application.
Inside your index.js file, add the following code snippets for the CRUD (Create, Read, Update, Delete) operation for our Todo application.
// Endpoint starts here
app.get("/todos", (request, response) => {
response.status(200).json(todos);
});
app.get("/todos/:id", (request, response) => {
response
.status(200)
.json({ data: todos.find((todo) => todo.id === request.params.id) });
});
app.post("/todos", (request, response) => {
todos.push(request.body);
response.status(201).json({ msg: "Todo created successfully" });
});
app.put("/todos/:id", (request, response) => {
const todo = todos.find((todo) => todo.id === request.params.id);
if (todo) {
const { title, desc, completed } = request.body;
todo.title = title;
todo.desc = desc;
todo.completed = completed;
response.status(200).json({ msg: "Todo updated successfully" });
return;
}
response.status(404).json({ msg: "Todo not found" });
});
app.delete("/todos/:id", (request, response) => {
const todoIndex = todos.findIndex((todo) => (todo.id = request.params.id));
if (todoIndex) {
todos.splice(todoIndex, 1);
response.status(200).json({ msg: "Todo deleted successfully" });
}
response.status(404).json({ msg: "Todo not found" });
});
//Endpoint ends hereEach of the HTTP methods used in the above code snippet corresponds with the CRUD operation mentioned above for creating a new todo, reading/retrieving all or single todo, and updating and deleting a todo from our data source.
Here’s a simple array data structure representing our data source:
// Data Source, could be replaced with a real database
const todos = [
{
title: "Todo 1",
desc: "This is my first Todo",
completed: true,
},
{
title: "Todo 2",
desc: "This is my second Todo",
completed: true,
},
{
title: "Todo 3",
desc: "This is my third Todo",
completed: true,
},
{
title: "Todo 4",
desc: "This is my fourth Todo",
completed: true,
},
{
title: "Todo 5",
desc: "This is my fifth Todo",
completed: true,
},
];
// Data source ends hereIn the next section, we are going to learn how easy it is to integrate Treblle into your Express.js APIs
Integrating Treblle
First, we will start by installing the Treblle Express package into our project. Run the following command to install:
npm i @treblle/express --saveNext, create a free account with Treblle to secure your API token ID and Project ID as they are required by the SDK.
After successful registration, create a new project (in our case DEMO) as shown in the screenshot below:
Next, copy your API Token and Project ID on your dashboard and include them in your .env file as shown below.
TREBLLE_API_KEY=your_********_here
TREBLLE_PROJECT_ID=your_********_hereLastly, open your index.js file and add the following code snippet under the respective comment section. You can learn more about the Express.js SDK from the official documentation.
const treblle = require('@treblle/express')
// …………
// Integrate Treblle starts here
app.use(
treblle({
apiKey: process.env.TREBLLE_API_KEY,
projectId: process.env.TREBLLE_PROJECT_ID,
})
)
// Integrate Treblle end hereThat’s all you need to do. However, with this simple setup, you have integrated Treblle into your API and you can start enjoying the benefits of API monitoring and observability using Treblle.
Now if you’re curious as we are, let’s open our Treblle dashboard and start monitoring and observing our API as we test them out.
Note that I have added a few widgets to customize my dashboard better. Below are some of the results after a few testing.
All Request Sent
Top Devices, Weekly and Monthly breakdown
Above are screenshots of different metrics you can generate while using Treblle to monitor and observe your API.
Summary
The process of constantly monitoring and observing your API either manually or automatically called API Monitoring and Observability has rapidly gained momentum in this ever-changing tech and API industry. In addition, because of the benefits and importance of monitoring and observing your API automatically in a timely manner many companies have developed tools to help teams with this process.
However, some tools fall short of the real concept of monitoring and observability thereby producing tools that are not easy to use and do not really get the job done.
However, in this guide, we have highlighted the different criteria to consider when choosing a monitoring and observability tool for your APIs.
Furthermore, we have also integrated one of the best tools in the market and demonstrated how to quickly set it up with your Express.js project.
