Bring Your Own Custom Monitor

When Catchpoint was founded ten years ago, we wanted to monitor as many things from as many places as possible. Over the years our list of monitors has grown to more than 20 to address our customers’ needs. But there is always that one service (or a custom one) you want to measure the performance of that we don’t yet support.

Catchpoint’s Custom Monitor lets you run your own code on an Enterprise Node.  You can create a script to monitor the performances of your service using the programming language of your choice.  Once the code is written, just upload it and schedule a test. Catchpoint will execute your code, process the output data, and collect the metrics you defined.

We take care of everything required to run your code and measure performance.

How it works

1. Script your Custom Monitor in your favorite scripting language.
2. Upload the script to your Enterprise Node.
3. Configure a Custom Monitor & Catchpoint Insights*.
4. The Catchpoint agent runs your custom script as defined in the monitor and collects metrics.
5. Analyze performance and insights.

* Catchpoint Insight allows you to create custom metrics that Catchpoint collects during testing. When you define Insight items, you are setting up rules that instruct Catchpoint to retrieve additional pieces of data when parsing the response, it receives from the Custom Script during testing.

General Custom Monitor use cases

Some potential use cases for Custom Monitors are below, but really the sky is the limit. If you can script it, you can monitor it.

Database Monitoring

You can create a Custom Monitor to check if your databases are accessible or not. A Custom Monitor can execute queries to check the performance of your database. The metrics from the monitor can then be correlated with application, network & infrastructure metrics.

For example, an ecommerce transaction test contains a step that performs a product search. There is a high “wait” time for this step in the test. You can now correlate this high wait time with the database’s execution time metric to see if it is the result of backend latency or not.

You can create Custom Monitors with alerts for any database like Oracle, MS SQL, MySQL, MongoDB, etc.

The above chart shows the response time of the query and the availability of a MySQL Database executing a query.

VoIP Monitoring

Most organizations today use VoIP for basic communication and rely on the quality of this service.  The quality of VoIP has improved over the last few years with ISPs offering more reliable & better connectivity. This has helped increase the subscription of VoIP service, but issues affecting VoIP quality persist. It is important to monitor your VoIP service to identify issues help to resolve them.

In the VoIP world, we often use Mean Opinion Score or MOS as a quality metric. MOS score is calculated by considering network latency, packet loss, and jitter. The score is a value between 1 and 5 with a good quality in the range of 3.2 and above.

With Custom Monitor you can proactively monitor for issues specific to VoIP service. You can build a script using the ping utility to capture important information like latency, jitter, and MOS to determine the quality of your VoIP service.

The above chart shows the quality of VoIP with a consistent MOS of 3.4 along with jitter & latency.

Google Lighthouse

Google Lighthouse chrome extension is Google’s website review and audit tool used by developers, performance engineers, and marketing. Lighthouse reviews a website for its SEO ranking along with other scores such as whether the site adheres to best practices defined by Google for mobile, progressive web apps, opportunities for improvement, and diagnostics.

The information provided helps users improve the quality of web pages. Google Lighthouse audits an individual URL by running a series of tests on the page and generating a report on how well the page performed.

Using Catchpoint’s Custom Monitor with Lighthouse, you can see how performance impacts the scores provided by Lighthouse overtime answering questions such as:

1. Did the increase in page size impact my SEO Score?
2. Did the introduction of a new CDN impact my end user experience and my Lighthouse score?
3. Did the number of page views at any point in time have an impact on my servers? And was there an impact on my SEO score?

The metrics collected from Lighthouse can be correlated with Synthetic & RUM metrics to see a wider performance impact.

The above chart shows how a website’s best practices have progressed and its effect on SEO

Sentiments Analysis using Twitter

Sentiments analysis, also known as opinion mining, is the process of computationally determining whether a piece of writing is positive, negative or neutral. Understanding user sentiment is useful for businesses to develop their strategies, understand a customer’s feelings towards products or brand, and how people respond to campaigns or product launches.  Using Tweepy, the python client for the official Twitter API, you can create a custom script to monitor positive and negative keywords of your business name and services that you offer. Customers have deployed this to correlate the performance/availability of their services with how their customers feel about them on Twitter.

The above chart shows the sentiments of @twitter with most tweets being neutral or positive

Conclusion

Today’s systems are complex, that just monitoring your web application is not enough. You need to monitor all the internal and external services, custom protocols, and custom workflows to ensure you deliver a good digital experience.  With Custom Monitors, it’s now possible to monitor these services and get notified of critical events.

These examples above are some of the many things one can do with Custom Monitor. The possibilities are endless. We cannot wait to see what scripts you will come up with.