Trend Shift Analysis

‘Trend’ is a pretty common business term, but its definition tends to vary by industry. In performance monitoring, trend, or trend shift, is a key metric that is used to indicate change.

Why is trend shift important?

Change is inevitable. Today’s websites must frequently update and change to keep up with competition and attract new users, but such changes can have a negative impact on the user experience if not managed properly. The dynamic nature of the Internet makes it necessary to constantly monitor different metrics. One of the biggest challenges in performance monitoring the lack of clarity when configuring alerts. Alerts that are set up using the wrong configuration can lead to an overabundance of false positive alerts or no alerts at all, which results in an increased MTTR.

Webpage response is an important metric that affects end-user experience. If the page takes too long to load, the user gets frustrated and will likely abandon the webpage. A high page load time can be caused by several issues, some of the common problems are:

1. The server that delivers the page content is unable to handle a surge in requests
2. The webpage has too many third-party ads/images
3. The network is sluggish due to maintenance

Monitoring webpage response values or keeping track of the trend shift can help identify any unusual webpage behavior and to ensure a positive user experience.

This is just one of the many scenarios where analyzing trend shift can be valuable. Trend shift alerts are also useful in:

• Detecting trend shifts on specific ISP or region or city
• Multi hosting or multi CDN environment

When can you use trend shift?

Trend shift alerts can be set up for metrics that are prone to constant variations in their values. In the image below, a trend shift was detected on July 18 at around 16:00, which continued for about 24 hours. Later, it returned to normal, indicating that the root cause was identified and fixed.

When is trend shift irrelevant?

Trend shift can be detected when there is a sudden change (for a few consecutive runs) in the usual pattern. But if this change is gradual rather than sudden, then trend shift cannot detect the problem. In such cases, warning and critical alerts should be set up for a specific value.

In the image below, we see that the webpage response has increased gradually. The webpage response is around 2500ms in the beginning and increased to around 3500ms over two days’ time. Trend shift alert is not triggered here as the value increased gradually.

Trend shift alerts using Catchpoint

Clients can configure trend shift alerts using Catchpoint; the alerts will help them take necessary action so that end-user experience is not affected.

Below is where you will find the trend shift alerts displayed:

1. Smartboard page
2. Alert Log page

Trend shift alerts in the Smartboard page

The assistant widget section in the Smartboard page displays trend shifts. The Smartboard page has been developed using artificial intelligence (intense computing algorithms) that detects the trend shifts. It computes the trend shift for the test time metric (total time taken to complete the test). The Smartboard page does not require any additional configuration to display the trend shift information.

Trend shift alerts in alerts Log

A trend shift alert is triggered and displayed in the alerts log page only after it has been configured in the test properties. Trend shifts can be detected for:

1. Timing metrics – test time, DNS, send, wait, DOM load, etc.
2. Ping metrics – ping packet loss, ping RTT
3. Request metrics – #Requests, #Connections, #Redirects, #Images, #Scripts, #CSS, etc.
4. Zone metrics – self downloaded bytes, %Self Bottleneck, %Third party bottleneck, etc.

Configuring trend shift alerts

Alerts can be set up in the advanced alerts section in the test properties page as shown below. The default recipients will be notified by an email when a trend shift is detected for the metric it was configured.

Where to find trend shift alerts?

The alerts log page displays a list of alerts; it can be filtered for specific tests and alert types (metric based) for a specified time interval.

How is Trend Shift alert different from other alert types?

For regular alerts, the user should configure the values over/below which an alert should be triggered. In the case of trend shift, however, the user should select the metric to be monitored for trend shift. Trend shift is based on historical comparison, therefore there is no need to set any specific values.

Advantages of trend shift analysis

Let’s take a real scenario where trend shift analysis can be critical. Ecommerce companies rake in major business during the annual sales like Black Friday and Cyber Monday. On such days, these websites experience a rapid hike in traffic with customers rushing in to grab the best deals. The servers managing the website may slow down considerably with the influx of requests. In the image below, we can see that the webpage response shot up during a special discount sale.

Trend shift detects such spikes in the webpage response and helps businesses identify the root cause of the issue. It can provide a lot of data around the incident such as:

1. Identifying the city with the most number of users
2. Identifying whether the ISP (Internet Service Provider) was capable of handling the load
3. Building a better infrastructure so that the load is distributed

Summary

Trend shift signals a change in the website performance. It provides a starting point to analyze the change (increase/decrease) in the metric being monitored. Performance analysts can use this data to identify the root cause and take corrective measures if necessary. Proactively identifying and fixing the issue will improve the user experience and have a positive impact on the company’s revenue.

By: Pradeep Thalanki and Ankur Vishwakarma