What is Digital Experience Monitoring?

In 2013 after a debate with a CMO I wrote this blog post where I made the case that Performance was the 5th P of the modern marketing mix and IT telemetry is not just for IT.

Fast forward six years, it is clear that the performance of a web site or app, namely whether it is available and how fast it loads, is critical in shaping the perception of those end users that interact with a brand’s sites or apps. As a result of this focus on performance, a new form of monitoring performance and availability from the perspective of end users across the globe has grown in popularity. Many refer to this form of monitoring as DEM.

DEM stands for Digital Experience Monitoring. To understand what this means we first have to define what digital experience means.

What is a digital experience?

A digital experience is an interaction between a user (customer, partner or employee) and an organization that is possible only because of digital technologies. Digital experiences can include:

• An employee using a SaaS application.
• A customer using your mobile app or browsing your website.
• A partner integrating your API into their application.

Every digital experience a customer has with an organization impacts the overall customer experience and their perception of the organization.

Delivering a digital experience to an end user is no small feat. Loading a single page involves a complex choreography that involves a multitude of stakeholders including:

• External network delivery spanning fiber, broadband, local ISP’s, and mobile networks
• CDN providers who deliver content close to end users
• DNS providers who resolve domain names into IP addresses
• 3rd party services and tags delivered via API’s
• Data centers that host application and 3rd party code
• The underlying application, database, and OS architecture and code

At Catchpoint, we refer to this complex choreography as the delivery chain. Any weakness in the delivery chain, whether with API, Networks (ISPs, Backbone providers…), DNS, CDN, or code itself can cause a ripple effect that can lead to an outage or slow performance. Another way to view the delivery chain in the context of digital experience is to imagine an iceberg. The top of the iceberg is the page elements that are visible to an end user, while everything below the water line are those elements are not visible to an end user but integral to the successful delivery of an application or service.

I have tried to show this complexity even back at DoubleClick to showcase the visibility gaps in our monitoring strategies:

So what happens when something goes wrong? According to Todd Paulino, an SRE at LinkedIn, we can consult what he calls the Donald Rumsfeld matrix of knowns and unknowns when it comes to detecting digital experience issues and responding to or resolving them. As the matrix shows, “good monitoring” occurs when issues are detected, and the response path is known. The result is a quick resolution to the problem. “Active incidents” occur when an issue is detected but the response or remedy is unknown. These require monitoring teams to work together to find the root cause of an issue. Where things get dangerous is when detecting a particular type of issue is unknown. This is known as a monitoring gap and can ultimately lead to the scenario no brand wants: tweets from users reporting an issue.

What is Digital Experience Monitoring?

Now that we have defined digital experiences, how they are delivered to end users, and the impact of known/unknown detection and response, we can dig into “What is Digital Experience Monitoring.”

In my opinion, it’s a culture first, a practice, a philosophy, a pact—that your company cares about your customers so much that you are going to put in place people, process and tools to ensure that you are not left in the dark when problems occur. You will have a proactive approach to detecting problems impacting those digital experiences and making your customers happy.

As an example, in the non-digital commerce world of the ’40s, companies deployed Mystery Shopping (mystery consumer, secret shopper). In the digital world instead of hiring people to randomly shop and provide feedback, we collect data from synthetic and real users accessing the site.

You also have to take into account the customer service view: In the old days, bad customer experience or a complaint was between the customer and the company. Today a bad customer experience is shared on Yelp, LinkedIn, Facebook, Twitter which is then amplified 100,000 times over thanks to the virality of these platforms. I have always loved this infographic which illustrates this evolution of Customer Service:

These bad experiences can have a devastating impact on a company’s bottom line and stock. For example, after the negative press and social media backlash in April 2017 United Airline’s stock fell 4%

Every experience matters

In the context where every customer experience matters, where brands and products are measured by how satisfied customers are, what do we do in a world where the experiences are all digital? How do you deploy mystery shoppers in a stream of 0s and 1s?

This is where digital experience monitoring solutions come in to play. DEM is not new; it was previously called EUM (End User Monitoring), UM (User Monitoring), and End User Experience Monitoring (EUEM).

The team behind Catchpoint pioneered this back in 1999 with the creation within DoubleClick of a team dedicated to what we called Quality of Services. Delivering ads fast was a priority from the top (board, CEO, CIO…) down. We talked about QoE (Quality of Experience) in many of the things we did.

We always asked the following questions to determine QoE:

• Did we deliver ads?

• Did we deliver ads fast?

• Did we count correctly?

We replicated this across everything we did from our reporting to our UI.

The mission was simple: build, deploy and maintain IT monitoring solutions to ensure the best quality of experience for the billions of ads we delivered every single day!

Creating an experience first culture

Digital experience monitoring is not a tool; it’s a way of thinking.

It starts with your company wanting to make sure you deliver an amazing digital experience to all of your customers (internal or external) and are not left wondering why people are complaining on twitter.

What I did not have back in 1999 are things like Site Reliability Engineering and DevOps. However, we did two things common in those practices.

• We included a performance aspect in our sprints. We allocated 20% of the budget to performance.

• We established strong Service Level Objectives (SLOs) and Operational Levels (today called SLI) between IT and Business.

These were always measured from the customer’s viewpoints not CPUs, and Database IOs.

DEM requires two ways of capturing digital customer experiences: Active and Passive monitoring solutions.

Active or synthetic monitoring is similar to those mystery shoppers, you have a robot (agent, node) that performs a continuous verification of the service (delivery of an ad, serving a css, buying a tv on a commerce site). The goal of this type of testing is to proactively catch problems before they impact real users. The benefits are immense since they can also measure everything in that digital delivery chain (DNS, CDNs, Cloud providers).

Passive or real user monitoring is like putting a video camera in a store to watch a long line of customers waiting in line to order. When someone orders a “Double Ristretto Venti Half-Soy Nonfat Decaf Organic Chocolate Brownie Iced Vanilla Double-Shot Gingerbread Frappuccino Extra Hot With Foam Whipped Cream Upside Down Double Blended, One Sweet’N Low and One Nutrasweet, and Ice “you can see the frustration as the line grows longer. Real User Monitoring captures every single interaction!

DEM as a solution must give you this Stereo Vision!

Bottom Line

IT systems are complex and will only continue to grow in complexity, creating more and more visbility gaps,  requiring all kind of monitoring solutions to be deployed to monitor and measure infrastructure, code, networks, and containers. But at the end of the day monitoring is only good if it tells you customers are having a problem. Your customers don’t care about your CPU utilization and cloud elasticity!

Bottom line: it’s not easy to deliver something to end users when you have so many things that can go wrong. This is why when we built our DEM solution we did not stop at just web synthetic tests. We offer over 20 tests including DNS, network, API, and SSL tests. All of these things are out of your control, and you must monitor them to mitigate the risks.

To see the power of Catchpoint’s Digital Experience Monitoring (DEM), platform and learn why companies like Microsoft, LinkedIn, Airbnb, L’Oreal, Best Buy and more choose Catchpoint for Digital Experience Monitoring, sign up for a free trial.