There are two methods for measuring webperf: Synthetic Monitoring (used for Lab data in Google Tools) and Real User Monitoring (used for Field data in Google Tools). Often opposed, they have the advantage of complementing each other.
In this article, we take a look at these two web performance monitoring methods. What are they? What are they used for? How can they be used effectively? What do you need to know to make the most of them?
Synthetic Monitoring (or Active Monitoring)
Tests are run :
- on servers in datacenters
- using a restricted connection
- to simulate the conditions that an average user might encounter under the conditions defined for the test.
Pages are actually loaded by a browser, in order to collect performance data corresponding to the reality of the user experience according to the parameters configured for the test:
- Browser
- Device type (mobile or desktop)
- Device model
- Geographic zone
- Network quality…
The benefits of Synthetic Monitoring
- Control of test conditions
- Test the same scenario several times and compare results before and after optimization
- Identify problems encountered by a category of users (in certain geographical areas, on certain types of devices or browsers…)
Examples of Synthetic Monitoring tools
Data collected with Synthetic Monitoring
Waterfall
This is a graphical representation, generally in the form of a waterfall, which visualizes the loading of the various elements that make up a page. This graphic is particularly useful for identifying blocking elements during loading (images, scripts, style sheets, fonts, etc.).
Warterfall WebPageTest showing a resource that significantly slows down page loading,
circled in red
The filmstrip
Presented in the form of a series of frames (or screens) on a timeline, the filmstrip is used to visualize what the surfer sees at each stage of a page’s loading, and how it is composed in the browser.
Technical data
Number of requests, page weight, server response time (Time To First Byte)…
Metrics related to page display and rendering
… and more generally to user experience: Speed Index, Start Render, Core Web Vitals Cumulative Layout Shift, Largest Contentful Paint…
When should Synthetic Monitoring be used?
For a technical benchmark
Synthetic Monitoring can be used to evaluate your own performance in a given browsing context, and compare it with that of the market or your competitors. For example, it can be used to answer the question: “Is my site faster or slower than that of my main competitor in France, in 4G on a mid-range cell phone?
To monitor a performance budget
The Performance Budget. consists in defining scores to be reached and/or thresholds not to be exceeded, for example: a minimum PageSpeed score of 90, dividing the Speed Index by 10, limiting page weight to 1MB…
Taking measurements at regular intervals under identical conditions enables you to assess progress and regression, and measure the impact of optimizations deployed.
To identify an SPOF (Single Point Of Failure)
Synthetic Monitoring tools can be used to pinpoint slowness, in particular through their visualization functions, to identify resources that may be causing problems during page loading and degrading the user experience (see WebPageTest waterfall above).
Real User Monitoring (or Passive Monitoring)
RUM data is collected continuously, rather than at a given point in time as with Synthetic Monitoring, using a script to measure loading times.
The benefits of Real User Monitoring
- Analysis of site traffic, taking into account the actions of real users
- Continuous measurement
- Takes into account all users, regardless of browsing conditions or geographical zone (no conditions to be defined before launching a test)
Examples of Real User Monitoring tools
Data collected with Real User Monitoring
Navigation Timing : RUM tools take advantage of theNavigation Timing API. Here’s all the data available as a page is loaded:
- Resource Timing: this API enables detailed measurement of the loading times of each static resource.
- User Timing: the User Timing API lets you mark and measure specific events in the browser.
- Conversion rates and other business data: some RUM tools allow you to correlate loading times and conversion rates. This is possible with Google Analytics, or Quanta.
In short, Synthetic Monitoring enables you to track your performance over time and position yourself in relation to your competitors, while RUM enables you to observe your customers’ experience on the ground.
As we said in the introduction, these methods are complementary.
Note: since the collection method is different, so are the results! Here are a few tips to help you interpret the data correctly.
Webperf monitoring: what you need to know to understand the results
It’s hard to compare results between Synthetic and Real User Monitoring, especially as they can be so far apart. Why is this?
Apart from the conditions under which data is collected, it’s also due to the fact that RUM tools combine first view and repeat viewdata in very different contexts, as the real user base can be highly heterogeneous.
In other words, metrics are derived from a mixture of those collected on pages viewed for the first time, and those collected on pages revisited with data from a cache (and therefore potentially faster).
In contrast, with a Synthetic Monitoring tool, measurements are generally taken on a first view in a single, homogeneous context.
Consequently, as this graph shows, the loading times indicated for the same page may differ:
Save
Save
Save
Save
You need to choose the data and the collection context according to what you want to illustrate. After all, loading speed can’t be summed up in a single figure!
Each KPI has its own interest, and the advantage of this plurality is that it can speak to all business lines: E-commerce teams will be sensitive to business gains, while SEO teams will be able to set objectives to optimize Core Web Vitals and PageSpeed score; as for technical teams, they will be able to focus on the metrics to be improved to ease the burden on the infra.
Everyone at their own level has their own references and requirements, and for a high-performance site, all teams have a role to play.
Which metrics should be monitored and improved?
Discover the essential webperf KPIs you need to know:
