6 Apr 2022

Core Web Vitals: 6 Ways to Measure It Using Google’s CrUX Data

6 techniques to extract Core Web Vitals directly from Google CrUX data, which will help you optimize your loading speed and support your SEO strategy.

You will be able to analyze the metrics that Google has integrated into its algorithm to evaluate the quality of the user experience:

Largest Contentful Paint (LCP) for display speed,
Cumulative Layout Shift (CLS) for visual stability,
First Input Delay (FID) for interactivity.

Since the announcement of the Page Experience update and its mobile deployment in 2021, then desktop in February 2022, the links between SEO and web performance have become even closer. Many SEO professionals have taken a closer look at the issue of loading speed, the means to measure and optimize it.

From a business perspective, the user experience must be optimal across all browsers and device types, so that customers complete the purchase funnel instead of abandoning their cart or going to a competitor.

SEO is not just about qualifying the user experience on the site, it is also about understanding how Google evaluates Core Web Vitals, and how to access this data.

Understanding how Google observes a site, and knowing the most important criteria for its robots, allows you to prioritize the corrections to be made to your web pages.

So, let’s take stock of:

data used by Google to measure Core Web Vitals,
the sources available to extract this data and their limitations ,
the best sources for SEO purposes ,
techniques for accessing these data sources , with examples.

What data does Google use to measure Core Web Vitals?

According to information provided by Google, the data collected in the Chrome User Experience (CrUX) report is used to measure Core Web Vitals for Search . Google has announced this on several occasions, including during John Mueller’s “Core Web Vitals & SEO” session at Chrome Dev Summit in 2020 , and most recently during the Web Vitals AMA at Google I/O 2021 .

John Mueller – Chrome Dev Summit 2020 – on Field and Lab Data Extraction

The Chrome User Experience (CrUX) report gathers information about the performance of web pages visited by real Chrome users who meet specific criteria.

So, to put things into context, Core Web Vitals metrics from Google’s perspective only look at a segmented subset of your entire user base.

Example of user distribution by browser for a website,
with 45% of Safari users, 25% of Chrome users,
15% of Chrome users that are included in CrUX and 10% of Firefox users.

Of course, we can’t know what percentage of Chrome users are part of the CrUX report for a given website, because this data is not disclosed by Google. How significant this subset is depends entirely on your actual users.

Ideally, you should track Core Web Vitals on your site for all users with a third-party tool, or by using the Javascript package provided by Google. However, CrUX data remains the best source of publicly available data.

What sources are available to extract Core Web Vitals from the CrUX database?

Knowing that Google’s search engine uses CrUX data, the next step is to figure out how to get your hands on that data.

There are 6 methods to extract and measure Core Web Vitals with CrUX, directly from Google:

l’API CrUX
l’API PageSpeed Insights
the CrUX Data Studio dashboard
PageSpeed Insights tool
The CrUX BigQuery Project
Google Search Console

Methods for measuring Core Web Vitals from Field data

Each of these methods has its pros and cons. Here is a comparative list to help you choose the best one depending on the type of analysis you want to do in terms of SEO.

For each technique, the following criteria are observed:

URL Extraction : Is it possible to extract Core Web Vitals data for a specific URL (if available)?
Domain/Origin : Is it possible to extract Core Web Vitals data from a specific domain (if available)?
Devices : Is it possible to sort data by mobile, desktop or tablet?
Network Connection : Is it possible to sort data based on users’ network speed?
Recent data : Is it possible to get the most recent data available (last 28 days from the day of extraction)?
Historical data : is it possible to access data from previous months/years?
Free : Is it possible to access the data without paying?
Scalability : Is it possible to extract this data easily for thousands of URLs or domains?
User interface accessibility : Does this data source have an easy-to-use interface?

Ranking CrUX Data Extraction Methods Based on SEO Needs

Comparison of Google Core Web Vitals monitoring methods based on SEO needs

L’API CrUX

The CrUX API is one of the easiest to access and most comprehensive to extract Core Web Vitals from CrUX. It is intuitive, easy to use, and contains all the information needed to understand, collect Core Web Vitals data, and monitor performance issues on a website.

CrUX API Highlights

URL and origin level data is accessible via the API when available;
you can sort the data according to the three devices (mobile, desktop and tablet);
network connection information is available, you can extract data for 4G, 3G, 2G, slow-2G and offline networks ;
you can pull the most recent data available , which is the average of the data collected over the previous 28 days starting from the last full day . This is (in theory) what Google Search uses to evaluate a website’s Core Web Vitals;
The tool is completely free and easily scalable . The only limit is the number of requests per minute, which is 150. However, the response time is very fast compared to other APIs, such as PageSpeed Insights.

CrUX API Weaknesses

There is no access to historical data, you can only access the data collection for the previous 28 days. But, it is possible to work around this by storing the data daily for later access;
there is no easily accessible user interface for the API.

How to access Core Web Vitals data with the CrUX API?

When it comes to using APIs to extract data, JavaScript, and more specifically Node.js, is a very interesting option. Here are some examples (if you don’t know how to run it, here’s how to install Node.js before practicing these examples).

[pastacode lang="markup" manual="%2F%2F%20Cr%C3%A9ez%20un%20fichier%20index.js%2C%20collez%20le%20code%20ci-dessous%20et%20ex%C3%A9cutez%20%60npm%20install%20axios%60%20dans%20votre%20terminal%0A%0A%2F*%20Modules%20*%2F%0Aconst%20axios%20%3D%20exiger%20('axios')%3B%0A%0A%2F*%20Variables%20de%20script%20*%2F%0Aconst%20apiKey%20%3D%20'YOUR-API-KEY-HERE'%20%2F%2F%20Obtenez%20votre%20cl%C3%A9%20ici%20https%3A%2F%2Fdevelopers.google.com%2Fweb%2Ftools%2Fchrome-user-experience-report%2Fapi%2Fguides%2Fgetting-started%23APIKey%0Aconst%20cruxEndpoint%20%3D%20%60https%3A%2F%2Fchromeuxreport.googleapis.com%2Fv1%2Frecords%3AqueryRecord%3Fkey%3D%24%7BapiKey%7D%60%20%3B%0A%0A%2F%2F%20Fonction%20personnalis%C3%A9e%20pour%20appeler%20l'API%20CruX%0Aconst%20getApiData%20%3D%20async%20(type%2C%20URL)%20%3D%3E%20%7B%0A%20%20%2F%2F%20Cr%C3%A9er%20le%20corps%20de%20la%20requ%C3%AAte%0A%20%20demande%20const%20%3D%20%7B%7D%0A%20%20demande%5Btype%5D%20%3D%20URL%0A%0A%20%20%2F%2F%20Envoyer%20une%20requ%C3%AAte%20API%0A%20%20const%20%7B%20donn%C3%A9es%20%7D%20%3D%20attendre%20axios(cruxEndpoint%2C%20%7B%0A%20%20%20%20m%C3%A9thode%20%3A%20'POST'%2C%0A%20%20%20%20en-t%C3%AAtes%20%3A%20%7B%0A%20%20%20%20%20%20'Type%20de%20contenu'%20%3A%20%60application%2Fjson%60%2C%0A%20%20%20%20%7D%2C%0A%20%20%20%20donn%C3%A9es%20%3A%20JSON.stringify(req)%0A%20%20%7D)%3B%0A%20%20renvoyer%20des%20donn%C3%A9es%0A%7D%0A%0A%2F%2F%20Ex%C3%A9cuter%20le%20script%20(IIFE)%20-%20Changer%20'type'%20%26%20'URL'%0A(asynchrone%20()%20%3D%3E%20%7B%0A%20%20const%20testOrigin%20%3D%20attendre%20getApiData('origine'%2C%20'https%3A%2F%2Fbuiltvisible.com')%0A%20%20const%20testURL%20%3D%20attendre%20getApiData('url'%2C%20'https%3A%2F%2Fbuiltvisible.com')%0A%20%20console.log(testOrigin%2C%20testURL)%3B%0A%7D)()" message="" highlight="" provider="manual"/]

L’API PageSpeed Insights

The PageSpeed Insights API is another option to extract Field data from CrUX.

It gives very relevant information, but it is missing some data compared to the CrUX API, which could be useful when diagnosing performance issues related to Core Web Vitals.

Points forts de l’API PageSpeed Insights

URL and origin level data is accessible via the API when available;
you can sort data by mobile and desktop ;
as with the CrUX API, you can fetch the most recent data , which is the average of the data collected over the previous 28 days starting from the last full day;
usage is completely free and easily scalable . The quota is limited to 240 requests per minute and 25,000 per day;
You can access this API through an easy-to-use interface with Google’s PageSpeed Insights tool (although it’s not as scalable).

Weaknesses of the PageSpeed Insights API

You cannot segment data for tablet users ;
no information about network connections is available, all different connections are aggregated when fetching Core Web Vitals data;
there is no access to historical data , you can only access the data collection for the previous 28 days. This problem can be solved by storing the data daily for later access;
This service runs Lighthouse in the background to get Lab data for the same queries, so the API response is a bit slower than the CrUX API.

How to access Core Web Vitals data with the PageSpeed Insights API?

Here is a small example of how you can extract CWV data from the PageSpeed Insights API. If you want a plug-and-play script to run, you can download this repository from Github.

[pastacode lang= »markup » manual= »%2F%2F%20Cr%C3%A9ez%20un%20fichier%20index.js%2C%20collez%20le%20code%20ci-dessous%20et%20lancez%20%60npm%20install%20axios%60%20dans%20votre%20terminal%0A%0A%2F*%20Modules%20*%2F%0Aconst%20axios%20%3D%20require(‘axios’)%20%3B%0A%0A%2F*%20Variables%20de%20script%20*%2F%0Aconst%20apiKey%20%3D%20’YOUR-API-KEY-HERE’%20%2F%2F%20Obtenez%20votre%20cl%C3%A9%20ici%20https%3A%2F%2Fdevelopers.google.com%2Fspeed%2Fdocs%2Finsights%2Fv5%2Fget-started%23APIKey%0A%0A%2F%2F%20Fonction%20personnalis%C3%A9e%20pour%20extraire%20des%20donn%C3%A9es%20de%20l’API%20PageSpeed%0Aconst%20getApiData%20%3D%20async%20(url)%20%3D%3E%20%7B%0A%20%20%20const%20endpoint%20%3D%20’https%3A%2F%2Fwww.googleapis.com%2Fpagespeedonline%2Fv5%2FrunPagespeed’%20%3B%0A%20%20%20const%20apiResponse%20%3D%20await%20axios(%60%24%7Bendpoint%7D%3Furl%3D%24%7Burl%7D%26key%3D%24%7BapiKey%7D%60)%20%3B%20%2F%2F%20Cr%C3%A9er%20un%20appel%20HTTP%0A%20%20%20const%20urlCWV%20%3D%20apiResponse.data.loadingExperience%20%3B%20%2F%2F%20Extraction%20du%20champ%20URL%0A%20%20%20const%20domainCWV%20%3D%20apiResponse.data.originLoadingExperience%20%3B%20%2F%2F%20Extraction%20des%20donn%C3%A9es%20du%20champ%20domaine%0A%0A%20%20console.log(urlCWV%2C%20domainCWV)%20%3B%20%2F%2F%20Consigne%20les%20donn%C3%A9es%20du%20champ%20URL%20et%20du%20champ%20domaine%20si%20elles%20sont%20disponibles%0A%20%20return%20%7B%20urlCWV%2C%20domainCWV%20%7D%20%3B%0A%7D%20%3B%0A%0A%2F%2F%20Appelez%20votre%20fonction%20personnalis%C3%A9e%0AgetApiData(‘https%3A%2F%2Fwww.searchenginejournal.com%2Fcategory%2Fseo%2F’)%20%3B » message= » » highlight= » » provider= »manual »/]

The CrUX BigQuery Project

The CrUX BigQuery project is a huge database of real user metrics records that dates back to October 2017. This project is full of very relevant information, but like all other sources, it has its pros and cons.

CrUX BigQuery Project Highlights

You can access data at the origin level;
you can sort the data by the three device types (mobile, desktop and tablet);
you can extract data for all types of network connections when available (4G, 3G, 2G, slow-2G and offline);
you can extract historical data beyond the last available month going back to October 2017;
you can adapt this system to as many projects as you want, and the data is very flexible, with the ability to create your own custom tables;
Some additional metrics and elements are available that may be useful for your analysis and are not available in the CrUX or PageSpeed Insights APIs, such as Time To First Byte or country segmentation.

CrUX BigQuery Project Weaknesses

You cannot access data at the URL level;
This dataset is updated every second Tuesday of the month for the previous month. So if you want to monitor Core Web Vitals more regularly, this is not the right source;
a good understanding of SQL language is necessary for in-depth data analysis;
There is a charge for it to work. While there is a free tier on BigQuery, you will need to add billing information to Google Cloud Platform to use it, but for small to medium scale reporting, the free tier will suffice.

How to access Core Web Vitals data with CrUX’s BigQuery project?

If you already have a Google Cloud Platform account, you can access the project using this link .

You will need to enable the BigQuery API to access data directly from your SQL editor.

BigQuery Project – SQL Editor Interface

You don’t need to be an expert in SQL, a little familiarity with the user interface will be enough.

Here are two resources that will help you kick off your data analysis journey: Rick Viscomi’s CrUX Cookbook and Paul Calvano’s Biguery for CrUX tutorial .

After seeing the different data sources that require some programming knowledge, we will see those that allow access to Core Web Vitals data from CrUX without needing to know how to code.

The PageSpeed Insights Tool

Google’s PageSpeed Insights tool is a great “no-code” alternative to accessing CruX data.

Home page of the PageSpeed Insights interface to analyze the performance of a web page

The benefits are exactly similar to the PageSpeed Insights API. The only downside is that this method is not very scalable. Indeed, to get data from multiple URLs, you will have to manually enter each URL into the tool.

How to access Core Web Vitals data with the PageSpeed Insights tool?

Enter the URL or domain you want to get data from in the PageSpeed Insights tool.

If information is available for the URL or domain (origin), you will find it at the top of the results displayed after running the tool.

Results page after entering a URL on PageSpeed Insights with Core Web Vitals metrics (LCP, FID, CLS, and FCP, INP and TTFB)

Ntd: This article explains in detail how to read PageSpeed Insights results , and why Field and Lab data are sometimes very different .

The CrUX Dashboard in Data Studio

CrUX Data Studio Dashboard is a powerful tool built by Rick Viscomi to access the CrUX BigQuery project very easily with an easy to use user interface .

Result of the analysis of the Largest Contentful Paint (LCP) of a website

Highlights of the CrUX Dashboard in Data Studio

You can access data at the origin level;
you can sort the three device types (mobile, computer and tablet);
you can extract data for all types of network connections when available (4G, 3G, 2G, slow-2G and offline);
you can extract historical data beyond the last available month;
the setup is completely free ;
it is very easy to configure and has a simple user interface;
Some metrics that may be useful for your analysis are not available in other APIs, such as Time To First Byte (TTFB).

Weaknesses of the CrUX dashboard in Data Studio

You cannot access data at the URL level;
This report is linked to the data available in the Big Query project which is updated every second Tuesday of the month for the previous month . This is not the ideal solution if you want to monitor your Core Web Vitals in real time;
This method is not really scalable if you plan to monitor a larger number of domains.

How to Access Core Web Vitals Data with the CrUX Dashboard in Data Studio

Here are the steps to follow to access the data:

Create a copy of the model directly on Data Studio via g.co/chromeuxdash .
add the domain you are interested in,
click on “Create report”
view the report for the last available month.

If you receive an error, verify that you have added the domain name correctly.

It is also possible that your domain is not included in the BigQuery dataset. You can find more information about how this report works in Rick Viscomi ‘s article on web.dev.

The Search Console Core Web Vitals report

Google Search Console has integrated a Core Web Vitals report . This report is relevant but quite specific because the metrics display is done by property.

Core Web Vitals tab in Google Search Console for mobile

Search Console Core Web Vitals Report Highlights

You can access aggregated data for a group of pages compiled by Google , a unique approach that does not exist in other methodologies;
you can sort data by mobile and desktop;
you can extract the most recent data , that is, the average of the data collected over the previous 28 days starting from the last full day;
90 days of data are available , but only based on the number of URLs affected per group (good/needs improvement/poor);
The Google Search Console interface is very easy to use.

Weaknesses of the Search Console Core Web Vitals report

You can’t access data at the URL or origin level. Data is collected by similar URLs and aggregate metrics, which is nice, but makes it impossible to track the progress of each URL individually;
it is not possible to download individual URLs classified as “similar”, but only the total number;
you cannot segment data by tablet users;
no network connection information is available;
It is not possible to access historical data beyond the previous 90 days.

How to Access Core Web Vitals Data with Google Search Console

For now, the only way to extract the data is through the user interface. The report is split between mobile and desktop .

Each device category contains individual Bad/Needs Improvement/Good reports for each of the Core Web Vitals metrics ( LCP , CLS , FID ).

Mobile LCP analysis report with Google Search Console, with the ability to export the report. The number of similar URLs and the “Agg. LCP” are shown at the bottom right

Each report has an export function (CSV, Excel or Google Sheet) that allows you to download a table with the different sections:

the base URL,
the number of similar URLs,
the value of the metric per group.

This report also allows you to know the number of URLs concerned in this subsection over the last 90 days.

Choosing the right method to measure Core Web Vitals, optimize UX and SEO

As you can see, whatever your development skills, there are many ways to extract Core Web Vitals data from CrUX to monitor your websites, and also those of your competitors to position yourself in your market.

If you are comfortable, not afraid of programming, and are looking to monitor your Core Web Vitals on a regular basis on a larger scale, then the CrUX API will be the best solution!

Conversely, if you are more interested in general industry trends and don’t often need the data to track a large number of sites, then Data Studio’s CrUX dashboard will be the most suitable solution.

Keep in mind that measuring the performance of your websites, against Google’s ranking criteria through Core Web Vitals, is the first step to optimizing them.

So, before you start collecting, define your goals to know what data to collect, and for what purpose you want to improve it.

We warmly thank Jose Hernando , Senior SEO Consultant at Builtvisbile, who gave us permission to translate this very comprehensive article ( in VO here ).

Feel free to contact Jose for more details on these data measurement methods via Twitter or on his website