Tech Report: Technologies - major.minor versions granularity

[max-ostapenko]

Related to HTTPArchive/httparchive.org#984

As the aggregation changes we have new schemas, and new tables for tech report.

I placed them in reports dataset:

• tech_crux (successor of core_web_vitals.technologies)
• tech_report_adoption
• tech_report_categories
• tech_report_core_web_vitals
• tech_report_lighthouse
• tech_report_page_weight
• tech_report_technologies
• tech_report_versions

Notes:

• removed a few columns from tech_crux (as compared to core_web_vitals.technologies):

  • category
  • origins_with_good_cwv_2023 and origins_with_good_cwv_2024 - deduplicated in origins_with_good_cwv

• all the metrics have 'ALL' version that aggregates at technology level and expected to match the current values:
  

• corresponding to the current approach tech_report_versions has full adoption data from crawl.pages and tech_report_adoption has the smaller absolute values because of the JOIN with CrUX.
  

• example of the technology versions:

SELECT
  version,
  origins
FROM `reports.tech_report_versions`
WHERE technology = 'WordPress' AND
  client = 'mobile'
ORDER BY
  origins DESC

[max-ostapenko]

@tunetheweb @rviscomi FYI
removed a few columns from tech_crux (as compared to core_web_vitals.technologies which is to be deprecated):

• category
• origins_with_good_cwv_2023 and origins_with_good_cwv_2024 - deduplicated in origins_with_good_cwv

[max-ostapenko]

Some test cases for the version pattern:

SELECT
  REGEXP_EXTRACT(version, r'(?:(?:0|[1-9])\d*)(?:\.(?:0|[1-9])\d*)?')
FROM UNNEST(['1.2.3', '01976.2.83', '0003.3.4', '0.0.1', '1.2', 'version 5.1.2', '8']) AS version

1.2
01976.2
0003.3
0.0
1.2
5.1
8