Skip to content

State of the Arabic Web

How do the homepages of the most visited sites on Arab country domains do when every one is checked by the same rules? Real numbers with confidence intervals, a method published before the crawl, and open data.

On this page
Soon

The first edition is coming soon

The crawl has not finished, and we will not show a number before it has: everything here will come from a real check of each site in the sample. Until then the sample and the method are published for anyone to review.

  • The headline numbers, each with its confidence interval and the number of pages judged.
  • A comparison of the Arab countries, and the share of platforms (Salla, Zid, WordPress, Shopify and others).
  • A data explorer where you pick the country, the platform and the metric.
  • The data files: Parquet, CSV and JSON with a JSON Schema, under CC BY 4.0.

The sample of sites

The sites are those of the Tranco list of the million most visited domains that sit on an Arab country domain. The sample is published before the crawl so that anyone can review it.

The sample is published before the crawl: 3,607 sites from Tranco list Y83YG (2026-10-01) on 19 country domains.

The sample of sites (CSV)The sites left out and the reason for each (CSV)

Sites in the sample, by country
CountrySites
United Arab Emirates734
Saudi Arabia568
Morocco521
Egypt332
Tunisia286
Algeria215
Qatar167
Jordan154
Iraq99
Oman92
Palestine89
Kuwait73
Lebanon57
Syria52
Bahrain49
Libya38
Sudan34
Yemen32
Mauritania15
Total3,607

We left 160 sites out of the list because they are not the Arabic web: link shorteners on domains sold to the world (such as bit.ly) and local mirrors of global companies (such as google.iq). Each of them, and why it was left out, is in a published file.

Method

What we measure

We check the homepage of each site, and only that, with the same Arablyzer rules any visitor checks their own site with: the code is deterministic, so the same page gives the same result. Each number is the share of the pages judged for that metric. A page a rule does not apply to (the direction check on a page that is not Arabic) is not in that number’s denominator.

  • Arabic rendering: letter-spacing that breaks the joining of letters, Arabic font problems, mixed digit systems, and the new Saudi riyal sign.
  • Direction and language: dir="rtl" and lang on Arabic pages.
  • Machine reading: words stretched with tatweel or holding invisible characters, text that is not in the HTML before scripts run, and AI search crawlers blocked in robots.txt.
  • Mobile: a right-to-left page wider than a phone screen, and a missing viewport tag.
  • Speed: the Core Web Vitals from the Chrome UX Report (real users on phones) where the site has data. We show no lab numbers, which change from run to run.
  • Security: HTTPS and five headers: HSTS, CSP, nosniff, framing protection and a referrer policy.
  • Structure: use of JSON-LD and whether it is well formed, the platform, and Arablyzer’s score.

«Letters pulled apart» means exactly letter-spacing on Arabic text, the one cause we detect with certainty in one browser. Shaping faults inside a font itself are not measured here.

The sample

From the Tranco list (a fixed list id, recorded) we take every domain that ends in an Arab country domain: .sa .ae .eg .kw .qa .bh .om .jo .ma .dz .tn .iq .lb .ly .sy .ye .sd .ps .mr.

If there are more than 10,000 sites, each country gets a floor and the rest is shared by what each has left, and inside a country sites are chosen by a hash with a published seed, so the choice is random with respect to rank and gives the same result on any machine. The real numbers for the list are in the «The sample of sites» section above.

Then we leave out what is not the Arabic web: 160 sites in this list, each recorded with its reason in a published file. We leave out every name directly under .ly (bit.ly and its like), so of .ly we keep only what sits under gov.ly, com.ly, edu.ly, org.ly, net.ly, sch.ly, plc.ly, med.ly and id.ly, and Libyan sites with a single name, picked one by one after reading their front pages; a few named hacks on other domains (lu.ma, cre.ma); and the country mirrors of global companies (Google, YouTube, Facebook, Instagram, Amazon, Microsoft, Apple, Wikipedia, Yahoo, Bing, Twitter and X, TikTok, LinkedIn, WhatsApp, Netflix and Booking). This comes before any sampling, so the size of each country’s list that weights the report is the size after it.

The sample file is a CSV with the domain, the rank and the country and nothing else, and the exclusions file gives the reason for each site left out.

How we crawl

  • The homepage only (https://, then http://, then www). We follow no link, submit no form and bypass no bot protection.
  • We read robots.txt as a crawler does: the * group counts, a block on ArablyzerBot is honoured, and a file we cannot read keeps us out. A site that blocks us is not checked.
  • We identify ourselves on every request with the user agent ArablyzerBot and a page that explains how to block it.
  • Visitors always come first: the crawl’s scan is last in the queue, and none is started while a visitor’s scan waits or runs or the server is busy, at a rate of no more than one scan every 20 seconds, one at a time.
  • Chromium alone by default. We keep no screenshots and no page text: for each site we keep a letter for the result of each rule and a few facts (the page’s language and direction, the platform, the CrUX figures and the score), and we delete the scan and its report straight away.

How we count

  • Every headline share is of the Arabic pages alone: a page whose declared language is ar (or one of its varieties, such as arz) or whose text is mostly in Arabic script. The other pages on Arab domains (English, French and the rest) are counted on their own, and their share is shown as a figure of its own: «of the pages on Arab domains are not in Arabic».
  • Each share has a 95% confidence interval by Wilson’s method.
  • The «whole sample» share is weighted by the size of each country’s list, not by how many of its pages we reached, so a small country that was given a floor does not outweigh a large one. Its interval uses the effective sample size that the weighted variance implies.
  • A country’s or a platform’s share is the share of its pages, unweighted.
  • We show no cell with fewer than 30 pages judged.

The interval measures the sample’s variation only. It does not cover the sites we did not reach (blocked, down, or behind protection) or a rule that misjudges a page. That is why we publish what happened to every site in the sample.

Limits of this report

  • A country domain is not the country’s audience: we leave out the link shorteners and the global companies’ mirrors we know of, and the sites that remain and are not in Arabic are in no share. On the other hand the .ly rule can drop a Libyan site we did not recognise, and most of the Libyan sites we put back were picked because their page title is in Arabic, so Libya’s share of Arabic pages is not comparable with another country’s.
  • We do not cover Arabic sites on .com, .net and the like, nor internationalised domains, nor sites outside the top million. A later edition may add a second source.
  • One homepage per site: it does not stand for the other pages or for what is behind a login.
  • The result of one rule is not a verdict on a site’s quality.
  • CrUX figures need a key and enough data for the site, so some sites have none.

Ethics and opting out

  • We publish no site’s result. The data files are aggregates only: no row in them is a site, not even under a hash. The one place with site names is the sample list, published before the crawl, with the domain, the rank and the country and no result.
  • If you do not want your site checked, add User-agent: ArablyzerBot and Disallow: / to your robots.txt, or write to [email protected] and we take your domain out of the sample, the crawl and later editions.
User-agent: ArablyzerBot
Disallow: /

The crawler judges by the site’s own rules, and each has a page in the rule library explaining how it decides.