Why shouldn’t we trust content just because it contains numbers?

ProcrustesPR

Numbers can make a claim look precise, but they do not guarantee that its source is genuine, that its calculation method allows a valid comparison, or that its conclusion is supported by evidence. The following three examples illustrate different problems: an author may deliberately invent percentages as an example; reporting periods and what gets counted can affect totals; and explanations for changes in data may lack experimental support. Assessing credibility therefore requires more than asking whether numbers are present. We also need to ask where they came from, what they count, and which conclusions they can support.

ProcrustesPRProcrustesPRHow should I divide my DIY SEO budget?

A claim that appears to be backed by research may be fabricated

At the beginning of a 2019 Semrush article, author Chris Lewis cites a “Stanford study” and percentages to claim that people are more likely to believe statements containing numbers. He then immediately reveals that he invented both the study and the figures, and that the accompanying link leads not to research but to content about a ball game.

This passage cannot be cited as “research confirming that numbers are more persuasive.” It is an example devised by the author to remind readers that even an institution’s name, a year, percentages, and a link together do not guarantee that a claim is backed by evidence. The author also notes that content creators sometimes fail to realize they are using misleading figures. This is a caution offered by the article, not a quantitative study showing how widespread the practice is.

Before comparing totals, check the reporting periods and what gets counted

In a 2024 article, Ahrefs’ Patrick Stox criticized what some of Semrush’s figures included at the time. He described a reporting period that covered six months and then extended back to the beginning of the preceding month. Depending on the date the data was viewed, the actual period could be roughly six and a half months or nearly seven months, rather than a fixed six months.

The author argued that the extra days, together with lost links and lost referring domains that were still being counted, could increase the displayed figures and make them misleading. He also saw no sufficient reason to include lost links. This was the author’s assessment, however, not an official statistical rule that applies universally. It also does not establish that the platform still uses the same approach today.

  • Do the two totals cover the same period?
  • Do they count the same things—for example, do they include lost links?
  • Does the larger total actually answer the question the comparison is meant to address?

Observing a change does not mean an explanation has been confirmed

A July 2026 Search Engine Journal report described an experiment that found a decline in organic search clicks when AI Overviews appeared. The decline was concentrated mainly in informational queries. The samples for navigational and transactional queries were smaller, and no change was detected, so the findings cannot be applied across all queries.

Google’s explanation was that some of the lost clicks were low-value visits in which users clicked through and quickly left. According to the report, however, the additional clicks gained when the overviews were removed performed no worse in terms of bounce rate, time spent, or returns to search. The experimental data therefore did not support that explanation. This does not prove that every lost click would have led to a conversion, either. At the time of the report, Google had not released segmented click data that could clarify the issue. What was observed, how it is explained, and what remains unverified must be considered separately.

ProcrustesPRProcrustesPRWhy is AI not mentioning my brand?

The credibility of numbers: perspectives from across the web

Does a low number of mobile visitors mean that improving the mobile experience would offer little benefit?

You cannot draw that conclusion from current visitor numbers alone. One comment on PTT suggested that a low share of mobile visits could also result from the desktop site being difficult to browse on a phone. If so, the current figures may not reflect demand after improvements. However, this is only a possible explanation; the comment does not establish that browsing difficulties caused the low visitor numbers.

If different tools show different IP counts, can we conclude that the tool with the larger count is deliberately inflating its figures?

No. The Ahrefs article quoted an unidentified speaker who suggested that tools might store data differently: some store a single canonical IP per domain, while Semrush appears to store IPs per link, potentially producing higher and misleading figures. However, the speaker also said they did not believe the difference was caused by deliberate inflation. The description of storage methods was speculative and should not be treated as a confirmed account of current practices.

ProcrustesPRDoes an AI brand mention bring website visitors?

References

  1. [討論] 公司的網站真的需要手機版嗎?www.ptt.cc — Discusses: Does a low number of mobile visitors mean that improving the mobile experience would offer little benefit?
  2. You Can’t Compare Backlink Counts in SEO Tools: Here’s Whyahrefs.com · 2024-03-13T17:14:56+00:00 — Discusses: If different tools show different IP counts, can we conclude that the tool with the larger count is deliberately inflating its figures?
  3. What Google Trends Teaches Us About When To Trust Datawww.semrush.com · 2019-08-30T07:00:00Z — Source used in the article above.
  4. Where Clicks Go, What Agents Skip, Who’s Leaving Bing – SEO Pulsewww.searchenginejournal.com · 2026-07-03T12:30:55+00:00 — Source used in the article above.

Related articles

How should I divide my DIY SEO budget?Why is AI not mentioning my brand?Does an AI brand mention bring website visitors?How Long Does It Take to Evaluate SEO Results?How do I write titles customers want to click?How is organic traffic different from paid traffic?Why do I have keyword rankings but no customers?Are Google Maps rankings the same as website SEO?
ProcrustesPR
ProcrustesPR