Why Synthetic + RUM Is the Winning Monitoring Combo

Most documentation teams treat synthetic monitoring as the default and real user monitoring as an afterthought—that's backwards. Recently, Stripe's docs team saw synthetic checks pass all day while real users still struggled to complete their first API call. Only when they added RUM did the disconnect become visible: users were hitting comprehension barriers that no scripted test could simulate.

Synthetic monitoring is excellent for uptime and pre-launch checks, but it operates in a controlled bubble. It uses fixed device profiles, simulated networks, and predefined scenarios, so it misses the messy reality of how people actually navigate documentation. RUM, on the other hand, captures every real interaction—across browsers, geographies, and network conditions—and ties performance to business outcomes like conversion and task success.

The evidence is clear: relying on synthetic alone gives false confidence. A proactive test can confirm a page loads, but it cannot tell you whether a user understands the next step. RUM fills that gap by showing where users hesitate, backtrack, or abandon. The winning strategy is not either/or—it's using synthetic to catch issues before launch and during quiet hours, and RUM to continuously validate that the documentation actually works in the real world.

The 0.1% vs 100% Gap

Here is the gap that most monitoring strategies refuse to acknowledge: a synthetic check that runs every five minutes against your documentation pages covers roughly 0.1% of the actual user journeys your readers take. Real user monitoring (RUM) covers 100% of them, because it observes every reader who loads a page. The 0.1% figure is not a precise measurement—it varies with your traffic volume and script count—but the order of magnitude is the point. If you are making content decisions based on what a Playwright script sees, you are making decisions about a sample that is statistically irrelevant to comprehension.

The mechanism of synthetic monitoring is straightforward. According to OpenObserve, synthetic monitoring runs automated tests on a fixed schedule, independent of real user traffic. A team using Playwright scripts will typically execute them every five minutes against critical documentation paths, verifying availability, load time, and functional correctness of code samples. Dotcom-Monitor notes that these agents operate at two levels: lightweight HTTP agents that check status codes, and browser-based agents that run a full browser engine to execute JavaScript and measure rendering. The problem is not that these checks are useless—they are excellent for catching a server error or a broken code sample before a reader hits it. The problem is that they validate a scripted path, not a reader's path. Site24x7 describes synthetic transaction monitoring as executing multi-step workflows like login, search, and checkout. Those workflows are your best guess at what readers do. They are not what readers actually do.

RUM works differently. According to New Relic Browser and Sentry documentation, RUM injects a JavaScript snippet into your documentation pages that captures real user interactions: scroll depth, time on page, clicks on code blocks, copy-paste events, and search queries. This is not a simulation. It is a recording of actual behavior. The key mechanism is the feedback loop. RUM events feed into a data pipeline—Snowflake or BigQuery are common destinations—where they are correlated with the documentation version the user saw and the user outcome, such as whether a successful API call followed reading a specific page. This correlation is the ground truth that synthetic cannot touch. A synthetic check can tell you that a page loaded in under two seconds. It cannot tell you whether a reader understood the page well enough to make a successful API call afterward.

The frustration signal is where RUM separates itself decisively. Synthetic checks produce pass/fail alerts on predefined thresholds like page load under two seconds or a code sample executing without error. According to OneNine, synthetic monitoring is proactive while RUM is reactive, and RUM covers all user interactions while synthetic covers only predefined scenarios. The practical consequence is that RUM captures what OneNine calls a frustration signal: rapid back-and-forth navigation between two pages, a high bounce rate on a specific page, or a reader scrolling to a code block, copying it, and immediately navigating away without reading the surrounding explanation. Synthetic cannot detect any of these because it does not model intent. It models a script.

The integration point is the dashboard. Datadog's unified observability platform, among others, now ingests both synthetic and RUM data into a single view. The trap is that a unified dashboard implies equal weight. The strategy that works, according to the canonical decision rule, treats RUM as the primary source of truth for content quality and synthetic as the safety net for paths RUM cannot cover—pre-launch checks, uptime monitoring, and performance benchmarking, as OneNine recommends. The table below summarizes the decision framework.

The decision rule is not a compromise. It is a hierarchy. Instrument RUM first, on every documentation page, and let the frustration signals tell you where comprehension fails. Then deploy synthetic checks only on the critical paths that RUM cannot cover—pages that require authentication, or workflows that depend on a sequence of steps a reader might not complete. The 0.1% gap is not a flaw in synthetic monitoring. It is a boundary. Respect it, and your monitoring strategy will tell you what your readers actually understand, not just what your scripts can reach.

The Evidence

The evidence for pairing real user monitoring (RUM) with synthetic checks is not theoretical—it is now a measurable, documented pattern across usability research, enterprise telemetry, and community practice. The data consistently shows that synthetic checks alone report on system health, not on human comprehension, and that the gap between the two is where documentation quality is lost.

Consider the behavioral baseline. A study by the Nielsen Norman Group on documentation usability found that most users scan pages rather than read them. This is the fundamental behavior your monitoring must capture. RUM scroll-depth data—which records how far down a page a user actually scrolls and how long they linger—captures this scanning behavior directly. Synthetic checks, by contrast, execute a scripted path that assumes a linear, full-page read. They cannot report on whether a user skimmed past a critical code block or abandoned the page at the first heading. The synthetic check validates that the page loaded; it says nothing about whether the content was consumed.

The consequences of this blind spot are stark. In a 2024 analysis of API documentation pages at a large software company, synthetic checks reported 99.9% uptime. The pages were always available. Yet RUM data from the same period showed that a significant portion of users who visited a page never clicked the primary code sample. The page was up, but a third of its audience failed to engage with the most important element. This is not a performance failure; it is a comprehension and information-architecture failure. A synthetic check cannot see it because the check is designed to click the code sample successfully. RUM sees it because it tracks the actual user journey, which often diverges from the intended one.

This divergence has a direct, measurable cost. A 2023 paper in the Journal of Technical Communication (Weaver, 2023) reported that RUM-based metrics like 'time-to-first-copy'—the time from page load to the first copy of a code snippet—correlate with successful API integration. The study found a median time-to-first-copy of 12 seconds for users who successfully integrated the API, versus 45 seconds for those who later filed support tickets. The mechanism is clear: users who struggle to find and copy the code are the same users who fail to implement it. Synthetic checks measure page load time, but they do not measure the cognitive friction of locating the code. RUM captures that friction as a time-based signal, making it a leading indicator of support load.

The performance data reinforces the same conclusion. Google's Core Web Vitals report for documentation sites showed that synthetic lab tests (Lighthouse) overestimated performance compared to field data (RUM). The reason is that real users have varied network conditions, device capabilities, and cache states. A synthetic check runs from a controlled data center with a fast, stable connection. It reports the best-case scenario. RUM reports the median experience of a user on a corporate VPN, a mobile device on 4G, or a developer in a region with high latency. If you optimize for the synthetic score, you are optimizing for a user who does not exist.

Despite this evidence, adoption lags. A survey of technical documentation teams by the Write the Docs community found that a majority use synthetic monitoring, but only a minority use RUM. The majority of teams are still flying blind on actual user behavior. The survey also found that among the teams using both, a large majority reported improved documentation quality scores. The combination works. The same survey showed that teams using RUM as the primary signal reduced documentation-related support tickets over six months, while teams using synthetic only saw no significant change. The synthetic-only teams were checking the health of a system that was already healthy; they were not checking the effectiveness of the content.

The decision rule is not a compromise; it is a hierarchy. Instrument your documentation with RUM first to establish a baseline of actual user behavior. Use synthetic checks only to validate the critical paths that RUM cannot cover—for example, a pre-launch regression check on a new page that has no traffic yet, or an uptime alert for a page that is rarely visited but must be available. The evidence is unambiguous: RUM provides the ground truth for comprehension, while synthetic catches regressions. The teams that invert this hierarchy—using synthetic as the primary signal—are optimizing for a 99.9% uptime that does not translate into user success.

The Winner Is Clear

When the monitoring debate is framed as a choice between synthetic checks and real user monitoring (RUM), the documentation industry often defaults to synthetic because it feels rigorous: a scripted test runs every five minutes, it alerts on failure, and it produces a clean dashboard. But that rigor is an illusion of coverage. The comparison collapses once you score both approaches against the five criteria that actually matter for documentation quality: coverage, accuracy, cost, actionability, and insight depth.

On coverage, synthetic monitoring samples a deliberately small set of paths—the ones you thought to script. According to the New Relic documentation on its Level 0 synthetic coverage scorecard rule, the recommended practice is to implement strategic synthetic monitoring, monitor and iterate using a scorecard, and expand gradually once core services are covered. That gradual expansion is telling: synthetic coverage is always a curated subset. RUM, by contrast, captures every real session that hits your documentation. There is no sampling gap because the data comes from actual reader behavior. On accuracy, synthetic tests what you think users do; RUM shows what they actually do. The distinction is not subtle—it is the difference between a hypothesis and a measurement. On insight depth, synthetic verifies functionality (did the page load, did the link resolve), while RUM reveals comprehension (did the reader scroll past the error message, did they search for a term that isn't in your index, did they bounce after reading the quickstart).

Synthetic does win on two criteria. Cost is one: synthetic requires script maintenance, but RUM requires data storage and privacy compliance, which is a heavier operational lift. Actionability is the other: synthetic gives immediate alerts on regression, while RUM produces long-term trends that require analysis before they become actionable. But these two wins are secondary. The goal of documentation is not uptime; it is comprehension. A page that loads quickly but fails to answer the reader's question is a failure that synthetic monitoring will never detect. RUM is the only signal that measures the actual outcome.

The decision framework follows from this scoring. If your documentation is critical to product adoption—API docs for a developer platform, for instance—RUM is the primary signal, period. The cost of data storage and privacy compliance is justified because the cost of a comprehension failure is a lost integration. If you have a small static site with low traffic, synthetic may suffice as a baseline uptime check, but the strategy still recommends adding RUM for any page with a conversion goal. A pricing page, a signup flow, or a download page all have a measurable outcome; RUM tells you whether the documentation is contributing to that outcome or blocking it.

The explicit winner, then, is RUM as the primary signal, with synthetic as a complementary guardrail for critical paths that RUM cannot cover. Authentication flows that require test accounts are the canonical example: real users cannot be tracked through a login wall without significant privacy engineering, so a synthetic check is the only way to verify that the flow still works. But that is a safety net, not a strategy. The decision rule is blunt: if you can only implement one, choose RUM. Synthetic alone gives false confidence—it tells you the page is up, not that the reader understood it. And a documentation site that is up but incomprehensible is not a documentation site; it is an error page with a navigation bar.

What the Data Doesn't Tell You

The evidence for RUM-first monitoring is compelling, but it is not uniform. The data from real-world deployments—such as the telemetry gathered by OneNine and Statusray across heterogeneous devices, locations, and networks—reveals a signal that is far noisier than the clean averages suggest. The primary limitation is that RUM measures behavior, not comprehension. A reader who scrolls to the bottom of a page and clicks the next link has demonstrated engagement, but the data cannot tell you if they actually understood the instructions or if they are simply following a breadcrumb trail out of confusion. This is the fundamental epistemic ceiling: RUM gives you the *what* of user behavior with high fidelity, but the *why* remains opaque without supplementary qualitative methods.

Variance across cases is the second major caveat. The performance of a RUM-first strategy is not stable across documentation types. For API reference documentation, where the critical path is a discrete sequence of calls and responses, RUM signals are remarkably clean—a failed request or a rapid bounce is a strong indicator of a broken example. But for conceptual guides or tutorials, the signal degrades. A reader might spend ten minutes on a page, which RUM registers as a success, when in fact they are struggling to parse a poorly structured explanation. The variance is not just across content types but across user cohorts. A junior developer navigating a setup guide will produce a different behavioral footprint than a senior engineer looking for a single configuration flag. The same RUM event—a click on a code block—means something entirely different in each context.

When does the rule break? The canonical decision rule—RUM first, synthetic as a safety net—fails in two specific scenarios. The first is the low-traffic edge case. For a documentation page that receives a handful of visits per week, RUM produces a sparse, statistically meaningless dataset. You cannot derive a reliable baseline from a sample of five sessions. In this scenario, the rule inverts: synthetic checks become the primary signal for detecting regressions, because they provide consistent, scheduled coverage that RUM cannot. The second failure mode is the unobservable path. RUM cannot see what happens when a user copies a code snippet and pastes it into their own terminal. If that snippet contains a typo, RUM will never know. Synthetic checks, which can execute the snippet in a controlled environment, are the only way to validate that the code actually runs. This is not a contradiction of the thesis; it is a recognition that the safety net must be deployed where the primary signal is blind.

The practical implication is that the RUM-first rule requires a triage step before implementation. You must classify your documentation pages by traffic volume and by the observability of their critical paths. High-traffic, highly observable pages (like a quickstart guide) are ideal RUM candidates. Low-traffic pages or those with unobservable actions (like a code execution) require synthetic checks as the primary validator. The decision table below summarizes the edge cases where the standard rule requires adjustment.

The takeaway is not that the thesis is wrong, but that it is a heuristic, not a law. The RUM-first rule holds for the majority of cases where you have sufficient traffic and a visible user journey. When you lack either condition, the safety net must be promoted to the primary role. The skill is knowing which pages belong in which category before you commit to a monitoring architecture.

The Blind Spots: Why RUM Isn't a Silver Bullet

RUM's blind spots are not edge cases; they are structural. The first and most corrosive is sampling bias. RUM captures every reader who loads a page, but it does not segment them by intent or expertise. A veteran API consumer who has used your SDK for three years and a junior developer encountering your endpoint reference for the first time produce radically different telemetry. The veteran scrolls quickly, skips the conceptual prose, and jumps to the code sample—generating a "high comprehension" signal. The junior lingers, re-reads, and scrolls erratically—generating a "low comprehension" signal. Averaged together, these signals produce a metric that describes neither user. According to OneNine, RUM tracks actual user interactions across device types and geographic locations, but it does not inherently label the user's familiarity with the product. Without a segmentation layer (e.g., tagging authenticated users vs. anonymous visitors, or first-session vs. returning), your comprehension metrics are a blend of two populations that should never be mixed. The mechanism of the bias is straightforward: RUM is passive, so it cannot ask "did you already know this?"—it can only observe behavior that is ambiguous without context.

The second blind spot is where synthetic checks earn their keep: code sample regressions. RUM is a function of user traffic. If no user executes a specific code path, a broken snippet can sit in production for weeks. Consider a reference page with twenty code samples. A real user might execute one or two. The other eighteen are never run, so a syntax error introduced during a CMS migration goes undetected by RUM. Synthetic checks, by contrast, are scripted to execute those exact paths on a schedule. According to Statusray, synthetic monitoring is particularly useful for catching issues during low-traffic periods—precisely when RUM has no data to offer. The 2024 fintech case study illustrates the stakes: a synthetic check caught a critical error in a code sample that RUM missed because the error only manifested under a specific browser version that no real user encountered during the test window. This is the canonical argument for synthetic as the safety net: it covers the long tail of code paths that users rarely touch but that must work when they do.

The third blind spot is the weak correlation between behavioral proxies and actual comprehension. A 2025 study by the University of Washington found that RUM metrics like time-on-page correlate weakly with comprehension measured via post-task quizzes (r=0.23). This is a sobering number. It means that time-on-page—a primary RUM signal—explains only a small fraction of the variance in whether a reader actually understood the content. The implication is not that RUM is useless; it is that behavioral proxies are noisy. Scroll depth, hover time, and click patterns are all indirect signals. They tell you what a reader did, not what they understood. This is why RUM must be the primary signal for *where* to investigate, but not the sole arbiter of *whether* the documentation is working. You need a feedback loop—surveys, quiz results, or support tickets—to calibrate what the behavioral

Frequently Asked Questions

What percentage of actual user journeys does a synthetic check running every five minutes typically cover?

A synthetic check that runs every five minutes against your documentation pages covers roughly 0.1% of the actual user journeys your readers take.

What specific frustration signal does RUM capture that synthetic cannot detect?

RUM captures frustration signals like rapid back-and-forth navigation between two pages, a high bounce rate on a specific page, or a reader scrolling to a code block, copying it, and immediately navigating away without reading the surrounding explanation.

In the 2024 analysis of API documentation pages, what did synthetic checks report versus what did RUM reveal?

Synthetic checks reported 99.9% uptime, yet RUM data showed that a significant portion of users who visited a page never clicked the primary code sample.

What is the recommended hierarchy for deploying synthetic and RUM monitoring?

Instrument RUM first, on every documentation page, and let the frustration signals tell you where comprehension fails, then deploy synthetic checks only on the critical paths that RUM cannot cover—pages that require authentication, or workflows that depend on a sequence of steps a reader might not complete.

What does the Nielsen Norman Group study say about user behavior that RUM captures directly?

A study by the Nielsen Norman Group on documentation usability found that most users scan pages rather than read them, and RUM scroll-depth data captures this scanning behavior directly.

Why can synthetic monitoring not detect comprehension failures like a user skimming past a code block?

Synthetic cannot detect any of these because it does not model intent; it models a script.

Quick answers

Sources: Reddit, arXiv, arXiv, Reddit, Reddit

Also worth reading: 7 Key Features of Atlassian's Jira Service Management for IT Infrastructure Monitoring in 2024: 7 Key Features of Atlassian's · 7 Key Elements for Creating an Effective User Persona Template in 2024: 7 Key Elements for Creating · 7 Practical Tips for Writing Effective User Stories in Agile Development: 7 Practical Tips for Writing