Sites Data Inconsistency Report: Purpose and Interpretation

The Sites Data Inconsistency Report helps to identify sites whose data distributions and/or time patterns differ from the study-wide reference more than expected by random variation. In other words, to find sites that “look different” (too high/low, unusual spread, odd visit timing trends), so you can check for process issues, measurement differences, or data problems. 

To access the report navigate to the Sites List page and click Data Inconsistency Report. The Combined Data Inconsistency score (Z-score) per site is displayed in the sites overview table and indicates standardized distance from the reference (the higher the score the more unusual the site behaves).

How to read the report (workflow)

1. Start with the Combined Z-score chart: pick the sites above the confidence threshold. 
2. Use the Overview heatmap table: see which test(s) drive the site’s overall score. 
3. Click a cell: the Statistical Test Details section updates to explain that specific signal (distribution or time trend). 
4. Interpret with context: confirm if differences are clinically plausible or suggest data capture / lab / operational issues. 

Chart 1 — Combined Sites Inconsistency Score (Z-score)

What you see 

• A bar chart by Site ID, sorted from highest to lowest. 
• A horizontal “Confidence threshold” line (value 3). 
• Background shading indicating “below vs above threshold”. 

How to interpret 

• Z-score = standardized distance from the reference (in SD units). 
  • Higher Z-score ⇒ more unusual site overall. 
  • Example intuition: Z = 3 means “about 3 standard deviations away” (rare under normal assumptions). 
• Use this chart to prioritize: 
  • Above threshold: investigate first. 
  • Just below threshold: monitor, especially if trending up over time. 

Table 1 — Individual Statistical Tests Overview (heatmap table) 

What you see 

• Rows = sites 
• Columns grouped by domain/parameter (e.g., Lab Hemoglobin, Platelet count, WBC count, Neutrophils, etc.) 
• Within each group you typically see: 
  • P-Val 
  • KS P-Val 
  • Adj. P-Val 
  • Percentile 
  • For some tests: Timeseries (TS) 

How to interpret the color + numbers 

• Darker cells = stronger signal (more inconsistent vs reference) for that statistic. 
• Click any cell to open the exact supporting plot below. 
• higher = more significant. 
• Once a Z-score exceeds conventional significance thresholds (around 3), further increases primarily indicate greater statistical certainty rather than proportionally greater practical severity. In applied site inconsistency, Z-scores beyond this range are therefore better interpreted as confirming the presence of inconsistency rather than scaling its magnitude. 
• Empty cells are shown for those sites and statistics that have too less values to be statistically meaningful.

Column meanings 

Export Data

The table with all sites can be exported as csv file by clicking the download button in the header of the table.

Section — Statistical Test Details (updates when you click a cell) 

A) Timeseries details — “GAM baseline vs actual … (TS)” 

What you see 

• Scatter points for the selected site across relative study day / data value. 
• A smooth baseline curve labeled GAM baseline (Generalized Additive Model). 

• A shaded 95% confidence interval band around the baseline. 

  
  

How to interpret 

• If points mostly stay within the band: site follows the expected time pattern. 
• If points systematically shift above/below the baseline: possible site-specific bias (e.g., consistently higher values). 
• If deviations appear only in certain time windows: possible phase-specific issue (startup training, device change, local lab change, visit scheduling artifacts). 

• Wider confidence band typically means less information / more uncertainty in that time region. 

  
  

Typical follow-ups 

• Check lab vendor / calibration, collection timing, unit conversions, visit window handling, data entry delays, protocol deviations at that site. 

B) Distribution details — “Kernel Density Estimation plot: Site vs Reference” 

(Shown when selecting P-Val or Adj. P-Val) 

What you see 

• Overlaid density/histogram-like distributions: 
  • Reference (all other sites or pooled data) 
  • Selected site 
• A small summary table with metrics such as: 

  • Central Tendency (mean/median-like) 

    Mean < Median: left/negative skew; investigate low values.

  • Variability (Standard Deviation, Mean Absolute Deviation) 
    • Standard deviation ≈ Mean Absolute Deviation: variability is evenly distributed; few extreme values.
    • Standard deviation >> Mean Absolute Deviation: large number of outliers and heavy tails
  • Quantile Difference (25% and 75%)

  • Sample Size, Missing/Unidentifiable 

    Number of data points with NA values

  • Outlier ratio (%)

    Percentage of data source values which are 3*1.4826⋅MAD from median

How to interpret 

• Shift left/right: site has lower/higher typical values. 
• Narrower/wider shape: site has less/more variability than reference. 
• Different shape (skew/multimodal): may indicate mixed populations, data mixing, measurement/process differences, or data quality issues. 
• Small sample size: treat signals cautiously (higher noise / instability). 

Plain-language checks 

• “Are site values consistently higher/lower than everyone else?” 
• “Is the spread suspiciously tight (too perfect) or too wide (unstable process)?” 

C) Distribution details — “Cumulative Distribution Function (CDF): Site vs Reference” 

(Shown when selecting KS P-Val and also in the Percentile example.) 

What you see 

• Two step-like curves: 
  • Reference CDF 
  • Site CDF 
• The KS test is driven by the maximum vertical gap between these curves. 

How to interpret 

• Curves overlap closely: distributions are similar. 
• Consistent separation across the range: systematic difference (location/scale). 
• Separation mainly in tails: site differs in extremes/outliers (e.g., unusually high values). 
• Largest vertical gap point: where the site diverges most from reference. 

Plain-language meaning 

• “At any given value, does this site accumulate observations faster/slower than normal?” 

  (i.e., “Does the whole pattern of values look different?”) 

Practical interpretation tips (quick rules) 

• High combined Z-score + multiple dark cells: 

  Broad inconsistency → investigate site operations. 

• High combined Z-score driven by one parameter: 

  Focused issue → investigate that domain (e.g., specific lab). 

• Strong signal + low sample size: 

  Verify with more data / monitor trend before escalation. 

• Always combine statistics with clinical plausibility and operational context (country, lab, device, training date, vendor changes).