Outlier Detector: Find Anomalies with IQR and Z-Score

What is Outlier Detection?

Outlier detection identifies data points that deviate so strongly from the bulk of the data that they deserve individual attention. Outliers matter because they can silently distort every statistic you compute afterward — a single mistyped value can double a mean, inflate a standard deviation, or flip a regression slope.

This tool runs two complementary methods at once. The IQR method (Tukey fences) flags anything below Q1 − 1.5×IQR or above Q3 + 1.5×IQR; because it is built on quartiles, it resists the very outliers it is hunting. The Z-score method flags values more than 3 standard deviations from the mean; it is intuitive and familiar but assumes approximate normality and can be masked by the outliers themselves.

Points flagged by both methods are strong outliers — investigate them first. Points flagged by only one method are borderline: usually a sign of skewed data (where Z-score over-flags) or masking (where Z-score under-flags). Either way, the verdict is "look closer", not "delete".

What to Do With a Flagged Value

Data-entry error? A human height of 17.2 m is a units or typo problem — fix or remove it, and document the change.
Measurement artifact? Sensor glitches and instrument limits create impossible values; exclude with justification.
Genuine extreme? A real billionaire in an income survey is valid data — keep it, but report medians alongside means and consider robust methods.
Wrong model? Many "outliers" can mean your data mixes two populations; try splitting the dataset or a different distribution.

Worked Example: House Prices

Load the House Prices sample: 18 values, mostly $250k–$600k, with 800, 1200, 2500, and 3000 (thousands) at the top. Sorted, the middle values give Q1 ≈ 320 and Q3 ≈ 525, so IQR ≈ 205 and the upper Tukey fence is 525 + 1.5×205 ≈ 832. Everything above ~832 — the four top values — is flagged by the IQR method.

The Z-score method flags fewer points, and for a reason worth understanding: those four expensive houses pull the mean up to ~735 and inflate the standard deviation to ~700, so even the $3M house sits barely 3 SD above the mean. The outliers are masking themselves. This is exactly why the IQR method is the safer default for skewed data.

The right response depends on context. If these are mansions in a dataset of suburban homes, they are genuine — but any average you report should be the median (~$380k), not the mean. If they came from a different city's luxury market by mistake, they are contamination and should be split out.

Common Misreadings

Deleting outliers by default. An outlier is a question, not an error. Removal requires a reason you can write down.
Repeating detection after removal. Re-running fences after each deletion can strip legitimate tail data one layer at a time.
Trusting Z-score on skewed data. Mean and SD are pulled toward extremes; use IQR or median-based methods for skewed distributions.
Ignoring borderline flags. A point just past one fence often signals skew or a mixed population rather than a bad value.
Forgetting the business context. In fraud detection and quality control, the outliers are the signal — the whole point is not to discard them.

Frequently Asked Questions

How are outliers detected?

Two methods run simultaneously: IQR-based Tukey fences (values beyond Q1 − 1.5×IQR or Q3 + 1.5×IQR) and Z-score thresholding (values more than 3 standard deviations from the mean). Points flagged by both are strong outliers.

Which outlier method should I use?

IQR is robust to non-normal distributions and is the industry standard. Z-score is sensitive to extreme values and works best for approximately normal data. When the two methods disagree, trust the IQR result for skewed data.

Should I delete outliers from my data?

Almost never automatically. An outlier is a signal, not noise by default: it may be a data-entry error (fix or remove), a genuine extreme observation (keep and report), or evidence that your model is wrong. Investigate the cause before deciding.

Why can Z-score miss outliers?

Because the mean and standard deviation are themselves pulled toward extreme values — an effect called masking. A single huge outlier inflates the standard deviation so much that its own Z-score shrinks below the threshold. IQR, based on quartiles, resists this.

What is a borderline outlier?

A value flagged by only one of the two methods, or one sitting just beyond a fence. Treat it as 'worth a second look' rather than 'wrong': check the measurement, the units, and whether the value is physically possible.

Key Terms

IQR Method: Data points outside [Q1−1.5×IQR, Q3+1.5×IQR] are flagged as outliers.
Z-Score Method: Values with |z| > 3 are considered statistical outliers.
Tukey Fences: The boundary defined by Q1−1.5×IQR and Q3+1.5×IQR.

Computation Method

Quartiles are calculated using linear interpolation (method 7 from Hyndman & Fan, 1996), consistent with Python's NumPy and pandas defaults.
Outlier detection uses the Tukey fences method: lower fence = Q1 − 1.5 × IQR, upper fence = Q3 + 1.5 × IQR.
Chart scaling caps the display axis at the upper fence value so the visualization remains clear even when extreme outliers are present.

References

Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley.
Iglewicz, B. & Hoaglin, D. C. (1993). How to Detect and Handle Outliers. ASQC Quality Press.
Barnett, V. & Lewis, T. (1994). Outliers in Statistical Data. 3rd ed. Wiley.