Why Row Counts Aren't Enough for Data Quality

A row-count check is one of the first controls engineers add to a data pipeline, and for good reason. It is cheap, easy to explain, and capable of catching obvious failures. But it is also easy to overestimate what it proves.

A dataset can have the expected number of rows and still be wrong.

This article presents a simple framework for moving from one-dimensional validation toward behavioral data-quality monitoring.

What a row count actually tells you

Suppose yesterday’s pipeline produced 100,000 records and today’s pipeline produces 99,800.

That looks reasonable. But the count alone cannot tell you that:

• 20% of a critical field became null;
• every timestamp shifted by five hours;
• duplicate keys replaced valid records;
• one source region disappeared while another duplicated;
• data stopped updating six hours ago;
• values moved outside their valid range.

The row count measures volume. Data quality has more dimensions.

Build checks in layers

I like to think about production validation in five layers.

1. Freshness

Ask whether the data is recent enough for its intended use.

Useful signals include the newest source timestamp, newest ingestion timestamp, and time since the last successful source change.

Freshness checks catch a pipeline that continues to run against stale inputs.

2. Volume

Row counts still belong here, but compare them against context rather than a single hard threshold.

Useful comparisons include:

• previous run;
• same weekday last week;
• rolling seven-day average;
• expected source-specific range.

A sudden change can then be scored relative to normal behavior.

3. Integrity

Integrity checks ask whether records obey structural rules.

Examples include unique business keys, required fields, valid foreign-key relationships, and expected one-to-many relationships.

These checks catch duplication and join failures that volume metrics may miss.

4. Distribution

A column can be populated but still behave incorrectly.

Track metrics such as minimum, maximum, mean, quantiles, category frequencies, or distinct counts for important fields. The goal is not to monitor every column equally; it is to select attributes whose behavior tells you something about the health of the pipeline.

5. Reconciliation

When possible, compare independent views of the same process.

Examples include source-versus-target counts, totals calculated through two paths, or aggregate business values that should balance.

Reconciliation is powerful because it can detect errors even when each individual dataset looks internally consistent.

Use history to distinguish change from failure

Static thresholds are useful for invariants such as “this identifier must never be null.” They are less useful for naturally changing metrics such as daily record volume.

For behavioral metrics, historical comparison provides context.

A simple approach is:

current value
    │
    ├── compare with previous run
    ├── compare with rolling average
    └── compare with expected variance

The sophistication can range from percentage thresholds to statistical anomaly detection. The important step is storing enough historical quality metrics to understand normal behavior.

Alerts need severity, not just pass/fail

Not every anomaly should wake someone up.

I prefer three broad outcomes:

Fail — the output is unsafe to publish.

Warn — the output may be valid, but behavior is unusual and should be reviewed.

Observe — record the metric for trend analysis without immediate action.

For example, duplicate primary keys might block publication, while a 12% daily count drop might generate a warning depending on the dataset.

This reduces alert fatigue while preserving useful signals.

Quality checks should travel with the data product

The best place to define a quality rule is close to the dataset whose contract it represents.

That does not mean every team needs a different framework. A shared engine can execute checks, while configuration defines dataset-specific expectations.

A configuration-driven model might describe:

freshness:
  max_age_minutes: 120

volume:
  compare_previous_run_pct: 25
  compare_7_day_average_pct: 30

columns:
  business_id:
    null_rate_max: 0
    unique: true

The particular syntax is less important than making the expectations reviewable and version controlled.

The goal is confidence, not the number of checks

It is easy to create hundreds of validations and still miss the failures that matter.

A better approach is to identify the few ways a dataset could become misleading and ensure each failure mode has an observable signal.

For a critical pipeline, I would rather have ten meaningful checks with clear ownership than one hundred generic checks no one investigates.

Row counts are a good starting point. They become much more useful when they are one signal inside a broader model of freshness, integrity, distribution, and reconciliation.