It Couldn't See Itself

Each Clarity Audit documents a real system engagement. An architectural readout. The goal is to surface structure.

The Problem

An automation implementing a standard “search before create” pattern, designed to be idempotent, created over 30,000 duplicate records across 1,000 work items. Despite checking for existing records before creation, every search returned “not found,” triggering new entries each run.

The system couldn't recognize what it had already created.

Root Cause

The search and create steps used slightly different formulas to generate lookup keys:

• Search: concatenated three fields
• Create: concatenated the same fields but inserted one extra space between two values

That invisible difference meant each newly created record was unrecognizable to subsequent searches. On average, each work item generated thirty duplicates before anyone noticed.

Contributing Factors

A single space triggered 30,000 duplicates. Nothing in the system's design would have caught it.

• No shared key definition. Search and create formulas were written independently.
• No normalization. No trimming, case standardization, or whitespace handling before comparison.
• No database guardrails. No unique constraint or key hashing to reject duplicates.
• No anomaly detection. No monitoring of create frequency or volume spikes.

Four layers where a safeguard could have existed. None did.

Resolution

• Standardized the concatenation formulas across both operations
• Added trim and case normalization before all key comparisons
• Ran a deduplication job using normalized key expressions
• Implemented alerts for unusual creation volume

Zero duplicate creations after deployment. Full cleanup completed within 24 hours.

What This Surfaced

The system had no definition of sameness.

It could search. It could create. But it had no shared, normalized concept of identity across those two operations. Each one constructed its own version of what a record looked like, independently, and nobody tested whether they agreed.

Any system that must recognize the same entity twice needs one method of identification, shared across every operation that constructs or compares keys. Tested. Normalized. Visible.

Without that, the system can't see itself. And when a system can't see itself, it duplicates, contradicts, and erodes trust in every record it touches.