No one plans to build an integration mess. It accumulates one reasonable decision at a time: the ERP needs supplier balances in the treasury tool, so someone writes a direct interface. HR needs new hires in the door-access system — another interface. The document management system needs project codes — a nightly export. Each one takes two weeks, works fine, and quietly adds a strand to a web that, five years later, means no system can be changed without breaking three others.
I have inherited that web more than once. The cost is not visible in any budget line; it shows up as the ERP upgrade that stalls for a year, the “simple” system replacement that requires re-testing eleven interfaces, and the integration developer who becomes the single human point of failure for the company's data flows.
The rule of N-squared
Point-to-point interfaces scale as roughly the square of your systems. Five systems fully connected is ten interfaces; ten systems is forty-five. Nobody builds them all — but even a fraction is enough that the interfaces, collectively, become a system of their own: undocumented, unmonitored, and owned by whoever happened to build each one.
What a middleware layer actually buys you
The answer is not always an expensive integration platform. The answer is a layer of indirection with rules. In practice, in the environments I run, that means:
- Every system exposes and consumes through APIs — no direct database reads across system boundaries. A database-level integration couples you to another team's internal schema, and it will break silently on their next patch.
- A middleware or automation layer in between — this can be an iPaaS, a lightweight workflow engine, or well-governed API gateways. Its job is transformation, routing, retries, logging and alerting in one place.
- Canonical definitions for shared data — one agreed shape for “supplier,” “project,” “employee,” “cost code.” Systems map to the canonical form, not to each other. When a new system arrives, it writes one mapping, not five.
On a recent program, moving reconciliation flows from manual exports and point-to-point jobs onto API-based middleware cut manual reconciliation work by around forty percent — but the larger win was invisible: the next system replacement touched one mapping instead of a dozen interfaces.
Master data: decide who owns the truth
Half of all “integration problems” are really master-data problems wearing a costume. If suppliers can be created in two systems, no middleware will save you — you will synchronize conflicts at machine speed. For every shared object, name a system of record, make every other system read-only for that object, and route creation requests through the owning system's workflow. This is an organizational decision first and a technical one second, which is exactly why it gets skipped.
Operate integrations like production systems
Integrations fail on weekends, at month-end, and during the vendor's maintenance window — never at a convenient time. Minimum operational standard I hold every interface to:
- A catalog entry: source, target, trigger, frequency, data objects, owner, and business impact when down.
- Monitoring with alerts to a human — not a log file nobody reads.
- Defined retry and reprocessing behavior, so a failed run does not require a developer at 11 p.m.
- A quarterly review of failures and volumes, the same way you review server capacity.
Migrating out of an existing web
If you already own forty undocumented interfaces, the answer is not a big-bang replacement program — those stall under their own weight. The workable path is opportunistic: first, spend the few weeks building the catalog, because you cannot govern what you cannot list. Then adopt a simple rule — every interface that must be touched anyway gets rebuilt to the standard: through the middleware layer, against the canonical model, with monitoring and an owner. System upgrades, vendor changes and new requirements will naturally route a surprising share of the estate through your hands within two or three years, and the web untangles without ever appearing on the budget as a standalone project.
Resist the temptation to grandfather “temporary” exceptions permanently. An exception is granted with an expiry date and appears on the quarterly review until it is closed — otherwise the new standard becomes one more layer on the old mess.
Integration gut-check
- Could we list every interface we run — today, from a catalog, not from memory?
- Does each shared data object have exactly one system of record?
- If our busiest interface failed on the night of month-end close, who gets alerted — and would they know what to do?
Integration architecture is unglamorous precisely because its success is invisible: systems change, and nothing else breaks. But it is the difference between an IT landscape you can evolve and one you can only add to — and in a transformation program, that difference is the whole game.