Operating consultancy for financial data

A financial data feed can run at 99.99% uptime and still lose you the client. Available is not correct, and correct is the only number that pays. In financial data the pipeline is the product: the feed that reconciles overnight, the corporate actions applied right, the prices that match the exchange. Everything the customer clicks is interface sitting on top of that. So the financial data operating model that works is simple to state and hard to run: treat the substrate as the product, resource data engineering like the product organisation it is, and put reliability where commercial decisions get made. Treat the UI as the product and the substrate rots underneath it.

The dashboard is not the business

Sit in enough steering meetings at a financial data vendor and the same pattern shows up. The roadmap slide is full of interface work: a new dashboard, a redesigned export flow, cleaner API docs. The thing customers actually pay for, the feed that reconciles overnight and prices that match the exchange, gets a single line item: maintenance.

That line is the business. The dashboard is a window onto it. Sell an asset manager a beautiful UI sitting on a pipeline that silently drops a rebalancing event once a quarter, and you lose that client the day their compliance team finds the gap, not the day the interface looks dated.

This is the founding mistake in how these businesses get run, and it is a grain problem before it is a product one. Leadership teams, often staffed by people who came up through product or growth, treat the interface as the product because it is the layer they can see, demo and iterate quickly. The substrate, the actual data engineering, gets filed under infrastructure: necessary, unglamorous, someone else's problem. Read the org chart and the data engineering lead usually reports two levels below the CPO, sometimes into the CTO, sometimes into ops. Nobody owns the substrate as a business function. Everybody owns it as a cost centre.

Treat the substrate as the product

Flip the framing and the operating model snaps into focus. If the pipeline is the product, then data engineering is the product organisation, governance is the quality-control line, and customer trust is the measurable output of both doing their jobs. This is not an abstract reframe. It changes who gets headcount, which work wins the roadmap, and who sits in the room when commercial decisions get made.

Three decisions move the moment you accept it:

• Roadmap prioritisation. A pipeline reliability fix that prevents one bad data day per quarter outranks a UI polish sprint every time, in a business where the data is the product. Most roadmaps get this backwards because reliability work does not demo well.
• Where the money goes. Budget follows whichever team leadership believes is the business. If that is still the front-end team by a factor of three, the substrate keeps degrading no matter how many all-hands mention data quality.
• Who sits in the executive review. If the person who owns pipeline reliability is not in the room, the substrate optimises for whatever engineering can measure internally, not for what the business needs from it.

That last one catches people out. A pipeline owner optimises ruthlessly for uptime, latency and throughput, because those are the numbers on their dashboard. None of them tell you whether the output serves the outcome the client is paying for. A feed can be 99.99% available and still be useless if it reconciles against the wrong reference data, or if it is fast but wrong on the single field a client's risk model depends on. Reliability without business context is a vanity metric wearing an engineering costume. The same failure shows up when AI pilots stall on the way to production: the demo proves the thing can run, not that the organisation can absorb what happens when it does.

Reliability is a commercial metric, not an engineering one

Most financial data businesses track reliability as an engineering KPI: uptime percentage, mean time to recovery, ticket volume. Useful numbers, wrong altitude. They live in the engineering standup, get reported up as a green dot on a dashboard, and rarely reach the conversation where the CEO decides where next quarter's money goes.

That is the wrong floor. In this sector, reliability metrics are commercial metrics, and the translation is worth making explicit rather than leaving it for a client to discover.

When a hedge fund's ops team catches a stale price feed inside an overnight NAV calculation, that incident does not stay in Jira. It goes to their CFO, and eventually to yours. So put reliability on the executive review with the same seriousness as revenue and churn. What did the last incident cost in renewal risk. What is the trend on data-quality escapes. Which clients depend on which pipeline, and what happens to the relationship if that dependency breaks. If leadership cannot answer those from memory, reliability is still being run as an engineering concern. It is the same mistake CTOs make about scale: treating a commercial fragility as a technical dashboard and being surprised when it bites on the commercial side.

The financial data operating model runs in one order: engineering, governance, trust

Every sector has a grain, and this is the sector operating lens for financial data. The grain here runs in a specific sequence: data engineering first, governance second, customer trust third. Most teams invert it or try to run all three in parallel, and both fail for the same reason. You cannot govern a pipeline you have not built to a defensible standard, and you cannot earn trust on governance that is not backed by engineering that actually works.

Data engineering first means the pipeline is correct and observable before anything gets layered on top. Not perfect, correct: you know what correct means for this specific feed, you can detect when it drifts from correct, and you can trace why. Skip this and governance becomes theatre, a policy document describing a process nobody can actually verify is happening.

Governance second means once the pipeline is observable you build the control layer: audit trails, lineage, sign-off gates on schema changes, a documented answer to "how do we know this number is right" that survives a client's due diligence questionnaire. This is where vendors either over-invest too early, building SOC 2 theatre around a pipeline that still breaks monthly, or under-invest permanently, treating governance as a checkbox for the sales deck rather than an operating discipline.

Customer trust third is the output. You cannot manufacture it directly and you cannot market your way to it. A client who has been burned once by a data-quality incident will discount every subsequent claim you make, no matter how polished the pitch. Trust is rebuilt slowly through a visible track record and lost in a single afternoon when the wrong number reaches a model. Operate every decision, especially the ones about where to cut corners under deadline pressure, with that asymmetry in mind.

How to build the control layer without the theatre

Governance is where most of the money gets wasted, in both directions. The over-investors buy the certification and build the audit deck before the pipeline is observable enough to audit. The under-investors write the policy and never wire it to anything the pipeline actually does. Both are theatre. The control layer earns its keep only when it runs on the same rails as the data.

In practice that means the governance is code, not a document. Lineage is captured where the data moves, not reconstructed in a spreadsheet after an incident. Sign-off gates on schema changes are enforced by the pipeline, not by a wiki page nobody reads. When we build this glue we tend to run it on n8n: SOC 2 and ISO 27001 compliant, self-hostable so the data never leaves your boundary, around £20 per builder seat per month, which matters when the whole point is that sensitive feeds do not travel. Where the pipeline has to classify or reconcile ambiguous records, the extraction is model-only, never a regex fallback. Regex looks like control and quietly corrupts the substrate the first time reality does not match the pattern.

None of this is exotic, and that is the point. The substrate gets mapped before the interface, the control layer gets wired to the pipeline rather than to the sales deck, and the team ships. It moves faster than the moonshot version because there is no theatre to maintain.

The point of that number is not the count. It is the order. Map the pipeline, name what correct means, wire the controls to it, and the build stops being a research project. Do it in the other order, interface first, and you get a demo the board loves and a substrate that keeps failing on the field nobody was watching.

The test worth running

You do not need a six-week audit to know whether a financial data operation has its grain right. You need one conversation. Find the person closest to the pipeline and ask them to name the business outcome it serves. The same substrate logic holds in adjacent infrastructure sectors, where in transit and mobility the operating cost hides in a licence nobody questioned rather than a feed nobody owns.

If the answers are fluent, the operating system is aligned and you can spend the roadmap on the interface with a clear conscience. If they are a shrug and a reference to an SLA, the substrate is optimising for the wrong thing, and no amount of dashboard polish will fix it. That single conversation is also the fastest way to scope where to start: pick the one feed whose failure would cost you the most, and take it apart end to end before you touch anything else.