The AI Center of Excellence in 2026: From Central Team to Federated Model

Two of the most-quoted findings on AI organisation seem to point in opposite directions. In Accenture's survey of 2,000 executives at very large companies, fielded in mid-2024, 57% of the "front-runners" had a centralised operating model, such as a centre of excellence (CoE), against 16% of fast-followers ^[1]. In BCG's 2025 study of more than 1,250 firms, the few that capture AI value at scale run "an AI-first operating model that combines strong leadership with decentralized execution and shared ownership between business and IT" ^[2].

The contradiction is mostly apparent. The same BCG firms had three times as many central AI platforms in operation as laggards ^[2]. Our thesis, and it is NKK's view rather than a finding: the useful question in 2026 is not "centralise or federate?" but "which layer does the centre keep?" A CoE that keeps the platform, the rules and the register, and releases delivery, is a stage on a path. A CoE that keeps everything becomes the queue that teams route around.

Three models, one trade-off

Microsoft's CoE guidance defines three operating models, which differ in who sets the rules, who delivers, and where the main risk sits: bottleneck, coordination complexity or standards drift ^[3].

Microsoft says the CoE "should evolve from a centralized control to an advisory team", but only when governance is embedded in platform operations ^[4]. BCG's August 2026 argument for a central AI hub agrees on the start: "Starting in a highly decentralized manner creates fragmentation." It is a conceptual argument, not a controlled comparison ^[5].

The failure on the other side is documented too. A study by Stanford researchers with LVMH-affiliated co-authors (the "Stanford-LVMH study") examined an unnamed multinational with more than 50 business units operating with significant autonomy. It found group-level frameworks "advisory in nature, lacking sufficient implementation details or enforcement mechanisms", and often no single escalation path for high-risk projects ^[6]. The authors propose three layers: a shared foundation, central advisory tooling and local implementation. It covers one organisation, is qualitative and is untested for outcomes ^[6].

A study of 23 Swiss companies suggests the few that scale generative AI are "moving beyond the classical hub-and-spoke models" to an "AI spine" for "implementing, evolving, and abandoning LLM use cases at scale" ^[7].

What the evidence does not show

No study we could find compares centralised, hybrid and federated models on financial or delivery outcomes. Microsoft calls central platform with federated delivery the most common arrangement at scale but cites no data ^[3]. Both headline surveys above are self-reported, and both publishers are consulting firms.

Maturity figures move with the definition. BCG's 2025 study put 5% of firms at "future-built" ^[2]; its 2026 Applied AI Index, a different survey of 1,330 leaders, reports 7.5% ^[8]. That is not a trend.

Maturity pays, but a structure is not the cause

MIT CISR's four-stage enterprise AI maturity model shows a steady gradient. Stage 1 firms sat 26.5 percentage points below their industry's average growth, Stage 2 6.8 below, Stage 3 4.7 above and Stage 4 13.9 above ^[9]. Read it with care: the briefing does not say what "growth" measures, and stages are assigned by a self-reported measure of AI effectiveness, so the link is partly circular. The stages are not operating-model types, and the briefing, based on a survey of 152 organisations, does not test whether any structure moves a firm up ^[9].

What it does offer is two cases. Guardian Life runs a fast-track board that can approve a new solution or reuse an existing one. Italgas embeds "business translators" in each business unit and keeps a group AI office overseeing model training, testing and monitoring. Both cases draw mainly on executive interviews, supplemented by company documents ^[9]. NKK reads the shared idea as a centre that decides and oversees quickly rather than doing the work; the briefing does not test it.

What a CoE keeps and what it must release

The CoE debate is easier once framed as decision rights. Microsoft's split gives the CoE platform strategy, security and compliance policy, architecture standards, release gates, risk tiers and autonomy limits; domains own prioritisation, design inside the standards, knowledge curation, day-to-day operation of lower-risk agents and domain KPIs ^[10]. MIT CISR's interviews with 48 leaders at 28 organisations recommend that central technology teams own the process of moving AI applications into production and that CoEs "define standards, tools, and architecture" ^[11]. The briefing also flags one bank's decentralised model, with end-to-end accountability in business units, as introducing "unsupportable applications and siloed data" risks ^[11]. Bain's technology report makes a similar split: central platform teams control the core agentic platforms while accountability for building and monitoring agents is distributed to business domains ^[12].

We turn that into a working tool, the NKK Retain/Release Layers. The artefact it produces is a decision-rights ledger.

How to use it. List every recurring AI decision, from "who approves a new model" to "who retires an agent", and assign each to a layer. Each ledger row has three fields: decision, layer, owner. An illustrative row: "set the autonomy limit for a customer-facing agent | shared foundation | CoE". Then test each layer. Foundation items must be enforced by the platform, not by a memo; one case study of an agentic rollout routed requests through a governed orchestrator so the controlled path was the default (one qualitative case, Accenture Research authors) ^[13]. Advisory items must be real products with an owner, otherwise they become the unenforced guidance the Stanford-LVMH study describes ^[6]. Released items need a named owner on the receiving side.

Where it breaks. The layers assume the registry and golden paths exist. Without them the released layer is just sprawl. The boundaries are NKK's synthesis, not validated against outcomes.

One accountable owner per system

MIT CISR's decision-rights briefing, based on 30 interviews with 27 executives at nine companies, reports one company's practice: One NZ names a business owner for each agent ^[14]. NKK extends this to every AI system, building on Microsoft's one-accountable-role-per-task rule ^[10] and Accenture's call for a named owner for every agent ^[15]. The NIST AI Risk Management Framework asks that roles for AI risk be documented and clear, without prescribing a CoE or any organisation chart ^[16].

An Adobe governance director argues that programmes have registries and dashboards but nobody with authority to stop a harmful system ^[17]. It is opinion from inside one company, not evidence, but the test is useful. If you cannot name who can stop a model, you have a register, not governance.

Intake, funding and the platform-versus-services choice

Intake. Microsoft's guidance is to capture business value, data sources, pattern, initial risk level, requester and sponsor; score every request on business value, technical feasibility and resource requirements; and route by risk tier so low-risk requests get light review ^[18][4]. Its warning: "Without a formal process, agent demand becomes noise." Publish the backlog and "record the decision and the reasoning for each request, not just the ranking" ^[18].

Platform versus services. NKK's framing: the platform is self-service and scales with usage, while services are bespoke delivery and scale with headcount. The NKK rule: services should shrink as the platform grows, leaving the high-risk or high-ambiguity work.

Funding. The evidence is thin. BCG quotes one anonymous retail executive: capital allocated centrally, amortisation carried by business teams "to ensure that they have skin in the game" ^[2]. The FinOps for AI overview describes showback as visibility into AI costs without immediately charging teams, stresses tagging, and notes there are no generally accepted frameworks for allocating cost across multi-agent workloads ^[19]. Showing back first and charging back only once attribution is reliable is NKK's sequence, not FinOps guidance.

Ownership is a rule, not a step: from the first use case a business owner holds the KPI and business case; the CoE reviews but does not own the benefit.

How CoEs fail

Microsoft names three failure modes. The gatekeeper "reviews everything, enables nothing, and becomes the bottleneck", and teams build shadow agents. The ghost is "launched with fanfare" but has no operating rhythm or decision authority. The perfectionist will not let anyone ship until governance is "complete" ^[20].

MIT's work on minimum viable governance supplies the balance. Above a "ceiling", governance creates bottlenecks and drives workarounds, including shadow generative AI; below a "floor", organisations face compliance gaps and lack of accountability ^[21]. MIT CISR's briefing sets out the framework and notes that generative AI's pace of adoption outstrips centralised review capacity ^[22]. The World Economic Forum's practice paper makes the same call for governance that enables rather than gates ^[23].

Two figures hint that agent governance is immature in many firms. They come from separate surveys with different populations and definitions.

A third figure concerns skills, not governance: 59% cite knowledge and training gaps as a main obstacle to responsible AI, from a McKinsey survey charted in the Stanford AI Index 2026, where respondents could pick several obstacles ^[25]. It shows skills gaps are a widely cited obstacle, not that firms sit outside the floor-and-ceiling balance.

Placing each AI pattern: the NKK Ladder and Decision Matrix

The NKK CoE Evolution Ladder

The ladder puts the direction of travel into five rungs, each with the signal that says it is time to climb.

What it is built on. Rungs 1, 3 and 4 follow Microsoft's centralised-to-hybrid-to-federated progression and its signals for when central control starts to hurt ^[4][3]; rung 2 is NKK's addition, built on Microsoft's intake guidance ^[18]. The Stanford-LVMH layering sits behind rung 3 ^[6], and rung 5 comes from the Swiss "AI spine" study ^[7].

How to use it. Place each AI pattern on the ladder, not the company as a whole. A domain analytics copilot built by a ready team can sit at rung 4 while a customer-facing agent stays at rung 1 or 2. Review placements quarterly; climb only when the exit signal is visible. Microsoft says to keep tighter central control for system-acting or customer-facing patterns until the platform enforces governance automatically; assistive patterns can federate sooner ^[3]. The example placements are NKK judgement; the matrix below is the placement test.

Where it breaks. It is not linear, and regulated sectors may legitimately stop at rung 3. The triggers come from vendor guidance and small qualitative studies; nothing links a rung to ROI.

Choosing a model: the NKK decision matrix

Microsoft says to choose a structure using five signals: maturity, risk, regulation, team readiness, and speed versus control. Most organisations blend models by use-case pattern ^[3]. MIT CISR's AI Decision Matrix sorts decisions by ambiguity and risk into Routine (automate), Consequential (manage), Exploratory (enable) and Strategic (lead) ^[14]. Our matrix keeps risk but swaps ambiguity for team readiness, because the question here is who can carry the work. Microsoft maps patterns to structures directly, recommending centralised delivery for employee AI enablement and federated for core business processes ^[3]; this grid weights readiness and risk instead, so it puts an employee assistant in "Enable and coach" and a core-process agent in "Hybrid with central gate".

How to use it. Score each use case, not each business unit. Place the use case on both axes with the team that would own it, read the quadrant, and record the choice in the registry. A customer-facing agent built by a team that cannot yet monitor it lands in "Central build" whatever the organisation chart says. As readiness grows, the same pattern moves right. Quadrants map to ladder rungs: Central build to rungs 1-2, Enable and coach to rung 3, Hybrid with central gate to rungs 3-4, Federate to rung 4.

Where it breaks. Placement is judgement, and teams over-rate their own readiness. Regulation can override the grid: Article 4 of the EU AI Act has applied since February 2025, and the amended text still requires providers and deployers to take measures to support their staff's AI literacy, but the Commission says no specific governance structure is mandated ^[26]. The AI Omnibus (the Digital Omnibus on AI), in force since 27 July 2026, moved high-risk rules to 2 December 2027 for Annex III use cases and 2 August 2028 for Annex I product-embedded systems ^[27].

Maturity and team readiness feed the matrix's readiness axis; risk and regulation feed its risk axis; speed versus control is the tie-breaker. In every row the first answer calls for more central control.

What to do next

1. Name one accountable owner for every AI system and publish a one-page decision-rights ledger using the three layers above. Exactly one accountable role per task ^[10].
2. Stand up a scored intake with risk tiers. Score on value, feasibility and resources, give low-risk requests a light path, and publish the reasoning with the backlog ^[18].
3. Put governance into the platform before you federate. Registry, identity, release checks and logging come first; write down the exit criteria for the CoE gate ^[4][3].
4. Start tagging and showback. Meter cost by team and use case ^[19]. In NKK's view, defer chargeback until attribution is reliable.
5. Place each AI pattern on the ladder and the matrix, and review quarterly. Expect to run several models at once.
6. Record who owns AI literacy evidence in the registry, so you can show how you meet Article 4 ^[26].
7. Measure the CoE itself. Track time to decision, the share of requests on the fast path, and the share of AI systems with a named owner in the registry. These are NKK suggestions; no source gives benchmarks.

Conclusion

A CoE is a stage in an operating model, not its end state. The sources broadly point to the shape: the centre keeps platform, data policy and standards; delivery and outcome accountability sit with named business owners, while the centre is accountable for the governance system. Delivery moves outward as the platform enforces the rules. The sources do not agree on how fast, because nobody has compared the models on outcomes. Choose by readiness and risk, make ownership explicit, measure the CoE's own gatekeeping, and revisit the choice as your teams mature.