Infrastructure and managed services that host and run AI workloads: cloud AI services, vector databases, model serving platforms and MLOps infrastructure.

Adopt

Mature, well-supported platforms ready for production use.

Foundation models

OpenAI, Anthropic, Google, Microsoft and Meta compete alongside a fast-growing open-weight field including DeepSeek, Alibaba, Moonshot AI and NVIDIA. Consolidation has begun too: Cohere and Aleph Alpha signed a definitive agreement in September 2026 to combine as a transatlantic sovereign-AI provider, subject to regulatory approval.

Providers differentiate across three tiers: smaller, faster models optimised for speed and cost; larger, more capable models balancing capabilities with response times; and specialised reasoning models for complex problem-solving. The distinction between general and reasoning models has largely dissolved, with the current flagships from OpenAI, Anthropic and Google integrating extended reasoning natively rather than exposing it through separate variants.

Foundation models warrant adoption for many business applications when paired with appropriate infrastructure (few-shot prompting, guardrails, RAG and evaluation frameworks). There is no universal “best model”, so benchmark against your own use cases and weigh factors beyond raw performance: pricing, reliability, data privacy requirements and deployment options. High-quality open weight models provide additional options for organisations with specific security or deployment requirements, though licensing increasingly varies: several 2026 flagships ship under custom, revenue-gated terms rather than permissive open-source licences.

Key considerations

• Performance & capabilities (accuracy, speed, and domain-specific strengths)
• Total cost of ownership (API costs, compute resources, and integration)
• Deployment options & technical requirements (cloud, self-hosted, edge)
• Data privacy & compliance (regulatory, legal, and security implications)
• Integration & lifecycle management (context limitations, version control, updates)
• Vendor stability & support (roadmap alignment, documentation, community)

Foundation model providers feature comparison (October 2026)

The capability gap between closed and open weights has narrowed to the point where it rarely decides the choice on its own. OpenAI, Anthropic and Google still set the frontier, and Microsoft now trains its own MAI models instead of relying on its OpenAI partnership alone. The open-weight field has drawn level on the coding and reasoning work our clients run day to day: DeepSeek, Alibaba’s Qwen, Moonshot’s Kimi and Tencent’s Hunyuan are all competitive against those benchmarks, and Meta has rejoined them with Muse Glimmer. NVIDIA is the one to watch: its Nemotron models are open, built for agentic work, and increasingly a base other labs fine-tune rather than a product chasing end users.

Open weights no longer guarantees permissive terms. Several of the strongest releases, Meta’s Llama and Moonshot’s Kimi K3, now ship under custom licences with revenue or user thresholds rather than Apache or MIT. Weigh the licence terms alongside the benchmark scores and your own testing before deciding whether a model fits your use case. For high stakes applications you may also want to check provenance: Ai2’s OLMo publishes its training data and code alongside the weights, so you can see exactly what went into the model. At the small end, Liquid AI and the sub-10B models have made on-device inference workable where data cannot leave the device.

National and regional programmes are the last group, and most compete on language coverage and jurisdiction rather than capability. A growing number post-train a shared open base instead of pretraining from scratch, layering local language, data residency and governance on top. Singapore’s SEA-LION builds on Gemma and Llama, and NVIDIA is positioning Nemotron as a base for sovereign builders. At the time of writing, two stand out for their unusual engineering: TII’s Falcon-H1 runs attention and Mamba-2 state-space heads in parallel within each layer, Aleph Alpha’s T-Free line drops the fixed token vocabulary altogether. Both are worth watching, especially for teams with data-residency requirements.

Feature definitions

• Open Weights: Models whose weights are publicly available for download and customisation
• Licence: How the open weights are licensed. Permissive means an OSI-approved licence (Apache 2.0 or MIT) with no revenue or user gate. Custom means open weights under bespoke terms, such as revenue thresholds, user caps or acceptable-use conditions. Proprietary means no public weights, available through an API only. Where a provider ships a mix, the classification follows its flagship open model.
• Enterprise Focus: Strong emphasis on governance, security, and enterprise integration
• Edge Deployment: Optimised for deployment on edge devices or resource-constrained environments
• Long Context: Support for context windows of 250K tokens or more
• Embedding API: Dedicated text embedding models and APIs for generating vector representations of text for semantic search and similarity tasks
• Agentic Workflows: Ability to autonomously plan and execute multi-step tasks using tools and external services. Goes beyond basic function calling to include complex workflow orchestration, error handling, dynamic planning based on intermediate results, and completing entire business processes without human intervention at each step

Weights & Biases

Weights & Biases tracks and visualises machine learning experiments, which helps when managing complex models and large datasets. By making experiment tracking light touch, needing just a few lines of code, it removes the friction that sometimes stops teams keeping up good measurement practices. CoreWeave acquired it in May 2025, and our recommendation stands, though we’re watching how the product evolves under new ownership.

Shared dashboards and reports make results visible to the whole team, so experiments become shared assets rather than knowledge siloed in individual notebooks, which speeds up iteration. The benefits only arrive, though, where a team already values measurement and experimentation.

Temporal

Temporal is a workflow orchestration platform that provides durable execution for long-running processes. Although not AI-specific, it has become increasingly relevant as organisations build production agentic systems that must survive failures and run reliably over extended periods.

Durable execution means that if a multi-step process fails halfway through, it resumes from exactly where it left off. This differs from Kubernetes, which restarts crashed containers but knows nothing about application state, and it differs from checkpointing as used by LangGraph, CrewAI and Google’s ADK. A saved checkpoint gives you no guarantee that the run finishes, that it recovers cleanly after a crash, or that it avoids repeating work already done. Temporal remembers that your workflow was on step 5 of 10, waiting for human approval, with specific context variables intact. The two are complementary, each handling failures at its respective layer.

Temporal’s programming model treats workflows as ordinary code in familiar languages (Go, Java, Python, TypeScript, .NET) rather than configuration or visual diagrams. For agentic systems specifically, it addresses failure modes that many agent frameworks ignore: LLM calls timing out, tool invocations needing retry with backoff, human approvals taking days. Built-in retry policies and the ability to pause workflows indefinitely handle these cleanly.

Other durable execution frameworks are available, such as Restate, DBOS and Inngest, and its LangGraph plugin, in public preview since mid-2026, runs LangGraph nodes as Temporal activities so the agent stays ordinary code while durability lives underneath.

We recommend Temporal for teams moving beyond prototype agents towards production deployments that have to run reliably. For simpler use cases, lighter-weight alternatives may suffice.

Data pipeline orchestration tools

Data pipeline orchestration has become essential infrastructure for managing complex data workflows, particularly those supporting AI initiatives. Whilst transformation tools such as dbt handle the “what” of data processing, orchestration platforms manage the scheduling, execution and monitoring of entire pipelines.

Apache Airflow is the established standard, with broad integration support across cloud platforms, though operating it well takes deliberate investment in DAG conventions and tooling. Prefect emphasises developer experience and dynamic workflow adaptation, with faster development cycles but fewer third-party integrations. Dagster takes an asset-centric approach where data assets become first-class citizens, providing built-in lineage tracking and data quality monitoring.

The choice depends on organisational context. Established enterprises with diverse toolchains often gravitate towards Airflow’s ecosystem breadth, teams prioritising developer velocity prefer Prefect, and organisations with complex lineage requirements consider Dagster’s asset-aware approach.

Cloud model hosting platforms

Model hosting has evolved beyond simple API access, with distinct platforms serving different needs from prototyping to enterprise production. Cloud-based hosting has become the default for most AI deployments.

Enterprise production environments typically use AWS Bedrock, Google Vertex AI or Azure OpenAI Service, which provide fine-tuning capabilities with enterprise security and integration with existing cloud infrastructure. For performance-critical applications, specialised providers such as Fireworks AI and Together AI focus on inference optimisation and custom model deployment, though teams must weigh simplified deployment against reduced ecosystem integration.

A separate class of providers now competes on latency. Groq and Cerebras run inference on purpose-built silicon, serving a small catalogue of open-weight models at token throughput the general-purpose clouds don’t match. The gain shows up in interactive and agentic workloads, where one task makes many sequential model calls and each round of latency adds up. The trade-off is coverage: fewer models, and little of the governance and integration tooling the larger platforms provide.

Development teams and startups often prefer Replicate, Modal or Hugging Face Inference Endpoints, which offer direct paths from trained model to production API with flexible pricing, at the cost of more limited enterprise governance.

Trial

Promising platforms with growing adoption, worth exploring for specific use cases.

Production AI monitoring platforms

Whilst experiment tracking tools such as Weights & Biases and MLflow excel at managing the development lifecycle, a distinct category has emerged to monitor AI systems in production. These tools detect drift and unexpected behaviour in deployed models that only surface when models encounter real-world data at scale.

Arize AI provides unified observability across traditional ML and LLM applications, continuously tracking feature and embedding drift (its open-source Phoenix handles run-level tracing; the platform adds drift monitoring at scale). Evidently AI offers both an open-source library and cloud platform, with over 100 metrics covering data quality and drift monitoring.

The payoff is early detection: you learn about performance degradation before it surfaces in support tickets. For teams already practising observability, AI-specific monitoring represents a natural extension of existing practices. This is the post-deployment counterpart to gating a release on evals: the gate catches regressions before they ship, monitoring catches the drift that only appears once live traffic arrives.

Open weight LLMs

Open weight models are now good enough for most production coding and reasoning, so teams usually choose one for control rather than capability. When you run the model yourself, your prompts and outputs stay inside the organisation, you decide when the version changes, and you can fine-tune it on your own data. For teams working with regulated or sensitive data, that control is usually the deciding factor.

You don’t have to self-host to get this. Inference providers such as Fireworks AI and Baseten will run open-weight models on infrastructure you control, under a private contract and often with no data retention, and because they carry no frontier training costs the price per token is usually lower than the proprietary APIs. Running your own GPUs takes serious ML engineering and is rarely cheaper once that work is counted, so most teams are better off starting with a managed open-weight endpoint and self-hosting only when they need to control the full stack.

Small language models

Small language models (SLMs) are the smaller cousins of the open weight LLMs above. The current crop sits in the 1B–7B parameter range and runs comfortably on a single GPU, a modern laptop, or in some cases a phone. Microsoft’s Phi-4-Mini (3.8B), Google’s Gemma 4 E2B and E4B (2–4B effective parameters), Meta’s Llama 3.2 1B and 3B and Mistral’s Ministral variants, joined by Liquid AI’s LFM2.5 series engineered specifically for on-device inference, now perform at a level that was out of reach at this size two years ago.

In our experience, the case for SLMs is strongest for organisations with their own data to fine-tune on. A few thousand labelled examples and a fine-tuning run produce a model that has the organisation’s terminology and edge cases built in. Hosted in-house, the data stays within the organisation, and the resulting model is small enough to deploy on the infrastructure the team already runs rather than dedicated hardware.

The natural fit is classification work where the language is nuanced and a wrong call is costly: routing customer messages by intent, triaging support tickets, tagging documents against a controlled vocabulary or extracting structured fields from free text. These are jobs that traditional keyword and feature-engineered approaches have never handled well, and where reaching for a frontier API on every request is overkill on both cost and latency. A fine-tuned SLM responds in the low hundreds of milliseconds or less and improves as more of the organisation’s examples are added.

The trade-off is scope and discipline. SLMs will not match frontier models on open-ended reasoning or breadth of world knowledge, and getting them to behave reliably means taking the task definition, the labelling pipeline and evaluation seriously rather than leaning on clever prompting. For teams prepared to invest the time, the result is a model that is cheap to run, fits the infrastructure they already have and is trained on the data they already own.

See also: Open weight LLMs, Local model execution environments.

AI-powered workflow automation

Visual workflow automation platforms allow teams to build AI-powered business processes through drag-and-drop interfaces rather than code. Zapier focuses on connecting SaaS applications with AI capabilities. n8n offers self-hosting, open-source licensing and extensive customisation for technical teams. Microsoft Power Automate provides native Office 365 integration with enterprise governance. Make.com emphasises sophisticated visual workflow design with AI agent functionality.

Common use cases in banking include onboarding document checks, statement and form data extraction, support ticket triage and transaction categorisation.

When evaluating, consider technical capability, data sovereignty requirements and scalability. Self-hosted solutions such as n8n offer maximum control but require expertise, while SaaS offerings reduce overhead but may have cost implications at scale. Teams should also assess error recovery and debugging capabilities, as AI components can fail less predictably than traditional integrations.

Digital twin platforms

A digital twin is a working model of a system that stays in step with live data, so you can test a change against it and see the effect before rolling it out. The most developed versions are physical, like NVIDIA Omniverse, which simulates factories and logistics networks in enough detail to plan from. Few banks run that kind of infrastructure, so the version that reaches this audience models a business process or the organisation itself rather than a plant.

A twin of a loan or onboarding workflow lets you find where it stalls and try a redesign before it affects production, and the same approach run against a portfolio shows how it copes under a rate shock or a liquidity squeeze, the kind of stress a regulator will eventually apply. Platforms such as Simudyne build these as agent-based simulations, where each customer or counterparty is represented as an agent and the behaviour you care about emerges from how they interact. This works best on top of process mining, which reconstructs the current workflow from event logs, so what you simulate against reflects how the process runs in practice.

We keep this in Trial because the platform is rarely the thing that stops a project. A twin is only as useful as the live data feeding it and the model behind it, and in a regulated firm a simulation that informs a lending or capital decision falls under the same model-risk governance and audit requirements as anything else the firm relies on. Most of the effort goes into the data integration and the validation needed before anyone can act on the result, which is why few of these programmes have moved past pilots.

Agent memory architectures

Stateless agents hit a wall quickly. An AI coding assistant that forgets what it learned yesterday or a customer service agent that can’t recall a conversation from last week force users to re-establish context on every interaction. Agent memory architectures address this with tiered storage: working context for the current session, long-term memory for persistent knowledge and episodic recall for past interactions. Over the past year memory has moved from a nice-to-have to a first-order design decision for anyone running agents beyond a single session.

Mem0 has gained the most traction, and ships as a native integration for AWS’s Strands Agents SDK, one of several supported memory backends. It sits alongside your agent framework, automatically extracting and retrieving memories without changes to orchestration logic. Letta (formerly MemGPT) pioneered treating context management like virtual memory, paging information in and out as needed. Zep differentiates on temporal accuracy, maintaining a knowledge graph that tracks how facts change over time. A theme this year is the move away from pure vector recall towards schema- and graph-grounded memory, which retrieves by structure rather than similarity alone and tends to be easier to audit.

Persistent memory is also an attack surface, and a nastier one than a stray prompt. Adversarial content written once into persistent memory can outlast the session then surface weeks later. OWASP now tracks memory and context poisoning separately from prompt injection. The defences differ from request-time filtering: scan content before it is written, make writes permission-aware, keep provenance on memory entries, and watch retrieval for behavioural drift. None is production-proven yet and reported attack success rates run high, so treat what an agent remembers with the same suspicion as any other untrusted input.

The frontier is memory that improves the agent. Systems such as Google’s ReasoningBank turn an agent’s own successes, and more usefully its failures, into reusable lessons retrieved on later tasks, so the agent gets better at the work it repeats. It is early, but it points at where memory is heading: not just recall, but steady improvement from experience.

For regulated industries, agent memory introduces governance questions your existing data policies may not cover. Where does the memory reside? How do you audit what an agent “knows” about a customer? How do you comply with deletion requests? Involve compliance and data protection teams early.

See also: Agentic tool use, Harness engineering, Microsoft Agent Framework, Agent identity, CaMeL, RAG.

AI governance platforms

The EU AI Act is phasing in. Its high-risk obligations, conformity assessments, risk documentation, human oversight and ongoing monitoring, were pushed to December 2027 by the Digital Omnibus, but the programmes to meet them take years to build. Banks and insurers already operate under model risk management frameworks (SR 11-7 in the US, SS1/23 in the UK), but these were written for statistical models with stable inputs and deterministic outputs. Extending them to LLM-based systems requires tooling that existing GRC platforms were not designed to provide.

Credo AI has emerged as the leading dedicated platform, providing AI risk classification aligned to the EU AI Act, NIST AI RMF and ISO 42001. It powers the compliance accelerators in IBM watsonx.governance through an OEM partnership, and Microsoft is integrating it with Azure AI Foundry. Others compose governance workflows from existing tools: model cards in Hugging Face, risk registers in GRC platforms, audit trails from observability tooling. Dedicated platforms offer structured workflows out of the box, while bespoke approaches require more engineering but can match how your organisation already works.

We move this to Trial. The space is young and consolidating fast, but the tooling is now worth the time to evaluate, particularly for financial services firms with EU AI Act obligations, even as the vendors keep shifting under them.

See also: LLM reproducibility, Production AI monitoring platforms.

Assess

Emerging or specialised platforms that merit evaluation for specific use cases.

Galileo

Cisco acquired Galileo in 2026 and folded it into Splunk Agent Observability, so the independent evaluation tool we assessed no longer stands alone. For current options see LLM observability tools; for open, self-hostable tooling, Langfuse or Phoenix.

Kubeflow

Kubeflow is an open-source ML platform built on Kubernetes, combining orchestration capabilities with ML-specific tools: Pipelines for workflow automation and KFServing for model deployment. This integrated approach helps bridge the gap between data scientists and operations teams.

Several factors keep Kubeflow in Assess. Implementing it demands expertise in both Kubernetes and ML engineering. Many organisations struggle with setup and ongoing maintenance, and report long timelines before seeing tangible benefits.

Organisations with established ML practices and Kubernetes expertise should consider it, particularly for challenges around model deployment, experiment reproducibility or resource utilisation. Smaller teams or those earlier in their ML journey may prefer managed options such as Vertex AI Pipelines.

Process mining platforms

You cannot reliably automate a process you do not yet understand, and its edge cases decide whether automation succeeds or fails. As agentic AI moves into the enterprise, process mining has become the preparation step: it reads event logs to show how a process actually runs, rather than how it was meant to. Task mining adds the desktop view, capturing how people handle the exceptions and workarounds that never reached the documented process. Together they map the reality an agent has to replicate.

Which platform you pick is mostly settled by the stack you already run. Celonis leads on depth for complex, multi-system processes, if you can fund the investment it takes. Otherwise the choice tends to follow your existing tooling: QPR ProcessAnalyzer for Snowflake shops, UiPath or Microsoft Power Automate Process Mining where you are already committed to that automation stack, and ABBYY Timeline for business users who want process and task mining without code. For a contained proof of concept, Fluxicon Disco runs on the desktop with no platform commitment.

As ever, we recommend starting in one business area rather than an enterprise-wide rollout from a standing start.

Taalas

Taalas hard-wires AI models directly into custom silicon (ASICs). Rather than running models on general-purpose GPUs, Taalas “prints” a specific model onto a chip, unifying storage and compute at DRAM-level density. The result is a fixed-function chip that holds one model and cannot be rewritten. Their first product, the HC1, integrates Meta’s Llama 3.1 8B, with claimed performance of 16,000 tokens per second per user at 10x GPU throughput and 20x lower production cost.

The trade-off is radical inflexibility. Each chip runs exactly one model. When that model is superseded, the chip becomes electronic waste. This makes Taalas most compelling for scenarios where a specific model will be deployed at scale for an extended period: edge inference, embedded systems or dedicated infrastructure for a known workload. For regulated industries, a model baked into silicon is inherently versioned and immutable, simplifying reproducibility concerns. Teams subject to Model Risk Management requirements may find this appealing.

We’ve placed Taalas in Assess because the technology is early and the economics only work at significant scale. The assumption that a single model will remain useful long enough to justify dedicated silicon runs counter to how fast the field is moving.

See also: Open weight LLMs, Local model execution environments.

Managed agent platforms

The counterpart to building your own harness is renting one. Managed agent platforms run the agent for you, offering a hosted harness with sandboxed code execution, persistent memory, scoped permissions and durable long-running sessions behind an API. AWS Bedrock AgentCore is the clearest example, a serverless, framework-agnostic runtime that hosts an agent built in any stack on isolated microVMs. Anthropic’s Managed Agents and Google’s Managed Agents take a more opinionated line, provisioning a sandbox where their own agent reasons, runs code and browses, while Vercel’s Agent Stack assembles the same pieces from durable functions and sandboxes. Coding-specific services such as Cursor’s cloud agents sit alongside, and Temporal provides the durable-execution layer several of these build on.

The trade is the familiar one: time-to-production against control and portability. A managed platform removes the undifferentiated work of running sandboxes, memory and retries, at the cost of tying your agent to a vendor’s runtime and pricing, typically billed per token plus a session-hour fee.

A hosted agent that runs code and browses still inherits the unsolved problem of prompt injection and the trust boundary now runs through a third party’s runtime. Least privilege still applies, and is harder to see from the outside: what credentials does the sandbox hold, and what can it reach on your behalf? For regulated teams there’s a data-sovereignty question too, since prompts, retrieved context and tool traffic all cross into the vendor’s infrastructure, which has a bearing on where regulated data is allowed to go.

We place managed agent platforms in Assess. The category is real and moving quickly, but most offerings reached preview or general availability only this quarter, pricing is unsettled, and portability between them is unproven. AgentCore is the most production-ready for teams that want to move sooner. Self-hosted, single-user runtimes such as OpenClaw sit at the opposite end and stay in Hold.

See also: Harness engineering, Temporal, Agent code sandboxes, AI-powered workflow automation, Vendor agent SDKs.

Hold

Not recommended for new projects; better alternatives exist.

Building against vendor-specific APIs

Tightly coupling applications to vendor-specific LLM APIs is risky in a market where capabilities and pricing shift monthly. Organisations that build directly against OpenAI, Anthropic or other proprietary APIs risk finding themselves locked in, facing painful migrations when a better or more cost-effective model emerges.

We recommend abstraction libraries that provide a common interface to multiple providers. AISuite or Simon Willison’s LLM CLI let you switch between models with minimal code changes, handling the nuances of different APIs behind a consistent interface. These abstractions add some complexity and may limit access to vendor-specific features, but the protection against lock-in outweighs these drawbacks in most cases.