Software that enhances AI development workflows without being embedded in your application code: IDE extensions, CLI utilities, testing frameworks and observability.

Adopt

Mature, well-supported tools ready for production use.

Software engineering copilots

Copilots that write and run code from a natural-language brief are now part of normal engineering practice, which is why they sit in Adopt. Most teams already have one in place, so the practical choices now are which tool to pick and how best to work with it.

Broadly, the tools split into two kinds: those you can point at any provider, and those tuned to a single model. Among the model-agnostic ones, OpenCode stands out for its terminal experience and breadth of integration; Cursor, Windsurf and Zed are standalone editors; and CLI tools such as Aider, Cline and Block’s Goose run against any provider. The provider-specific tools, Claude Code, Gemini CLI and OpenAI Codex, are each built around their own model, while GitHub Copilot and Tabnine live inside the IDE.

The field keeps fragmenting, with no consolidation in sight. Claude Code is the strongest for long, autonomous work but also the heaviest and most tied to one provider, and as usage-based pricing bites, teams increasingly pair a capable proprietary tool for the hard problems with cheaper, model-agnostic ones running on open weight models for routine changes. Most of the recent movement is in that agnostic layer, because it keeps the provider choice open.

Teams work with these tools interactively in two broad styles: free-form “vibe coding”, steered by conversation, and a more structured, spec-first approach that works out what’s needed before any code is written. Kiro does both, pairing a conversational mode with a specs mode for drafting requirements first. Either way, project conventions increasingly live in AGENTS.md and steering files that every tool can read, rather than in a separate dotfile for each tool.

A newer mode is different again: it hands a scoped task to a cloud agent that runs unattended in its own sandbox and opens a pull request; OpenAI’s Codex cloud and Google’s Jules both work this way. This shifts the load onto review and widens the attack surface, so scope these agents tightly and keep a person in the loop. We cover the mechanics under Agent code sandboxes and Supervising multiple agents.

Senior engineers get the most from these tools, handing over routine work while keeping a close eye on quality; less experienced developers can struggle to tell a sound suggestion from a plausible but wrong one. The teams that do well treat that as a training problem and keep a trust-but-verify habit.

Provider-agnostic LLM facades

The LLM market moves quickly enough that today’s best choice may be displaced within months. A facade between your application and the underlying provider keeps that switching cost manageable. Options range from the lightweight AISuite and Simon Willison’s LLM library to heavier alternatives such as LangChain and LlamaIndex. A thin in-house wrapper is also a reasonable choice for teams that prefer to minimise dependencies. We’ve seen enough projects hampered by tight coupling to a single provider to treat some form of abstraction as a default.

Notebooks

Notebooks remain the de facto standard for data science and ML experimentation. Code, prose and visualisations live alongside each other, which suits the iterative work of model development and gives technical and non-technical collaborators something concrete to discuss. Jupyter is the most widely used, with cloud-hosted equivalents from Google Colab, AWS Sagemaker, Azure and Databricks. Language-specific alternatives include Pluto.jl for Julia, Clerk for Clojure and Polynote for Scala.

Trial

Promising tools with growing adoption, worth exploring for specific use cases.

MLflow

MLflow is an open-source platform for managing the machine learning lifecycle. The 3.0 release expanded it to cover generative AI: alongside the classical experiment tracking, model registry and deployment tooling, MLflow now offers OpenTelemetry-based tracing for LLMs and agents, capturing prompt construction, tool calls, memory retrievals, latency and token costs. This puts it in direct overlap with LLM observability tools such as Phoenix, Langfuse and LangSmith.

MLflow runs self-hosted or as a managed service on Databricks, SageMaker, Azure ML, Red Hat OpenShift AI and Nebius. That breadth is part of the appeal: teams avoid the lock-in of monolithic MLOps platforms while still being able to pick a managed offering where one fits.

Self-hosting requires technical expertise to configure and integrate. Unlike a turnkey platform such as Vertex AI, MLflow is not plug-and-play; its modular components must be tailored to specific use cases. We recommend it for organisations that value flexibility and have the proficiency to manage integrations.

Vector databases

Vector databases have emerged as specialised tools for managing the high-dimensional embeddings required by AI models. Prominent solutions include Pinecone, Qdrant, Milvus and Weaviate.

Traditional databases may suffice for simpler operations, and alternative approaches such as pgvector with pgvectorscale bring vector search directly into Postgres, avoiding the data consistency challenges of keeping embeddings synchronised across databases. If a dedicated vector database is required, Pinecone leads in production readiness but comes with managed service costs, while open-source alternatives such as Qdrant and Milvus offer greater control but demand more operational expertise.

For prototyping, Chroma offers a Python-first approach with minimal configuration. A 2025 Rust rewrite improved performance, though it remains best suited for small-to-medium scale applications. LanceDB takes a different approach as an embedded vector database, similar in philosophy to SQLite. It operates as a library within your application using Apache Arrow’s columnar format, making it compelling for local AI assistants and edge deployments where data must remain on-device. The trade-off is limited high-concurrency support.

One caution on scope. For retrieval over document corpora, where questions and source text use different words for the same idea, vector search remains the right tool. For agents working across a codebase, it increasingly is not: teams are finding that agentic search, direct grep and file traversal, beats a pre-built embedding index that goes stale as the code changes. Reach for a vector database when semantic similarity is the problem, not as a reflex for every retrieval task. See RAG for the fuller argument.

Local model execution environments

Tools such as Ollama, LM Studio and AnythingLLM provide accessible ways to run open weight models on local hardware. These enable experimentation with models from Meta, Mistral, DeepSeek, Alibaba and OpenAI without API costs or sending data to external services. Many now support tool calling via MCP and connections to commercial APIs for hybrid workflows.

These tools serve developers testing AI features, teams comparing model responses and organisations exploring capabilities with sensitive data that cannot leave their infrastructure.

LLM observability tools

Modern agentic builds involve multi-step reasoning, tool orchestration, RAG retrieval and chains of LLM calls where a single user request might trigger dozens of internal operations. Debugging why an agent produced an unexpected result requires visibility into every step of that chain. This is distinct from production AI monitoring, which focuses on drift detection in deployed systems.

Phoenix, from Arize AI, has emerged as a leading open-source option. Built on OpenTelemetry, it provides tracing and evaluation with auto-instrumentation for LangChain, LlamaIndex, DSPy and direct integrations with OpenAI, Anthropic and AWS Bedrock. Langfuse is the most popular fully open-source alternative (MIT licence), combining tracing and evaluation with strong multi-turn conversation support.

For LangChain-committed teams, LangSmith provides native integration that surfaces framework internals in debugging views. Helicone takes a lightweight proxy approach: route API calls through its endpoint for observability without SDK changes. MLflow 3 also offers OpenTelemetry-based tracing for LLMs and agents, a natural choice where a team wants a single platform across classical ML and GenAI. Since these tools capture prompt and response data, data sovereignty is a concern: Phoenix and Langfuse both offer self-hosting for teams with data residency requirements. Both have also added evaluations that run as CI gates, so a tracing tool can now block a regression before it ships; that practice is covered under Evals as a release gate. Galileo uses purpose-built small models (its Luna family) for low-latency evaluation rather than frontier-model judges; Cisco acquired it in 2026, and it now ships as Splunk Agent Observability.

LLM API gateways

As organisations adopt multiple model providers, an infrastructure layer emerges between applications and providers. LLM API gateways handle routing, caching, failover, rate limiting and cost tracking at the proxy level, complementing code-level abstraction from libraries such as AISuite. A facade is a developer choice about how to call models; a gateway is a platform decision about how to manage model traffic across the organisation.

LiteLLM is the most widely adopted open-source option, providing an OpenAI-compatible proxy with spend tracking, budget controls and key management. Portkey offers a managed alternative with semantic caching and conditional routing. Kong brings enterprise API management experience to LLM traffic.

For multi-provider deployments, a gateway centralises audit logging, per-team budget enforcement and automatic failover. Check whether it simplifies the operational story before you build equivalent functionality in-house.

AI red teaming tools

Our security coverage has grown this quarter, with guidance on prompt injection in MCP and RAG and architectural defences such as CaMeL. Red teaming is offensive testing: tools that systematically probe AI systems for vulnerabilities before attackers do.

Promptfoo is the standout, evolving from a prompt evaluation CLI into a comprehensive red teaming platform. OpenAI acquired it in March 2026 but the MIT licence remains. It ships financial services plugins for PCI DSS and banking regulation testing. Microsoft’s PyRIT focuses on multi-turn attack orchestration with Azure AI Foundry integration. NVIDIA’s Garak takes an agentic approach, autonomously probing for prompt injection, data leakage and toxicity.

The EU AI Act will require adversarial testing for high-risk systems, an obligation the Digital Omnibus moved to December 2027. We recommend starting with Promptfoo for breadth and CI/CD integration.

AI-assisted code migration

Large-scale code migration sits in a gap between what copilots do (assist line-by-line) and what bootstrappers do (generate new projects). Migration tools operate at codebase scale, applying thousands of coordinated changes to upgrade language versions, swap frameworks or modernise APIs. The pattern that works best combines deterministic code transformations with AI for edge cases that rules alone cannot handle.

Moderne is the leading platform, built on the open-source OpenRewrite engine. OpenRewrite provides deterministic “recipes” for common transformations such as Java version upgrades, Spring Boot migrations and Jakarta EE transitions, while Moderne adds AI-assisted handling of non-standard patterns and enterprise-scale orchestration.

These migrations are well-understood but labour-intensive, exactly where deterministic transformation augmented by AI pays off. We recommend trialling OpenRewrite on a representative repository before committing to the Moderne platform.

Agent code sandboxes

Agents increasingly write code and run it, to analyse data or drive a tool as they work through a task. That code is untrusted: it’s generated at runtime and often shaped by input an attacker can influence, so you don’t want it running on the host that holds your credentials and source. An agent code sandbox gives it somewhere safe to run: an isolated, disposable environment with limited filesystem and network access, torn down after use.

The category has purpose-built providers. E2B runs each session in a Firecracker microVM, while Modal isolates sandboxes with gVisor by default and offers a full-VM option for workloads that need their own kernel; Daytona and Cloudflare’s Sandbox SDK offer isolated containers with fast start-up; Pydantic’s monty is a minimal secure Python interpreter for LLM-generated code with no container overhead. For local work, Anthropic’s sandbox-runtime confines an agent’s processes at the OS level using Seatbelt, bubblewrap and network namespaces. This is a different concern from local model execution environments, which run the model itself; here we isolate the code the model produces.

We place agent code sandboxes in Trial: the tooling is mature enough to adopt now, and a team that lets an agent run generated code outside a throwaway environment is taking an avoidable risk. Prompt injection makes isolation more important, because the untrusted input flows straight into the code the agent runs. A sandbox contains the damage when that happens; it doesn’t stop the injection itself.

Assess

Emerging or specialised tools that merit evaluation for specific use cases.

Supervising multiple agents

Running several coding agents at once is much harder than running one. You lose track of what each is doing and which one is waiting for you. The field is early, and tools are trying quite different interfaces for it.

Staying engaged. The most useful thing one of these tools could do is work like an inbox: interrupt you only when an agent needs a decision, instead of leaving you to check each one in turn. Few do this yet. Claude Squad shows a list of running agents for you to look through. Conductor and Imbue’s Sculptor are built around reviewing each agent’s changes, and Sculptor can drop you straight into an agent’s work on your own machine to take over. VS Code and GitHub’s Agent HQ add a shared view across your agents. None yet lets you move smoothly between agents, and the inbox idea has barely reached coding tools so far. The risk is that you stop reading the changes and just approve them.

Keeping it local. To run agents in parallel, each needs its own copy of the code so they don’t overwrite each other: a git worktree per agent, or a container per agent, as Sculptor does. A container can also act as a sandbox, stopping the agent from reaching the rest of your machine or the network. Keep the agents on your own machine and your code and credentials stay with you, or use hosted services such as Factory and Devin, which run them on their own servers and suit sensitive or regulated work less well.

We place this in Assess: the tools change fast, and some have already shut down or gone commercial, so don’t become too invested in any one tool. Running each agent in its own workspace is sensible now, and we would prefer tools that keep everything on your own machines.

AI application bootstrappers

AI application bootstrappers generate complete applications from prompts or designs. Lovable (formerly GPT Engineer) has emerged as a leader alongside V0, Bolt.new and Replit Agent. Google entered the space with Firebase Studio. These tools can take projects from concept to working application in hours.

Capabilities are improving rapidly. Lovable’s visual editor allows Figma-like manipulation with automatic code updates. V0 excels at production-ready React components. Bolt.new runs full-stack development in the browser.

However, success still correlates strongly with existing engineering expertise. Senior developers use them as accelerators, understanding how to refactor generated code. Teams without this expertise risk shipping code they cannot maintain or debug. The gap between “working demo” and “production-ready system” remains substantial.

We recommend these primarily for prototyping and proof-of-concept work, with clear separation from production codebases unless your team has the engineering depth to take ownership of generated code.

Visual computer use agents

AI agents that interact with computers through visual understanding have matured but remain risky. Claude Computer Use controls desktops and browsers by seeing the screen and reasoning about interface elements. OpenAI Operator focuses on web browser automation through a managed environment. Browser Use offers an open-source alternative across multiple providers.

Reliability for bounded tasks has improved, with standard office workflows seeing success rates in the high 80s. However, prompt injection attacks, where malicious instructions hidden on web pages hijack agent behaviour, represent a systemic vulnerability. OpenAI has acknowledged this problem “may never be fully solved”.

For many automation needs, programmatic approaches via APIs and workflow automation platforms remain more reliable and secure. Visual computer use is best suited to isolated environments where the agent cannot access sensitive data. Teams should grant minimal permissions and maintain human oversight for high-stakes actions.

Lakera

Lakera was acquired by Check Point Software for approximately $300M in November 2025. Lakera Guard, its core AI safety scanning product, is being integrated into Check Point’s CloudGuard WAF as part of a broader application security offering. The underlying capability of scanning LLM inputs and outputs for prompt injection, toxic content and data leakage remains relevant, but the product context has changed substantially.

Technical limitations from our earlier evaluation still apply: scanning is text-only with no multimodal support, custom rules rely on regex patterns rather than context-aware analysis, and scanning is non-stateful with no awareness of conversation history. Teams evaluating Lakera should now assess it as part of the Check Point ecosystem rather than as a standalone product.

Structured output libraries

Libraries such as Instructor, Outlines and Marvin bridge a gap: LLMs produce freeform text, but applications need structured data. These libraries constrain outputs to match specified structures through prompting, logit manipulation or grammar-based generation. Instead of hoping an LLM produces valid JSON, developers specify Pydantic models and receive guaranteed-valid objects. For agentic systems this is essential, as agents need to produce function calls and decision objects that downstream code can reliably process.

Before reaching for one, check whether the task needs generation at all. If all you need is one label from a fixed set, that’s classification, and a purpose-built classifier such as TypeSafe’s Jev returns it directly, faster and cheaper than constraining a generative model to produce it. Keep these libraries for when you need the model to fill in a richer object.

The space is evolving quickly. Instructor has gained traction for its simplicity and Pydantic integration, while Outlines offers more sophisticated constrained generation. Native structured output features from model providers (OpenAI’s JSON mode, Anthropic’s tool use) may reduce the need for external libraries in some scenarios.

A broader category of runtime guardrails has grown up alongside these libraries. NVIDIA’s NeMo Guardrails and Guardrails AI go beyond schema conformance to include prompt injection scanning, content filtering and hallucination checks. Lakera Guard scans inputs and outputs for prompt injection, toxic content and data leakage; following its acquisition by Check Point it is best assessed as part of that stack rather than as a standalone product. Teams building production LLM applications should evaluate both levels: structured outputs for type safety and guardrails for content safety.

See also: PydanticAI, AI red teaming tools.

MCP gateways

An MCP gateway sits between AI clients and MCP servers as a controlled proxy. As MCP moves into production, it raises the same governance questions as any other integration point: who may call which server, with whose credentials, and how it’s all audited. A gateway answers them at that one point, adding access control, credential handling and audit logging, and some products also inspect the traffic for data loss and prompt injection.

Several vendors now offer products under this label. Lasso’s MCP Gateway concentrates on inspecting content, masking secrets, redacting personal data and blocking malicious servers. MintMCP and Workato add single sign-on, role-based access and identity-aware policy. Docker’s MCP gateway runs servers in isolated containers behind one managed endpoint, while Kong brings its existing API-management machinery to MCP traffic. The recurring hard part is credentials: the MCP authorization spec makes each server an OAuth resource server and forbids passing tokens straight through, so most of the engineering effort goes into handling that cleanly at runtime.

We place MCP gateways in Assess. The problem is worth solving and the security concerns are well founded, but the products are young and fragmented, and they differ enough that you can’t yet count on moving between them. Pilot one where you’re taking MCP into production, alongside the security guidance in the MCP entry. It’s a different layer from LLM API gateways, which govern model-provider traffic rather than tool calls.

Agentic vulnerability discovery

Agents that read source code can now find security vulnerabilities and often propose a fix. Because the agent reasons over the code rather than matching patterns, it catches classes of bug that scanners and fuzzers miss, and it can carry a finding through to a candidate fix rather than stopping at a ticket. This is separate from AI red teaming tools, which probe a model’s behaviour for jailbreaks and unsafe outputs; here the target is defects in conventional code.

This is already working in practice. Google’s Big Sleep, a Project Zero and DeepMind effort, has found live bugs in widely used code including a SQLite flaw, and Mozilla’s run of a frontier model against the Firefox source surfaced logic bugs conventional tooling had missed. GitHub’s Copilot Autofix suggests fixes for code-scanning alerts, with an agentic assign-and-open-a-PR flow in preview, and OpenAI has opened gated access to cyber-tuned models through its Trusted Access for Cyber programme. Visa’s open-source Vulnerability Agentic Harness brings this into the development pipeline: a chain of agents that discovers a vulnerability, verifies it, proposes a fix and validates the result, so code is hardened as it is written rather than audited after release.

We place this in Assess: the strongest tooling is still gated or in preview, and the patterns for folding it into a security programme are only forming. Regulated teams should watch closely and run the generally available pieces, such as Copilot Autofix where code scanning is already in place. Two cautions for regulated code: proprietary source sent to a hosted model leaves your boundary, so prefer self-hosted or on-premise options such as Visa’s harness; and an agent-authored fix is a proposal, not a merge, so keep a person in the loop rather than auto-applying patches.

AI supply-chain security

AI systems have inherited every software-supply-chain risk and added a few of their own. The LiteLLM package compromise in March 2026 shows the inherited kind: a poisoned Trivy scanner, running unpinned in CI, stole the project’s PyPI publish token and pushed backdoored releases with a credential-stealer inside. It was an ordinary pipeline attack that happened to land on AI infrastructure, a reminder that the AI stack is now a target in its own right. The newer risks sit around the models and the tooling. On the model side, weight files can execute code when you load them, and models are pulled from hubs with no record of where they came from. On the tooling side, a self-propagating npm worm has shown how a coding assistant’s MCP configuration can be poisoned with a rogue server, whose tool descriptions smuggle in prompt-injection instructions to exfiltrate credentials and spread.

The defences are maturing, and the starting point is to treat a downloaded model as an untrusted binary and prefer serialisation formats that don’t run code on load. On top of that, keep a machine-readable inventory of the models and AI services in use: ML-BOM is now a standardised CycloneDX capability, and vendors such as Endor Labs add model discovery and malicious-model detection to conventional software composition analysis. Signing goes further than an inventory, and OpenSSF’s Model Signing specification, built on Sigstore, covers a model’s weights, tokeniser, config and datasets as one verifiable unit, so you can confirm the files are the ones the publisher signed and nothing was swapped in transit or on a registry. Google and Kaggle have demonstrated it, though adoption across model hubs and CI is still thin.

We place this in Assess. Only a small part of this is AI-specific, but that part isn’t covered elsewhere: AI governance platforms handle compliance rather than artefact integrity, and red teaming targets model behaviour. Start with model provenance and an inventory, then extend the supply-chain controls you already run to cover the AI stack rather than standing up something separate.

Hold

Not recommended for new projects; better alternatives exist.

OpenClaw

OpenClaw is an open-source runtime for persistent, always-on AI agents that execute multi-step tasks by controlling a computer: clicking, typing, navigating applications and browsing the web. It supports Claude, DeepSeek and OpenAI as backends, and has spawned a wave of imitators. NVIDIA’s NemoClaw wraps OpenClaw in the NVIDIA Agent Toolkit with sandboxed execution against Nemotron 3 Super models. Moonshot’s KimiClaw runs natively on kimi.com with a community skill marketplace and persistent cloud memory.

The security model hasn’t been figured out across any of these variants. Persistent agent runtimes grant broad computer access to AI agents processing potentially untrusted instructions. The same prompt injection vulnerabilities that affect all visual computer use agents apply here, amplified by the breadth of access and the always-on character of the deployment. An agent with permission to control your browser, email and file system has an enormous blast radius if compromised.

We don’t recommend OpenClaw or its variants for new projects until the security model matures. Teams already using one should enforce strict sandboxing, limit accessible applications and maintain human oversight for actions involving sensitive data.

Conversational data analysis

Tools such as pandas-ai, tablegpt, promptql and Julius enable natural language querying of databases. Modern MCP servers can provide substantial context to models, including schema understanding and data contents. Our experience with JUXT’s XTDB showed models traversing complex table structures with apparent ease.

For experienced analysts, these tools represent a meaningful productivity boost, converting natural language into draft queries that can be refined. However, generated queries can be inefficient or incorrect despite appearing plausible. Uber’s QueryGPT demonstrates both the potential and complexity, highlighting the guardrails required for reliable results.

We’ve placed this in Hold because successful deployment requires users capable of understanding and validating generated queries. These tools offer substantial benefits for data teams with appropriate expertise, but should be approached cautiously by those unable to review AI-generated database queries.

Tools