Anyone introducing AI-supported workflows in a mid-sized company soon runs into the same question: are the data allowed to leave the building? With contracts, personnel files, calculations or customer data, the answer is often no — and that rules out many cloud services. A recent step by Perplexity aims squarely at this gap: according to its own account, the company is providing a model and an associated agent that can be run locally on your own hardware.
What is remarkable here is less the individual product than the reasoning behind it. Perplexity argues that agents for local operation need different tuning than models that run in a data centre with virtually unlimited computing power. In a competitive comparison, the provider claims the agent outperforms the Hermes and Pi models. Such self-assessments should be read with caution: Perplexity has its own interests in a locally deployable agent, and benchmark claims from manufacturers are no substitute for your own testing in the specific use case.
What distinguishes an agent from a chat model
A language model answers queries. An agent — that is, an AI system that plans multi-step tasks, calls tools and evaluates intermediate results — works in loops: search, read, summarise, write a file, check the result. Each of these steps triggers its own model query.
That explains why local agents need a different profile. In the cloud, an elaborate reasoning process can be afforded across many intermediate steps. On a workstation with a single graphics card, by contrast, every additional step determines whether a task is finished in seconds or in minutes. A model that decides concisely and reliably which tool to call next is more valuable in this environment than one that formulates at length.
The real advantage: data stays in-house
For companies, the central point is organisational, not technical. If an agent runs on a machine in your own network, a whole class of questions disappears:
- Data processing agreements: There is no external service provider processing personal data, and therefore no additional contract including review.
- Third-country transfers: The question of which country the processing takes place in does not arise.
- Training use: There is no uncertainty about whether inputs flow into future model versions.
- Availability: An outage or a price change at the provider does not immediately interrupt the workflow.
This opens up an area that was often blocked until now: analyses of internal document collections, preparation of quotations, research in your own knowledge bases, processing of minutes. All tasks where the data situation is sensitive but the benefit is immediately tangible.
Costs shift, they do not disappear
Moving from the cloud to your own hardware is not a cost-cutting programme but a relocation. Instead of ongoing billing per query, the bill covers the purchase and operation of graphics cards, power consumption, cooling and administration. On top of that comes knowledge that must be built up internally or bought in: updating models, estimating memory requirements, measuring response times, managing access rights.
Quality is also a trade-off. Models that fit on a single workstation are smaller than the largest cloud models. For structured, recurring tasks with clear input, that can be enough. With open-ended questions and long context, the difference is more noticeable. Anyone working locally should therefore not aim for the largest possible task area, but specifically select the workflows where data protection is the deciding factor.
How to test the approach sensibly
Instead of a fundamental decision, a narrow test setup is advisable:
- Select a specific workflow that currently fails because of data protection questions — not the most demanding one, but one with a clear, measurable result.
- Define in advance what counts as good enough: hit rate, acceptable response time, effort for post-checking.
- Run the same workflow with a local and a cloud model and compare the results against each other, rather than adopting manufacturer claims.
- Connect the agent in such a way that the model remains interchangeable. The interfaces to your own systems outlast any current model generation.
The last point is the most important. The market for locally operable models moves quickly, and what counts as the top of a comparison today may be overtaken within a few months. Anyone who hard-wires their automation to a single model buys into a dependency that they were precisely trying to avoid by moving to their own infrastructure.
For web and software projects, this means: architecture determines the benefit more than the choice of model. Anyone who encapsulates AI functions behind their own interface can switch between local operation and cloud service, depending on how sensitive the data is in the respective use case. And data protection thus turns from an exclusion criterion into a decision that is made per workflow.