The market for permanently running AI agents continues to fill up. After OpenAI and Anthropic, Perplexity is now also offering agents with Teammates that are supposed to work through projects independently on their own virtual machines. This refers to systems that not only respond to individual inputs but pursue tasks over longer periods, execute intermediate steps and deliver results.

The offering requires Perplexity's Max subscription at 200 dollars per month. According to the report's assessment, this places it well above comparable offerings from OpenAI or Anthropic. For companies that want to equip several employees, this quickly adds up to a relevant budget item.

A benchmark dampens expectations

Almost simultaneously, SemiAnalysis published the GDPx benchmark, which confronts AI agents with simulated knowledge work – including chip design, quant trading and cyberattacks. In this context, a benchmark is a standardised test setup that allows different models to be compared under identical conditions.

The result is sobering: no model breaks the twelve percent success rate mark. Claude Opus 5.5 leads narrowly ahead of Gemini 4 Carbon. The order at the top is therefore close, but the overall level is low.

There is also criticism of the test itself. The small number of tasks per area and inconsistent test conditions are criticised. The figures are therefore an indication of the order of magnitude, not a precise measure of the performance of individual systems. Anyone drawing conclusions from this should keep both in mind: the success rates are low, and the methodology is disputed.

The gap between offering and evidence

Taken together, both reports paint a contradictory picture. On the one hand are products that are marketed as permanent employees and priced accordingly. On the other hand is a measurement attempt that finds very low hit rates for demanding specialist knowledge work.

This contradiction partially resolves when the task types are separated. GDPx tests specialised expert work in domains such as chip design or quantitative trading. That is not the same task class as research, compilation of information or recurring documentation work. Low values in highly specialised fields do not automatically mean that agents are unusable in simpler routines – but nor does the opposite.

What can realistically be delegated today

For practice in medium-sized companies, this yields some sober guidelines:

  • Prefer tasks with verifiable results. If a person can verify the result in a few minutes, the risk of an error remains manageable.
  • No unsupervised decisions. Wherever an agent touches contracts, payments, customer communication or production systems, human approval is required.
  • Grant limited rights. An agent on its own virtual machine is convenient, but also an additional access point. Access rights should be set as narrowly as possible.
  • Calculate costs per result, not per subscription. At 200 dollars per month per seat, it is worth asking how many usable work results are actually produced.

Pilot instead of rollout

A limited test run with clearly defined tasks and a metric determined before the start makes sense. This can be the number of usable results without rework or the processing time saved. What matters is that the evaluation is not subsequently adjusted to the result.

Documenting failures is equally important. Which tasks an agent reliably cannot handle is often more valuable for planning than the list of successes. It defines the boundary at which a process must remain staffed by humans.

It is also helpful to keep dependence on a single provider small. The market is moving quickly, and prices and scope of services are changing. Anyone who builds workflows so that the underlying model remains interchangeable retains room for manoeuvre.

Assessment for web and software projects

For project work, this means: always-on agents are a serious tool for preparatory and repeatable activities, but not a substitute for professional responsibility. Plan agents as an additional work layer whose results fundamentally go into a review rather than being written directly into production processes. As long as robust success rates for demanding knowledge work are lacking, the economic lever remains in cleanly delimited subtasks – and not in the complete delegation of entire project roles.

Sources