Getting a Model Running on Unfamiliar Hardware

The model server said it was listening. Requests still hung. In a September report to the llama.cpp project, which makes software for running local AI models, an operator using the name asksven described this problem on a Jetson Orin NX, a small NVIDIA computer. Neither a health check nor a request for generated text returned the expected response. The operator had hardware, model files and a running process, but still could not use the service.

The intervening work matters when deciding what to delegate to an AI agent. An agent might do more than write code that uses a model: it could configure the machine that makes the model available. My reading of new deployment experiments is that this expertise is becoming a capability worth evaluating in its own right. It could let someone put unfamiliar hardware to work without personally solving every installation problem. The value would depend on how reliably the agent finishes, and whether a good installer already does the job more cheaply.

Anthropic’s October 12 report supplies a concrete result. In its setup evaluation, an agent using Qwen3.5-9B, one of the smaller AI models tested, established a working model server at least once in 8 of 9 hardware categories. The machines ranged from cloud computers to small physical boards. The agent had hardware information, tools and time to retry; success was uneven across machines.

Hardware changes the installation

A model’s files do not specify every part of its installation. Software must use the available processor, fit the model into memory and expose a service that another program can actually call. Ollama’s hardware documentation illustrates the variation: processors need compatible software and drivers, and operating-system behavior can change which processor does the work. Its Linux documentation describes a case where waking a computer from suspension can leave Ollama using the CPU after failing to rediscover an NVIDIA GPU.

That distinction affects what counts as a successful installation. A server that produces an answer may still be using a slower execution path than intended. A process that announces its network address may never answer at all, as asksven reported. There are several possible stopping points between possessing a machine and having a service worth using.

The September operator tried different builds, CPU and GPU execution, memory-related settings and container conditions. The next day, asksven reported working health checks and completions after a clean restart; by September 28, the earlier failure could no longer be reproduced. That changed the initial diagnosis. The account documents the work and uncertainty of troubleshooting, rather than a confirmed cause or a test of AI assistance.

For an agent, the useful possibility is adapting its next step to what the machine actually does. A prepared installation procedure can encode known combinations. An agent could investigate a combination the procedure does not cover, choose another supported path or discover that its first explanation was wrong. To earn its cost, that reasoning has to change the installation in a useful way.

Anthropic’s report describes one Qwen agent debugging package versions for hours when the actual problem was an incompatible service endpoint. The service did not match the evaluator’s expected interface. More persistence with the wrong explanation was not enough. The researchers’ surrounding software periodically restarted an agent with its workspace and notes, giving a new instance room to try another route. Agents also received hardware information and a sample profiling script, with up to 6 hours for setup. The result belongs to that whole arrangement.

The setup results reveal substantial variation. Qwen succeeded in 13 of 15 attempts on the tested Xeon CPU environment, but only 1 of 5 on Google’s TPU, a specialized AI accelerator. The Jetson Nano used a smaller target model because of its memory limit. Reaching 8 categories shows range, while the failures show why choosing an agent for installation still requires attention to the particular machine and task.

Packaging can change what intelligence is needed

There is a strong alternative to having an agent reason through each installation: package the setup well. Ollama’s documentation provides supported paths for several processor families. A fleet of identical machines can use a maintained procedure. The difficult September report does not establish how often such help is needed, and Anthropic did not compare its agents with an experienced administrator or a mature installer on the same tasks.

The value of an agent might lie in handling exceptions around a good installation procedure. A team could retain a reliable standard setup while delegating diagnosis when a machine falls outside its assumptions. Once the agent finds a working route, that route could become part of the procedure. Adaptive reasoning could help create ordinary automation, rather than being invoked afresh for every identical machine.

A separate June study of AI-controlled computer worms shows why packaging remains part of the story. In 32 trials on GPU-equipped hosts, the researchers supplied administrator credentials. Launching a new agent succeeded in about 94% of attempts; completing local model serving as well succeeded in about 69%. Their main network experiments used a shared inference service, so they tested local serving separately. Establishing that service added ways for deployment to fail even after access was available.

Those researchers propose stronger packaging, supervision and recovery as ways to improve their system. Those improvements remain untested in their comparison. They point toward a practical choice: change the agent, change the software it inherits, or combine both. If a maintained package can remove a recurring obstacle, requiring the model to rediscover the solution would add work without adding a useful capability.

Starting a service is not yet sustaining a replicator

Anthropic’s security question makes the setup task especially consequential. A self-replicating system would need to establish the conditions for another copy to keep working. Installation competence would become part of how it acquires further usable computing capacity.

A separate demonstration divided Gemma4-26B-A4B across 15 purchased TV boxes. The model ran at 0.39 tokens per second, slowly generating the pieces of text called tokens. Sharing the work across several boxes made physical inference possible. Those boxes did not perform the propagation experiment.

In the separate contained propagation test, GLM-5.2 established serving on all 8 machines in 4 of 5 runs. The acting model ran on a separate server, and access to the targets was deliberately simplified. The targets were emulated Android machines serving a different, smaller model. This demonstrated deployment and propagation under those conditions, while leaving the acting agent dependent on computing resources outside the target fleet.

Keeping these demonstrations separate exposes the engineering problem. A model can fit on weak hardware without being competent enough to deploy the next installation. A more competent model can perform the deployment while depending on computing resources elsewhere. A self-sufficient system must join both abilities and keep operating at the speed its hardware permits. The experiments have not yet joined them on the physical TV boxes.

Slow hardware also changes the trade-off. An installation process working in the background might tolerate a wait that would make a conversational assistant frustrating. It still needs to survive interruptions, finish successfully and justify the resources it consumes. A machine able to run a model for a demonstration may not remain available long enough to build the next installation. Neither experiment establishes persistent operation on an uncontrolled real-world network.

For someone administering their own machines, the nearer opportunity is already concrete: delegate an unfamiliar installation and judge the service that results, along with what it cost to establish. The useful output could include a working model and a repeatable setup that avoids another round of diagnosis. That may make a spare machine worth keeping in service; it may also show why standardizing the fleet is cheaper. Hardware becomes a resource someone can use only when the intervening work is finished.

Sources

• William Zhou and Daniel Freeman, AI models may be capable of self-replicating without access to a datacenter, Anthropic, October 12, 2026.
• Anthropic, Inference setup across hardware and End-to-end replication, accompanying figures. The setup grid labels its data July 30, 2026.
• asksven, llama-server hangs before serving any request on Jetson Orin NX, September 26–28, 2026 issue report and updates.
• Ollama, Hardware support, documentation accessed October 14, 2026.
• Jonas Guan et al., AI Agents Enable Adaptive Computer Worms, June 2, 2026 preprint, version 1, Appendix C.2–C.3.