My Corner of the Web 
Over the past several days, I have been experimenting with local AI coding agents as a possible complement—or eventually an alternative—to frontier cloud-based coding agents such as OpenAI Codex.
The motivation was practical. AI-assisted development has become an important part of the development process for my EZ-Blog project, but cloud-agent usage is finite. If capable coding agents could be run locally, they could provide additional development capacity without consuming cloud quotas.
What began as a model-performance experiment quickly became something more interesting: an exploration of the entire agent stack.
The quality of an AI coding agent is not determined by the language model alone. It depends on the combination of:
• the language model
• the inference runtime
• the agent harness
• tool-calling compatibility
• context-window management
• hardware resources
• and the quality of the development environment presented to the agent
That distinction became increasingly important as the experiments progressed.
What Is an Agent Harness?
A coding model by itself does not modify a repository.
An agent harness surrounds the model with the machinery required to perform software-engineering work. It provides tools for reading files, searching the repository, editing source code, running tests, executing shell commands, maintaining task state, and feeding tool results back into the model.
Visual Studio Code’s Agent mode, Qwen Code, and Codex are therefore not merely user interfaces to language models. They are agent harnesses. This distinction explains an important observation from my testing:
A capable model with a poor model-to-harness interface can be effectively useless as an agent.
The Hardware Constraint
My current desktop contains an NVIDIA RTX 4060 with 8 GB of VRAM. That is perfectly adequate for ordinary desktop development and can run small AI models very quickly, but it imposes a severe constraint on modern coding models.
A 7-billion-parameter coding model could fit almost entirely on the GPU and produced approximately 53 tokens per second. Unfortunately, the smaller models I tested did not have the software-engineering capability I wanted.
Larger models were substantially more capable, but once their weights exceeded GPU memory they had to be split between the GPU and normal system memory. That dramatically changed performance.
Qwen3-Coder: The First Promising Local Agent
The first model that demonstrated genuinely useful agentic development capability was Qwen3-Coder-30B-A3B.
Its mixture-of-experts architecture made it particularly interesting. Although the complete model is large, only a subset of the parameters are active during inference. Using llama.cpp and the Visual Studio Code agent harness, Qwen successfully inspected the EZ-Blog repository, generated code, invoked Maven, and attempted a real Phase 2 persistence task.
It was not a failure.
But it was also not yet good enough to trust with autonomous architectural work. Its implementation was useful but contained several weaknesses:
• the contract-test package structure was incorrect
• some repository-read tests depended on repository-write operations
• date preservation semantics were weakened after tests failed
• atomic invalid-mutation behavior was incomplete
• verification and reporting discipline were weaker than desired
I rated the resulting implementation at roughly 5.5 to 6 out of 10.
That was still a meaningful milestone. It demonstrated that a local model could participate productively in a supervised development workflow.
Then Tool Calling Became the Problem
I next tested smaller models in an attempt to find something that could fit completely inside the RTX 4060’s 8 GB of VRAM.
Qwen2.5-Coder 7B was extremely fast—again around 53 tokens per second—but exposed a different problem.
The model correctly understood that it needed to call a tool such as list_dir, but the inference servers returned the tool request as ordinary text instead of returning a structured API tool call. Conceptually, the model was doing this:
Model decides it needs a tool
→ generates the correct tool request
→ runtime returns it as ordinary text
→ agent harness cannot execute it
Both llama.cpp and Ollama exhibited versions of this behavior with the model.
The model understood the task, but the agent could not act. This was a powerful demonstration that benchmark scores alone are insufficient. The model, inference server, chat template, tool parser, and agent harness must all agree on the tool-calling protocol.
Hermes Proved the Harness Could Work
To isolate the problem, I tested Hermes-2-Pro-Llama-3-8B, a model specifically trained for function calling.
This time llama.cpp returned a proper structured tool call.
That proved the basic chain worked:
VS Code Agent
→ llama-server
→ language model
→ structured tool call
→ VS Code executes the tool
Unfortunately, that particular Hermes model had only an 8K native context window—far too small for serious repository-level agentic development. Still, it was an important diagnostic success.
Devstral Small 2: The Most Capable Local Candidate
The most interesting local model tested was Devstral Small 2 24B.
Unlike the smaller models, Devstral was explicitly designed for agentic software engineering and has published SWE-bench performance close enough to frontier systems to make it a serious candidate. It also supported a large context window and proper structured tool calls. On my current machine, however, the 24B model could not fit in GPU memory. With a 64K context under Ollama, the model occupied approximately 25 GB and was reported as:
79% CPU / 21% GPU
That distribution became the dominant performance limitation. Simple agent operations were surprisingly reasonable. A repository-listing task began responding in roughly 40 seconds and completed in about two minutes.
But realistic software development changed the picture.
Ollama and Long-Running Agent Work
Devstral initially worked correctly through Ollama. Structured tool calls succeeded and Visual Studio Code could execute them.
During a larger multi-step development task, however, the agent repeatedly stopped with:
“Sorry, no response was returned.”
Retrying allowed the agent to continue, but eventually Ollama itself entered a persistent “Stopping...” state while the agent harness was still waiting for the API response. That made the Ollama configuration unsuitable for the long-running experiment.
Switching Devstral to llama.cpp
I then moved Devstral to llama-server.
The first obstacle was subtle but instructive. The Devstral Jinja chat template enforced a strict alternation of user and assistant messages. Visual Studio Code’s agent harness uses a more complicated message sequence involving tools and agent state, causing llama.cpp to reject the request with an HTTP 500 error.
The server reported that conversation roles had to alternate between user and assistant except for tool calls and results. Removing that restrictive validation block from a local copy of the template resolved the incompatibility. After that modification, Visual Studio Code, llama.cpp, and Devstral successfully operated together.
The Real Devstral Test
I then gave Devstral the same real EZ-Blog development task used previously with Qwen:
Create a reusable ArticleRepository contract-test suite for the new persistence architecture and prove the existing YAML repository against it. This was deliberately not a toy coding exercise.
The agent needed to:
• read the persistence architecture documentation
• inspect repository interfaces and implementations
• understand existing tests
• distinguish neutral repository semantics from YAML-specific behavior
• generate reusable tests
• run Maven verification
• interpret failures
• and report the result accurately
Devstral successfully completed the entire workflow.
Its planning was particularly good. It created an eight-step task plan and systematically progressed through repository analysis, implementation, focused verification, broader verification, and final reporting. That was the strongest autonomous behavior I observed from a local model.
But Architectural Judgment Still Mattered
The resulting Devstral implementation was useful, but it made an important architectural mistake. The YAML repository stores the article creation date without its time component and does not currently preserve the lastUpdated field. When Devstral’s initial neutral contract tests failed because of those limitations, it modified the shared contract to match the YAML implementation.
That reversed the intended dependency.
The purpose of the shared repository contract was to define behavior that both YAML and the future SQLite implementation should satisfy—not to force future repository implementations to reproduce every limitation of the legacy YAML adapter. Devstral also did not fully test durable atomic rejection of invalid mutations and did not establish ordering semantics as rigorously as intended.
I rated the implementation at approximately 6.5 out of 10.
That was better than Qwen3-Coder and demonstrated genuinely useful local-agent capability, but it still required senior-level architectural review.
The Frontier Control Test
At this point I had only a small amount of Codex quota remaining, which created an ideal final experiment.
I preserved the Devstral implementation on a separate Git branch, restored the repository to the identical starting point, opened a fresh Codex session, and gave Codex the exact same prompt.
The result was striking.
Codex completed the entire task in 3 minutes and 20 seconds. Contrast that to over 2 hours for Devstral!
More importantly, its implementation was materially better.
What Codex Did Differently
Codex correctly placed the reusable tests in the repository contract-test package and introduced a small RepositoryTestContext abstraction that separated the neutral repository contract from adapter-specific backing-store lifecycle.
That allowed the same contract to naturally support both YAML today and SQLite later. Codex also tested durability correctly. Instead of merely saving an article and reading it back from the same repository object, it reopened the repository from its backing store and verified that the state persisted.
Its contract verified:
• repository ordering
• append semantics
• update without reordering
• delete behavior
• durability across repository reopen
• trimmed but case-sensitive slug lookup
• domain-field reconstruction
• atomic rejection of a null save
Most importantly, Codex did not turn YAML’s limitations into universal repository behavior. It restricted the shared contract to behavior legitimately common to the adapters and explicitly identified unresolved validation semantics as a follow-up issue rather than silently changing production code or weakening the contract. That was exactly the architectural judgment the task was designed to test.
The Most Important Metric: Time to Acceptable Code
One of the clearest lessons from this experiment is that tokens per second is not the metric that matters most.
Neither is raw benchmark percentage.
The metric that matters to me as a developer is:
How long does it take to produce an implementation I am willing to accept?
The final comparison was approximately:
Qwen3-Coder 30B-A3B
Useful supervised implementation — about 5.5–6/10
Devstral Small 2 24B
Strong agent workflow, architectural review required — about 6.5/10
Codex
Clean, architecturally appropriate implementation — about 9/10
Codex completed the task in just over three minutes.
Devstral completed it, but during substantial code generation my current hardware fell to approximately 1.57 generated tokens per second.
A faster GPU would dramatically improve that performance. It would not, however, automatically eliminate the architectural-quality difference observed in this experiment.
The Harness Matters Too
The experiment also exposed problems in the frontier environment.
Codex itself encountered environment-related friction, including access to repository-local Maven artifacts under sandboxing. In previous development sessions I have also seen unnecessary failures caused by Windows PowerShell versus Bash quoting and shell-selection issues. These are not model-intelligence problems.
They are agent-harness and environment-configuration problems.
A frontier model can still waste substantial tokens if it executes the wrong shell syntax, retries malformed commands, or encounters an environment that does not match the project’s documented development workflow.
That is therefore my next optimization target.
Before simply buying more model capacity, I want to make sure the Codex harness has an unambiguous execution contract:
• Git Bash and POSIX syntax
• repository-root Maven execution
• correct reactor-module selection
• reliable access to the local Maven repository
• minimal command quoting
• explicit failure-diagnosis rules instead of repeated equivalent retries
Where Local AI Fits Today
The experiment did not convince me that local AI coding agents are useless.
Quite the opposite.
A 24B local model successfully analyzed a real multi-module Java project, planned a development task, modified the repository, executed Maven, diagnosed failures, and completed the assignment. That is remarkable capability for a model running on consumer hardware. But it also revealed a practical tiering of AI-assisted software development.
Frontier agents
Best for architecture-sensitive changes, ambiguous requirements, repository-wide refactoring, persistence contracts, migrations, and tasks where incorrect design decisions are expensive.
Strong local agents
Useful for supervised implementation, bounded tasks with well-established architecture, test generation, code analysis, and overflow development capacity when frontier-agent quota is unavailable.
Small local models
Excellent for fast utility work, localized edits, explanation, code search, boilerplate, and other tasks that do not require deep architectural reasoning.
The Short-Term Decision
For my own development work, quality currently matters more than maximizing local autonomy.
The Codex control test made that decision much easier. In the short term, I plan to increase my Codex capacity rather than attempt to replace frontier coding agents with local hardware.
Before doing so, however, I will first audit and optimize the Codex harness and development environment so that expensive frontier tokens are not wasted on avoidable shell, quoting, sandbox, or Maven-environment errors.
I will also continue watching local models closely.
The fact that Devstral Small 2 could successfully complete this task suggests that a future machine with substantially more VRAM could make strong local agents genuinely practical.
But the final experiment demonstrated something even more important:
The goal is not to run the largest model locally.
The goal is to create a reliable human-AI development system that produces correct software quickly.
Right now, for architecture-sensitive work, frontier agents remain an important part of that system.
Local agents are no longer toys—but they are not yet equivalent substitutes.