That is the second memo the place I describe my current experiences on operating
small fashions domestically on my developer machine for agentic coding. In
the primary memo, I lined the various components that may affect the viability of that setup — {hardware}, mannequin alternative, runtime, harness. Right here I give attention to the concrete experiences, the duties I gave the fashions, what occurred, and my ultimate conclusions.
Scope
To recap: my focus is on agentic coding particularly, not simply auto-complete. The machines I used had been an M3 Max with 48GB RAM and an M5 Professional with 64GB RAM.
Viability funnel
Evaluating small fashions is kind of tedious to be trustworthy: downloading takes some time (on my non-fibre web connection in the midst of Berlin), then configuring the brand new mannequin in a harness, utilizing it for duties, decoding the outcomes, …
I considered it as working by a funnel of accelerating viability:
- Does it match into the RAM? If I haven’t got sufficient RAM, I will not even be capable of run
the mannequin, interval. For the overall evaluation of broader viability, let’s assume as a
baseline of 48GB out there RAM. - Does it run at affordable velocity? As soon as a mannequin is operating, the primary smoke take a look at is
to see how rapidly it will probably reply to a easy request. - Can it deal with instrument calling? As soon as I noticed that the request velocity was bearable, I
gave it a easy job from a coding harness that concerned studying and altering recordsdata, to
see if it will probably deal with the instrument calling. - Does it construct functionally right code?
- Can it deal with a continued dialog / extra context? As soon as a easy job labored,
I continued the dialog for longer, to see how a lot backwards and forwards it might soak up
phrases of size of the context. - Can it deal with a bigger or extra complicated job? If a setup survives 1-5, the subsequent degree could be to present it a extra complicated job to unravel and see what occurs.
- Is the code high quality acceptable? What’s the steadiness like between velocity of coding and overview effort?
The journey
It has been fairly a curler coaster journey, which is an statement in itself!
- Section 1 – handbook evals: I began with selecting a number of duties, after which doing them again and again with totally different fashions and configurations, “manually”. I wished to see what it feels like by way of the person expertise, which is difficult to do with an automatic setup.
- Section 2 – automated evals: Nonetheless, after I shared these outcomes with different colleagues in Thoughtworks, I instantly realized new issues from their enter and experiences. So to retry a few of my duties with totally different configurations, I vibe coded a small automated eval setup in spite of everything, which gave me extra knowledge.
- Section 3 – day-to-day use: In spite of everything of this, I labored with essentially the most promising mannequin (Qwen3.6 35B MoE) for some time on the duties that saved popping up in my daily, to see if I might combine it into my workflows.
Duties
Which job to present to the fashions makes a big distinction, and is among the key challenges of analysis. The primary duties I utilized in my extra systematical comparability had been each JavaScript/Typescript, so not loads of tech stack variety, hold that in thoughts.
In a much less systematic method, I additionally wrote a number of shell and Python scripts with it, which labored fantastic.
Because the duties are so vital, I am providing you with some particulars right here for 2 of them, to assist inform your interpretation of my outcomes. The selection of job is finally one of many largest components that determines viability of small, domestically run fashions – it is all about expectations. It is about complexity of the duty (How good is the mannequin at reasoning?), concerning the variety of recordsdata we estimate the agent must learn and write (How good is the mannequin at instrument calling? How large is the context window?)
Activity 1: Kind and cumulate an present bar chart
I need to change the diagram within the frontend titled, “Messages per
nameless poster”
– It needs to be “per poster”, not “nameless”
– I would like the bars within the bar chart to be sorted by variety of messages, with highest bar
first on the left
– I would like the x-axis to not present numbers (“#75”), as an alternative I would like it to indicate what
share of the general messages a sure bar is at. E.g., if the primary 10 bars add
as much as 240 messages, and the general messages in that point interval are 1000, then I would like
to see 24% on the x-axis on the tenth bar. I would like this share to be proven each tenth
bar.
- Wants some code search, albeit trivial (I give away the lede within the immediate, it must search for a selected chart title)
- Wants adjustments to 1-2 recordsdata
- The cumulation of values on the x-axis was a factor that fashions most regularly struggled with
- No pre-existing exams for these recordsdata, no specific expectation of exams so as to add
Section 1: Guide analysis
I attempted this with Qwen3.6 35B, Gemma 4 31B, Gemma 4 26B and Qwen Coder Subsequent 80B MoE, with each OpenCode and Pi because the harnesses.
- Qwen Coder Subsequent did reach a functionally right implementation of this inside these 2.5 minutes on my 64GB M5 Professional. Nonetheless, after I added one other message to the dialog after that, the runtime crashed – so it is succesful, however not realistically runnable.
- Qwen3.6 35B and Gemma 4 31B adjusted the sorting with out problem, however then it took me quarter-hour to commute with them concerning the cumulative percentages till that performance labored. Within the spirit of my “funnel” described at first, at this stage I simply wished to give attention to performance, and did not even take a look at the standard of the code – if the fashions can’t even give me performance proper, the standard does not matter a lot but.
- Gemma 4 26B was essentially the most profitable right here. It did implement the total set of issues I requested for. However then after I continued the dialog to ask for a refactoring, I acquired the “textual content wall of doom”… My understanding is that this may be mitigated with activating “presence penalty”, however I’ve not tried that but.
Section 2: Automated analysis
Frustratingly, the automated setup didn’t affirm the handbook expertise. Gemma 4 26B didn’t ship a functionally right resolution 3/3 occasions, wheras Qwen3 35B MoE succeeded 2/2. The failure was at all times that it did not correctly implement the x-axis labels I requested for. They both weren’t displayed in any respect, or improper.
Observe that for the automated analysis, I anticipate the agent to “one-shot” the issue, which isn’t absolutely sensible. I gave it entry to the browser as the one sensor to self-correct, but it surely by no means known as it. Increasing sensors and sensor use might result in self-correction that will make this extra viable in actual utilization.
Activity 2: Create a bar chart of nations primarily based on access_log knowledge
I need to add a horizontal bar chart to our visualisation that exhibits which nations the requests have come from. The title of the chart needs to be “International locations”. It ought to present the highest 10 nations, sorted by variety of requests, with the nation with essentially the most requests on the prime. All different nations needs to be grouped into an “Different” class on the backside of the chart. For entry entries with no nation worth, they need to not present up as their very own class, however be lumped into “Different” as nicely.
As a reminder, the entries now appear to be this:
{”ts”: “2026-05-24T01:10:41+02:00”, “ip”: “x.x.x.x”, “standing”: 200, “ref”: “https://the-referer.com”, “nation”: “Malta”}
We are going to solely must learn and alter this one file, don’t be concerned about the remainder of the appliance. We’re studying a bunch of *.ndjson recordsdata with entry log entries such because the one given for example above, and visualising them
@scripts/visualise_access_logs.mjs
- Modifications to precisely one file, a script that generates a HTML web page
- Pre-existing use of D3 for diagrams already within the file
- No take a look at setup for this script, no expectation of exams being written
Section 1: Guide analysis
I attempted this with Gemma 4 31B and Qwen 35B, and it was surprisingly tough! I noticed my reminiscence utilization enhance fairly a bit, very lengthy reasoning chains adopted by excruciatingly gradual makes an attempt to lastly edit the file. I attempted some totally different variations, like refactoring into smaller recordsdata with a giant mannequin first. I attempted with each OpenCode and Pi, but it surely didn’t make a lot of a distinction. My notes say issues like, “Gave up after 11 minutes”, “Gave up after 12 minutes”, “Stopped after 8 minutes”. No thought what was totally different right here in comparison with the opposite duties, it does not appear that difficult.
As this job required change to precisely one standalone file, I additionally tried to go old style right here: I began a plain chat with a mannequin in LM Studio, with none harness use, and pasted the total file in there – that gave me a working resolution! Nonetheless, the entire thing took about 6 minutes, and most of that point was spent by the mannequin regurgitating my 450 traces of earlier code, plus the added traces for the chart.
Section 2: Automated analysis
Within the automated setup, I ran this job with Qwen 35B MoE 7 occasions, and it didn’t correctly resolve it 5 out of these 7 occasions… The problem within the failures was at all times that there have been no labels displayed on the bars to depict the nations.
Right here is the place it will get attention-grabbing although: I then ran the evals once more on the M5 machine with 64GB, and it failed solely as soon as! Which was actually shocking to me, as a result of I used to be anticipating the 2 machines to ship totally different speeds, however I didn’t anticipate this huge distinction in high quality of output with the identical mannequin settings. This stays a thriller to me.
Day after day use
Activity traits
Along with these structured comparisons, I additionally used Qwen3.6 35B MoE commonly for daily duties, each work and private.
- Just a few bash and Python scripts – typically okay
- Including new content material entries to my private web site – good
- Making small, very nicely outlined adjustments to present codebases – typically okay
- Constructing a recreation from scratch (planning with Claude Sonnet, then delegating coding execution to the native mannequin) – began nicely, however fell aside for extra complicated logic
These are a few of my reflections on the selection of job, as of now:
- Will it require code analysis, or can the immediate level on the particular recordsdata to vary? The extra discovery the agent has to do, the extra it requires instrument calling, and the extra it fills up the context window, and due to this fact treasured RAM.
- What number of recordsdata would possibly it must edit? Even when we are able to level the agent at particular recordsdata and skip among the code search, the larger these recordsdata are, the extra strain on the context window.
- How particular are the directions? We have all develop into used to being much more imprecise with our coding agent directions, as the large fashions have gotten so succesful. Prompting brokers with native fashions feels a bit like a mixture of the early auto full days, and my first makes an attempt at utilizing Copilot’s “multi-file edit” mode in November 2024…
- What’s the tech stack? It is arduous to inform from this very small knowledge set, however my impression was that I used to be normally far more profitable with the Bash and Python duties than the JavaScript ones. However there might have been different components at play within the job sort.
Again to fundamentals
In my day by day utilization, I’ve truly fairly loved the journey again in time, to weaker capabilities. It felt virtually like a detox. A small mannequin has larger sensitivity to adjustments within the setup, so I acquired extra indicators about what helps and what does not. And the temptation to simply take a look at and never perceive the outcomes anymore is lots decrease, I went again to reviewing outcomes far more – and I imply that in a great way. My expertise with giving robust fashions plenty of autonomy, and surrendering to them in a method, has typically led to fixed rework and surprises at later phases. So I’ve loved that the weaker fashions make me go slower, and take extra care within the second, as an alternative of deferring that comprehension and care to the longer term.
A second perspective
My colleague Jigar Jani works with Qwen 35B MoE (4BIT) commonly on a 48GB Macbook, and does virtually all of his coding with it, on a “actual world” Python and React codebase. He’s constantly enhancing his harness with expertise, and has discovered Graphify and Perceive Something to be notably helpful to assist the fashions with code search and understanding. Jigar finds it fairly helpful and sees enhancements as he improves the harness, however he additionally stresses that code overview is tremendous vital.
To sum up…
It has been a irritating expertise with typically complicated outcomes, I due to this fact discover that it is undoubtedly not a plug-and-play sort of expertise but. However, I really feel like I perceive a bit higher which sorts of duties have the potential to work with these small fashions at this level, and I’m incorporating Qwen 3.6 in my workflows to additional enhance my instinct when to succeed in for it.
Total although, the agentic coding capabilities are undoubtedly very distant from what I’ve now develop into used to with greater fashions.
On the time of publishing, my default setup is:
- Mannequin: qwen3.6-35b-a3b, 4BIT quantization (48GB of accessible RAM are stretching it with that mannequin, I’ve to shut different excessive RAM functions throughout utilization)
- Reasoning off (LM Studio: Inference > Settings Customized Fields > Allow Pondering > disable)
- Max context window (LM Studio: Load > Context and Offload > Context Size > drag slider to max)
- OpenCode or Pi as coding harnesses
- I attain for it when I’ve small, easy duties, typically pre-planned by an even bigger mannequin
