I’ve used LLMs to augment my programming since 2022 (with pendejo GPT-3.5). But let me water my ego plant for a minute: I do actually know how to program and I know how computers work.

I had to take a break from LLMs from September-January. I was working exclusively in Zig, which hasn’t reached 1.0. The changing language specification (and the fact that the main support forum is on Discord) mean that LLM training data for Zig is incorrect, so I had the pleasure of digging through the std source and coding by hand for a while. (It legitimately was a pleasure, Zig is a small language, and I enjoyed the opportunity to dig deeper and learn about the corners of Zig).

So like most programmers, I’m riding this wave and experimenting, but still trying to get stuff done. It has also been fascinating to see how different programmers react and conceptualize the tech. I plan to do a longer post about LLMs and coding later.

But right now, I thought I’d just do a quick run-through of my first dive into the next “level” of LLM programming with Aider.

LLM coding “Levels”

It looks like Aider is on L2 of this classification system, but I’m not sure I agree with that. At any rate, Aider was a new “level” of LLM tooling for me:

• Level 1: Consumer chat completion, e.g. https://chat.deepseek.com/
• gptel (Level 2): Editor/IDE integration, allowing precise control over context, redirecting output, alternative model APIs, and similar features
• Aider (Level 3): Limited autonomy, automated context management, multi-agent cooperation, git integration, etc.
• Level 4: ????

Previously I was using gptel to efficiently chat with LLMs while programming. gptel is actually super cool and will probably have Aider-like features soon, but right now Aider presents some serious advantages:

• Aider will actually edit the files (if you tell it to), so you don’t have to manually make the changes it suggests
• Aider uses a repo map, which allows the LLMs to effectively request needed context files without sending the whole codebase
• Aider has an Architect mode where multiple LLMs collaborate to complete tasks
• Aider has the ability to execute arbitrary commands (e.g. make) and view the output, inching towards a more “agentic” workflow (I personally don’t let Aider execute commands yet)
• Aider has community insights and tools to improve developer productivity. For instance, the leaderboard page ranks model combinations and includes commands you can copy/paste to replicate the setups

Loss of Control

When you first try to use Aider, there’s this “loss of control” feeling that I think is really unpleasant and probably turns a lot of people off. But it’s really a product of how Aider is configured by default.

Basically, what happens is you start Aider in code mode, request some help with a bug, and a bunch of text scrolls past your window too fast to read. Then you realize that Aider has not only edited file(s) in your repository, but it has actually staged and committed them to Git! I was not prepared for Aider to start creating commits or touching my Git history in any way.

It’s smart of Aider’s authors to make Aider work so well with Git. For one thing, it positions Aider as a great CI/CD bugfix bot. You can watch one example workflow where Aider automatically submits PRs for Asana tickets when the user assigns them to a specific Bugfix agent. Note that each PR probably costs less than USD $0.01. I can easily imagine this becoming almost standard in the near future.

The other reason it’s smart for Aider to be so git-native is that it takes credit for its commits, enabling the Aider creators to count public Aider commits on Github.

Regardless, I don’t want Aider making commits on my repository at all. I’d argue that the feature should be opt-in, since most users probably don’t want it. I especially don’t want it when Aider writes code that doesn’t work…

Luckily, the feature is easy to disable.

You can tell Aider wrote itself

When you go to start using Aider beyond the bare minimum tutorial level, it becomes clear that you are using LLM-developed features and reading LLM-developed documentation. Don’t believe me?

Is it really a problem? Hesitant “Sort of?”. For instance, the documentation lacks cohesiveness and there’s no single source of truth for some things. The authors of Aider clearly took advantage of Aider’s ability to generate documentation for every tiny new feature. But it also kind of works, because there’s a good search function:

Being a normal human programmer, I immediately tried to configure everything in the suggested .aider.conf.yml file. Then I saw this warning in the file:

# Note: You can only put OpenAI and Anthropic API keys in the yaml
# config file. Keys for all APIs can be stored in a .env file
# https://aider.chat/docs/config/dotenv.html

Seems like an arbitrary restriction. I prefer to use other LLM providers because

1. they give more options,
2. they allow me to use open source models,
3. they’re cheaper and sometimes faster, and
4. fuck S@m Altm@n.

So, ok. I can’t use the YAML file. I’m sure as hell not storing my settings in the command-line switches to Aider, or setting them in the env for every single project. It looks like I can do everything in a .env file in my home directory. Great. I’ll just ignore the fact that I may now be leaking my API keys through other random utilities which also use that file.

Officially there are only 4 ways to set every option

But then you see that there are technically SIX ways to select ‘architect’ mode:

At this point I’ll admit I’m being kind of silly. It’s necessary to switch chat modes during a session and it’s good to give power users multiple ways to do things so they can build their own workflow. But I find that all of these little things add up and make Aider annoying to use sometimes.

Without diving into details, some other issues I found:

• Workflow for switching between models during a live session ranges from confusing/inconsistent to straight-up broken
• By default, long lines of input you type in a prompt are word-wrapped (ok) with a > added in front, so you can’t cleanly select your own prompt for reuse (not ok)
• Model selection for a given API is not always correct

Pessimist Takeaways

Aider has an impressive array of features and configuration, and I think it also kind of serves as a case study in why LLM coding doesn’t necessarily produce better outcomes in every situation.

LLMs can chew through a certain class of TODOs: simple features or bugfixes, documentation, changelogs, style fixes, and so on – but there’s still the core “skeleton” that needs to be structurally sound, and I’d argue it’s more important. I don’t know. I’m not saying Aider itself doesn’t have a solid skeleton, I just think you need to be careful of overdoing the adornments, because they come at a cost.

In other words, I still see a lot of value in coding without AI for certain tasks during the working day. I.e. use it when it helps, but not too much, and don’t waste time waiting on AI answers when you have important design and programming work. Some things are just important, and doing it yourself is the best way to get it right. Meanwhile you can collect LLM tasks and batch them later.