It's very capable on almost any coding task I've thrown at it, and very good for easy-to-medium hard scripts, new code bases.

It struggles on some complex tasks in larger code bases, e.g. using to debug and fix bugs in llama.cpp it gets close to working code but often introduces errors. For such tasks its still very useful as a search/explore tool and drafting fixes.

I'd run the following 5-10 times with one model, then again with a 2nd model.

"Verify the correctness and completeness of all security configs/rules in SETUP.md. Consider if anything is missing, and if anything is not needed. Do not modify any files; only write potential findings to report.txt"

"Verify all findings and claims in report.txt."

Replace "SETUP.md" with whatever you're working on.

It's both terrifying and incredible watching what the models get correct and what they get completely wrong.

However, after enough runs they tend to settle on a state they claim does not need any more edits. And that result is generally useful with much fewer errors/hallucinations compared to a single run.

It's called review-anvil and does find significant amount of problems that might pop up:

https://github.com/mrshu/agent-skills/#review-anvil

Or does "verify all claims in report" counteract that?

E.g. some findings first classified as moderate priority often get reclassified as low priority even if the finding itself is correct.

The exact phrasing doesn't seem to matter as much as keeping the prompts short, simple and to the point.

However some models seem to do a bit better when adding ", if any" to prompts such as "List potential improvements".

The increase in compute (decrease in brute-force cost) combined with price increases in many crypto tokens means brute-forcing old wallets can become worth it years after passwords were forgotten.

And of course even smaller, local AI models can now easily write optimized scripts to brute-force any given KDF function.

Moore law did its thing, now you can do it with a lot less computer power.

s/power/time/ maybe? Or on second thought: so energy-efficicient that it actually uses less power in the same-or-shorter time… which brings me back to "less compute power".

One set of models run on 8GB VRAM / 16GB RAM and another set runs on 24GB VRAM / 64GB RAM. Both are very useful for easy and easy-to-moderate complex code, respectively.

The latest open, small models are incredibly useful even at smaller sizes when configured properly (quant size, sampling params, careful use of context etc).

Bit of a hype madhouse whenever a new model is released, but it's pretty easy to filter out simple hype from people showing reproducible experiments, specific configs for llama.cpp, github links etc.

I've used this to debug some moderately complex bugs in golang and godot code and it works really well - the combo of having a new context with the (sometimes overly) granular debug logging and only the required, specific source code.

I use the pi-mono coding agent with several different new open models running locally.

The simpler and more precise the prompt the better it works. Some examples:

"Review all golang code files in this folder. Look for refactor opportunities that make the code simpler, shorter, easier to understand and easier to maintain, while not changing the logic, correctness or functionality of the code. Do not modify any code; only describe potential refactor changes."

After it lists a bunch of potential changes, it's then enough to write "Implement finding 4. XYZ" and sometimes add "Do not make any other changes" to keep the resulting agent actions focused.

"Do not modify any code; only describe potential changes."

I often add it to the end when prompting to e.g. review code for potential optimizations or refactor changes.

Interesting pros/cons vs the new Macbook Pros depending on your prefs.

And Linux runs better than ever on such machines.

Then again, I was looking in the UK, maybe prices are extra inflated there.

The 5090RTX mobile sits at 896GB/s, as opposed to the 1.8TB/s of the 5090 desktop and most mobile chips have way smaller bandwith than that, so speeds won't be incredible across the board like with Desktop computers.