Artisan Hand-Crafted Coding

Writing code by hand is soooo 2022.

I’ve been doing some less startup-y, more professional engineering things over the last ~6 months (invariably mixed with startup things), and one problem that I’ve consistently had is attribution of code to LLMs vs myself.

LLMs are amazing tools, they allow you to breeze through CRUD endpoints, mock interfaces with plain english, and ship tailwind apps faster than I can count to 10… But, one of the major problems is that LLMs like to lie a lot, especially via making stuff up.

I was talking with a senior engineering leader at large fintech last week and mentioned to him that some of his team might not be using the tools. His response was something along the lines of “yeah I see that - we want to use the tools but we want to make sure to use them in the right places (especially not places moving money in fintech)“. This is so true, and I agree. When we did higherrrrrrr, I spent maybe 1-2 days on interface and app, and 7 days (or more) on the initial version of the smart contracts. Moving money was risky, so it needed to be done by hand. I had a similar experience designing a real-time streaming pipeline for phone AI - the LLMs just lack the context and creativity to architect these apps correctly.

Thinking about it more though, our current tools for tracking code changes were never built for this abstraction. I’d like my git command line to be able to attribute:

1. lines that I wrote

2. lines the LLM wrote

3. lines that the LLM wrote but I reviewed and understood and approved

Then this should be shown in apps like github etc. as a way to both reflect the safety of the code but also reliability. I’m sure it would also be helpful in incident retros etc.

Also, I’m sure this goes for many other things besides code. I’m not hiding that the LLM does a lot of work for me now (it’s a feature, not a bug) but overall I’d like to be intellectually honest about what is mine, what I understand, and what is the machines. It also helps with trust in a world where so much content will be generated by AI.