Personally I've found "carefully review every move it makes" to be an extremely unpleasant and difficult workflow. The effort needed to parse every action is immense, but there's a complete absence of creative engagement - no chance of flow state. Just the worst kind of work which I've been unable to sustain, unfortunately. At this point I mostly still do work by hand.
It's unpleasant for me at normal speed settings, but on fast mode it works really well: the AI does changes quickly enough for me to stay focused.
Of course this requires being fortunate enough that you have one of those AI positive employers where you can spend lots of money on clankers.
I don't review every move it makes, I rather have a workflow where I first ask it questions about the code, and it looks around and explores various design choices. then i nudge it towards the design choice I think is best, etc. That asking around about the code also loads up the context in the appropriate manner so that the AI knows how to do the change well.
It's a me in the loop workflow but that prevents a lot of bugs, makes me aware of the design choices, and thanks to fast mode, it is more pleasant and much faster than me manually doing it.
This is my biggest problem with the promises of agentic coding (well, there are an awful lot of problems, but this is the biggest one from an immediate practical perspective).
One the one hand, reviewing and micromaning everything it does is tedious and unrewarding. Unlike reviewing a colleague's code, you're never going to teach it anything; maybe you'll get some skills out of it if you finds something that comes up often enough it's worth writing a skill for. And this only gets you, at best, a slight speedup over writing it yourself, as you have to stay engaged and think about everything that's going on.
Or you can just let it grind away agentically and only test the final output. This allows you to get those huge gains at first, but it can easily just start accumulating more and more cruft and bad design decisions and hacks on top of hacks. And you increasingly don't know what it's doing or why, you're losing the skill of even being able to because you're not exercising it.
You're just building yourself a huge pile of technical debt. You might delete your prod database without realizing it. You might end up with an auth system that doesn't actually check the auth and so someone can just set a username of an admin in a cookie to log in. Or whatever; you have no idea, and even if the model gets it right 95% of the time, do you want to be periodically rolling a d20 and if you get a 1 you lose everything?
I don't think there's anything difficult or unpleasant about the process of letting the LLM run free, that's the whole point, it's nearly frictionless. Which includes not reviewing the code carefully. You say "need" but you mean "ought".
Reviewing isn't hard when the diff is what you asked for. It's when you asked for a one-line fix and get back 40 changed lines across four files. At that point you're not even reviewing your change anymore, you're auditing theirs.
This comment is completely out of touch with how typical office workers use their computers. "Package manager" is your feldspars. But it's even worse than that, because you don't train for the typical employee, you train for the least-technical employee lest they become completely useless overnight.
It seems strange to you? It's natural to how I write - intentionally avoiding politeness would be weirder to me.
But aside from that, an LLM is only a roleplayer. Treat it like an idiot that makes mistakes and it will act like one. Treat it like a coworker who you respect and it will act like one, and it will find better results.
Obviously nothing about how they act is set in stone but as a general rule this seems to me to be both wise and, in my experience, true as well.
I think if you treat it like a coworker who you respect, it will speak to you like a coworker who respects you, but will still make some idiotic mistakes...
Yeah that's increasingly been my feeling as well. I have to keep prefacing my Kagi recommendations with, "web search is less and less useful every year, but..."
I still appreciate being able to customize rankings, bangs, and redirects. But with how utterly shit the web is overall, any web search is basically only good if you know the site(s) the answer(s) will be on. When you're searching for something novel-to-you, even Kagi is just going to show you a full page of unregulated slop on the dumbest, just-registered-this-year domains. Real information is increasingly limited to small islands of trust.
As a rule, "pop history" is full of shit and is probably better considered misinformation than anything else. I probably don't I know of a single general-audience history/anthropology book that doesn't horrify scholars of the field.
As unfortunate as it is, studying cause-and-effect is extremely complex. If it's even theoretically possible to distill it down to easily digestible ideas, that's well outside our current technical capabilities.
There's usually going to be some true and interesting information in these books, but it will be too deeply embedded in a narrative that is misleading.
I'm an avid reader of history books (antiquity, middle ages), and I'd say I'm very picky, trawling through reviews and recommendations from trusted sources before deciding on a book.
I have already come across books that were a slog to read because of the author's simplistic worldview or obvious contrarian agenda (so I can definitely relate), but I've also read some masterpieces (for example, Kaldellis I believe is solid).
Unfortunately I don't count any historians among my friends, so I'd welcome any recs from you for authors that are the least bad, or a teardown of main antiquity/middle ages historians.
Interesting... I first went to the linked recent post What the Longevity Experts Don't Tell You. Sorry to be harsh: it was nonsense. It just lists a few weird, unscientific behaviours of John D Rockefeller and tries to draw lessons (to what end? longevity? is Rockefeller still alive?) from them despite there being no indication those behaviors even had any effect, let alone positive impact on longevity. It also doesn't bring up things "the longevity experts don't tell you," it's just summaries of topics in a single biography.
Still I gave this article a shot. I don't understand what it's doing. Like, one of the points about Thiel is that he destroyed Gawker to cover up his vampirism. He actually destroyed Gawker to cover up his relationship to Epstein, the pedophile and saboteur of US social/economic integrity. Why put a silly spin on that? I guess the entire thing is just a little joke... just doesn't feel like it belongs on the HN front page. I had higher expectations.
Oh, I admit that I didn't finish the Rockefeller article, since it looked like more of the same. I can see now how it's satire.
Honestly though, I'm still not sure what the point of the vampires one is. Satire relies on the reader drawing some conclusions that aren't laid out, and I don't really see where it's trying to lead me in that respect. Is it that these billionaires are fools for following bunk research?
Also weird it didn't mention Peter Attia's connection to Epstein outright. It did this weird tongue-in-cheek thing for a few paragraphs referencing Epstein only in the foot notes. I still can't tell whether what I read was actually praising these guys or extremely subtly sardonic.
A bit of a meta lesson for me here: Writing a short, pointed, opinionated blog post is blogging. If I care about blogging my thoughts, I need to just do it, not worry about rigor or depth ahead of time
That feels arbitrary as a measure of quality. Why isn't new research simply devalued and replication valued higher?
"Dr Alice failed to reproduce 20 would-be headline-grabbing papers, preventing them from sucking all the air out of the room in cancer research" is something laudable, but we're not lauding it.
But that seems almost trivially solved. In software it's common to value independent verification - e.g. code review. Someone who is only focused on writing new code instead of careful testing, refactoring, or peer review is widely viewed as a shitty developer by their peers. Of course there's management to consider and that's where incentives are skewed, but we're talking about a different structure. Why wouldn't the following work?
A single university or even department could make this change - reproduction is the important work, reproduction is what earns a PhD. Or require some split, 20-50% novel work maybe is also expected. Now the incentives are changed. Potentially, this university develops a reputation for reliable research. Others may follow suit.
Presumably, there's a step in this process where money incentivizes the opposite of my suggestion, and I'm not familiar with the process to know which.
Is it the university itself which will be starved of resources if it's not pumping out novel (yet unreproducible) research?
> In software it's common to value independent verification - e.g. code review. Someone who is only focused on writing new code instead of careful testing, refactoring, or peer review is widely viewed as a shitty developer by their peers.
That is good practice
It is rare, not common. Managers and funders pay for features
Unreliable insecure software sells very well, so making reliable secure software is a "waste of money", generally
> Presumably, there's a step in this process where money incentivizes the opposite of my suggestion, and I'm not familiar with the process to know which.
> Is it the university itself which will be starved of resources if it's not pumping out novel (yet unreproducible) research?
Researchers apply for grants to fund their research, the university is generally not paying for it and instead they receive a cut of the grant money if it is awarded (IE. The grant covers the costs to the university for providing the facilities to do the research). If a researcher could get funding to reproduce a result then they could absolutely do it, but that's not what funds are usually being handed out for.
Universities are not really motivated to slow down the research careers of their employees, on the contrary. They are very much interested in their employees making novel, highly cited publications and bringing in grants that those publications can lead to.
reply