@gurupanguji<p><strong>🔗 My AI agents are all nuts</strong></p><blockquote><p>But I’ve been the first responder on an incident and fed 4o — not o4-mini, 4o — log transcripts, and watched it in seconds spot LVM metadata corruption issues on a host we’ve been complaining about for months. Am I better than an LLM agent at interrogating OpenSearch logs and Honeycomb traces? No. No, I am not.</p><p>Feeding error logs to AI is a game a hit and miss. Sometimes, you find a clue, and sometimes, the issue is so out of its reach that it sends you off on a wild goose chase where you waste hours testing and verifying every single one of the agent’s leads.</p><p><a href="https://nikoheikkila.fi/blog/my-ai-agents-are-all-nuts/" rel="nofollow noopener" target="_blank">My AI Agents Are All Nuts</a></p></blockquote><p><a href="https://nikoheikkila.fi/blog/my-ai-agents-are-all-nuts/" rel="nofollow noopener" target="_blank">Niki Heikkilia</a> has a breakdown of arguments in response to “<a href="https://fly.io/blog/youre-all-nuts/" rel="nofollow noopener" target="_blank">My AI Skeptic Friends are All Nuts</a>” – a post by Thomas Ptacek that made the rounds last week about why adoption of LLMs for coding assistance must be adopted. </p><p>Given the perverse incentives of traffic and attention, I know it will take time, but serious people should read <a href="https://knightcolumbia.org/content/ai-as-normal-technology" rel="nofollow noopener" target="_blank">LLMs as a normal technology</a>. Figure out where it’s beneficial to you and use it as a tool. </p><p>Related to the blockquote here though I find the ability to think about LLM usage from a <a href="https://en.wikipedia.org/wiki/Phase_space" rel="nofollow noopener" target="_blank">phase space</a> perspective useful. If your question (and context window) is something that has established patterns – a lot of usage and writing in the web in the last 2-10 years, then it’s highly likely that the LLM is going to be useful to you. </p><p>In that spirit, I found these list of arguments about <em>why</em> an AI agent might not work to be a great list to keep in mind when leveraging LLMs:</p><blockquote><p>Here’s a summarised list of everything that still requires improvement regarding agentic programming.</p><ul><li>Never take a rule for granted. Agents are more than willing to bend and break them.</li><li>The more rules you impose on agents, the less they obey you. Talk about a robot uprising!</li><li>Agents get stuck easily, retrying the wrong fix again and again.</li><li>By default, agents touch every file and run every shell command unless you tell them not to. This is a hazardous risk.</li><li>The code agents write for production and tests is incredibly bloated and complicated. To continue from that point would take a significant amount of time, converging towards net zero in productivity gains.</li><li>Agents optimise for the number of lines delivered, which makes reviewing and maintaining their code risky and expensive.</li><li>Agents fill their context window and burn through tokens faster than you realise. This leads to context-switching as you switch to a new thread, requiring an agent to relearn everything.</li><li>Agents rapidly dispatch parallel API requests, often causing your computer to become rate-limited. This abruptly stops the flow since you must wait until the rate limits wear off.</li><li>Most people won’t throw away their AI-generated prototypes but continue to use them in production instead. We have witnessed this long before AI and will continue to witness it for the foreseeable future.</li><li>Lastly, working with agents is far from fun. I acknowledge it might affect my overall opinion, but I stand behind it.</li></ul><p><a href="https://nikoheikkila.fi/blog/my-ai-agents-are-all-nuts/#:~:text=Here%27s%20a%20summarised,stand%20behind%20it." rel="nofollow noopener" target="_blank">https://nikoheikkila.fi/blog/my-ai-agents-are-all-nuts/#:~:text=Here%27s%20a%20summarised,stand%20behind%20it.</a></p></blockquote><p>So leverage that and keep that in mind to ensure that a stochastic system doesn’t harm you in your experiments. Good luck. </p><p></p><p><a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://gurupanguji.com/tag/ai/" target="_blank">#ai</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://gurupanguji.com/tag/coding/" target="_blank">#coding</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://gurupanguji.com/tag/development/" target="_blank">#development</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://gurupanguji.com/tag/guidelines/" target="_blank">#guidelines</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://gurupanguji.com/tag/models/" target="_blank">#models</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://gurupanguji.com/tag/software-development/" target="_blank">#softwareDevelopment</a></p>