There’s reasoning traces, and then there’s reasoning traces.
Understanding a really simple code agent
Just a quick note to start off the week. For those who may not have tried code agents. If you are familiar with them, this post is not for you.
We are really used to reasoning in LLMs. We see these chains of thought quite often in the LLMs, where it goes “Thinking about user’s request”…“Trying to understand the intent”…“Going through the steps to get the insight.”
Apple’s paper on the “The Illusion of Thinking” released in June 2025 (https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdf), took a look at whether reasoning effort in LLMs correlates with performance. They found evidence that “reasoning traces” could be more performance theater than real problem-solving.
There have been other papers since then on this topic.
This is where code agents shine. While hallucination is certainly possible (trust me, I have been burnt), they are much more useful for tracing the reasoning steps as you can see the code actually being executed. Plus, there is the advantage of having code really fail when the code agent does something wrong.
Instead of ‘I’m analyzing young customers...’, you see: filter_by_demographics(age_min=18, age_max=30) → 122 rows returned. If the code is wrong, or something is missing, the code fails - a very good thing when you want to avoid hallucinations.
1️⃣Code agents don’t generate plausible-sounding explanations to please you. They show the exact operations performed on the data, with concrete inputs and outputs at each step.
2️⃣You can also constrain them to specific tools - functions you write for them to use.
3️⃣And when things go wrong and they fail, you see them fail. You don’t get made-up reasoning steps and conclusions.
It’s hard to see the details in many of the AI services that offer Agentic AI with all the bells and whistles. Note though that not all AI agents are code agents. Some may just be conversational agents, not too different from a more autonomous chatbot.
Here’s a super simple code agent that analyzes some synthetic data. Give it a spin and see the real steps that a code agent takes to answer questions. To
#AIAgents #AgenticAI #AIRiskManagement
Note: To run it: Open as a Google Colab link → Click “Runtime” → “Run all” → Enter your OpenAI API key when prompted (get one at platform.openai.com if needed)
https://drive.google.com/file/d/1YaOkFsRwNpHdDmu5RgDp5AphwDOVs7-f/view?usp=sharing