Gradient Descent, Again
Reflections - from independent life to art
I wrote about gradient descent three months ago on LinkedIn. I was figuring out how to wander the uneven terrain of independent life. The failures. The plans that drifted. Finding the right angle to navigate.
I’ve learned to be comfortable with the uncertainties and disappointments of this new phase. Even relaxed enough to pick up the brush again.
But somehow I met gradient descent again. This time when trying to learn to learn art again.
A light week of meetings. A bunch of regulators from around the world I am mentoring for a course on AI in financial services. A collaborator on the Agentic MRM paper who was in town. A consultant thinking of moving to a senior AI role in a bank, asking for my views. An ex-colleague from my strategic planning days, now in communications. A senior risk manager between jobs, developing scented cleaning supplies as a hustle.
Lots of building otherwise. And art. I’ve built up the discipline of doing watercolors every night before bed. Which started another round of gradient descent. And a bit of angst.
The Gap
A retrospective I wrote last week made the art sound resolved. An interest that could be an uncorrelated asset in this world of exhausting AI hype. (Side-note: I’m starting to think I have a love-hate relationship with AI. I’ve always loved the discipline, but it has increasingly become populated by hypesters and scammers that rile me.)
I thought I had hit the ground running with my watercolors. The practice had returned. A satisfying piece here. A passable piece there. Enough momentum to feel like my old friend was back. I thought the next five pieces would be even better.
The next five pieces across the week made me realize it had not. Almost none of them worked. My old friend had not forgiven the gap.
Gradient descent is how a model learns. It makes a prediction. Measures the gap between the prediction and the right answer. Then adjusts in the direction that reduces the gap. Then repeats.
It’s called gradient descent because of how it finds that direction. Imagine a hilly landscape. You’re standing somewhere on it, blindfolded. You want to get to the lowest point - the valley. You can’t see the whole landscape. But you can feel the slope beneath your feet. Gradient descent says - follow the slope downhill. One small step at a time. Measure the ground. Adjust. Step again. Eventually you reach the bottom. Or at least somewhere lower and better than where you started.
The gradient is the slope. The descent is following it.
Three months ago I wrote about gradient descent in independent life. Misses. Proposals that went nowhere. Plans that drifted. All of it telling me the slope and which way to go next. Lots of uncertainty. But I got reasonably comfortable with that. Not because it got easier. Just because I could detach from the results. And go with the flow.
Gradient descent in learning to learn art seems different.
Somehow, I find it harder to detach from the gap. The gap between what I see in my head and that muddy mess that appeared on paper in front of me.
And there’s the obsession and ambition problem. When you restart a practice you start looking at the masters. Chung Chien Wei and his effortless wet-on-wet scenes. Thomas Schaller’s architectural washes, luminous and controlled. Alvaro Castagnet’s explosive loose strokes that somehow resolve into something complete.
You know what good looks like. You can feel the gap between that and what’s on your desk. And the gap isn’t abstract. It’s a piece you spent an hour on that looks nothing like what you intended.
One piece almost hit the spot. For a moment something was clicking. Then I lost it. The next piece drifted back toward the same problems as before.
Getting close and losing it is worse than consistent misses. When nothing works you can tell yourself you’re still finding the slope. When something almost works you know exactly what you’re reaching for. But then it becomes elusive again.
The Way Forward
The difference between knowing gradient descent and feeling it is considerable. I wrote about it in the context of independent life.
What I didn’t write about is what it actually feels like to sit with a miss. To look at something that isn’t working and try to stay patient. To not force it. To trust that the signal is in there somewhere even when you can’t see it yet. And to let go if it really just needs to go.
Three months ago the gradient descent was in independent life. Now it’s in a practice. More immediate. More personal. More uncomfortable in a different way. But not very different.
And I think the way forward is simple.
Pick up the brush again tomorrow. That’s all gradient descent really requires.
#GradientDescent #Art #IndependentLife #AIRiskManagement #Transitions #Reflections