When do we stop being a meat proxy and start becoming a useful bottleneck instead?

A short spillover from my last note here.

Looks like this trend even has its own website:

There are a couple of things about this AI thing that have been sticking with me lately. And I mean this AI thing: the one that suddenly feels like it’s everywhere, even though people like Hinton and Bengio were already deep in it in the 1980s and 1990s, and even though, before the ChatGPT moment (November 2022, for the record), we already had Word2Vec, BERT, and a whole research field quietly building the foundations before AI became commoditized into a consumer-facing product with much broader access. I think that’s great, and it’s getting better and better but this transition period, this strange middle we’re living through, is… weirdly great, and also a little spurious.

Three notions help guide my thinking as I navigate this strange transition space:

• The first comes from a line Karpathy keeps returning to (this): you can outsource your thinking, but you can’t outsource your understanding. I keep translating it for myself as you can’t delegate your understanding. Same idea. You can fool yourself with these self-regularizing systems that do everything while making you feel like you were the one who did it. That is super misleading. When you’re not really involved, the agent just does it better, and you become a non-useful bottleneck: slower, and definitely not sharper.

• The second is the idea of being a useful bottleneck. I don’t want to give up on being a bottleneck. I want to keep being a useful one, and I think where that usefulness lies will change over time. When coding-agent harnesses reached what was arguably their first genuinely useful phase (Claude Code, Codex, and the rest), I was a master’s student. Being a bottleneck in the code still had a decent payoff when I could catch assumptions that were not rooted in the domain, ask why that constant was assumed, ask why we needed ten functions for something that could have been one clean, Pythonic line. Those agents have improved so much that I don’t see much value in being a bottleneck in coding anymore (like Karpathy announced a year earlier), unless the work is safety-critical or demands a similar level of care. Personal and corporate privacy, regulation, and all of that are part of a much bigger conversation that I can’t honestly do justice to here as a normal citizen and PhD researcher. Setting those issues aside, right now I’m usually not a bottleneck in code. I’m a bottleneck in questioning the outputs, questioning why we needed to write that code, questioning whether there is a better way, and asking the research question early. I fully expect this to change in the very near future, and my response is simply to keep evolving until I find the next place where I can be a useful bottleneck. But whatever it takes, I want to keep being a bottleneck, and I want to keep being useful.

I still think of these models as probabilistic by nature, no longer merely a collection of matmuls, but perhaps a much better-packaged collection of matmuls wrapped in very nice-looking decorations and colors. Even with a lot of post-training (and I really don’t know what’s going on under the hood in the frontier labs), I am convinced that they still tend to produce ideas closer to the mean of the Gaussian: more mediocre ideas that feel like the result of interpolation within what already exists rather than extrapolation, which is where many major ideas in research and innovation seem to come from. I find it super hard to push them out into the right tail, where the extraordinary ideas, the black swans, live. Those still seem to come from over-100-h-index researchers and elite engineers who learned their domains, and their problems, the hard way; who know what’s been tried, where the limitations are, and which questions remain open. So for now, humans can still be very useful bottlenecks in problem formulation, research-question formulation, and feedback. And if that stops making sense later? Fine. Move somewhere else. Maybe the useful bottleneck shifts to physical execution: a robot, or some generative brain, tells you to do something and you still get to say why? I don’t think this makes sense. Whatever advances, the formulation I’m proposing for myself is simple: still be a useful bottleneck. Just keep evolving where that bottleneck sits and don’t ever reach a point where you can’t even question the necessity of doing what the generative model is telling you to do.

• The third term is meat proxy. I think I first saw it in a newsletter, then found Gruhn’s post, and I wish the phrase would take off the way it probably would if Karpathy tweeted it. I see it myself and I really hate it when people give up on being a useful bottleneck in anything: pasting AI output into a team chat, skipping the human pass, skipping the check for whether a UI still has sloppy wording in it. I ask a question, I put some time into writing and framing it, and someone drops a wall of probabilistic prose in response. Sometimes people even say “wow, nice work” to those nonsensical collections of bold bullet points. I still hope that isn’t, and can’t become, the new normal. Even if you feel like you are saving time, you are not. You are taking my time. Parsing AI-slop wording is a pain, and processing whole AI-slop paragraphs is an even bigger one (poor reviewers!).

I don’t know how this transition period will end up, but in the meantime, while watching everything unfold from the shore, I hereby promise myself to stick to these three notions:

I will never be a meat proxy, and I will keep being a useful bottleneck instead.

Because a useful bottleneck still owns the understanding.