The Intern With a Time Machine.

Not long ago we hired a machine learning intern to work on improving our object detection models. She was a phd + postdoc in machine learning, significantly overqualified for the role, and did excellent work in the short time she was with us. Then she got an offer she couldn't turn down, and moved on.

We were left with her plans and her machine.

Rather than let the laptop sit idle I decided to try setting it up with Claude Code. I wired it into our repos, our model training scripts, our build tools, and the physical test devices we use for development. I wanted to see if it could take on some of the work we kept passing over.

Every product team I have worked in has a category of work that never gets done. Good ideas that come up time and time again, but are never making it to the top of the priority list. The value of implementing them is either too small or too unknown to justify the work. Polishing parts of the UI that we aren't certain will stay in the product long term anyway, experiments to improve the accuracy of our realtime basketball tracking when the current version is "good enough" and there is no guarantee that the time spent will actually yield results. At Kima these are things coming up in product syncs, discussed at length, but ultimately dismissed because we can't justify pulling a developer off higher priority work.

But the threshold for what we throw to an AI machine is totally different.

What the system actually does.

The way I think of our Claude Code set up is as a junior developer with no common sense, who is in possession of a time turner.

It can write code across our model training scripts, our object detection library, and the main Kima codebase. It can decide what experiments to try given an objective. It can run those experiments, evaluate the results, and decide what to continue or abandon. It can train new models, build and publish new library versions locally, pull those into the app, build and deploy to attached devices, run test sessions, collect the data, and report back the results in a tiny fraction of the time it would take a human. But it will also get lost in absurd rabbit holes, and confidently claim completely false conclusions built on entirely ridiculous premises.

The core loop looks something like: idea, implement, build, run, measure, repeat. Most of that now runs without me. Up front I need to define what we want to achieve and by what metrics we can measure success. Once that is done it can work independently to try different tracks and report back. I check in on it periodically to review and steer, and it will stop to ask for input on things that are unclear, like what direction to take after an inconclusive test. I also remote into the terminal from my normal machine so I can intervene when needed.

Same shot, same angle, every time.

We are building a basketball training app. To use it you need a real player on a real court. To test it you need the next best thing.

We have a basketball court at the office specifically for real world testing.

Using physical devices has always been essential because emulators and simulators cannot reliably replicate the hardware constraints of actual phones. When I work from home I had already been using a similar setup to what we now use for the automated system: a phone on a tripod pointed at a looping video on a screen.

The difference with the automated version is consistency. The phone is in the same position, at the same angle, under the same lighting, watching the same video, played from the exact same moment, every single time.

That sounds trivial but it is the only reason any of this works.

Much of what we do operates on fine margins. A small tweak to how we process frames might improve accuracy by a few percent, or it might make things worse by a similar amount. If the test conditions change between runs you cannot tell which it is. You are just guessing. With a repeatable setup the system can make meaningful comparisons between approaches. Without it, it just thrashes.

Ideas I didn't suggest.

Given a clear objective and something concrete to measure, it works better than I anticipated.

It tries more approaches than I would have considered. Not just variations on what I suggest, but different directions entirely. In one case it restructured how we run object detection and pose detection, switching them from sequential to parallel processing. I did not suggest that. The result was a 2x improvement in frames per second on Android, it made no difference on iOS so it abandoned the idea in the iOS code.

That was the moment it became clear we could get real value from this set up. The system had independently identified an architectural change, implemented it, tested it against the baseline, measured a significant improvement, and presented the comparison. The whole cycle probably took less time than it would have taken me to write the ticket.

It also explores dead ends much faster than a person would. It can try an approach, measure the outcome, determine it didn't help, and move on in the time it would take a developer to set up the experiment. The failures cost almost nothing.

Confident conclusions from meaningless data.

It gets things wrong a lot.

The main problem is that it does not know what it does not know. If the problem is not defined precisely, it will optimise the wrong thing and not realise it.

At one point it was comparing models and concluded that a larger model was worse because the test session took longer to complete. That makes no sense. Inference time affects how long each individual frame takes to process, which affects the frame rate of the live analysis. It has no effect whatsoever on how long a session runs. The session length is determined by the video it is watching. The analysis was technically correct in that the session did take longer, but the conclusion drawn from it was completely absurd.

The mistake was not in the code. It was in what the system thought it was measuring.

I had to go back and be much more explicit about what metrics actually indicate success for the thing I was asking it to improve. That meant thinking carefully upfront about what to measure, what comparisons are valid, and what a good result actually looks like. The system cannot do that thinking for you. It will happily measure whatever you tell it to and draw confident conclusions from meaningless data.

You have to do the thinking.

This only works with constant oversight. Not just at the start. Throughout.

I think there is a misconception that spec driven development should work here. You write a detailed brief, hand it over, and come back to a finished product. That is not how this works, and frankly it is not how development works with humans either. You gain insights and clarity during the work. The direction shifts as you learn things. When you hire a competent human you trust them to take on that learning and make sensible choices, you can't trust the AI, at least not yet. You have to do the thinking.

My job in this system is to define what success actually means, check whether the results support the conclusions, spot when the system has gone off track, and notice when something important has happened. That last one is more subtle than it sounds. Sometimes it stumbles across a significant improvement without recognising the significance. If I am not watching, that gets lost.

It behaves exactly like a junior developer. Just one that can compress weeks of exploration into minutes. And like any junior developer, it needs a conscientious supervisor reviewing all of its work. The difference is that because it lacks common sense entirely, you actually need to be more attentive than you would with a real junior. I caught mistakes that would have been disastrous if they had made it into the codebase. Which is why I will not let it work on anything that I or someone on my team did not have the expertise to review thoroughly.

A new tier of work.

The biggest change is not speed. It is what we now consider worth doing.

In our product syncs when we discuss priorities, we essentially have a new tier. Things that are important enough to let the machine have a go at, but not important enough to pull a person onto. Before this system existed, that tier did not exist. Those ideas just died.

A good example is our make/miss detection. The app needs to know whether a shot went in or not, and the existing version works well enough to give useful coaching feedback. But it could be better. I had ideas for things that might improve accuracy but no certainty that any of them would actually work. Before, that combination of "might help, might not" would have killed it in prioritisation. Instead we gave it to the machine with a test video where we knew the ground truth for every shot: when it happened, whether it was a swish, a long make, a short miss bouncing off the front rim. The system could run its changes against that video repeatedly, under identical conditions, and get a comparable accuracy score after each attempt. That kind of structured iteration would have been tedious and time consuming for a developer. For the machine it was just the next cycle.

That changes how you think about building. You can explore things you would not have touched before without pulling anyone off the work that actually matters. The cost of trying has dropped enough that the uncertain return becomes acceptable.

If this were fully mature it would probably look like multiple machines running in parallel, integrated into our project management tools so we could assign work to them and get progress updates directly alongside the human work. We are not there yet, but the direction is clear enough.

This setup does not replace developers. It does not work without oversight. It needs clear direction, repeatable test conditions, and someone paying attention the entire time. Without those things it will produce convincing results that are wrong.

But with them, it compresses the exploration cycle enough that a whole class of work becomes viable. Things that used to get discussed and dismissed now get tried.

Would you hire a junior developer with no common sense, and a time machine?

I would.