Recently, I’ve gone down the rabbit hole of local LLMs, and one thing became clear from the start: if you can run inference locally, you should. While large language models like Sonnet and Opus can handle difficult tasks with ease—and have become essential to everyday work—I’d argue that their greatest strength is also their biggest drawback: they are almost too good at what they do.

Because these models seem to have all the answers and can complete most tasks in a single attempt, we, as end users, are bombarded with information, leaving little room to think about and act on their output. Smaller models, by contrast, with their limited context windows and lower token output rates, operate at a level that is cognitively much closer to humans. Smaller context windows force us to focus our work and break it down into digestible tasks before giving it to the model. Even the fact that most of these models have limited tooling support and web search capabilities—meaning they only know what they were trained on—can be a plus. A good model trained on a well-curated corpus can recall fundamentals while cutting through the noise of current affairs.

This brings me to the main point I want to make: local LLMs as learning partners may be the most important breakthrough we have seen in learning in years. Where Google falls short is in vetting information and explaining it—in other words, bridging the gap between knowledge as a set of encyclopedic facts and knowledge as conceptual understanding that has been internalized. Your personal tutor can spend long hours explaining concepts repeatedly until the fundamentals become clear. It can help identify gaps in your understanding or revisit past questions you have asked, and it can even give you a test and assess your level. It can create personalized, self-paced courses to introduce you to even the most demanding concepts. These models’ limitations—the fact that they can, and most likely will, hallucinate—are the razor your thinking will hone itself against, like finding a mistake in a coursebook. The fact that they can’t fit the full conversation into their context window, forcing you to ask and explain the same questions repeatedly, is what will help you internalize this knowledge. You need to be concise and specific with these tools to elicit valuable output. All this effort benefits the learner more than fancy tooling and extensive reasoning.

The best learning tools are not always the most powerful ones. Sometimes they are the ones that resist us, slow us down, and force us to meet them halfway. Local LLMs are valuable not despite their limitations, but because of them. In a world optimized for effortless answers, that friction may be exactly what meaningful learning requires.