Back to reading room

Chikocorp websites

Jim Keller

Jim Keller: Moore's Law, Microprocessors, and First Principles

Rank #5 | Hardware | Watch on YouTube

One of the strongest episodes on first-principles engineering, systems constraints, and technical leadership.

Curated Summary

A concise editorial summary of the episode’s core ideas.

Thesis

Jim Keller argues that progress in computing comes less from flashy invention than from excellent craftsmanship: getting fundamentals, interfaces, modularity, and organizational dynamics right so ideas can reliably become real systems. He extends that view from CPUs to AI hardware, suggesting the next major shift is from machines built to execute serial code or pixel shaders toward systems that natively execute graph-structured computation.

Why It Matters

For technical readers, the episode is a compact philosophy of how hard systems actually get built: performance is constrained by prediction, memory locality, scale, and human coordination as much as by abstract architecture. Keller's perspective also frames AI as a systems transition, where future advantage may come from compilers, dataflow, and scalable hardware-software co-design more than from any single algorithmic breakthrough.

Key Ideas

Practical Takeaways

Best For

This episode is best for hardware engineers, systems programmers, AI infrastructure builders, and technical leaders who care about how ambitious products get made under real constraints. It is especially valuable if you want durable heuristics about architecture, scaling, team design, and the transition from classical computing stacks to AI-centric ones.

Extended Reading

A longer, section-by-section synthesis of the full episode.

Engineering over novelty

Jim Keller argues that great computing systems come far more from disciplined craftsmanship than from chasing constant "innovation." He draws a distinction between occasional breakthrough ideas and the much larger body of work required to make them real: most computer design is reduction to practice, getting fundamentals right, and building clean abstractions. His example is branch prediction, where a single published idea significantly improved performance and ended up used across the industry, but the larger lesson is that breakthroughs only matter when they land inside teams capable of executing them well. He is especially critical of organizations that reward patents and novelty for their own sake, because that tends to neglect basics like libraries, tools, validation, and design hygiene. He returns several times to the idea that engineering is a craft. A system built from "great bricks" is more likely to become a great building, and good teams are often distinguished less by flashy invention than by their care with fundamentals. That same lens shapes his view of software history: technologies such as JavaScript and PHP may have looked crude, but simplicity, timing, accessibility, and the ability to spread quickly often beat technically cleaner alternatives. Keller also says instruction sets matter less than people think; what limits performance more fundamentally are the predictability of branches and data locality, plus the quality of tools and interfaces built around the architecture. "good engineering is great craftsmanship"

How big companies win, plateau, and miss pivots

On Intel, ARM, Apple, and Nvidia, Keller emphasizes that dominant companies win when they align technical capability, product fit, and organizational flexibility - and lose when bureaucracy or rigid assumptions take over. He credits Intel's long success not to marketing magic but to real innovation in process technology and repeated strategic pivots, starting with the move from memory to microprocessors under Andy Grove. But he also says Intel's later culture became too insular and too focused on its own tools and assumptions, especially compared with ARM's much more synthesis-friendly, ecosystem-oriented approach. ARM's strength, in his telling, was not one perfect product but a broad palette of designs spanning tiny microcontrollers to high-end processors, enabling many market experiments rather than one giant centralized bet. His discussion of leadership is unusually blunt: companies naturally drift toward too much order, because order feels productive, scales easily, and rewards managers who formalize process. Over time, bureaucracy captures the organization. In that framework, forceful leaders like Steve Jobs and Elon Musk function as a counterweight against the slide into stasis. Keller describes Jobs less as an engineer than as a first-principles selector of talent and possibility, someone who rejected "can't be done" as a category, while Musk is more likely to read the rocket manuals himself. The common theme is that exceptional organizations need both vision and a disruptive force against procedural gravity. "you are not along for the ride, you are the ride"

Modularity, beauty, and the limits of human design

Keller's notion of beauty in hardware and software centers on abstraction layers: systems are elegant when components can work together cleanly without needing to know each other's internals. He uses network stacks and AMD's Zen design as examples of modularity enabling quality, verification, and independent progress. In Zen, he pushed to define interfaces before RTL implementation, and that modularity reduced bugs because subsystems could be tested in isolation rather than as a giant tangle of hidden interactions. For him, beauty is not decorative; it is the practical alignment of human cognitive limits with the scope of a design problem. That cognitive framing matters because, in his view, humans are not getting smarter while systems keep getting more complex. A single person can create beauty only within a bounded design space, so scalable complexity requires decomposition into modules with clean interfaces. Even then, no one fully understands the total systems we now depend on. He imagines an alien looking at Twitter not as an app but as one layer in an incomprehensibly vast stack of networks, protocols, servers, power systems, and social dynamics that no individual human grasps. Human achievement, then, is increasingly collective and emergent: the colony can build what no ant understands.

Moore's law, scale, and why future computing may look inefficient

Keller treats Moore's law not as a single phenomenon but as a bundle of overlapping S-curves: hardware gets better steadily, but software methods, scaling strategies, and workload decomposition can improve much faster for a time. He points out that recent leaps in AI came from several vectors at once: neural-net efficiency improvements, better algorithms, better decomposition across many machines, and scaling from one computer to thousands. Hardware may still offer something like doubling every two years, but he sees larger gains coming from how many computers can be effectively brought to bear on a problem and how software evolves to exploit that scale. One of his most counterintuitive claims is that large-scale computing often advances through managed inefficiency. As workloads move from 100 machines to 10,000, per-machine efficiency can go down while total capability and value go way up. Search, social platforms, and AI training all followed this pattern: scale exposed bottlenecks, forced redesigns, and justified expensive infrastructure because the economic returns were large enough. He also predicts computation will become nearly ubiquitous, to the point where tiny embedded systems are cheaper than simple dedicated components. He gives a Tesla example where using a computer plus sensor became cheaper than a resistor, with the embedded computer still stronger than a machine he worked on in 1982.

Why AI hardware may need to move beyond GPUs

A major section of the conversation covers Keller's move to Tenstorrent and his thesis that AI workloads are not just "more parallel compute" but a distinct computational model centered on graphs. He contrasts classic CPUs, which extract parallelism from serial programs, with GPUs, which exploit given parallelism such as millions of pixels running shader programs independently. Neural networks, especially as expressed in frameworks like PyTorch, are graph-structured: matrix multiplies, convolutions, data movement, merges, pooling, and transformations connected in explicit dependency patterns. GPUs can emulate this model, but Keller's claim is that they were fundamentally built for pixels and shaders, not for native graph execution. Tenstorrent's bet, as he explains it, is to build hardware and a software stack that run graphs more naturally. The system compiles PyTorch models into graph representations, partitions large operations into appropriately sized chunks, schedules data movement and kernels directly, and extends across chips using packetized communication and Ethernet links. A key goal is programmability: a user should be able to take reasonable PyTorch code and run it without heroic, architecture-specific tuning. Keller contrasts this with GPU optimization culture, where naive matrix multiplication may achieve only 5-10% of peak performance and experts - the "CUDA ninjas" - are needed to approach hardware limits. Tenstorrent aims to reduce that gap by aligning hardware primitives with the actual semantics of AI workloads. He describes the company as still early, but already notable for having built a test chip and two production chips with relatively modest capital. The broader reason he joined is that AI hardware seems less like incremental processor work and more like a new computing frontier. He repeatedly comes back to customer basics, though: memory bandwidth, local bandwidth, compute intensity, power management, network behavior, drivers, and time-to-first-model matter more than hype. His target is simple and practical - plug in a PCIe card, install the driver, and get a model running in about an hour.

Self-driving, software 2.0, and the shift from code to data

Keller links Tenstorrent's direction to Andrej Karpathy's "software 2.0" idea: computing is shifting from human-written code toward systems specified largely by data and learned models. He sees that trend in autonomous driving, where the hard problem is no longer just writing logic but building a data engine that continually discovers failures, gathers relevant examples, retrains, and compresses improved models into shippable form factors. On Tesla's full self-driving effort, he says progress is real but slower than expected, and he remains unsure whether current onboard compute is exactly sufficient or off by 2x, 5x, 10x, or 100x. What matters is that the system keeps getting better as data grows and training improves. He thinks current driving pipelines remain constrained by labeled data, which is human-limited, and that the next major jump would come from more self-supervised or unsupervised learning, analogous to large language models. But he also points out a deeper gap: humans do not learn to drive from driving data alone. They bring an enormous prior world model involving language, physics, surfaces, intentions, and social context. That means autonomous driving may require broader conceptual grounding than current task-specific training captures. Still, he expects self-driving systems to remove huge classes of accidents even if they retain some failure modes humans would avoid, because the cars also have advantages humans lack: 360-degree sensing, no fatigue, no attention lapses, and fast emergency response.

Consciousness, dreaming, and minds as graph-like systems

The conversation moves from chips to consciousness without much transition, but Keller treats the jump as natural. He describes consciousness as a strange post-hoc, single-threaded narrative emerging from a massively parallel brain. Our experience feels unified and sequential, yet the substrate is billions of neurons running in parallel. Consciousness also appears delayed - roughly half a second behind events - and deeply constructive: there is no inner theater, only signals transformed through layers into a world-model we can inspect, narrate, and manipulate. In that sense, he suggests an AI might appear conscious if it built a world, modeled alternatives, maintained emotional or priority structures, and generated stories about past and future within that world. He is especially interested in dreaming and incubation as tools for thought. He says he processes ideas slowly and benefits from front-loading work, then letting it "soak" over time, including during sleep. Before bed, he deliberately loads important problems or questions into his mind, increasing the chances he will dream about them or wake with useful intuitions. He also advises against over-filtering ideas too early: some concepts need room to sit for days or years before they reveal value. That method mirrors his broader engineering philosophy - spaciousness first, reduction to practice later. On the brain as hardware, Keller notes features that seem almost arbitrary: a cortex only six or seven neurons deep, working memory of roughly seven items, and severe limits on direct high-dimensional thought. He wonders why humans cannot natively hold 100 or a million numbers in mind, or think fluently in 4D or 8D, and suggests future minds - biological, artificial, or hybrid - could be much stronger in these ways. Brain-computer interfaces therefore interest him not just as medical devices but as possible expansions of action, perception, rendering, and imagination. He expects AI-based rendering and immersive interfaces to become far more direct and realistic, whether through screens, headsets, or eventually more intimate links to cortex.

Life advice: know yourself, escape cages, and build real skill

Keller's advice to younger listeners is practical and psychological. First, get good at something you genuinely care about, because depth requires real interest and sustained attention. Second, do not let institutions or inherited scripts trap you in debt, boredom, or groupthink. He warns against young people merely repeating ready-made views from media or peers; that may feel safe, but it produces a narrow, borrowed life. The better route is self-knowledge, a clear sense of what energizes you, and deliberate confrontation with your fears. He speaks candidly about depression, embarrassment, and emotional baggage, arguing that many people carry invisible cages built from childhood patterns, humiliation, or unexamined anxieties. Meditation, physically intense activity, and honest observation of one's triggers can help. A strong line running through the interview is that agency comes from recognizing those patterns and deciding to break them rather than accumulate them. He also stresses balance: work alone is not enough, even for ambitious engineers. Love, family, friendship, and meaningful non-work space are not distractions from achievement but part of the mental ecology that makes sustained creative work possible. On love, he gives a distinctly functional engineer's answer: love keeps parts of the world from becoming invisible through habituation. We adapt to almost everything, but not to what we love; those things remain vivid, surprising, and worthy of attention. That applies to children, partners, work, and communities. In that sense love is not a sentimental add-on but a mechanism for preserving novelty, care, and depth across time. Keller treats it as essential both to good engineering and to a successful life, because without it there is no reason to keep looking closely enough to notice what matters.