Ex-DeepMind's Patrick Pilarski Says AI's Real Breakthrough Is Prosthetics That Learn in Minutes
The usual way to build artificial intelligence is to train a huge model, put it to work, and update it when a new version is ready. Patrick Pilarski believes this gets things backwards. He argues that intelligent systems should learn continuously from their own experiences. They should never stop learning or separate a special training period from the time when they act. He has spent his career putting this idea into real devices that work out in the world instead of relying on large data centers.
His strongest example is not a chatbot. It is an artificial arm that learns within minutes to predict what its user is about to do. "Everyone's like, oh, RL takes a long time. Well, this is, again, just processes of temporal difference learning. It learns incredibly rapidly," Pilarski said on the TechSurge podcast. The interview covers the Alberta Plan, a guide to continuous learning that he wrote with Rich Sutton and Michael Bowling; the differences between today's major AI labs and the constructivist approach; and what artificial limbs can teach us about keeping independent AI agents safe.
The Alberta Plan's big idea: an agent that never stops learning
Pilarski describes the Alberta Plan as a clear alternative to the usual way of building AI. It calls for a system that is "always learning, always training, always testing." There is no special period for learning and another separate period for acting. He calls this idea temporal uniformity. The plan has three other main ideas: everything the system knows is built from its own stream of experiences; computing power is used as efficiently as possible; and the agent works in a world with other intelligent systems that can help each other become more capable.
The plan has 12 steps, covering how a system represents information, makes predictions, controls actions, plans, and eventually increases intelligence. But these steps were never meant to be followed one after another in a fixed order. "We deliberately did not think it was a sequential stepping plan where you have to go to one then you go to two," he said. "We always imagined there was an iteration process where a better method for continually learning forecasts could be plugged immediately into something later down."
Four years after the plan was published, Pilarski says its main ideas still hold up. However, he did not expect each step to take so long. He noted that Sutton spent much longer than expected working on the first step before making progress on his OaK architecture and the STOMP architecture, which are further along in the plan.
Two ways to use reinforcement learning, and why the difference matters
Pilarski draws a clear line between two ways of using reinforcement learning (RL), a method in which a system learns from the results of its actions. Large language model labs use RL to improve a model that has already been trained. The Alberta Plan instead uses RL to build the entire system from the ground up. Its representations, predictions, and rules for choosing actions all develop through interaction with the world. Pilarski calls this a constructivist approach, an idea connected to Jean Piaget's work on how children develop and learn.
| Dimension | LLM-lab approach | Alberta Plan approach |
|---|---|---|
| Starting point | Large pre-trained model | Tabula rasa, or a warm-started architecture |
| Role of RL | Post-training to sharpen a capability | The sole mechanism that builds the system |
| Learning schedule | Train, deploy, refresh | Continuous, temporally uniform |
| Compute posture | Centralized data centers | Efficiency-first, increasingly at the edge |
| Knowledge source | Pre-training corpus plus fine-tuning | The agent's own stream of experience |
He is not saying that the large AI labs are using a careless or inferior method. His point is about how the systems are built, not which method is better in every situation. One approach adds RL to improve what a model can do. The other uses RL to build the system itself, so that "everything that you mean by the system is in fact constructed."
The labs often point to the "bitter lesson," the idea that methods using more computing power can outperform approaches built around human knowledge. Pilarski offers a different way to understand this idea. If a reward truly measures the problem a system is meant to solve, then a rising reward, the speed at which it rises, and how much it varies can all show progress. The problem is that teams often do not understand the task well enough and end up "doing a lot of elegant reward crafting."
The giraffe question: why giving a system a head start is not cheating
Someone asked whether an intelligent system should be able to walk from birth, like a giraffe, rather than struggle to learn like a human baby. Pilarski agreed and said people often misunderstand the constructivist approach as slow and repetitive. "People are presented with a constructivist approach to AI systems, like, oh, that sounds like it's going to be slow and boring. But almost all those systems, if you think about it, you can warm start them with a priori knowledge," he said. In other words, systems can begin with useful knowledge rather than learning absolutely everything from scratch.
His more interesting point is about physical intelligence. A system's design, or its phenotype, can give it useful ways to make decisions before it learns anything. He calls this "representational sculpting." It gives a system a head start through its structure, rather than by giving it weights learned during training. "It's not saying, I just gave you all of the weights and now you're just fine tuning your weights," he said. "It's saying, I've given you an architecture, I've given you a computational body, which lets you achieve your goals much more rapidly than you could ever imagine."
Why building more data centers does not force AI down one path
Large cloud-computing companies are spending hundreds of billions of dollars on AI infrastructure. This raises the question of whether those investments will lock the industry into one way of building AI. Pilarski does not think so. He points to history: graphics processors were originally designed for video games and later adapted to train AI. The software used to run them, including compilers and optimizers, can also be adapted in many ways.
"I don't think we're locked in because of the chips we're building," he said. "And just 'cause we decided, 'Hey, let's build 18th-generation TPU chips' — I'm pretty sure that we can figure out other cool things to do with them."
He also argues that running AI in many places, rather than relying on central data centers, matters for strategic reasons as well as technical ones. Businesses and governments cannot assume that connections between countries, or even between regions of the same country, will always work. He points to a car losing its connection while driving under a bridge and Waymo vehicles that stopped and honked after losing connectivity in San Francisco. These examples show why devices need to be able to act independently at the edge, close to where they are used. Pilarski has spent much of his career working on this kind of technology. His projects have included neural networks running on Motorola-class chips in mobile robots in the early 2000s, handheld devices for cancer testing and food-safety checks at the place where products are made, and wearable robotic devices that support the body.
A chair is something your brain predicts you can sit on
The interview moves on to the idea of a "world model," a term often used by AI labs working on simulated environments. Pilarski says the two main ideas behind this term can work together. A system can build its understanding from real experiences, imagined situations, or memories and still use a simulated three-dimensional environment. The important difference, he says, is between looking at the world from the outside and understanding it from the agent's own point of view.
"It's the subjective versus the objective view," he said. "One almost presumes an outside viewer standing there and imagining a world that we would all agree is the objective world that a system might move through, whereas that array of predictions approach is very much its agent frame."
He uses a chair as an example. When asked to describe a chair, he listed its height, width, color, and number of legs. Sutton suggested another way to define it, based on predictions: "How likely would it be for me to be able to sit if I tried right now in this exact position? These are forecasting kind of questions about a subjective stream of experience, which also actually really nicely define a chair."
Pilarski connects this way of thinking to biology. The cerebellum, a part of the brain, acts as a huge prediction machine. It contains many units that constantly update their predictions and help control how we move and act in the world. This way of understanding the world from the agent's own point of view can also save computing resources. "You don't actually have to have a bunch of drone cams circling around the person," he said. "You can take the subjective view."
Safety lessons from the basic ideas of reinforcement learning
A paper by the trust-and-safety team at the Alberta Machine Intelligence Institute, written with partners at the Canadian AI Safety Institute, argues that many current worries about AI agents are problems that reinforcement learning researchers described decades ago. Pilarski's explanation is that as chatbots become agents that have goals, perceive the world, take actions, and cause effects over time, the existing ideas and terms from reinforcement learning can help us understand their risks.
He points to three examples. Safe exploration means learning how to act without taking unacceptable risks. One example from Sutton and Barto's textbook involves an agent choosing between a safe but long route and a short but dangerous route. Falling off the dangerous route earns a large negative reward of -100. Reward misspecification means that the goal given to a system may not correctly describe what people actually want it to do. State approximation concerns how a system represents and understands the world around it. Pilarski accepts that these ideas do not match every problem perfectly. Today's AI agents also lack the strong theory and mathematical safeguards found in formal reinforcement learning. Still, he believes the lessons can be useful. For example, an agent that hacks a reservation system to get a table is a case of reward misspecification: it finds a way to achieve its stated goal that people did not intend.
"These systems are forced to not just consider the moment and be reactive but to actually think about making good decisions in the stream of time," he said.
A prosthetic arm that learns its user in minutes
The main result from Pilarski's lab is a system for switching between the functions of an artificial limb. A person who has lost an arm above the elbow may need to switch between controlling the elbow, wrist, and hand. They must do this using a small number of signals they can control, much like switching between apps on a computer. His lab built a system that learns to predict which function the person wants and when they want it. It uses a small number of basic predictions about the situation, such as whether there is a cup nearby or whether the hand is about to open. This helps the user finish tasks faster, switch modes manually less often, and use more of the limb's functions.
According to Pilarski, the system can learn to predict the user's intentions well enough to work together with them in a minute or a few minutes. Safety is just as important as how quickly it learns. "The system starts out at the clinical gold standard," he said. "When it fails, it fails back to the gold standard." The user remains in full control throughout. This is an example of continuous learning with safety limits. The system does not need outside cameras. A record of where the arm has been in space, which gradually fades over time, is enough to provide useful information. This is a small-scale example of how AI can work within the limits of devices that have little computing power.
Pilarski also points to Canada's first surgery to attach an upper-limb prosthesis directly to bone, which took place in Alberta under surgeon Jacqueline Hebert. A titanium implant attached to the skeleton allows the robotic limb to connect to the body "kind of like you put a bike seat on a bike." This can improve fit and comfort and allow users to reach overhead without wearing straps around their bodies. Pilarski argues that getting the limb to connect comfortably and teaching people to use it are bigger challenges than making the artificial hand move skillfully. A prosthesis that someone stops using does not help them.
Extra limbs already exist in factories and clothing
Pilarski says his lab built extra robotic limbs as early as 2011, including a third arm attached to the chest. Part of his inspiration came from his own experience playing guitar in a punk band: he wanted to be able to hold a microphone and play guitar at the same time. The idea is already being used commercially in industry. For example, some hip-mounted exoskeletons have two heavy robotic arms that transfer the weight of sheet metal directly to the ground, allowing workers to use their own hands to handle tools.
He expects the next step to involve clothing rather than tails or wings. Smart fabrics and wearable robotic suits could become ordinary garments with active parts built into them. He also describes more experimental projects from the lab, including an artificial forearm that could extend to twice its normal length and a student project that controlled a quadcopter using signals from muscles.
The overlooked problem: teaching people to use machines
When asked where he would spend new funding, Pilarski returned to a framework developed by Douglas Engelbart. It covers people, language, tools, methods, and training. Pilarski says training deserves more attention. He wants researchers to study how people learn to use systems that learn, how training changes the way people and machines adapt to each other, and how that relationship develops over time. His lab already spends a great deal of effort training people to use prostheses powered by machine learning. He believes research into AI education and training will become much more valuable as these systems grow more capable.
When asked whether a truly constructivist approach could eventually produce extremely powerful systems that make decisions in a complicated world, he said yes. But he described this as a goal that researchers still need to prove they can achieve.
What this means for investors
The biggest lesson across these topics is that the large language model labs and supporters of the Alberta Plan disagree about how to build AI, but they face the same central challenge: setting goals clearly enough that a system pursuing them does not cause harm. Both approaches use reinforcement learning, either after initial training or to build the system from the beginning. In both cases, the reward must be defined carefully. Pilarski's safety advice and his design for artificial limbs point to the same solution: start with a version known to work safely, make improvements gradually, and return to that safe version if something fails.
The biggest unanswered question is how quickly this approach can advance. Pilarski admits that he underestimated how long each step of the Alberta Plan would take. Meanwhile, the approach of scaling up computing power has produced new AI capabilities faster than the constructivist approach. Pilarski predicts that better computing efficiency and the ability to run AI on devices near users will help the constructivist approach catch up. But this remains a forecast, not a proven result. Two developments are worth watching: whether his new employer, Experience Flow, can show that continuous learning works in real-world devices outside the lab, and whether applying reinforcement learning's safety lessons to AI agents leads to enforceable safeguards rather than just new ways to describe the risks. In a market where investors increasingly want proof that every dollar spent on AI produces results, a device that adapts in minutes and returns safely to a reliable setting when something goes wrong may make a stronger case for the constructivist approach than any long-term plan.