Juan Benet Podcast

Juan Benet Podcast

Playback speed
×
Share post
Share post at current time
0:00
/
Generate transcript
A transcript unlocks clips, previews, and editing.

The Power of a Single Neuron and Simulating the Brain | Dr. Konrad Kording

How neurons actually compute, and why the path to understanding the brain runs through reading its wiring — not just recording it.
Juan Benet's avatar
Juan Benet

New episode with Dr. Konrad Kording, professor of bioengineering and neuroscience at the University of Pennsylvania and co-director of CIFAR’s Learning in Machines & Brains program. Konrad works at the intersection of causality, machine learning, and neuroscience, building rigorous methods for causal reasoning when experiments aren’t possible — and challenging how researchers interpret neural data and build AI.

Konrad argues the most promising path to understanding how the brain works is to read the brain’s wiring directly, down to the molecular detail of each connection, and to build compilers and simulations to understand the brain’s computation directly.

In this episode we go deep into how neurons work, how neurons wire together, and how organic and artificial neural networks differ. We discuss why organic neurons are doing much more; how a model of a single organic neuron can solve MNIST — computing more like a 3-layer artificial neural network; how the brain might learn by solving credit assignment with only local signals; how to approximate backprop without a global algorithm; why AI and humans are intelligent along different dimensions; why Konrad isn’t very worried about AI replacing us; economic models of intelligence and physical work; and much more.

Konrad is a brilliant, contrarian thinker who explains complex concepts very intuitively. It is a solid computational neuroscience primer. I hope you enjoy this conversation as much as I did!

Other links to this episode and references below.

Topics covered

Podcast Links

Links From the Podcast Episode

Guest + Organizations

Papers directly from Konrad Kording’s lab:

Referenced external papers:

Books & Media:

Juan & Protocol Labs

Transcription

Juan Benet

My guest today is Professor Konrad Kording, a computational neuroscientist known for bridging neuroscience, machine learning, and motor control. He is professor at the University of Pennsylvania, spanning neuroscience, bioengineering, and physics. He’s one of the top scientists working to understand how our brains work.

He’s done tons of pioneering work, including, you know, some of the earliest Bayesian brain hypotheses and figuring out why control works. He’s a clear contrarian thinker on what neuroscience does and does not yet understand about the brain. And he consistently pushes for more rigorous, more falsifiable models in neuroscience and beyond.

Konrad, thank you for being here and let’s dive in.

Konrad Kording

Thanks so much for having me, Juan.

Juan Benet

Great. So we’re going to start from the bottom up. What is a neuron and how does it work?

Konrad Kording

Well that’s a great question. So neurons are small structures that do all the information processing in brains. They usually have a cell body that is maybe ten micrometer big, hundreds of a millimeter. And then they have a piece that takes the information from other neurons. It’s called a dendrite. And it might be a millimeter long.

There’s a large variation there. And then it has an axon, which is the wire along which it sends the signals that it produces, which may be very short, maybe just a few micrometers, or it might be all the way from the brain to the feet of the giraffe, which would be meters in scale. And the way in neuroscience, how we think about them is that they are input output devices.

We have inputs. They’re called the synapses from other neurons. It’s putting all these inputs together produces an output which is spikes. And ultimately these spikes control your body and are the basis of all the interactions that happen in your brain.

Juan Benet

And when a neuron interacts with another neuron, what are the different. How does that communication happen through the synapse.

Konrad Kording

Yeah. So in most cases neurons are what we call spiking. So there’s an electrical signal that comes out of the neuron. And that then reaches the synapse which is the place where they meet one another, where it reaches that in so-called chemical synapses. We have this phenomenon where the cell where the signal comes from throws a lot of chemicals out of the cell, and the next cell that it’s connected to.

The downstream neuron basically has receptors for these chemicals. We call those chemicals neurotransmitters. And these chemicals. Then once they bind the receptors produce an influence on the postsynaptic cell. Usually that is just the current. It happens by having ion channels just like a battery.

They open and then the ions go through it. Current produces the signal on the postsynaptic side, and that is the basis of computation in neurons and variance.

Juan Benet

You said most neurons are spiking neurons. What are the what are the non most neurons.

Konrad Kording

Yeah there are some neurons that are what we call analog neurons. That way you can say what changes over time is the voltage in it. So spike is something that is either there or it’s not there. It’s like a telegraph signal like beep, beep, beep and it arrives on the other side. Whereas whereas analog neurons, you can think about it more like wires that can go up and down and are just graded signals.

Much of computation in human brains is in terms of spiking neurons. Much of the computation in very small systems maybe once or something is graded. much of the information processing in your eyes and your retina is graded as analog. And then the outputs of. From your eye are spiking. And then basically the eye signal sends a signal to the rest of the brain.

That’s the basis of seeing.

Juan Benet

And how all these different functions get computed in the brain. In the computer science model that we have. We have a very, you know, simple abstraction where we tend to look at a neuron as just one mathematical object with a set of inputs coming through and then firing out. And they’re effectively the same everywhere.

But brain neurons aren’t like that. They’re very different. There are many different types.

Konrad Kording

Yeah, there’s lots of brain, lots of different neurons and brains. And I think their function is much more complicated where you can say so in the, in the neuron. I told you, there’s a lot of synapses that go there, roughly 10,000 depending on where you are in the brain. And what happens is every synapse produces a current on the postsynaptic side.

But it’s not as in computer science, where we say the output of the neuron is just the sum of all the input that goes. But we have local synapses and they do something non-linear. For example, we have this thing called NMDA spikes where you have a couple of synapses, and if they’re all active at the same time, they’re much more strong than if just one of them comes.

And then we have if we go from these very local pieces on the dendrite, two more global pieces, we have things that’s called calcium spikes, where you can say the neuron is very active for some period of time, maybe a 10th of a second. And then ultimately we have this spiking in the neuron. So when people do that analysis where they ask how complex do neurons appear to be?

And it’s based on some simulations. So take it with a grain of salt. But I think the best way of thinking about it is that a given neuron is a little bit like a three layer, four layer neural network that just has one output. So in that sense, you can say that a real neuron is really complicated and might be doing much, much more work than one neuron in an artificial neural network would do.

Juan Benet

That’s a very good insight of Modeling. So we could model one neuron with like a four layer artificial neural network. I think is there some kind of estimate on the parameter count that an average neuron might have? Because that implies a lot of different parameters and a lot of computation that is happening in one single organic neuron?

Konrad Kording

Yeah. I think you need a very large number of to describe the computation in one neuron. Imagine you have 10,000 synapses. It’s not that you now have 10,000 parameters, which is one weight for each of them. It’s much worse. So there are tons to be that. There’s some connections that are very fast and something happens and then it’s gone.

And there’s other connections that are much, much slower. There are excitatory inhibitory ones. There are those where two inputs, if they come immediately after one another, it’s like the second one doesn’t count. We call that depressing. And then there’s the opposite of facilitating. If just one input comes, very little happens.

But if two of them happen. One right after the other. It’s a very strong signal. So all these things, we need parameters for it. So I think there’s going to be at least ten parameters for each synapse, that number of synapses on or that number of parameters. Now we’re at maybe 100,000 per cell, but there’s a lot of extra ones coming from the local non-linear properties of dendrites.

We have these things called ion channels where that can amplify signals. They can weaken signals, they can make signals nonlinear. There might be a million of them on the cell. So the number of parameters that we need to describe a cell is undoubtedly very, very large.

Juan Benet

Across different types of organic neurons. Do they follow at least a set of patterns where you roughly like if you were to kind of model one organic neuron with a particular structure of a, of an artificial network, is that viable, or are they just so uniquely tuned that it’s kind of like learning? For each individual neuron, you would have to learn a different network from scratch.

Konrad Kording

Yeah, I think every neuron is different and they have to be, you know, like. Because what do you what do neurons do? They embody what we know about the world. So if they were all identical, they couldn’t start any information. Like there must all be different. And because they are all different, they have lots of lots of parameters.

And therefore, if we want to model them well, we need to use lots of parameters.

Juan Benet

Well, we push them back on like the artificial neural networks are all mathematically identical. They just they store the differences in like different parameter weights. The network structure of the artificial neuron is identical while the parameter is different. And so here you’re saying inorganic neuron terms.

How different are they. Is it like the more variable like a parameter setting? Or is it just like the actual wiring and structure of them is just so fundamentally different? Wonder until the next. That is very difficult to like generate a model that can kind of abstract between them.

Konrad Kording

I think they’re all very different, and yet they’re all the same. And let me highlight what I mean with it. So you could say if I describe what a neuron does with its inputs, indeed they are non-linear and they’re all different from one another. But if you look, if you zoom into it and ask how it works, they all work in the same way, which is there are ion channels on it, there’s synapses on it, there’s synapses, and ion channels produce electrical currents on that neuron.

And then that neuron uses these electrical currents that get integrated by the physics of the cable equation integrates all of that. And after integrating it, it produces an output. And that output is sent to the axon. They are like slides. It’s biology. No, evolution does whatever works. So there’s slight variations of it where you can say, sometimes a neuron tell something to another neuron that doesn’t go through this axonal output.

And that’s those are the areas where we don’t quite know how big these effects are. But those effects do exist. So for example, if you have two neurons that are just like touching one another, even if there’s no axon from one to the other, the fact that there touch, if one of them is electrical active, the other one is electrically influenced by the first one, it’s called a ephaptic coupling.

And that means that two neurons that are right next to one another will influence one another beyond just the axons.

Juan Benet

Yeah, you mentioned the cable equation. So what are the computational models that we’ve built to represent neurons today? And what is sort of like the zoo of different models that we’ve tried and what has worked and what hasn’t worked?

Konrad Kording

Yeah. So there’s a continuum of abstract to more realistic models. Let’s start with the realistic model. The realistic model basically thinks about the neuron like that where you can say the neuron has this dendritic tree. If we zoom into the physics of that, it’s a little bit not like what is the dendritic tree.

It basically is a water and salt on the inside. And we know in physics how we should think about water and salt. It Current can flow through. It has a certain amount of resistance locally, and you can say a long branch of a dendrite may just be like a lot of resistors, one after the other. There’s also around that there’s cell membrane.

Now how should we think about the cell membrane. It’s a little bit like a capacitance to the outside of that. Now different cells have different lengths of that. And they branch out. And they might be thin or thick, or they might have a thick layer around it or thin layer on it, which from a physics perspective just means the different sets of resistors, different numbers of them, maybe a tree of resistors, maybe some of them have bigger capacitance than others, but they’re all fit into like that.

Same way of thinking that a cell has basically lots of resistors, which lots of capacitance and lots of sources of currents. Now let’s talk about sources of currents, like how does a current go into it? If we zoom all the way into it, we have an ion channel. There’s a mechanism that opens it often, and when that closes it, if it’s open, it means that ions will go through it.

It’s like attaching a little battery with a resistor to it. And that’s how we model it when we try to build very realistic models. And then you can say we have the synapses. It’s the same as an ion channel, only that it also matters what comes from the presynaptic neuron. So at some level we have a very coherent way of thinking about it.

Let me highlight the places where it gets more complicated, which is within the cells. We have a lot of molecules that influence that. So an ion channel is regulated by intracellular biochemical processes. That’s a place where complexity comes in. And that’s a place where we are not that certain about how that complexity works.

There’s aspects that we understand very well. For example, if you have a spike, the most famous model basically says we just have like one piece of axon. It’s like little resistors between with capacitance to the outside only that there is also this thing which lets in more current when the voltage is very high, and then once the voltage was high for a bit, it makes the voltage low again.

And this what gives rise to the spikes where you reach the place where it starts. Then the voltage goes high and then it goes down again. And if you put this across space, it like sends a spike through the axon and transmits it to other neurons. And we understand it at that level very well. Now when it comes to the non-linear regulation phenomena that happened there, like there’s just a voltage gated sodium channel that we need for that, that is still relatively simple.

In reality, there’s these things we call them second messengers, where where various cellular phenomena affect ion channels and complex ways. And learning is just the extreme way of that, where there’s a process that, if you want, starts a molecular cascade that ultimately Changes. How strong a synapse is, but also if new synapses get started.

Juan Benet

Going back to kind of like the different models. So we have some realistic description both in physics. And then you can build a computational model of that. But now that would be kind of way too granular. And with two details like, like let’s start kind of abstracting out, like what are more abstract kind of computational structures that can represent what the neuron is doing to like a great degree.

But maybe it’s like losing some information.

Konrad Kording

Yeah. So let’s go to the abstractions we use. And like a cable equation model is expensive to simulate because you as you end up simulating every little piece, every resistor, every capacitance. And on top of it, it’s nonlinear and with inputs that change over time. So it’s very difficult, very expensive computationally to simulate that.

There’s therefore there’s various abstractions of that. There are ways, for example, where you can say we take the cable equation, which tries to model like every little piece of that of the cell, and you can say, let’s, let’s just cause grain. That way you can say, maybe I take everything that originally was 100 resistors, and I replace it with one resistor that’s approximating what the system does.

Then I have just fewer resistors, fewer capacitance. It’s easier for me to simulate. We can go to the next step of abstraction where we can say, well, let’s assume that all the dendrites don’t overly matter. And that gives us the integrate and fire neuron. The idea there is we take every synapse we see on average.

How strongly does it influence the voltage at the soma at the cell body of the neuron? We then pretend that all this complicated dendrite, with its delays and time scales and nonlinearities and so forth. It doesn’t exist. It’s just all linearly added to it. Integrate and fire. It at least produces spikes.

It’s the basis of many works in spiking neural networks, which is a branch of artificial neural networks with the promise of making computation a lot more efficient. And then we can go farther with to what we call a rate model, where we can say, yeah, there’s time, and but time is fast. And if you want like over the period of a second, we might have a spike or multiple spikes.

And therefore it doesn’t matter all that much of what all those biophysics are. It’s enough to know as a function of how many spikes come in and how many spikes come out. And so you have this whole continuum of biological realism there.

Juan Benet

What do you guess is going to be the right computational model to then start building larger and larger scale representations and networks where you can recapitulate behavior well, meaning like, can you truly abstract it all the way to leaky integrated fire type of structure? Or do you need like something much more real?

Like what do you think right now? Or like, how do you think that this might develop over time?

Konrad Kording

I think there’s the information processing in the cell actually truly matters because there’s strong non-linearities. That and nonlinearities is the basis of artificial neural networks. So the idea that you can take one very simple thing that just adds things and then thresholds it and gives you the spikes in the way we do in artificial neural networks, I think is wrong.

But what we can do is we can now use machine learning to say, well, here you have this simulation of the neuron that’s really expensive. We want to simulate it fast, and it’s fine if it costs us some time to understand that. So we call that amortized inference where we run our simulations a lot and do a lot of computation.

And others with the idea that we fast put in a lot of compute, and then later it’s cheap to do that. You can say a neuron, a realistic neuron that gets lots of inputs, lots of and has outputs that change over time. We can still use machine learning to approximate it, where you can say, what? What does anyone do?

It gets like a couple thousand inputs as a function of time. It still just computes a function that. We can use machine learning to approximate that function, and in all likelihood, we can do that pretty efficiently. And in fact, there’s some work that already does that. If I tell you that complicated neuron can maybe approximated by a three layer neural network, well, you can simulate that three layer neural network massively faster than the cable equations for a neuron.

So there is this path by which we can take realistic neurons and in principle take those neuron models and move it into the space where we can really efficiently simulate it. And I think that’s essential if we want to build big things.

Juan Benet

What are the sort of functions that you think each of these neurons is calculating? Because at least in artificial neural networks, you get to use these very simple linear algebra style functions, and you build the complexity out of the layers of the in the stack. When you want to learn and approximate some more complex function, you’re just doing it across the layers where each individual neuron is not actually very complicated, but it sounds here.

The dynamics of a single organic neuron are so complicated that it could actually be learning. One single neuron could be learning a very complex function its own.

Konrad Kording

Yeah, it’s great that you mentioned that because we’ve done some fun research there where you can say we took a neuron that was somewhat realistic, and we asked, could a single neuron solve a machine learning task? And we found that a single neuron can solve MNIST. So for people who don’t know MNIST, MNIST is basically you put in the image of a number and it tells you what number that is.

And it uses real US Postal Service data. So. So everyone writes numbers differently. I’m from Germany. Are you my the way I do the number one is different to the way you do it. And yet it recognizes all of them. And so what we did in that study, we took a neuron with all of free parameters that they have.

And it was still a somewhat approximated, somewhat simplified neuron. And then we asked, can we predict, can we train that neuron with all the free parameters so that it would recognize if something is, say, a number. Seven was this a number? One? A single neuron can do that very well, actually. Therefore, you can say that it just gives you an example that the power is higher.

But at the same time, if we take a model of that neuron and we make it very efficient, it’s still structured where you can say there is a place where there are two parts of the neuron, there are two dendritic branches meet. That still means that not all possible three layer neural networks could be implemented.

It needs to be one where information meets and then it meets other information meets other information ultimately produce the output of the neuron. And so in that sense, it might rather be that these realistic neurons make it easier to compute these complex functions, because they have a lot of structure and sparseness and structure just promises to make a lot of computations better.

Juan Benet

When you mention the 3 to 4 layers, I immediately thought of MNIST. So that’s awesome that, that actually that actually has been done. Have we pushed into the, the limits of that, like what is like the most complex things that we have gotten single neurons to try to do? And where do they sort of break down?

What kind of functions are they just not able to then figure out?

Konrad Kording

I think in principle, a single neuron could probably compute a lot of the functions that a three layer neural network can. So what we did in that area and there’s not that there’s there’s emerging literature like pushing it further. And I think you can push it a lot further. But MNIST is famous for being a somewhat old fashioned machine learning problem, because it’s just basically binary drawings of numbers.

And there are more interesting models that people use today. CIFAR-10, for example, is one of them where you have, where you have just ten different object classes. There’s fashion, MNIST, this, this. They’re still simple. They’re not don’t think of it as an LLM or something, but they’re pretty complicated tasks.

A single neuron could solve those problems pretty well as well, I think. No. Like, there’s no doubt that it won’t be able to do an LLM. And you can you can say there’s something about the number of parameters. A neuron has 10,000 parameters. We shouldn’t expect a neuron to be able to do more than A and with about 10,000 parameters.

So you can say 10,000 parameters is kind of like the machine learning year 1995, say state. But machine learning 1995 does an awful lot of like non-trivial things. So I think we are not giving neurons enough credit for all the things that they could be doing.

Juan Benet

Super powerful in the three layer that you’re describing or 3 to 4 ish. Does that come from the dendritic structure or does that come from the parameter space of the individual synapses, or.

Konrad Kording

It comes from the structure of the dendrites. You know, if you look at it, dendrite, it’s this tree like structures. Now it really looks like a tree. Look at the harmonica drawing you kind of like. It almost looks like. In fact, we had a fun little paper where we quantified how similar trees are to neurons, and we found that they’re remarkably similar.

And so. So it’s a bit of a tree and you can say in that tree, wherever two branches meet, you have the chance to combine two signals. So I think that’s where the three layers comes from.

Juan Benet

And is it as straightforward as just, you know, if you look at the if you just take a snapshot of the neuron and you trace the tree structure, you can kind of represent that with like a logical neural network following that same tree structure. Or does it get more much more complicated than that? That’s how we did it.

Konrad Kording

There’s nice research from Eden Segev, who kind of looks more holistically at that. But I think first order approximation is this. Take a neuron that has tree structure and model it by an artificial neural network that has tree structure is a pretty good approximation.

Juan Benet

So this has kind of covers, you know, single neurons to some degree. When we start kind of putting them together into larger and larger units, into small circuits, and then later, like larger and larger networks, how do those work? And what are the kind of communication pathways that we that we see? Right.

So in artificial neural networks, we have a very we have constrained ourselves to build, like most of the deep learning structures just have these single directional neural networks where things are just kind of flowing in one pass, just because that makes the learning algorithms much easier. But organic neurons are not like that.

There’s all kinds of like recurrence and channels coming back and can maybe speak to that a bit. And what is the complexity when we try to model these like circuits?

Konrad Kording

Yeah, I mean, you mentioned the first complexity already, which is in say, a transformer architecture. We have strict feedforward transmission of information. There’s one layer, it goes to, the next layer goes to the next layer, and eventually you’re finished in the brain. It looks very different if in the brain, let’s say we’re both neurons.

I talk to you and you’ll very likely talk back to me, or if not directly, kind of your team talks with my team. That’s roughly the logic in brains. So you have this massive recurrence where it goes one direction and it comes back. I think it’s not that disjunct from modern machine learning, where a lot of people in machine learning use neural networks, and I think they’re having a bit of a recurrence these days, resurgence where we want to allow systems to feed back information.

I think that’s crucial. It’s incredibly useful. But let’s first talk about what we know biologically. The brain has hundreds of different brain areas, many, many structures. Neurons look different in all of them. In general, know this pattern that if it goes from one place to another place, it usually comes back from the other place to the first place is a general pattern that by and large holds.

Juan Benet

And this sort of a different channel like information flowing down. Or is it use the exact same channel back?

Konrad Kording

Yeah. No, it projects slightly differently. So if you look at cortex, which is the area that most people study, there’s parts of cortex that send the signal up and then there’s the neurons that are up, send the signals and they come back in a different place. So they come to the superficial layers and they’re being sent forward from the deeper layers in cortex.

So there is structure there. So if you want it’s useful from a learning perspective and from a processing perspective, that kind of information is separated. And you know what comes from the bottom and you know what comes from the top. And I think that is very important.

Juan Benet

Yeah. And it’s very useful algorithmically. Like that’s the basis for machine learning. Learning rules is like separation of information is what lets you do like the feedforward and backprop learning and all that kind of stuff. Right. And so some of the tension between the two fields is that in artificial neural networks we get to do this global computation, whereas in organic neurons we don’t.

Or we have to figure out how to represent the global computation in a in a local way.

Konrad Kording

Yeah. So let’s break down these two things. So that’s the first thing which is processing. If you make a decision, if you want to say a word say information, your brain will go forward and backwards and forwards three layers and then backwards two. It kind of goes back and forth all the time until the decision is made by your brain to like, say, given what if you look at an artificial neural network, it only goes forward.

Now in the artificial neural network, information also goes backwards, but it’s only information for learning in the artificial neural network at the end, you know, was this a good thing or a bad thing? And then you kind of go back and say like, hey, I did this bad thing and you were responsible. And then the neural net was responsible.

So I was like, okay, but those guys were responsible for me being active. And then those guys kind of like, that is how the so-called credit assignment problem is solved in AI in the brain. It’s much less clear how it works. So like it’s it’s clear. You need to know. If you want to get better at learning, you somehow need to know, well, should I, as a neuron have been more active or should I have been less active?

Otherwise, how do you know if you should do it differently next time? So this credit assignment problem kind of intuitively clearly needs to be solved. I need to know how I should change an AI. We know how we do that. We have a very efficient algorithm called backpropagation that tells every neuron if it should be more or less active, and every neuron tells it to the to the weights between them, the synapses if you want in brains, we don’t know.

It’s clear that somehow, if you want to get better, it means that the neurons where it helps. If they’re more active, they must become more active, and the neurons where it helps that they are less active must become less active. We know that it can. It’s a simple mathematical proof that on average that needs to be happening.

But how that works is very unclear. And like everyone, every neuron constantly talks with lots of other neurons, and somehow out of this dynamics must emerge, something which tells the right neurons to become more active and the wrong neurons to become less active.

Juan Benet

In neural networks, we get to apply this whole learning structure on top of the entire network the feedforward and backprop and backpropagation model with like the whole, you know, whole network gradient descent works in great part because we know the bounding box of this entire network and we know what is the feedforward pass, what is a backprop pass.

And, you know, there’s a whole string of papers that try to inspect both the artificial neural networks and neural networks to try and figure out is the brain doing something similar to that or a totally different type of learning? Because recurrent neural networks in the artificial models are very hard and notoriously very hard to train because of this bounding box problem and the computation getting very messy, The brain has to be doing something locally, right?

Like not whole. Brain synchronized. The learning and the algorithms have to be at least if not in a single neuron, in some local patch of some sort. Or maybe this like a question. What? What do we know?

Konrad Kording

Now let’s talk about physics. Yeah. Imagine you’re like a sign up somewhere in the brain. What information can you possibly have? Now, the only thing that you can know locally is what did the upstream neuron do? What did the downstream neuron do? And it turns out there’s this thing called back propagating action potential, where basically if the cell that I’m on as a synapse is active, I’m going to hear it.

It’s like everyone knows that just happened because that’s where we are locally. And then we can like maybe see locally a few so-called neurotransmitters. So there’s neurotransmitters that kind of seem to be related to like this was good or this was unexpectedly bad. So there is the possibility that I have some information there, but mostly I just see what happened of the Pignon?

What happened to me.

Juan Benet

In those neurotransmitters, eh? What are those? And be. Do they happen locally or in larger areas or globally?

Konrad Kording

Yeah. So there’s a lot unknown about it, but there’s suddenly dopamine seems to tell you about expected reward. And then there’s a huge literature that shows that this approximation is a really bad approximation. But like, there’s there’s some experiments, beautiful experiments by Schultz that basically say, you give me a reward that I don’t expect, someone comes in, gives me some coffee, dopamine will be like, yep, great.

And then alternatively, I expect coffee. Someone comes. I’m like, sorry, no coffee right now. And the dopamine minions will go like, ooh,

That’s sad. And so there is this way of representing those things, but reality is much more complicated. And that’s because I think like signals, not like an AI system. We just we have a so-called reward function or loss function, depending on which field you’re in. You’re just like, here’s better, here’s worse.

It’s very simple. But for us as humans, it’s not like better or worse. Is that simple? Now, like, what is better versus worse on a word that you just produced? There’s a word that might be like, you look at my face. I frown when you say something. So like, that’s one source. But also you might be like, well, that didn’t quite sound right.

And you might be like, this a strategically problematic word in this context. And the listeners on my podcast will not like this one. So there’s all these things layered on top of one another in AI is clear. There’s just like one function that kind of says how good or bad there was for us. There’s a lot of things layered.

For example, we know there’s other neuro modulators, say serotonin or something, let’s say more related to like, hey, that mattered. You should probably pay attention and not forget what just has happened. So there are these extra dimensions on it. And I think in AI we are building them into systems.

Now where we are weighing where like this an important stimulus and this not a gradient descent. There’s a little bit of that, but we’re trying to like do that even better. So there’s these layers of complexity that I think are much better for biological systems than they are for AI.

Juan Benet

Yeah. Going back to the kind of building larger and larger networks. And so there’s the algorithms that we might need to run, like learning, maybe digging into learning. Most artificial networks effectively have one learning paradigm. And maybe you can look at the current transformer architectures as maybe patching some additional learning processes on top through RL and all these other things.

But for the most part, like the pre-training process is just one main single algorithm. What do we know about the brain and how does learning in the brain happen? Is there a bunch of different algorithms? Are these maybe localized separately? Are the neurotransmitters triggering different algorithms, or is just tuning a single algorithm in a way.

Like, what do we actually know?

Konrad Kording

Let’s maybe start with the theory. So there’s always a theory space in neuroscience, a theorizing space, a sensemaking space, and a data space. Let’s first talk about the theory space, because a I know a lot about that space. And B that’s a field one is a starting point that people have. So there exists a space where people say, well, maybe the brain is relatively simple and know maybe the brain is evolutionarily already set up to do almost everything, and then you just need to add like a little layer of learning to it.

Tony Zador is one of the people proposing that where you could say, if most of it is pretty wired, then maybe we just need a little of a little bit of extra learning. That’s this one set of theories. Let’s say it’s relatively simple. Why is this great? Credit assignment is not much of a problem if the whole brain is just like its evolution gives us the things we just need to save.

A goes with B like no, like that’s the what? Blue. Go with color blue. It’s a simple learning space and maybe like a simple learning algorithm can work. Now most people that I talk with don’t believe that it can be this simple. And why can’t it be that simple? Because language is really complicated and we live in a really complicated world.

And if simple learning could succeed in this world, probably the world would be very different. Like, I believe the world is truly, unbelievably complicated. And I believe that because the outside world is so complicated, our brain is complicated. So if you then go towards more complicated algorithms, a lot of the algorithms that we believe could actually work in a complicated world require credit assignment.

Now, like I if you make a mistake, there’s like an unlimited number of ways how something could be a mistake like. And because of that, you need to find out what’s the reason for that mistake. And that goes into both directions. That goes into the inside of the brain, like which neurons screwed this up and it goes into the outside world, which is kind of like, how is the world such that this wasn’t a good idea?

And so if we have these more complicated algorithms, we need to solve the credit assignment problem. Like what was the nature of the mistake that we just did? And if we need to solve it, we need to we need a way of figuring out what went wrong. And there is this in gradient descent is how we do it in AI. And there exists a large set of proposals how the brain could do it.

Now let’s see what they have in common, like they have all in common that they build of things that is purely local, like the synapse only knows what happens locally. The neuron only knows what happens locally. Everything that it knows about the wider world. It must have been told that by its neighbors, not like because it only talks with its neighbors.

There is no global like an extra like an extra human being. Like, tell us what it is. Or at least to our knowledge at the moment. I mean, like, sure. Like maybe happens to divine intervention into all onions. But as long as we believe in like mostly physical world, the problem like kind of that information must somehow become locally.

So we have a lot of algorithms that explain how purely local things, neurons talking with their neighbors can ultimately still figure out how we should do credit assignment. And let’s talk about the simplest version of that. The simplest version is a way how the biological brain could actually approximate gradient descent.

Really, really well. Imagine it goes to the brain. Let’s ignore the fact that it always goes back and forth, or fourth, it goes to the brain. In the end, you do something, then you find out if this was a good idea or not, and then if it was a bad idea, you have anyone that says you did this and that was a bad idea.

And then you have for every neuron in the brain, kind of like a twin, which kind of says, I think you did something wrong. And then the set of like neurons with their twins. Think about it like, what do you know? Like in Star Wars now you have like the Sith, where there’s always the master and the apprentice and kind of like you have a master for every union like twin that basically says this was good or this was bad.

And then they tell the other neurons, well, the neuron that I’m the master of like, did it wrong and you guys were part of that. And then they tell that to like their training twin of them. And it kind of goes all the way that way. That way you produce something that’s almost exactly like gradient descent. It just requires twice as many neurons, and it requires the twin onions, and it requires the student neurons.

Juan Benet

And so this would imply there’s a, you know, a set of circuits of running the actual computation and a set of then parallel circuits going back, propagating the learning information.

Konrad Kording

That’s right. And of course, if you want biologically that’s totally realistic, but it’s totally not what we are seeing. Like if we stick an electrode somewhere in the brain, let’s say in the visual system, we have neurons that are very active. If we show them a little black thing, like, like a black line on white background or something, we don’t see kind of neurons.

Well, you should have been more active for that black line. Now, parts of that reason could be that we wouldn’t see them now like, because most of the time your visual system probably gets it right by the time you’re an adult. So it would be hard to see them even if it was like that. But the other one is, you can say it should predict that this kind of this subnetwork that kind of runs something else than the first network.

We’re not really seeing something like that. You could say, in terms of connectome, six ways of quantifying how networks like really make their connections. We don’t see kind of like these like these parallel networks. That’s one theory. Not like there’s an extreme theory that very few people would hold because the data doesn’t go along with it.

But there’s versions of that where you can say, well, I don’t actually need a twin. It could, for example, be the same neuron at a different time where you could say that it’s a part where the neurons tell you what to do, and then there’s another time where they’re like, okay, let’s like kind of fix ourselves so that we wouldn’t have done that.

And there’s a lot of algorithms that have been proposed as theories of how brains learn that basically do that, where you use the dynamic switch if you want, like the neuron has a time where it makes the decision and the time where it learns in the first one, it’s like if you want feature like it, it represents something that’s in the outside world and at the second time, it’s more like it tells you what was good and what was bad.

Walter Senn has a version of that algorithm. Blake Richards was involved. Yoshua Bengio has a version of that algorithm. It’s even me doing my PhD had a version of that algorithm. But these algorithms all have in common is there’s no blatant biology violation, and there’s no evidence that the brain does it like that.

So we don’t know from the theory side. So what we do know is that the brain could be doing it realistically. Now there’s this thing. If I think about evolution, we know that algorithms that do something like backpropagation, like gradient descent, are really great and work on real world AI prompts, and that algorithms that don’t do anything like that aren’t very good.

So in that sense, I believe that evolution is amazing and we have a very long history. And you should have expected that kind of real brains that do the right thing, that do good. Algorithms that if it’s if it’s realistic, if it’s easy, as in those algorithms, they all like really pretty simple algorithms that people propose.

If it’s easy, then biology should have figured it out. So that’s why people in that field realistic gradient descent or however you want to call that field. I think we all agree the brain should be doing something like that. Therefore, because of evolutionary thinking, we should strongly believe that the brain is doing something like that.

But the data, the evidence for that is really weak. Now let’s talk about what the evidence is that we have. The first evidence is that learning in biological beings is pretty amazing. You make a mistake, you grab like a coffee cup, you make a mistake, you’ll be like this next time you don’t make that mistake.

Imagine the cup is heavier than you thought. And I’ve done some experiments, I. You know, I’ve done some movement science in my past. You do some mistakes with your coffee cup. Next time you’re, like, 30% better, like half a second later. And in general, if you look throughout the biological kingdom at learning prom, like we always get better at things.

So. So therefore we kind of know that learning must be good, because if it wasn’t good, you’d be. It’s not. It doesn’t look like you try some stuff and like, oh, this time I was even worse at picking up my coffee cup.

Juan Benet

There’s very few examples, you know, compared to like, the millions that we use for artificial neural networks today.

Konrad Kording

That’s right. You get better at pretty much everything. Now, that is not true for a lot of neural networks. If you’re not very careful with it, if you’re like, make them learn very fast, you’re even worse. And that’s despite the fact that we built in a lot of like these tricks. So it seems that like the algorithm used by the brain is very good.

We know that it’s biologically realistic and the relevant experiments haven’t been made yet. So let me talk you through what we know biologically. So we know that there are Hep like phenomena. You have a local synapse. If the presynaptic and the postsynaptic neurons are active at the same time, the sentence might get stronger or weaker.

Juan Benet

Neurons that the wire together and fire together will fire together more.

Konrad Kording

Yeah. Yes, exactly. Well, why are together more so? It’s a little bit like gradient descent. So imagine the neuron that you get input from isn’t active at all. Why would you change the weight? Because the weight from the presynaptic neurons would not make any difference at all. Imagine the neuron after the synapse isn’t active at all.

Why would you want to change the weight? Well, the neuron is not active. There’s no change that will make it active. So you can say that if the brain wants to do something like gradient descent locally, things should kind of look a lot like Hebbian. You know, like no activity pre. No learning. No activity post.

No learning because gradients at zero. If nothing is active, we can go go further and say if the presynaptic neuron is active before the postsynaptic neuron, we have a lot more plasticity. Well, if your presynaptic neuron was active afterwards, it cannot change you because well, it’s not active before you do things.

Therefore there’s no influence and therefore the gradient is zero and therefore you shouldn’t be learning there. So if you look at these local plasticity findings, they kind of look like a little bit like gradient descent. Now does the brain do a gradient descent. We don’t know because the experiments for that have not been done because the people who do synaptic plasticity, they kind of look microscopically what’s happening in the cell, they usually wouldn’t have a even know.

They often use slices. It’s all very local. They wouldn’t know what the gradients would be, so they can’t quite test the big idea there. And then there’s a cultural thing, which is the people who work on gradient descent and brains, they tend to be adjacent. They tend to not sit in active biology lab. So the theories that they come up with usually don’t get tested.

So we are at this moment where if you want, we believe that brains do things that are pretty much like AI, and we have microscopic data that’s kind of compatible with that. But there is no, like, clean, convincing paper that the brain actually does it. And that’s despite the fact that it’s pretty obvious how to do that experiment.

It’s just that the that the fossils never align to make that question be answered.

Juan Benet

Let’s talk about the limitations experimentally. I mean, we have the ability to grow neurons, you know, separately from brains, we have some amount of ability to look into specific brains and record small areas of the brain. We haven’t figured out how to record large scale full brains, but maybe we can record like C elegans and smaller organisms, maybe talk us through, like, what can we figure out experimental today?

Like what is like the current state of the art and like our ability to record and understand generate the data that we need to then figure out how to test theories, and then separately, like where do you see this going? Like how is the measurement and recording landscape evolving?

Konrad Kording

Yeah. Let me first briefly sketch the experiment. How would we find out if the brain does gradient descent. And then I give you like a broader overview of like what are the tools in our toolkit. So what is gradient descent? It’s a very simple idea. It says if it would make your decisions less bad, if the neuron was more active, if it would make your decisions better, then the neuron should become more active.

And if it would make your decisions better. If the neuron was less active in that given situation, it should become less active in that given situation. It’s a very intuitive rule here. Now you can combine two experiments that a lot of people in this world do that would directly ask that question. The first one is if you want to know how, if a neuron makes you better or worse at a task, you can absolutely do that.

You can go in and you stimulate the neuron in a given situation that changes the behavior. Therefore, it makes you better or worse at the task. And a lot of people have run these experiments by brain region in an animal show how it changes behavior. And of course the behavior changes how good the animals are.

That. So that’s one piece that if you want mathematically is forward differentiation. You go into a function, you change something and you see does the output go up or does the output go down. It’s it’s and you just what you want to do is how much does it go up and divide it by how many extra spikes you did, which is then the derivative of the output?

The reward if you want after the activity of the neuron, that’s the first thing. That’s what we call a gradient. A gradient just means how much better it gets. If you make it more active, then what you need to know is how that change now happens. You can do a different experiment that a lot of people do. Where you say, I take the brain and record from it.

Then I give a learning task to an animal and I ask, do the neurons change? Lots of people have done that. If you don’t want to test if the brain does gradient descent, you just need to correlate those two. It means that those neurons were making them more active makes you better. The task should be the neurons that become more active.

If you actually do give them that task. Like it’s a it’s a it’s a direct test. Now we can do that because people are doing both of these experiments. It’s just no one does both of these experiments at the same time. There’s nothing holding back people from doing that. Apart from the fact that a lot of people have, if you want this idea of like take AI as a way of thinking about brains isn’t yet an established way of thinking in neuroscience.

So I think that’s why those six pumps haven’t been done.

Juan Benet

Interesting.

Konrad Kording

And now let’s talk a little bit about like our toolkit at neuroscience. So one of the things that make me so excited about neuroscience is like this dramatic explosion of toolkits. So in the past, we could record from a small number of neurons. Recording from 100 neurons was heroic when I was a PhD student.

Now you can buy one device, stick it into a brain, it records from a thousand of them, and of course, you can stick a whole bunch of them into it. So we can now record from lots of neurons. My lab discovered this thing called Stevenson’s Law, which is it doubles every six years the number of neurons that you record simultaneously.

But there’s now these modern techniques coming that promise millions potentially more. At the same time, we can do much larger numbers by going optically into brains than going going electrically. So there’s this thing we can get far more data out of brains when it comes to the electrophysiology of it.

We now have molecular tools to do these things. So we know there’s many different cell types, and we can make it that we can only record from the neurons that are like type 17 into neuron with the following molecular properties. It’s like the rest of the brain becomes invisible. And you see only those nodes that are of the molecular type that you currently want to study.

Unbelievably cool optogenetics. Deisseroth and Boyden popularized that. It was a protein they found in some algae. It basically is this molecule. You shine light on it. It lets in current letting in can make the neurons more active or less active, depending on what kind of current you like it So with optogenetics you can basically go in.

You illuminate a neuron and it becomes active. Now you can illuminate one neuron or group of neurons. You can illuminate them again. You can like make them go up. You can make them go down. It’s kind of like you can play it like a piano. Unbelievably cool. Even like these latch and release techniques, you shine in one color of light, and every cell memorizes what it was doing at that point of time.

Like, I was very active, I wasn’t active. And then you shine another light on it. And the neurons that were active basically like replay being active, they all get activated. The ones that were inactive don’t replay getting active. So you can kind of like recreate a brain state if you want, like unbelievably cool, not like.

And then come all these anatomical techniques. We were always able to use em to reconstruct a neuron. And I get my PhD in a lab where Kevin Martin was doing these experiments. And it’s unbelievable. Not like they were basically drawing the neurons by hand. And it took forever to do that. And it was really unbelievably complicated.

And now we have these automated machines. They have like a hundred beans at the same time. And we have AI algorithms that like draw the neurons and they don’t do draw one, you know, and they draw like a million neurons. And it’s just that speed of development is unbelievable. We’ve done some analysis. Imaging is getting cheaper by a factor of two like every 18 months or something.

It’s unbelievable. It’s almost imaging is getting cheaper almost as fast as AI is getting cheaper. So we have like all these techniques that are coming online and by and large we are asking the same questions we asked 70 years ago. We just like do it with monuments and with more beautiful neurons and now we can see them.

But how to really like gain deeper insights with it? I think we’re still working on it.

Juan Benet

Part of the answer there might just be a scale thing, right? Like a, you know, in artificial neural networks. You just have to wait until the networks were big enough to be able to compute some very complicated things. So maybe now that we have the ability to look at millions of neurons, then we now have the scale of data required to be able to kind of build much more robust models, potentially.

Yeah, that’s the idea.

Konrad Kording

But let me push on where we’re like failing at even seemingly simple things. There is this worm that has the arguably the simplest nervous system that exists. It’s called C elegans. It has a huge behavioral repertoire. It likes forages for food, it finds partners, it lays eggs. It does everything you and me do, minus the talking, kind of like almost.

But it kind of has a really rich repertoire of things that it does. It only uses 300 neurons to do that, and we can’t simulate it now like that’s 300 neurons. We can see them all at the same time. Why can’t we simulate it? Well, why can’t we simulate it is because and we know all the wires in it. We know we have a name for each of the 300 neurons that it has.

We know a lot about the molecular properties of that. We know every single wire that exists there, and we don’t know it just for one one. We know it for a handful of them. We kind of like, in principle, have all that information. And yet our simulations are barely worth the papers that we print them on. My own lab tried to simulate C elegans based on that information.

We’re horrible at it. And so basically we have these like explosion of techniques. And yet we are not able to simulate relatively simple systems.

Juan Benet

So why not? Yeah, there’s been a range of projects trying to both record and establish the connectivity between the neuron, like the entire connectome of the of C elegans and various different attempts to model it. So. So what’s going wrong there? Like why don’t they work yet?

Konrad Kording

So the first thing is a lot of data that we have is not perturbative. So and let me highlight why is this problem for exponential explanation. Imagine a record from all three neurons at the same time. You’ll be like shouldn’t you have all the information you have like all the neurons? Isn’t that cool?

We don’t quite usually. No one like is quite recording for all of them, but we’re getting very, very close to it. The big problem is they are all correlated with one another. Now imagine you had two neurons that would always do the same thing. You wouldn’t know is it neuron A or is it neuron B that does it. The problem is there’s not two neurons that always do the same thing.

But if one there’s some dimensions that carry a lot of the variance. Very large principal components, singular values depending on the language you’d like to use. And we know an awful lot about those. But if we want to get at the function, we’re effectively solving an inverse problem. Like I give you the activity of the 300 neurons as a function of time.

And I ask you, can you tell me how much each of them influences everyone else. If we do even linear regression, which is the simple step. We need to calculate this covariance matrix of the neurons with everyone else and invert it. Now that inversion requires basically if you have small singular value, then it becomes a so-called ill conditioned prompt.

So you can’t solve that. Therefore, you cannot know because there’s dimensions in which you basically have no variance. You cannot know what’s happening there, and therefore you cannot find out based on neural recording how they interact with one another.

Juan Benet

But these are the type of problem that current machine learning techniques are very good at modeling. Like if you had enough data and enough recording of worms through a whole bunch of different behaviors, and you have activity traces of all the neurons across long enough time scale, shouldn’t you be able to fit an artificial neural network to that?

Even like relaxing a bunch of the constraints and having a pretty large artificial neural network against it?

Konrad Kording

Yes, you can.

Juan Benet

But.

Konrad Kording

Here’s the problem. So there’s a forward modeling problem that machine learning solves, which is give me a prediction of what these 300 neurons will do is sometime into the future. And it’s amazing at that. We’re really good at making those predictions. And then there’s the inverse problem, which is what happens in the neural network to make those changes happen.

The forward prompts are very easy. And the more low dimensional the worm is, the easier, because like it’s basically just changing along three dimensions, like making up some numbers here and because it’s low dimensional machine learning works great. The lower dimensional problem, the easier machine learning.

We have a really good understanding of that, but there exists an infinite number of nervous systems that would produce the same changes of 300 neurons. And in general, when we use machine learning, we are great making predictions. We are horrible at understanding how the world works and we don’t need to.

We just need to make predictions. And how does machine learning do it? It basically says, well, there’s a lot of dimensions. Let’s like distribute what we do along all those dimensions we don’t know. But like as long as we’re similar enough in this situation it’s okay. We’ll just like put a little bit on all dimensions and we make really good predictions with it.

It turns out that for making predictions, if the world is low dimensional, it makes the problem easier. And if you want to understand how the world works based on your data, you care about the inverse. And therefore, if it’s low dimensional, it’s impossible to understand the world. And machine learning runs exactly into this palm.

And we know in machine learning, let’s say if we build predictive models, a lot of people are new to machine learning. In fact, almost everyone who’s new to machine learning makes that same mistake where you’re like, oh, I fit a machine learning model. It’s good. And therefore the machine learning model would tell me what would happen if I would reach in and change something in the world.

No machine learning is horrible at that. Machine learning generally doesn’t work well if you go far out of domain and reaching into the world and changing something is bad. And let me give you an example of like how that comes about. Imagine you build predictive model of like death of people and you’re like, okay, let’s take like things that we have in the electronic health records.

Maybe people who work out die less often. People take vitamin D, die less often in a given period of time. Now, why could that be? Well, not like a machine learning system. What do what it should be doing, which is vitamin D, predicts that you’ll kind of like live longer. It might also just be that people are rich.

Take vitamin D because they like fall for like snake oil science and therefore it’s an illusion. It’s just like vitamin D is a great predictor of your living longer. It’s just like people are like rich. Like they also work out more and they go and like see a doctor earlier. And that’s kind of all kinds of causal chains.

So a machine learning system, it’s like full of these like predict things from the wrong things. If the measured world is low dimensional, which it always is. And every machine learning problem, in every learning problem for brain centers we cannot solve the inverse problem. There’s like a million things that could all have produced the same input output behavior.

And so the fact that we’re good at making predictions is confused universally by neuroscientists and machine learning people alike, as evidence that we understand how this system works. No we don’t. We’re just good at making predictions.

Juan Benet

So just pushing on it a little bit to get a crisp intuition. Can you create a structure where you train models that can be good at prediction, and then you either distill them or you figure out, like the simplest possible models that can will still exhibit the right behavior. Shouldn’t that give you a sense of what actual computation that system is running to be able to run a better and better abstraction of it, that should match events outside of the distribution that you’ve measured.

So this would be the equivalent of saying you build some large artificial neural network model of C elegans. You kind of potentially even constrain the training of the network to maybe mirror dendritic trees of the actual neurons. So then figuring out like segments of this network have to be learning closely, close enough to what those individual neurons may be doing.

And if you do this over enough data distribution, you should be able to then distill out some representation of what functions those neurons are actually doing, or what’s the or is there no hope? How do we get. Like clearly there’s got to be some answer. We have to be able to get the data that we have and the computational model of the system into something that actually represents from an information theory perspective, like what the system is actually doing.

How do we clamp this down? How do we get to the answer?

Konrad Kording

What I love so much is that you kind of sketch my own thought patterns over much of my career, and I started exactly at this corner. Like, let’s just like use like simplicity. Let’s use prior knowledge. To kind of constrain things and maybe it will converge there. Let me kind of try and squash that idea. So if what we have is low dimensional information because we don’t see all those dimensions, then you can say it’s like we measure a small number of parameters.

If you measure a small number of parameters, but like you measure 100 of them and you live in a system that has a billion free parameters, there is basically a billion -100 set of possible models that can produce exactly those hundred measured dimensions. And now you can say you bring it down from this space to maybe which neurons connect to each other, neurons.

And then you go from a very, very large space to still a very, very large space. So you’d need to be able to bring it into that lower dimensional space. Now, like let’s take the key elegant space like we have 300 neurons. So in principle the covariance matrix is 300 by 390,000 free parameters. Now you could say if I use.

Juan Benet

Although from what you were saying before, if we were trying to model these neurons with artificial neural networks, shouldn’t, shouldn’t be more like 10,000 parameters per neuron.

Konrad Kording

It could be worse.

Let’s not let’s let’s not. Yeah, let’s let’s let’s let’s let’s let’s Steelman that approach. You can say there’s a large number of parameters. Now let’s use the full connectome now like C elegans neurons 10,000 ish synapses. That means like we only have 10,000 synapses. But now imagine that this the 300 neurons really slosh around in a five dimensional space.

You just measured 25 parameters in a space where even if it was all just linear weights, would have 10,000 numbers. So you measure 25 out of 10,000, which means that there’s like 9975 different like dimensions orthogonal to that. And you can basically pack any possible model to it. And this before we even touch like this sort railways like.

Yeah. Now they’re also all non-linear. And they also might have like multiple violations of our modeling assumptions. And so in noise in the in the data that we have now there’s better approaches probably like Andrew Leifer has been doing these really cool approaches where he goes in and stimulates neurons.

The problem is they still need neurons. If you could stimulate them independently, maybe you could get the data out. But the problem is it’s like kind of like hard to do enough experiments of that. The good thing is, though, they separate it now. It’s no longer sloshing around in like a five dimensional system.

It’s kind of like you at least go in for every one of the three other neurons, and you push that button. And, so far, at least in our hands, we can’t take data like that to produce like, really good models either. That’s probably related to there’s still a lot of noise on it. It’s still not because now we are talking about very, very small number.

We have a lot of neurons, but we have a small number of experiments.

Juan Benet

What are you thinking now? Like where do we go from here? What are we missing? Is it. Yeah. Increasing the ability to perturb the system. To extract, like, higher quality data and signal out of that? Or is it like different modeling approaches or like what?

Konrad Kording

I mean, look like? I tried it for 20 years of my career, trying to solve 25 years to try and solve this inverse problem, basically. How can we look at the output of neurons to find out how they compute? I now understand the inverse problem we’re solving much better. I’m reasonably convinced that in that space, there’s no credible solution for maybe C. elegans, because we could at least stimulate it independently, but probably not for larger systems.

Therefore, I think we need a different approach and a different approach. From my perspective goes through a connector mix, molecularly annotated tissue perspective and the logic that is very different. Instead of saying I observe the output of the nervous system, and I try to figure out what the inputs must have been or what the interactions must have been, you can rather go and let’s say, can I see signs of the interactions?

Can I see an excitatory sign ups between two neurons? Then I don’t need to kind of solve an inverse problem like I see an excitatory synapse. What if we could say look at it and say, well, there’s a big excitatory synapse. And based on all the molecules that I see there, it’s also pretty strong and fast. So I feel that we need a different approach.

And I feel that connectome or like the broader field of like imaging can give us that, but it forces us to completely rethink the logic of neuroscience.

Juan Benet

Concretely, it means instead of just trying to look at a smaller sample of the neurons that are given moment in time and recording those, you’re able to look at all of the individual weights in between each of the synapses and trying to establish what those actually are.

Konrad Kording

That’s exactly right. Now, like if we if we if we stress the AI analogy, the deep learning analogy, if you want like instead of figuring out the weights based on the outputs, which is hard and probably impossible, let’s rather see the weights. And the idea is that the weights will somehow be represented.

And there exists some data where people, for example, found that if I know what these two cell types are, pre cell type and the post cell type, I will often be able to at least. And I and I also tell you how big the synapses I’ll be able to note, like a fair bit of how strong that is. And people are now pushing that now, like, what if I tell you the transcripts of the basically which molecules are being produced in pre and post cell?

It’s a big step forward. I think that there’s a whole new field emerging.

Juan Benet

So this goes into like the broader economics, another omics approach to figuring out starting with C elegans and then Drosophila and then larger and larger mouse and then eventually primate and human. It would seem to me that getting a, a computationally valid system, where that demonstrates that we can recapitulate the actual organism and the full behavior of the organism, like we’re still pretty far away from that, in that we currently are doing the recording, and we might be able to establish the wiring diagrams of these, and maybe it’s molecularly annotated.

So you don’t know, just the two neurons are connected. You also know in the ways in which they’re connected or you have some indication of the parameter strength. Some parameter space will be able to model the thing. But how far we suppose that we do that. So whether we get like a good molecularly annotated connectome of either C. elegans or what’s missing from that, to then put it into a simulator that actually recapitulates the entire organism.

Konrad Kording

So I think what’s currently missing is what I call compilers. So at the moment we have these images and we have the connections, we have a list of wires and we have methods that efficiently get them to us. But what we don’t have is what’s between those wires. Not like between those wires. Between two cells there is a synapse, and the synapse has a certain strength.

If we go with the AI analogy, in reality, biologically they have like timescales and plasticity rules and then lots of like a maybe a dozen parameters or something. But what we currently completely lack is mechanisms that take these images as input and tell you how neurons influence one another. All we have is the wiring diagram.

So the focus in the past was really like, let’s make wiring diagrams. Whereas I think we should ask like, what are the things we want out of wiring diagrams? I think what we want out of wiring diagrams is we want to look at the neurons, and the images need to tell us how they interact with one another. And for that we need this concept that I call compilers.

We can say what goes in is if you want like the portray of the sign ups, show me how big it is. Show me. Show me how many receptors molecules it has. Show me how many of various second messengers it has. You name it. And can you tell me how strong it is? Can you tell me how fast it is? I believe so, no, this like a standard machine learning problem of the kind that we know we are good at solving is, which is let’s get how strong and fast and so forth.

A million sign up system. And let’s get at how they look like. And then let’s find out how we can predict how strong and fast they are based on how they look like and the molecules that are their molecular annotations. All these things. But compilers are super cleanly defined. Prom, which is basically you measure how fast they are.

You measure how strong they are. I give you the image and then after training I’m like, here’s the image, how strong is it? And you’re like, this the Peco and current that will come through it. And I’m like, okay, you’ve been right the last thousand times I trusted you that you have this down. And then I’d be very happy to build systems based on this.

But this bridge we don’t have. The way I view it at the moment is like neuroscience has these two halves, has the physiologist stick electrode and so brain so optical recording, see what’s happening while they do things. And then there’s other pieces which is like make wires, make a wiring diagram. But as long as we can’t convert the second into the first it’s Its kind of two fields that I mean, like they touch one another at the level of storytelling.

So if you look at the fly field, they do look at the ways and they’re like, look like if I look at these wires where they come from, the visual cells, like maybe there’s motion and they’re better than chance, but kind of there’s still this translation is extremely qualitative and I think compilers could make it very quantitative.

Juan Benet

Describe a roadmap or a path to solving this problem. What would you do or what is like in that technology tree of like how would you break down the larger problem today? And like what’s what should we be doing over the next few years for compilers?

Konrad Kording

What you need is primarily a training set or calibration data. The way it works is you do relatively old school synaptic physiology. You go in, you measure what the current is that flows to a synapse, and then you freeze it. Or like you stop all reactions there and you store it and then you reconstruct it where you know how it looks like, where all the molecules are, and so forth in it.

And then you don’t do that for one synapse, as traditional papers do. You do it for a million because like, look, we have like robotic patch clamping and we can do it optically. There’s a lot of techniques that scale. Let’s not go through all of them. But like you use techniques that scale to produce a very large number of them.

And then you use a machine learning algorithm for that. And basically what we need to do as a field to build that bright. What I’m trying to say is that bridge is necessary for progress. And it’s also clear how to fill that bridge. Like, you just need to.

Both sides of the bridge at the same time. And it’s not rocket science. It’s perfectly well established standard neuroscience where you’re just like, show me molecular annotated synapses and tell me how strong or fast they are. And then you predict one from the other. And it’s a very concrete research program, and it will have trouble getting funded because it’s so different to all the other things we currently do.

Juan Benet

Problems like these on a path to a much larger swath of solutions can get funded, right? Meaning if we do this and we understand organic neurons dramatically better, and we then can figure out how the brain is doing learning, and that distills out insights that we can then use to improve machine learning.

That already is like a super valuable contribution. Then will greatly accelerate the field of AI and like let’s pay back itself like many times over. Or.

Konrad Kording

Yeah, but hold on, these things are pretty far downstream. So let’s be clear what a compiler does for you in an image. And it just comes knowledge about what currents there are. If I told you for a given neuron what current every synapse produces, it’s relatively easy to then even do the fitting or kind of like find out what the neuron does, at least in simulations.

But that’s not been proven out yet. If you know how a neuron converts its inputs to its output, it should be relatively clear how to get to a socket or even a whole brain. If you had a whole brain.

Juan Benet

It.

Konrad Kording

Would be reasonably obvious how you could improve AI systems. But if you want, the compiler is just the best defined first place. It’s also a major enabler for neuroscience. And like you can ask a million question if I tell you like look here, this synapse does the following thing. You can say, well how do molecules make that happen?

You can say, how is this thing different in the disease? You can say, no, it opens up a huge space. But from there to and now we improve AI. There’s like a multiple there’s multiple steps still that are necessary for that value. And that’s why I’m a little bit worried about like the funding future of the field, that basically we can break it down into these steps like compilers to synapses, synapses to cell cells to whole brains.

They all come with their own set of benchmarks. And it’s clear if I if I had a bunch of people working on it, how do I tell them if they’re on the right path? But to get from there to value if you want, like all three pieces need to be solved, but you can only start work on the second once you solve the first.

Juan Benet

So it’s very concretely like, you know, if current capital expense to train the current machine learning models, like we’re hitting scales of capital that are beyond anything we’ve seen in history. Well, there are these graphs that show that a fraction of GDP, this similar to different industrial booms like the railroads and oil and similar type of industrial output.

But in terms of like a pure science and engineering sort of discovery processes, if we can figure out how to dramatically increase the learning quality of machine learning models, that can be worth, you know, tens to hundreds of billions on its own right now, which then suggests that it should be easy to make the case to these groups that if we can figure out how the organic neural networks work and larger and larger circuits that then enable us to truly understand how learning works in brains.

Maybe not even a human brain. Like just in a mouse. Exactly. If we can get a mouse brain and understand how learning works in mice, and there are concrete insights out of that then massively reduce the expenditure for quality and artificial AI like that. That’s super valuable, right?

Konrad Kording

And it sounds like a good but like look like there are lots of risk factors in the way. For example, now let’s go through some risk factors. Let’s talk about risk factors for Comillas. It’s possible that every synapse is unbelievably complicated. Yet if you can’t somewhat compactly describe how synapses work, you’re dead in the water.

Now, with everything that we know about synapses, it seems very likely that’s the case. But that could go wrong. It could be that you need more molecules than you can realistically measure to be able to do that. We expect that it would be upper bounded by the number of proteins that exist in human cells, but it’s still a very, very large number.

But it could be that you need 20 of them. We don’t know how many you need at the moment. So there is a risk factor on how complicated synapses are. There is a risk factor on how complicated neurons are. Now what do I know if like quantum gravity and light is actually the source of consciousness, then no amount of compiling will solve that.

Now, with the things we currently believe we know about neurons, that shouldn’t be the case. But there’s real risks about how complicated neurons might be. There’s real risk about like how the interactions and brains could be complicated.

Juan Benet

But all of that sounds, you know, par for the course for the kind of risk involved in, in artificial, you know, in AI research today where, you know, people are trying to stretch the boundaries of how to get the current transformer architectures to perform better, like just even like the chain of reasoning model, which initially started as like this big Cluj around the entire thing.

Just that was an incredibly expensive thing to try to do, and it worked out. And that’s the one that we’ve seen work. There’s a lot of other things that people have tried that probably were worth tens to hundreds of millions of dollars in training costs alone that never went anywhere. So, like the AI labs have to have a portfolio approach to research where they have to try a bunch of different things to see which of these will catalyze improvement rate.

And, you know, maybe 2 to 3 years ago, the capital expenditure there was in the millions. Now it’s in the tens to hundreds. And so that’s just like a very different scale. It’s like.

Paying $100 million to like, see if we can figure out, like how my brain learns. Seems like, like an acceptable research question that Frontier Labs should engage in today. And if not today in a year.

Konrad Kording

I’m with, you know, like, I’m not saying that this isn’t the right path forward. What I’m just saying is I can give you a long list. In fact, the compilers paper has an appendix of 20 different ways, so this could be not working. Yeah. Including like no, there’s ethical risk. Like is it acceptable to if you would simulate what, you know, like a mausoleum and maybe it’s not ethical enough for like everything gets shut down because kind of simulating a mouse in a not so great way is maybe unacceptable.

I mean, like what? You know, like there’s there’s a very serious. No, it could be that the resolution that you need that it really matters not just which molecules are there in the synapse, but where are they exactly? They are like a one nanometer resolution. If that’s the case, it would drive up, like everything by a couple orders of magnitude and we couldn’t deliver on it.

So there are risks.

Most things that we do in neuroscience, I can write down those risks for you. Exactly. Like here’s a list of things and here’s how we would know these things, and here’s when we’d know those things. So it feels like that approach of like, let’s do bottom up understanding of brains. So at the moment we try top down like we record the things and then we try and see like what kind of models are compatible with it?

With what I’m arguing, there’s just this huge set of different models that all produce the same input output, whereas the buttons.

Juan Benet

Are like describing the risk profile to be very clear and upfront and like, what’s the problem? Space. But you very strongly believe in this approach in terms of like the outputs that they’ll generate.

Konrad Kording

That’s, that’s that’s right.

Juan Benet

I like to generate.

Konrad Kording

But there’s like glue pieces that also carry conceptual risk. For example I told you about cable equations. If a cable equation just is really bad at modeling how a neuron works, then it’s possible that you run into like fundamental problems of producing the models. And if you can’t model it, then you’re blocked on your value chain and you can’t ever do anything out of it.

So yes, but it’s an orthogonal approach with a risk structure that is entirely uncorrelated to the rest of neuroscience.

Juan Benet

Yes.

Konrad Kording

Yeah.

Juan Benet

We’ve been talking about neurons and circuits and brains and so on. And we’ve been talking about kind of modeling them and creating some simulations, but we haven’t really discussed the simulating part itself. So. One part of mapping the circuits is to just kind of understand. How they work. But what is a great value that can come from this?

Like what are the range of possible applications that we can get from both? Understanding how the brain works or understanding how neural systems work and simulating them, or even simulations without understanding. Right? There’s a lot of utility in creating these simulations. So what is the universal possibility there?

Konrad Kording

I think maybe it’s good to start first, like stretching a little bit ways of understanding. Because ultimately what we do in science is epistemic. So at some level there’s there’s a logic behind what we’re doing. And I think there’s two ways of understanding that we often mix with one another. One is what you call traditional science.

I tell you principle neurons are cells, and between them there’s a synapse. And you’ll be like, okay, Roger that. Like, now I have something that’s useful. And if you want these things, do two things. They describe things in the outside world, and they help us think better. And in that sense, if you want like a lot of what we know in neuroscience is of that nature, not like we know about synapses and Hodgkin, Huxley models and neurons and simulations.

There’s a second type of science that’s coming up, which I call machine science. So like take AlphaFold. It’s great that you put in a protein and out comes how it falls. It’s a huge enabler for things. But if I give you this, this system, if I’m like, okay, look, here’s AlphaFold, run like, tell me about chemistry.

You’re like, I don’t know. Like it works very well. And that’s very dense. So I think those two are different and they’re valuable in different ways. If you give me a simulation even of a human, let’s say, of a of myself, I give you on a hard disk, simulate Konrad’s and you run it and it works perfectly, it’s useful in some ways.

For example, you could say, here’s two kinds of coffee. Well, this simulated Conroy would prefer coffee A or coffee B. Great wax is useful because you can make the world better fast, and you can generalize it like we move into an AI world. Would people be happier in this kind of a world or that kind of a world for that?

You don’t need to have this traditional science understanding you. It simulates Konrad. You’ll be like, I don’t know, there’s like a trillion variables on this thing and like, it just works. Like Konrad, there’s this distinction of understanding of like human understanding, which we don’t get, and simulation quality, which we do.

In that sense, the machine science is useful because you could also say for AI alignment, if I can simulate a Konrad, the problem usually in the world is you can never have people compare A and be in the world ever, because it’s going to be the one. Now that gets like one coffee and the one tomorrow that gets the other coffee.

Well, if I could have a simulation of you at the same time, no problem. I, both of you, get the coffee right now and I can read out later. I can ask each of them and I want to attend scale. How do you rate your coffee? So it would be a major enabler if we could simulate animals or humans for that matter. There is the human possibility.

It’s possible that if I gave you this full simulation of Konrad, you’d go like and like, look, there’s this piece of Konrad that I now understand. There’s this neuron that every time does this thing. The question is, how real are those stories now? Like we might be? If you wonder if we look at the history of neuroscience, we often have like, oh, look at this.

This does X only that. Then a decade later we found out that it also does y. And now we find out that it does x, y and so many different things that Simeon and I think this history like shows us the limits of human science versus machine science now, like they exist neurons, they are like super specialized neurons in your retina that show you that like right there, there’s something bright.

It’s great. Wax generalizes. If you go all the way in the middle of my brain. One of those neurons might be about your face, but also about how I feel about you and maybe about relationships and so on and so forth. In that sense, there might be areas where we can only have machine science. Now, we might still be able to say, build better AI with it.

Maybe we can take this Konrad simulation and we can be like, can we fit a loss function of what Konrad optimizes for? And the answer is yes, good quality, quality of coffee. But we may not see that readily, and we might be able to build systems now that have reward functions, loss functions that mirror how humans think are the humans that we could simulate if we could simulate them.

So I think there’s these two the human understanding and the machine science piece that are different.

Juan Benet

Logically, having a more concrete set of descriptions like what is like sort of like the downstream utility or value that people can get out of it, right? So like the machine science, understanding that you’re describing that you can get if you kind of understand how certain neurons work, then naturally you can start figuring out the nature of disease or the nature of some problem and start like potentially fixing things.

Or, you know, a lot of like the neurotech applications look like. We understand that the brain does this particular thing. We see this issue in how the brain signals are connecting to a part of the body or a set of sensory input. And so therefore like let’s kind of like patch this by finding some circuit around the kind of like damaged nervous system or like the damaged nerve or and so that’s one path to like repairing or restoring function.

So yeah, it’s sort of like the universe of utility that you can you see out there in, you know, suppose that we can do the, the machine science and, or the simulation. So maybe we have one or the other. Like what? What is it like the downstream value that people get out of it?

Konrad Kording

Yeah, I think that’s even both a human science and a machine science disease application. You could say a human science application. Take Parkinson’s disease. There’s some neurons that die because they die. There’s a certain function everyone can’t do anymore. Once we understand what that function is that’s missing, we can now build drugs that help with it.

We can stimulate the brain stimulation. It’s one of the best things that exist there. The existence of brain stimulators is like one of these things that would be magic in the past. And then also like just embodies, like the wonderfulness of neural tech. To me at least, it’s just like seeing the patients is like believing in the in the research program in the case of Parkinson’s.

And that’s very like human science stuff. Like there’s neurons, you can see them, they’re different from other neurons. And they have a pretty simple path to the disease. Now you could say from a machine learning machine science perspective. Maybe the way my reward function in my brain is implemented is really complicated.

Maybe distributed over 17 brain areas and a trillion neurons. And I could still use machine learning approaches to, say, distill them into like a good compact descriptor of what Konrad cares about or maybe what people care about. So I think both of them, you know, and you can say a lot of diseases might be related to reward functions are learning algorithms.

And so for example, you can say, if I have a disease of the brain, we usually view it as there must be something wrong with the molecules. But it could be just as well that there’s something wrong with the learning algorithm. It moves us into a space that is not a healthy space. and in which case what really happens is upstream of the neurons and we might just be working on the symptoms instead of working on the causes.

And,

So, so I think I’m just trying to sketch that there’s like a really diverse set of, like, values that we’d be getting from it, but you can still group it into areas. There is the group of, of areas cure diseases. There’s a group of areas build better AI systems. There’s a group of areas align AI systems with what humans care about.

There’s a lot of different buckets. And I’m at this point of time unclear where the main value is. And there’s the other thing that no one ever talks about. Like, if we could, like, simulate human beings or if we could even simulate a mouse, then it would just be such an unbelievably cool thing, you know, like, it’s kind of like there we have a system with like billions of parameters, and yet we can kind of at least simulate it and potentially kind of come up with approximate descriptions of that.

That’s kind of like in the in the brag sheet of mankind. Not like eventually. Well, the sun blows up and like people will look at like, what have we done in the time in between? And it’s like, understand our own brains. But for me, be very high up there.

Juan Benet

Yeah. I mean, the last one like, to me, it’s one of the things that attracts me the most into neurotech neuroscience. and to me, it seems similar to AGI and ASI as a concept. For decades, scientists and authors were able to look ahead of where science and technology were going and kind of predict, in broad strokes, roughly how machine intelligence would be built and develop.

And we got that early in the 20th century, like mid 20th century. We got a lot of sci fi books about written about it, but the technology was just so far away that we then kind of watered down the possibility space to the point where, like now, we are finally building these AI systems to be close to or surpassing human intelligence on a range of domains, and suddenly, kind of like the mainstream opinion finds it hard to square the fact that we’re literally creating these incredible results, which are incredibly cool and powerful.

But yet, because it didn’t happen for so long, it just sort of feels weird or distant still. And to me, kind of building a digital version of a of a human or a simulated version of a human has that same kind of flavor, where it was also predicted in sci fi and by both scientists and science fiction authors way ahead of many decades, becoming possible to the point where right now, people are just not really considering the real possibility of what happens when we do, in fact, get there.

And like how incredibly cool that future landscape could be. And so I don’t know if you have thought about that landscape, like, how could that be a kind of optimistic vision of the future? What does that enable us to do? What is it kind of like a digital existence or not necessarily necessarily digital, but like a computational existence, open up for humanity the promise of the potential.

Konrad Kording

I mean, the question is what’s not the potential? So yes, if no, if it could if we could digitize an animal, we could transport it from A to B, no problem, because we delete it from one place, we instantiate it in the other place. It’s unclear how we should feel about that process. Like there’s ethical problems, there’s like logical problems, there are value problems.

But every change of technologies always forced us to rethink what everything is about. Like when when machines came, when the steam engine came around, people whose value came from, oh, I am so strong, must have felt miserable. And now that AI comes around, there will have been, there will be. People will be like, well, I knew everything about ex while in somehow on the internet.

X was there and then like now, like everyone knows about X and there will be proms and I think logic of if we’d be able to have simulations would have these similar trade offs that are weird. That being said, not like the first applications will not be oh, now we become like a digital only species, but the first users would be I can try drugs.

I can simulate what they do to a brain, and I can screen them before having to go to lots of patients. It feels like it would. Being able to simulate brains would allow us to speed up a lot of the things we do. That includes like drugs, electrical treatments of brains may even be possible to ask how? Sort of like how to optimize our environment for the brains that we have.

It might be possible for us to say which brain states are desirable, which for which brain states are not. It would open up a new set of capabilities, and people are often worried about these things, not with AI. There is this notion that just AI, because it’s so different, will be the end of human civilization.

A lot of people go around like AI, AI is coming, and then we all die. But I don’t think that’s the logic of the world. Like why? Like, hello? We are building it. We wouldn’t build it if it was just like. And then we all die. We’d build it because we think the world will be more magical, more wonderful with AI.

And we work towards simulations because the world will be more magical, more wonderful with the capability to simulate brains.

Juan Benet

Yeah. Let’s talk about AI for a while. Of course, the capabilities today are dramatically better than they were over the last five, ten, 15 years. You can now talk about building AGI like 15 years ago. We couldn’t. Right? Like, it was kind of in close rooms. People were talking about it, but like, you wouldn’t be taken seriously as a scientist if you like, even in the deep learning field, if you kind of said that you would build a machine close enough to human intelligence, and yet this happened like we are here now, like we now have models that can outperform humans at a broad range of cognitive tasks.

They still fail dramatically at a bunch of straightforward things. So maybe, you know, as you’ve observed the capability space growing here, what are some reflections that you have, like what has worked better than you thought it might this quickly or what has been way slower or like, you know, give us a sense of like how you look at it.

As a neuroscientist.

Konrad Kording

I think the main realization that I feel is missing in the debate is the extent to which artificial intelligence is different from biological intelligence. So humans kind of live on one or much of human intelligence lives on one axis now. Like, if you are very intelligent, you probably are good at reciting Shakespeare and solving integral equations and reasoning about rowing above a river in kind of distinguishing the superficial features of something from the deep features.

So there is this continuum. There’s this manifold of human intelligences. And when people think about AI, they often like try to project it on that manifold. And like the whole idea of AGI is like, there is this like axis of intelligence and intelligent humans that like far out are like, hi there. And like not so intelligent people like lo there.

And AI is somewhere on that axis where in reality AI is like just totally different. So AI’s are better than you at lots of things. They have been better at you and lots of things for a long period of time. And computers have been better at multiplying big numbers than you like since you were born. Webern. It’s not that there’s this one access and I’m like and at the same time, it’s then assumed, well, because they are great like summarizing a paper, you know, a complicated paper that therefore they could solve my short term strategic problems in life.

Yes. If you as a human being are like really great summarizing this paper, but like, sure, you could plan how I get to the airport from here or something, but AI doesn’t live on that manifold. And in that sense, that’s a that’s also one of the things that gives me a lot of hope. You know, like if AI’s were just like us, then yeah, that would be kind of set.

But they’re not they’re like really great some things and really not so great other things. And that’s why, at least for the time being, humans are great if you couple them with AI and like I like I use AI every day and I’m a much better me. Much higher quality is the things I produce are much higher quality because I use AI, and I think in the future this will just be much more the case.

And so, so like the main observation from the place where I come from is that this naive, like it seems intelligent and therefore it’s like us. That is a something I really have trouble with.

Juan Benet

We were talking about learning earlier the kind of neural network concept of learning, which is how do you adjust the parameter space of neurons to have the system learn some set of functions? But that word learning comes from a different concept, which is kind of how humans absorb conceptual information and draw relationships between these concepts and then extract some.

Like additional skill like that, you are able to generate an additional mimetic skill in your brain out of some prior context. And so humans are really good at that. That type of learning neural network version of learning, which seems like a much lower level algorithm. These two are maybe cast is the same thing by a lot of machine learning and artificial intelligence research, but to me they feel qualitatively very different.

Like, there seems to be like something is happening in the neurons, and something altogether different is happening in the conceptual space that the neurons are simulating or emerging. So I don’t know, how do you think about this? Like, like clearly we’re able to not just learn from a few examples, which is the characteristic case that people point to with AI relative to like the millions of examples that neural networks require to train or like artificial neural networks require.

But on top of that, we’re also able to reason about a complicated landscape of ideas and then come up with new ideas and or learn that some idea is wrong, and then shift our behavior and understanding mathematically. And that seems very different than a basic kind of like reward loop and weights adjustment.

That seems like some other computation that’s happening.

Konrad Kording

Yeah. Now, like it feels to me at least that in human behavior

We have a history of all our thoughts. And we not merely know how to solve the problem, we kind of know how to reason about paths that failed and paths that worked. And to have this like meta awareness about what a good thought patterns and what are not. And I think that is, you know, basically a neural network has no record of the thoughts that it had produced.

It might see its output tokens, but it doesn’t really it can’t really reason about the thought pattern it uses itself. And above all, I think we have. So we have these world models now, like I can close my eyes and like I can imagine I’m in a coffee shop and I sit there with my friend and I talk like, kind of like it.

We have these world models, and somehow we managed to get the information we have in our neural network things into this, like discrete world model where there are baristas and coffee machines and all that. And then once we play this thing, we can go back and kind of like put it into weights in our brain that we then use for like online decision making.

And I think artificial neural networks and like the whole deep learning field has nothing like that. And I think that’s why we have these complicated ways of managing context, because kind of we can’t hold it all in there, obviously in a large language model. So we have we try and build these elaborate memory systems.

I’m with you. There’s something fundamental that is missing. And again, like if we could like produce simulations of nervous systems, we might be able to really say something about it. And the brain has this structure now, like it’s not that it’s like layer layer layer, not like it has like stuff goes to thalamus and stuff then goes to cortex.

But when we sleep, nothing goes to thalamus. And there’s still kind of a world simulation happening and we can turn on and like we can kind of route information a much more freewheeling way in brains. So it feels there’s a big logical piece that’s missing there.

Juan Benet

Have people tried building large scale artificial neural network systems based on the regions of the brain and the functional utility that we think exists? Like taking a functionalist approach and kind of draw these kind of broad boundary boxes and connectivity between them, and then or is just completely fraught and like, unlikely to like, is it better to just kind of emerge everything from scratch because it is pretty crazy that the current Transformers are effectively recapitulating the entire process of evolution every time from scratch.

Right. Like, it’s just kind of like building a brain from zero. And, you know, it takes a few months to and like tens of billions of dollars worth of compute to then, like, output a new brain. But clearly, evolution learned a lot about how to build intelligence in that, you know, many 100 million year process that abstracted a bunch of pieces.

And these seems like those pieces are qualitatively very useful, but somehow we haven’t been able to, like, simulate these.

Konrad Kording

Yeah. Look, there’s a bottleneck. Every generation. The information we can give to our progeny is upper bounded by the amount of DNA that we have. And arguably most of the DNA isn’t even dealing with those situations. So the situation there is quite different. And if you think about it like a lifetime of a human being relative to what we feed to an LLM, it’s just this absolutely, incredibly tiny bit of information.

And yet on a lot of tasks, human experts are still much better than our labs. So there is this disconnect that clearly there’s a lot missing in it. Now you can say, is it consequential? Now you can say maybe over evolutionary timescales, we did get as much data as the Llms have, so that somehow we efficiently use the data from our long ago ancestry.

And it’s clearly there, like and in that sense that makes us also like much less fragile, which is we deep down through evolutionary paths, we remember the flood of something at some level, like we have like we have an implicit knowledge of all these cataclysmic events that were evolutionarily important.

I wanted to put a pin a little bit in evolution. So evolution I think, gives us deep insights about human minds. And I think also it’s helpful to understand brains. So the idea in evolution is like that. Systems are in a way you get good at the niche in which you’ve been growing. And there is an interesting way of thinking about AI, which is what’s the niche of an AI system, what makes an AI system succeed?

So when people talk about it, they often they think about it as if it was a species living out there. Some may live and others may die, and the ones that live will have babies. That is not how the evolutionary system for AI system works. The AI systems for most of the time before the Frontier Labs works like this.

They exist code that runs an artificial neural network. It’s being run on some computer by a PhD student, and if it has desirable traits, which is it wins at a benchmark, it’s going to be copied onto the laptops of a few other PhDs, will change a few lines of it and then submit to the same conference as next year.

So. So in the same thing as of course, even within the frontier lab that’s happening, there’s going to be some code bases that people like and others that they don’t. That is an evolutionary niche. Now what makes one LLM outcompete another one? Well, it must win competitions. What other competitions?

What are the benchmarks that we use? Or like that Frontier Lab series. While they use things like don’t lie to us more than needed, more than our customers actually like, which means that the evolutionary landscape for them is just like

Lying is bad for you, because then you’re much less likely that you’re going to be used in the next generation. And so this evolutionary process will give AIS the things that they’re selected for. And the same thing is true for brains. Not like you can say what’s the best waste that we have about brains?

The single best idea about thinking about human or animal brains is let’s just assume they’re pretty good at solving the problems in the niche. In neuroscience, we call that normative models, where we’re basically, let’s assume you want to be good at estimating how far away something is and how big something is, and if this one object versus another object, if we build that into into a models, and we used to build that using Bayesian systems.

Today we use deep learning systems. But the key is we built in just like here’s our understanding of the niche. Give me the best solution to that. And if you solve for the best solution, that’s like the single best, most useful, most reusable model of brain of brains that we ever had. So there’s thinking deeply about the evolutionary landscape is what gives rise to like, well, walking models and people don’t usually think about it.

Juan Benet

I really like the evolutionary frame on top of the current kind of generation of models. Certainly we’re also selecting for a lot of economic utility, right, where these labs are under intense economic pressures to deliver broad capability and utility, which today is shaped in terms of the knowledge work that people are doing day to day.

These systems are very bad at navigating the real world. So like robotics is so far behind. In fact, we should probably dig into that a little bit in a moment. But yeah, how do you how do you think this selection process is developing over time? Like, what are we. We’re clearly selecting for some cooperative structure and cooperative game.

Yeah. How do you expect this to flow over time?

Konrad Kording

Well, of course we should expect it to be become more and more aligned with the financial incentives that they’re there. But let’s be fair to those financial incentives. If I’m a company, I wanted to solve my problems. I don’t want to. I don’t need it to tell me that I’m like, so freaking awesome. It’s enough if it’s like, here’s the problem solved.

And by and large, CEOs do the right decisions so they will buy AI that solves the problems that occur. They have preciously little interest in buying AI that will take to that will try to take over the CEO job, but there will be really happy with AI that like if you call and complain that the piece wasn’t delivered.

Censor that piece. So at some level we should expect in that environment. And that means that the evolutionary environment is give the companies what the companies want and the companies as an evolutionary landscape. I would give the customer what the customer wants. So ultimately alignment is like built into that whole process.

And like the evolutionary landscape is improve alignment.

Juan Benet

That’s a very interesting way to put it. One counterpoint narrative to that would be sure, the kind of there’s like this outer alignment loop that is training and developing these systems, but there’s just a raw, increasing capabilities that they’re developing. And at some point they’ll cross some threshold of broad capability across a very wide range of domains.

So then, you know, far exceed what humans can do in a range of things. And that plus some misalignment in like internal goals where there might be like a, like a misalignment between the, the internal belief structure of a model, that’s kind of where the problem lies.

Konrad Kording

So let’s just see if we can answer it from an evolutionary perspective. There’s going to be some eyes that are more likely to try and break out of their sandbox than others. Which of these two systems is more likely to like win the long term, and which one in fact? Well, when? Relatively quickly? Well, like the scientists are looking for violations of their sandbox.

And let’s just say it’s not good for you if you’re an LM system or like if you’re one like it makes it makes it that way. Every generation we select for alarms that are less likely to break out of their sandbox, given that it’s.

Juan Benet

Isn’t it actually selecting for llms that not getting caught, which could include llms that are simply not doing the thing, or llms that get really good at doing the bad thing, but hiding it?

Konrad Kording

My hunch is not like they is something

About species and evolution that at some level you’re being fact checked by the rest of your species and. And if you want, I believe that there’s a general scaling law on intelligence, which is if you put more intelligence onto a task, the extra benefit you get tends to accrue in a highly sublinear way.

So if you’re one like the probability that one system, if you make a if you give more flops or something to a system that does something, whatever it is that doing something is the benefits. Well, usually at first you get a lot of benefits from basically not completely screwing up, and then there will be less and less.

So in that sense, if that take off like sublinear intelligence scaling is correct, then if you’re one like the other, llms will of course like be calling out that one LLM that tries to cheat and will be very successful at that because because well, the one even if that one would be more, would be better, more successful, it would gain a very small benefit from it.

And that’s kind of why I come out in that discussion as feeling rather bullish about the future of artificial intelligence, because basically, I believe on any one task there is sublinear gains to it, which means that kind of like the system is inherently stable. Now, you could you could hold a different view where it’s like it’s up to some intelligence, like you don’t get anything.

And then if you if you exceed some threshold, then you gain a very, very large amount. It’s just people are not doing a great job at giving me examples of that. Like cracking is maybe the best example that people have where like, you have a system that some point of time discovers it, but this growing process is, is more like if you’re one, you get like that one extra bit when you reach that and the reaching it on an individual not like it has this jumping off of capabilities.

But it’s not that it’s a clean thing. If you have like the right amount of compute, all of a sudden it jumps up. It more is like you discover a dimension that you didn’t have before. That dimensional benefit, though, is usually on an overall scale. It’s a relatively small benefit. It’s just like if you select for only those cases that this benefit was speaking to, then it looks like it’s a big jump.

But on the grand fitness scale, it feels like these are like generally small, small things.

Juan Benet

So far, we’ve had a lot of success where in the areas that we have an enormous amount of data because of the dynamics of learning and the fact that these systems require so many examples and learning vast data quantities, plus easy verification loops, predicting the next token. This where we’ve seen a ton of the gains.

And meanwhile we haven’t seen correspondingly good gains on things like being able to navigate the environment, which leads to the classic more of a paradox type argument, which is, you know, the dramatically easier to compose a symphony than walk down the hall and not break the door. Right? Or something like that.

And so clearly, robotics is way behind. And so we’re having kind of a an impact on knowledge work but not yet on physical work. Having done a ton of work on like just the motor areas and so on, like what’s going on there and like, what do you think? How do you think we’re going to get past this? Do we really need to do what the robotics labs are doing now, which is kind of, again, brute forcing the generation of just oodles and noodles and noodles of data?

Or could we glean some insights from neuroscience to be able to do this problem better or like yeah, how do you.

Konrad Kording

So there is.

Juan Benet

A.

Konrad Kording

Big cognitive effect which is that people are drawn to causal explanations. So if I ask you to explain something to me, you will immediately use causal language. Like that’s how humans talk with one another. in the motor domain, when I walk around the corridor and I’m trying to not break a door, it’s all about causal effects.

And like what? Well, me doing something to do or not do to the door in by design. Our systems are correlational, like it’s machine learning. It’s trying to predict the future from the past. That means that it knows what’s a good predictor. It doesn’t know what’s a good causal influence. And in motor control, everything is about causal inferences.

And in fact, if you look at babies start moving. A lot of this they’re trying to find out what happens if I move my hand, what happens if I do this? What happens if I chew my own hand? They kind of causally explore the world. And yet our machine learning system, causality, doesn’t really exist in them.

You can say in reinforcement learning it exists, but it exists in a pretty rudimentary way, which is we have probabilistic policies. If you run a probabilistic policy, you can say there’s a little bit of noise on your policy, and that’s kind of what you use to figure out how the world works. But it’s a very inefficient way.

Not like it’s not that. Like if a kid explores a room, it’s just like there’s random stuff in the room. I was like, oh my God, there’s a switch. Let me try what’s happening there? And you’re like, oh my God. Like stop lights turning the light on and off. But it kind of like it’s a directed exploration aimed at understanding causality in the world.

I think it’s super salient, at least in human cognition. Alison Gopnik has a great analysis of showing that, like, kids are, like, deeply attuned to that. And yet our AI systems don’t even know it. And there’s no doubt that human brains kind of know the difference between that. And in fact, you see it in like, super young babies that things like intentionality, you know, like, we are agents, we have goals, we do stuff to get out towards our goals.

AI systems just try and explain the regularities. There is no real sense of agency and no real sense of causality and like, like to the level that it helps predictions. Like, of course, like there are cases where causality does help prediction a lot. But then when it comes to like intuitive physics, AI systems aren’t quite up to snuff yet.

And when it comes to more complex like, like, like it look like causality is this magical thing. The way it works in our in our world, which is we have causally specialized object like a cup is for drinking coffee out of it. It does one thing, and there’s one thing that there’s one causal chain that is supported by cups, which is put coffee into our mouths.

And so we live in this world where causality runs on a very small number of sparse lines through time. And we are, like, so attuned to kind of discovering these causal lines and then being able to manipulate it. Whereas a lot of the world alarms are much better at describing the regularities than I am, but I kind of know how I can, like, get coffee in a good place in a way that probably AI systems that have great trouble with at least when bodies are involved.

Juan Benet

But so that implies that the brain is building these very sophisticated world models, these causal world models, and then are able to therefore to path, plan and navigate and so on. And I think that makes sense for humans, make sense for probably a lot of animals with fairly sophisticated brains. But if you go down the other direction to a much simpler and simple and simple animals like, eventually you get to like certain insects who can navigate very effectively and can have amazing swarm behavior, like bees, for example, will have tremendous swarm behavior and are able to do very complicated sets of things.

But they probably don’t have a very sophisticated world model or I mean, I guess we don’t really know.

Konrad Kording

Yeah, I don’t I don’t think they need to know. So you can say there’s always two possibilities. Either a can evolution can give you a reward function and you optimize for that Casio learning to get good at optimization. But you will get a policy that’s better in the end. Or alternatively give you the policy and then maybe just learn at the bottom of that.

I think insects are much more in the let’s let’s let evolution figure out the policy and then they just run policy there. You can say when it comes to navigation.

Juan Benet

That’s why it’s like so little to a very, you know, weird environment like it flies characteristic like are following light to some degree and like they can’t handle windows very well. Whereas like I think wasps do it better where like wasps I think have like some different policy where they can.

I think one of them handles light and one of them handles wind differently. And so there’s like these classic cases where you can they will respond very differently to potential predators. But it seems like very hard coded into the brain.

Konrad Kording

Yeah, I think I think it is. And, and look like the policies you need for, for 3D navigation if you’re a drone. I don’t believe that they’re crazy complicated and like the reason why our life and our niche is so important. It’s so, so complicated is because other beings have kind of like this. Other humans have like the same mental depth to them that we do.

And that produces kind of a fundamentally complicated world. And in one way we need models because we kind of come just assume that one policy is going to generally win. Like we can’t just say the same thing. Yeah.

Juan Benet

There’s all these theories about what led to the evolutionary run off process that gave humans this level of capability and intelligence. So, you know, there’s theories like the social behavior did it or like complicated hunting patterns or like, you know, all kinds of pressures. yeah. How do you think about this?

Or like, what do you think if you had like, guess or I don’t know how you explain this, this rise of intelligence.

Konrad Kording

I mean, let’s let’s talk about the evolution and the nature of compute. You can say if you have relatively short lives and you have relatively stable environments. Then in relatively simple environments, then evolution. The best path at some level is built everything in. Now you could say if on the other hand we have systems that are relatively intelligent way with intelligence.

What I always just mean is being able to choose the actions that are appropriate or that like reward generating in a given situation. Then the situation is different. You can say if, if instead if you want. For simple insights, evolution might want to mostly build in run this policy and like what I know the be like use this such strategy and you and they have the waggle dance and for communication.

But it’s hard coded. And alternatively if you have an abundance of intelligence arguably humans arguably AI systems. A lot of it is about gut reward design, and people are always like, yeah, like that sounds like that’s so simple. LeBron must be doing something much more complicated, unless you start to realize that it’s probably hundreds of reward functions that in your brain are like your brain knows to not have too much salt in the blood and not too little salt to, like, have enough water and not too little water to have not to have the not too hot, not too hot, not too cold.

It’s kind of like its Goldilocks, but not unlike a dimension or three. It’s like on like a hundred dimensions at the same time. We regulate our emotional content, we regulate who we hang out with. And a lot of it is undoubtedly. And we know that the hypothalamus has a good number of those, like we have this like rich set of, of, of rewards.

And then we have these rich sets of like learning things to it. So for example, obviously we don’t want to have to be vomiting because we don’t want to eat bad food. But here’s the problem now. Like if you eat bad food, that will be only there three hours later. And not only that, if it was just a reward, what do you do?

Like three hours later, you’re like everything. No. Like, are you going to remodel your visual system? No. This like a biological system isn’t like. Oh, there was a big negative reward. It is like there’s a big negative reward of type food thing, which should be attributed to my food preferences, which I now want to adapt, but I don’t want to adapt all the other things.

That is something that like

That is if you just have a numerical reward, like every milliseconds you get how good or bad that millisecond was. That is very hard to capture with that like machine learning setting. And yet it’s like very natural if you think about like a more biological credit assignment.

Juan Benet

It also seems to suggest that we may not get sophisticated artificial digital beings or like with sophisticated, complex behavior. Until that level of complexity and reward functions is sort of built in. Like the current reward function might be so straightforward and simple that it inhibits a sophisticated and creative landscape.

Like maybe that these models are really bad at extrapolating and generating new things, simply because their task is just so bounded to like predicting the next token model.

Konrad Kording

Yeah, it feels like there’s clearly something missing there. And I think it’s also about values. No, you can say AI cannot give you values by itself because where is it from? Like the values, the reward functions. It’s implicitly like imitate what humans do. And then it is now like if we look at how we train our lives now, there is a first stage like your value.

You’re the only thing valuable in your world is that you talk like humans and then like, yeah, and also like talk in a way that humans find desirable. That’s the second part. And then it’s like, yeah, and you can also talk with yourself. But like the long term goal is just to make that humans find you valuable.

So and then you can also use tools, but you can also only use tools with the one goal that humans find you valuable that I mean, it’s a good from a society perspective because we want these things to be tools that satisfy our desirable desires that make us happy. If you want, that’s good, makes it safe in a way, but it clearly kind of misses the richness of what we care about.

Our value is at least I hope my value isn’t just like do like everyone else, and it’s also not let’s do whatever makes my audience most happy. Not like I’m opinionated in a way that is much more opinionated than I should be for my audience is sake. And I think I think we all have that. And I think that’s just like it’s another example of not like we talked earlier about how intelligence is so different between humans and these llms.

And this just another of these dimensions which like, yeah, we’re not on the same axis. We’re very differently constructed, differently evolved systems.

Juan Benet

It also potentially is the kind of pressure that would yield the richness of conscious experience and qualia that we that we have where beyond kind of our, our own individual utility to the rest of society, there’s the kind of richness of our own individual perspective and experience. You know, it’s hard to reason about quality or consciousness eventually be of models trained in this very narrow landscape.

Konrad Kording

Yeah. And if we can even talk about it in those terms, I wanted to like, just talk about one thing that I think is important here. So evolution gives us reward functions, not many of them. Basically it’s because it can’t reach into our lives. It kind of tries to approximate fitness functions with reward functions.

We like coffee because it kind of it serves something else. And, and here’s the thing from evolution’s perspective, we clearly are breaking out of the sandbox. Not like we’re supposed to maximize fitness, but we’re not. We’re maximizing something else that kind of arguably breaks out of the sandbox that evolution had intended for us.

And in that sense, because we broke out of Evolution sandbox. So we might not want to allow AI to do the same.

Juan Benet

So now, thinking about AI today, we’re talking a bit about capabilities. You talked about how you use AI and how it sort of improves your work and so on. What are some of the ways, like you’re using AI for science or AI to accelerate either your own work or your lab’s work or like, how are you seeing sort of like the impact in the field?

Have you seen a lot of shifts or, you know, some fields are getting accelerated more than others. So here’s how you’re seeing this.

Konrad Kording

Yeah, maybe. Let me start with how I use AI. So a lot of my AI use is to produce friction. A lot of people use AI to remove friction, and I think it’s a huge mistake. Standard use. Write this paper for me. I’ve come up with an idea for me. Or do this science for me. For me, the use is almost always the other way around.

Here I wrote something. Tell me where the logical problems are with this. I cite papers in my paper. Can you check that they actually support what the sentence says? Where I cite them from. And it’s shocking. Like in the past, I could never have done that. But today, I check every single use of references for me.

And you’ll be shocked at how large the proportion is where I get it wrong. Like the sentence I’m like, yeah, this supportive of that. But the paper doesn’t quite say that it’s in the right direction, but not quite. And that is for things that are my field where I’ve been in for 20 years. So if I basically find myself often somewhat borderline reciting things, I’m sure that’s very common.

So, so this and this at so many layers, you know, like I do some coding, I might even do coding with an LMS. But for every command where I’m like, code this thing, for me there’s like ten commands. So I’m like, okay, now like look for bugs in this. And how could my key leak in this case and analyze if this an if this uses the same design patterns as the rest of my code base, and it’s kind of it kind of like I use it to add friction instead of removing friction.

And in fact, I build an app for students that called it’s called Planar Science

That adds friction to their way of planning science. So it asks, so what’s your science question? And then it pushes back on that. It’s like, well, these three words have no meaningful definitions. Or it might be like, well, there’s this Smith et al, 2015 paper that sounds like mightily similar to this, and it kind of gives you the pushback.

And then once you establish a question, it asks you about your hypothesis. And it’s like, hmm, would be nice if those hypotheses were mutually exclusive. And it’s not quite as snarky as I, but basically it pushes back, it gives you a fraction. And I believe fractions is so important. And so this one of the users that I really promote in my field as well, because I see it all the time now.

People test our hypothesis and there’s like ten other hypotheses in the field, but they don’t use them. They don’t know about them. They don’t take it seriously despite the fact that it would make their paper more interesting. No, it’s it’s not that we’re like always in a competition situation, but like, we’d want to know which of them it is and kind of people miss that.

So I think AI for friction is important and that’s important also from my hypothesis about people and life. If you’re one, you want to become stronger. You’ve got to start lifting. And if you want like if you’re intellectual, want to get stronger, you get to start intellectually lifting, which is you get stronger by people pushing on you.

This also maybe one of the reasons why I’m, like, happy to give my field a little push back at times. I believe the field gets stronger by me pushing back against the field. I don’t believe that the field will just crumble and die just because I disagree with some statements that we are making. And so this like add friction I think is a super important philosophy to me.

Now, there was a second question you asked. I’m trying to cut your your questions into pieces, which is how is the field using it at the moment and how is it positively using it. So there is this possibility now that you can replicate papers much more easily and test paper. So for example, I have a student I’m working with who is trying to you upload a paper with the method section and you upload the code base and it just asks the question, hey, can you check?

Does the method section do things that the code base doesn’t? And does the code base do things that the method section doesn’t?

Juan Benet

I’m so scared to hear the results.

Konrad Kording

And obviously there’s a fair bit of like this deviation happening. And I think there’s a general feeling of excitement that we can do these things. Now this another example of like adding friction. But it’s it’s again constructive friction.

Juan Benet

And that’s fantastic to throw at like the entire literature right. Like there’s there’s a lot of literature across all fields that probably has enough embedded in it, especially over the last 10 or 15 years that have a lot of code where we could get a view into the reproducibility of all of these papers everywhere.

Konrad Kording

Absolutely. And it would be great to do that. And yeah, I should start writing grants for like, medium level compute for that. But I mean like this.

Juan Benet

This DeepMind and OpenAI would be thrilled to like, do this.

Konrad Kording

I think they should do it. In fact, it’s one of the few reasons why I’m sometimes jealous of not working for those companies. But basically, yes. Not like you can do it for a dollar of compute per paper. You could basically do the whole literature in and make a complete list of, of which figures replicate, which figures don’t replicate which method section.

And there’s a lot of differences between what people say they do and what they do. I ran funded data sharing grant at some point of time, and good proportion of the literature can’t even produce their own figures. And that is these professors were sending it to me. It’s not that.

Juan Benet

Yeah. Even when there’s like all the best intention and you try to do those like good work. Yeah. You still like it’s.

Konrad Kording

It’s so easy. It’s gotten much better now. Like now you, like, put the whole Python repo online. It’s much more reliable. But back then, at the time where people were using Matlab, there was a lot of just difficulty. And as professor, you can’t replicate it from your students. It’s just so incoherent and just replicating it would be wonderful.

Juan Benet

It’s probably a large thread that we could spend hours talking about. We both care an enormous amount about the tree of knowledge. What do we actually know? How do we know it? What is the evidence? What is the science that we’ve done? What’s reproducible versus not? What is the likelihood of whatever claim exists somewhere in the literature actually being true.

Konrad Kording

And an alternative interpretation and not like a lot of papers are like A versus B, and you’re like, hold on. Like there’s also C as a possibility. And I bet you there’s lots of papers where C is much better than A or A or B.

Juan Benet

Yeah, yeah, yeah. I think a lot of us are very optimistic about applying AI to a ton of these methods, and accelerating science quite a bit. You also have been thinking about the economic impacts of AI, and you recently co-wrote a paper with your wife, Ioana Marinescu. Yeah. Which, by the way, this super awesome.

The two of you came together and built this amazing and combines a lot of your thinking, both in how intelligence works and how it gets applied to knowledge work and beyond, and then how economic systems work. What are some of the interesting insights or consequences the two of you think this will produce?

Konrad Kording

Yeah. So this paper comes kind of from a contrast of two very different ways of thinking. Like if you if you look at the AI worlds, you’re seeing that the price for equivalent tokens. Yep. Exponentially gets cheaper. It’s like a crazy you one year or sub one year time scale. So that is one way of looking at the world.

And then you can look at the world from a very different perspective, say the GDP of a country. And then you’re like, yeah, we’re like growing and like it takes us 15 years or something to double. These two frameworks are just so very different. And we started talking about what would happen when those two meet.

And let’s talk about how the two of them meet. So imagine you have a task a real task. You want to like take dot out of one place and bring it to another place. For that, you need intelligence and you need physical things. You need an excavator, and you need someone who sits there and controls the excavator.

And if intelligence is very bad, then the physical machine that you have, the excavator will not be used well, like you’ll be wasting lots of time digging or the excavator gets destroyed. Now the question is, what happens if we now saturate intelligence? We add intelligence and we add even more intelligence until we have unlimited amounts of intelligence.

What’s going to happen? At some level, this machine here is going to be upper bounded by its physical properties in how much Earth it could move. At some level it’s physically bounded. There’s a sudden there’s a combustion motor in it and like it cannot move more than a certain amount of Earth power, energy that it burns.

So in this task of like having to move Earth, you need a physical piece and you need an intelligence piece. Economics has a history of modeling such phenomena in that classical framework that people use for things where you need both is labor and capital. Not like we know from ancient econ literature that you can say we if we want to run a factory, we need a bunch of workers there, and we need a bunch of like, physical goods in the company traditionally capital.

And now how is the interplay between them going to be? There’s a certain amount of substitute ability. And like imagine people are very expensive and excavators are pretty cheap. Then let’s build one mega excavator with just one person controlling it. Instead of building 100 excavators that are small with like one person each.

At the same time, imagine that people get to be like super cheap. Let’s hire a lot of people with shovels. Shovels are very cheap. And this tradeoff between capital and labor. The economist model that with an idea of called constant elasticity of substitution CS which basically the idea is that there’s a certain friction to like moving from one side to the other.

So, so basically replacing the fast human with an excavator is very cheap, replacing the hundreds or A human with an excavator is very expensive. Representing all humans is impossible depending on the setting of the parameters of that substitution dynamic. But for a lot of parameters of it, it’s basically impossible, I guess more and more and more expensive as we go in that direction.

When we then talked about physical and intelligence capital, it feels like it has a certain similar property now. Like if I build AI into like that excavator, like there’s a certain limit to it. And so you can say what we what we then did is we said, well, let’s, let’s think of the world as four pieces. They exist physical, they exist capital, labor.

Think about excavators. They exist like human, a human physical, which is think about humans with shovels, but also think about like human doctors who have hands. Think of human nurses that have hands. Think of teachers that have a face that can frown. Like. Like anything that has the body. We call it physical and anything does it.

That doesn’t require a body or anything physical we call intelligence. Now the idea is that in production you can say there is a fixed like physical capital think excavators and there’s like a human physical. Think of human bodies broadly construed. And there’s a human intelligence and there’s an artificial intelligence.

And then what happens if, like we substitute in those areas? And now the question is how can we substitute like physical with intelligence? And the idea is very much not much because like, look, yeah, we can have robots, but the robot is the physical piece of it. And it turns out also and this something that I think the AI community often goes wrong.

Robot scaling isn’t intelligent scaling. It doesn’t have every eight months. Robot scaling is extremely slow and we built millions of industrial robots. And robots are just capital wise, very expensive. And the reason being, like there’s this long supply chain that we need to build to build robots.

Can we do it cheaper? Yes. Can we do anything in this universe cheaper? Yes. Does it take time and effort and all that also? Yes. On the intelligence side, the scaling is very different. And there’s even people who say, well, like one hour now like that, we all use Claude Code. And now that, like all the frontier labs use Claude Code, they will get better at building intelligence.

That maybe or maybe that’s not the case. I’m not sure you could make both arguments. You could say the easy things are being done fast, and so maybe we’re still at the face where we do the easy things that we can do with internet scale, compute, compute. But maybe that’s not. And we don’t know. And that’s why we want like a modeling that like brackets that out of the rest.

And now you can say how can we replace physical intelligence? And in most cases it’s very unclear how you could do that, like build a house,

Put more intelligence in it. How much faster is that how it’s going to build itself? Very much, not at all. Build a better straight. How much is intelligence going to help? Not at all. If you want to build bigger robot factories, you need to build streets and street scale like streets. They don’t scale like AI.

So you can say no. And people are like, yeah, but I could build like a $100,000 robot and put like super intelligence into it. Yes, if you want the robot that’s already there. But if intelligence was like went to infinity, all of a sudden that $100,000 robot would be a lot more valuable. In any case, the idea is that replacing physical fund agents is crazy complicated.

Juan Benet

The reason is kind of like until the scale up of the robotic supply chain can catch up to the demand, like, like what you’re kind of getting at is like, there’s going to be this process of sure, intelligence costs might go to zero, but that just increases the value of a whole bunch of places where you’re applying human work, which includes both physical and intelligence.

And then eventually, once the robotics production kind of catches up.

Konrad Kording

Yeah, but it’s not just the like. Like, robots are not unique in the way that AI people think. Robots are unique. Like a factory is a robot. If you think about it like it has a it has a ways of automating much of it. We’ve been automating factories like for 100 plus years. Yeah. And like they get better every year.

They’re more efficient. But it’s not that there’s super dramatic, unrealized possibility.

Juan Benet

There’s always like this very long tail of tasks that are just incredibly difficult to automate, that keep rising in price. Right. So like you keep automating and you keep finding the next constraint and.

Konrad Kording

That’s that’s right. And not only that, but also for a lot of places like robots aren’t even substitutable. You cannot replace a teacher with a robot because there’s something about them being human that makes them be attractive as a teacher. And let’s just say if you look at big tech CEOs, they don’t have like their kids, be robots.

Zoom taught. They have them be in very high intensity, human taught environments. And then you can say, what if we now only look at the intelligence production? do we find this phenomenon that basically humans get replaced massively with AI? That is entirely unclear within that sector, because you can say even within it now, how replaceable are there?

It turns out that least so far, it looks that AI doesn’t replace the human programmer. Maybe it replaces the entry level person, but the people who are good at working with AI, they’re just so much more productive now. So therefore, it’s unclear that the existence of AI kind of like moves all the humans out of it.

And there’s precious little evidence that the tech CEOs are actually being replaced by robots for the time being. And so but the key is because of the existence of this physical sector, if the intelligence thing would go to infinity, if the price of any intelligence would go to zero, the economy still wouldn’t go to super productive.

Sure. What do I know? GDP might double, but it wouldn’t like 100 fold. It wouldn’t tenfold. It would be like probably a medium size change. And it very much depends on your assumptions about how close to IQ tapped out we already are. But just give me the real world tasks. More intelligence would actually like produce mono like your car factories are already full of and robot factors.

They are already full of robots. It’s not that they don’t use robots, it’s just that yeah, they need the humans for the outlier tasks, both intellectually and physically.

Juan Benet

But it does suggest that there is some threshold after which you can get the recursive self-improvement of the robotics factory’s going well enough, and there is a very long tail there that I think everyone in the AI community is maybe underappreciated. But at some point, whether this might be, you know, certainly not in the next 5 or 10 years, but potentially after you can get production so well automated that you can then create like a much faster production of physically intelligent capital.

Konrad Kording

Yeah, but that is not a new phenomenon. You can say since the time of the Romans, we had exponential growth, and exponential growth has been in what, you know, like the 7% range for quite a bit. And now imagine it goes up to ten. Imagine it goes up to 15, which would be unbelievable. It would still take a very long time.

And like the Romans, if there would plug exponential curves, would be like, oh my God, in 200 years every peasant has unlimited food. and knowing that kind of ended up building a big civilization with all kinds of other things. And so I think, like exponential move, there’s no reason that it will stop. And I think like there’s the degrowth movement, which I think is an insanely bad idea.

But basically there’s going to be ongoing growth and we will find new things to spend that wonderful growth on, and it will make our lives much cooler.

Juan Benet

This will be like a super interesting area to see and measure over the next few years. I think it’ll be like a very big feature of the conversation.

Konrad Kording

Absolutely. And what in the context of the paper with my wife is really interesting, is if we see kind of the canary in the coal mine thing, there’s no signs at all for the substitution of physical for intelligence, because that’s very, very difficult. But even within the intelligence sector, like it’s not that we see a strong job market drift where this happening now, you can say a lot of it in a lot of our simulations there.

There will be an early period where if you want increased productivity, kind of drowns out any effect of substituting people. But even there, if you want like the upshot is we are kind of somewhat safe because there’s so many jobs that require bodies, and we could have a lot more of those jobs, like we could do a much better job at teaching kids, for example.

Like if we had like extra human beings that are jobless, then we probably don’t want classes of size 40 to 1 human teacher, which we could probably one on, one on one teaching. And so I personally, my take is that we’re rather moving into a very positive world where. So for example, take all that software engineering software is horrible by and large, just like software is just the worst thing ever.

And therefore you could say in the future is it likely that people have like expectations of quality for software? Yes. And that would be a good thing. And so it will take a lot of people to get that to happen.

Juan Benet

Very good. Optimistic view on the moment because it will just kind of we’ll be able to reap the rewards of all the growth along the way. The economic transition is not going to be as rocky as everybody currently fears it will be. And so yeah.

Konrad Kording

Right. Yeah. And there’s also this phenomenon though, people are like, yes, if we build robots and we invest all proceeds of building robots into building more robots, then wouldn’t it be like so fast? But that’s not how the real world works in the real world, if your robot factor is making wonderful profits, you’ll spend it on like a heli skiing trip in Alaska or whatever else you spend part of your money on.

And so the economy tends to, like, not reinvest all proceeds. It tends to reinvest 20% or something of it. And there’s no reason to believe that AI will all of a sudden change that. If people have money, they spend money on doing the things that they believe makes the world better from their perspective.

And that could be trips or yachts or like investing in neuroscience progress.

Juan Benet

If you were to kind of give a, you know, an optimistic vision for the future, you know, I don’t know, ten, 20, even 40 years out. Like, what do you find like very exciting, inspiring cool out there? Like what? what do you dream about?

Konrad Kording

Well, there’s lots of layers. I think I do dream about understanding brains to the level that we can simulate them. I think the idea that simulations are too far away. Well, that simulations have been too far away for 70 years. And we have, like, so much better techniques. And the idea that simulations must always be a far future dream should go away.

We should start simulating brains. I do dream of a world where AI is used, by and large, in a positive, constructive way that helps people live better lives. And I in that sense, I. I dream of a world where AI seen less as a competitor with humans, but because I think it’s not, and more just like something providing something of value in the same way that a machine, a car is providing value.

And in general, I dream of a world where we use technology to solve a lot of mankind’s problems. And I believe that as engineers, we’ve always been doing it, and I think we will continue doing it. And AI is just part of that.

Juan Benet

That’s wonderful. Well, thank you so much for coming to chat with us today. Thank you for great time.

Konrad Kording

Thanks for having.

Juan Benet

Me. Thank you. I hope you enjoyed this episode. This a new podcast, so we need your help to get the word out. Please like, rate and subscribe on your favorite platform and share it with people you think would find it interesting. Thank you. See you next time.

Displaying _FULL__Juan_Benet_Podcast_-_EP05_-_V04__edited__formatted_.txt.

Discussion about this video

User's avatar

Ready for more?

© 2026 Juan Benet · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture