YouSaid · the spoken record

Elon Musk

lines on the record: 134
first: 2024-08-02
most recent: 2024-08-02
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“And maybe Said something that I think is worth exploring there a little bit. You said it's primarily a UX challenge. And I think a large component of it is, but there is also a very interesting machine learning challenge here, which is given some data set including some on average correct behavior of asking the user to move up or move down, move right, move left. And given a data set of neural spikes, is there a way to infer in some kind of semi-supervised or entirely unsupervised way what that high resolution version of their intention is? And if you think about it, there probably is because there are enough data points in the data set, enough constraints on your model, that there should be a way with the right sort of formulation to let the model figure out itself, for example, at this millisecond, this is exactly how hard they're pushing upwards. And at this millisecond, this is how hard they're trying to push upwards.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“So, if in the ideal world you could get a signal of behavioral intent that is ground truth accurate at the scale of sort of one millisecond resolution, then with high confidence I could build a mapping from my neural spikes to that behavioral intention. But the challenge is, again, that you don't observe what they're actually doing. And so there's a lot of nuance to how you build user experiences that give you more than just sort of a course on average correct representation of what the user is intending to do. If you want to build the world's best mouse, you really want it to be as responsive as possible. You want it to be able to do exactly what the user is intending at every sort of step along the way, not just on average be correct when you're trying to move it from left to right. And building a behavioral sort of calibration game or sort of software experience that gives you that level of resolution is what we spend a lot of time working on.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“And that imagined behavior map one to the other. And then at test time, you take that same pattern matching system. In our case, it's a deep neural network, and you run it and you take the live stream of brain data coming off their implant, you decode it by pattern matching to what you saw at calibration time, and you use that for a control of the computer. Now, a couple sort of rabbit holes that I think are quite interesting. One of them has to do with how you build that best template matching system because there's a variety of behavioral challenges and also debugging challenges when you're working with someone who's paralyzed. Because again, fundamentally, you don't observe what they're trying to do. You can't see them attempt to move their hand. And so you have to figure out a way to instruct the user to do something and validate that they're doing it correctly such that then you can downstream build with confidence the mapping between the neural spikes and the intended action. And by doing the action correctly, what I really mean is at the level of resolution of what neurons are doing.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“This key down. And so that mapping is fundamentally what the app is responsible for. But there's a lot of nuance of how that mapping works that we spend a lot of time to try to get right and we're still in the early stages of a long journey to figure out how to do that optimally. So one part of that process is decoding. So decoding is this process of taking the statistical patterns of brain data that's being channeled across this Bluetooth connection to the application and turning it into, for example, a mouse movement. And that decoding step, you can think of it in a couple different parts. similar to any machine learning problem, there's a training step and there's an inference step. The training step in our case is very intricate behavioral process where the user has to imagine doing different actions. So for example, they'll be presented a screen with a cursor on it and they'll be asked to push that cursor to the right. Then imagine pushing that cursor to the left. Push it up, push it down. And we can basically build up a pattern or using any sort of modern ML method of mapping of given this brain data.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so maybe working backwards from the goal. The goal is to help someone with paralysis, in this case Noland, be able to navigate his computer independently. And we think the best way to do that is to offer them the same tools that we have to navigate our software because we don't want to have to rebuild entire software ecosystem for the brain, at least not yet. Maybe someday you can imagine there's UXs that are built natively for BCI. But in terms of what's useful for people today, I think most people would prefer to be able to just control mouse and keyboard inputs to all the applications that they want to use for their daily jobs, for communicating with their friends, et cetera. And so the job of the application is really to translate this wireless stream of brain data coming off the implant into control of the computer. And we do that by essentially building a mapping from brain activity to sort of the HID inputs to the actual hardware. So HID is just the protocol for communicating like input device events. So for example, move mouse to this position or press.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. So maybe first, what is decoding? I think there's probably a lot of folks listening that just have no clue what it means to decode brand activity.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“You need the screen, like 120 hertz just doesn't work anymore if you're trying to have something respond at something that's at the level of one millise”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, a couple things. So, kind of hilariously, Bluetooth low energy protocol has some restrictions on how fast you can communicate. So the protocol itself establishes a standard of the most frequent sort of updates you can send are on the order of 7.5 milliseconds. And as we push latency down to the level of sort of individual spikes impacting control, that level of resolution, that kind of protocol is going to become a limiting factor at some scale. Another sort of important nuance to this is that it's not just the Nerling itself that's part of this equation. If you start pushing latency sort of below the level of how fast screens refresh, then you have another problem. You need your whole system to be able to be as reactive as the sort of limits of what the technology can offer.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“The brain to actually impact your command of your hand is about 75 milliseconds. So if you look at those numbers, you can see that we're already competitive and slightly faster than what you'd get by actually moving your hand. And this is something that if you ask Nolan about it, when he moved the cursor for the first time, we asked him about this. It was something I was super curious about. What does it feel like when you're modulating a click intention or when you're trying to just move the cursor to the right? He said it moves before he is actually intending it to, which is kind of a surreal thing. It's something that I would love to experience myself one day. What is that? I like to have the thing just be so immediate, so fluid that it feels like it's happening before.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Definitely the case that you have a higher ceiling of performance Because you don't have to buffer your intention through your arm, through your muscle, you get just by nature of having a brain implant at all, like 75 millisecond lead time on any action that you're actually trying to take. And there's some nuance to this. Like there's evidence that the motor cortex, you can sort of plan out sequences of actions. So you may not get that whole benefit all the time. But for sort of like reaction time style games where you just want to, somebody's over here, snipe them, you know, that kind of thing. You actually do have just an inherent advantage because you don't need to go through muscle. So the question is just how much faster can you make in? And we're already faster than what you would do if you're going through muscle from a latency point of view. And we're in the early stages of that. I think we can push it sort of our end-to-end latency right now from brain spike to cursor movement. It's about 22 milliseconds. If you think about the best mice in the world, the best gaming mice, that's about five milliseconds-ish of latency, depending on how you measure, depending on how fast your screen refreshes. There's a lot of characteristics that matter there. But yeah, and the rough time for like a neuron.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, this is a passion project of mine. So I want to build the best mouse in the world. Don't want to build like the Chevrolet Spark or whatever of electric cars. I want to build the Tesla Roadster version of a mouse. And I really do think it's quite possible that within five to ten years that most esports competitions are dominated by people with paralysis. This is like a very real possibility for a number of reasons. One is that they'll have access to the best technology to play video games effectively. The second is they have the time to do so. So those two factors together are particularly potent for esport competitors.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“And you can look at sort of across many channels how that sort of background noise is behaving, and you might be able to get more juice out of the signal that way. But it comes at a cost. That signal is now a floating point representation, which means it's more expensive to send out over a power. It means you have to find different ways to compress it that are different than what you can apply to binary signals. So there's a lot of different challenges associated with these different modalities.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“That extremely efficiently in hardware, which means that you can run it in a low power across 1,024 channels all at once. Another approach that we've recently started exploring, and this can be combined with the Spark detection approach, is something called Spike Band Power. And the benefits of that approach are that you may be able to pick up some signal from neurons that are maybe too far away to be detected as a spike, because the farther away you are from an electrode, the weaker that actual spike waveform will look like on that electrode. You might be able to pick up population level activity of things that are maybe slightly outside the normal recording radius, what neuroscientists sometimes refer to as the hash of activity, the other stuff that's going on.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. So we've tried many different versions of basically turning this raw signal into sort of a feature that you might want to send off the device. And I'll say that I don't think we're at the final step of this process. This is a long journey. We have something that works clearly today, but there can be many approaches that we find in the future that are much better than what we do right now. So some versions of what we do right now, and there's a lot of academic heritage to these ideas. So I don't want to claim that these are original Nerlink ideas or anything like that. But one of these ideas is basically to build sort of like a convolutional filter almost, if you will, that slides across the signal and looks for a certain template to be matched. That template consists of sort of how deep the spike modulates, how much it recovers, and what the duration and window of time is that the whole process takes. And if you can see the signal that that template is matched within certain bounds, then you can say, okay, that's a spike. One reason that approach is super convenient is that you can actually implement”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“And for many applications, that's a totally reasonable trade-off to make and can simplify the problem a lot. And as you sort of scale out channel count, the relevance of distinguishing individual neurons becomes less important because you have more overall signal and you can start to rely on sort of correlations or covariance structure in the data to help understand when that channel is firing, what does that actually represent? Because you know that when that channel is firing in concert with these other 50 channels, that means move left. But when that same channel is firing with concert with these other 10 channels, that means move right.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so there's a whole field of sort of academic neuroscience work on exactly this problem of basically given a single electrode or given a set of electrodes measuring a set of neurons, how can you sort of sort, spike sort which spikes are coming from what neuron? And this is a problem that's pursued in academic work because you care about it for understanding what's going on in the underlying sort of neuroscience of the brain. If you care about understanding how the brain is representing information, how that's evolving through time, then that's a very, very important question to understand. For sort of the engineering side of things, at least at the current scale, if the number of neurons per electrode is relatively small, you can get away with basically ignoring that problem completely. You can think of it like sort of a random projection of neurons to electrodes. And there may be in some cases more than one neuron per electrode. But if that number is small enough, those signals can be thought of as sort of a union of the two.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Spike trains, meaning how often are spikes firing in any given window of time. And so that allows us to do sort of a crazy amount of compression from this very rich high density signal to something that's much, much more sparse and compressible that can be sent out over a wireless radio, like a Bluetooth communication, for example.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“And so we sample across all 1024 electrodes about 20,000 times a second, 20,000 times a second means for any given one millisecond window, we have about 20 samples that tell us what that exact shape of that action potential looks like. And once we've sort of sampled at super high rate the underlying electrical field nearby these cells, we can process that signal into just where do we detect a spike or where do we not. sort of a binary signal one or zero do we detect a spike in this one millisecond or not and we do that because the actual information carrying sort of subspace of neural activity is just when our spikes occurring essentially everything that we care about for decoding can be captured represented in the frequency”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Sample that signal, or sample the local field potential nearby that neuron much more frequently than once a millisecond. You need to sample many, many times per millisecond to be able to detect that this is actually the characteristic waveform of a neuron producing an action potential.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“So maybe before diving into what we do, it's worth understanding what we're trying to measure because that dictates a lot of the requirements for the system that we build. And what we're trying to measure is really individual neurons producing action potentials. And action potential is, you can think of it like a little electrical impulse that you can detect if you're close enough. And by being close enough, I mean like within, let's say 100 microns of that cell. And 100 microns is a very, very tiny distance. And so the number of neurons that you're going to pick up with any given electrode is just a small radius around that electrode. And the other thing worth understanding about the underlying biology here is that when neurons produce an action potential, the width of that action potential is about one millisecond. So from the start of the spike to the end of the spike, that whole width of that sort of characteristic feature of a neuron firing is one millisecond wide. And if you want to detect that an individual spike is occurring or not, you need to”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and I think the UX design component here is underrated for BCI development in general. There's a whole interaction effect between the ways in which you visualize an instruction to the user and the kinds of signal you can get back. And that quality of sort of your behavioral alignment to the neural signal is a function of how good you are at expressing to the user what you want them to do. And so, yeah, we spend a lot of time thinking about the UX of how we build our applications, of how the decoder actually functions, the control surfaces it provides to the user, all these little details matter a lot.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“That is actually representing. And so he played with it for a bit. He was like, I don't quite get it yet. He played for it a bit longer. And they said, oh, when I move this finger, I see this particular Duran start to fire more. And I said, okay, prove it. Do it again. And so he said, okay, three, two, one. Boom. And the minute he moved, you can see like instantaneously this neuron is firing. Single neuron, I can tell you the exact channel number if you're interested. It's stuck in my brain now forever, but that single channel firing was a beautiful indication that it was behaviorally modulated neural activity that could then be used for downstream tasks like decoding, a computer cursor.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Body mapping is where you essentially present a visual to the user and you say, Hey, imagine doing this, and that visual is a 3D hand opening, closing, or our index finger modulating up and down. And you ask the user to imagine that. And obviously, you can't see them do this because they're paralyzed. So you can't see them actually move their arm. But while they do this task, you can record neural activity. And you can basically offline model and check, can I predict or can I detect the modulation corresponding with those different actions? And so we did that task and we realized, hey, there's actually some modulation associated with some of his hand motion, which is a first indication that, okay, we can potentially use that modulation to do useful things in the world. For example, control a computer cursor. And he started playing with it the first time we showed him it. And we actually just took the same live view of his brain activity and put it in front of him. And we said, hey, tell us what's going on. You know, we're not you. You're able to imagine different things. And we know that it's modulating some of these neurons. So you figure out for us what.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah. So we are quite excited to move as quickly as we can. And Noland was really, really excited to get started. He wanted to get started actually the day of surgery. But we waited till the next morning very patiently. It's a long night. And the next morning in the ICU where he was recovering, he wanted to get started and actually start to understand what kind of signal we can measure from his brain. And maybe for folks who are not familiar with the Nerling system, we implant the Nerling system or the Nerlink implant in the motor cortex. So the motor cortex is responsible for representing things like motor intent. So if you imagine closing and opening your hand, that kind of signal representation would be present in the motor cortex. If you imagine moving your arm back and forth or wiggling a pinky, this sort of signal can be present in the motor cortex. So one of the ways we start to sort of map out what kind of signal do we actually have access to in any particular individual's brain is through this task called body mapping.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“That's a good question. My answer is going to be pretty lame here, but it was boring. Seen it so many times. Yeah. That's exactly how you want surgery to be. You want it to be boring. I've seen it so many times. I've seen the robot do the surgery literally hundreds of times. And so it was just one more time”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“During that procedure, there was actually a lot of cameramen in the room, so they also were curious and wanted to see. There's several neurosurgeons in the room who are all just excited to see robots taking their job. Crowded around a small little iPhone watching this live brain data stream out of his”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, so actually in the operating room right after we sort of finished all the thread insertions, I started collecting what's called broadband data. So broadband is basically the most raw form of signal you can collect from a Neuralink electrode. It's essentially a measurement of the local field potential or the voltage essentially measured by that electrode. And we have a certain mode in our application that allows us to visualize where detected spikes are. So it visualizes sort of where in the broadband it's very, very raw form of the data. A neuron is actually spiking. And so one of these moments that I'll never forget as part of this whole clinical trial is seeing live in the operating room while he's still under anesthesia beautiful spikes being shown in the application, just streaming live to a device I'm holding in my hand.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“The room, I got to sit there and think to myself, wow, you know, that brain is moving a lot. When you look inside little craniectomy that we stick the threads in, one thing that most people don't realize is the brain moves. The brain moves a lot when you breathe, when your heart beats, and you can see it visibly. So that's something that I think was a surprise to me and very, very exciting to be able to see someone's brain who you physically know and have talked with at length actually pulsing and moving inside their skull.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, I think for me, this is something I've been excited about for close to 10 years now. And so to be able to be even just some small part of making it a reality is extremely exciting. A couple Be special moments during that whole process that I'll never really truly forget. One of them is during the actual surgery, you know, at that point in time, I know Nolan quite well. I know his family. And so I think the initial reaction when Nolan is rolled into the operating room is just, oh shit kind of reaction. But at that point, muscle memory kicks in and you sort of go into body just do all the talking. And I have the lucky job in that particular procedure to just be in charge of monitoring the implant. So my job is to sit there, to look at the signals coming off the implant, to look at the live brain data, streaming off the device as threads are being inserted into the brain. And just to basically observe and make sure that nothing is going wrong or that there's no red flags or fault conditions that we need to go and investigate or pause the surgery to debug. And because I had that sort of spectator view of the surgery, I had a slightly removed perspective than I think most folks.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Or who don't know somebody with ALS, it's not often obvious why you would want a brain implant to be able to connect and navigate a computer. And it's surprisingly nuanced to the degree that I've learned a huge amount just working with Noland in the first Nerling clinical trial and understanding from him in his words why this device is impactful for him. And it's a nuanced topic. It can be the case that even if you can achieve the same thing, for example, with a mouse stick when navigating a computer, he doesn't have access to that mouse stick every single minute of the day. He only has access when someone is available to put it in front of him. And so a BCI can really offer a level of independence and autonomy that if it wasn't literally physically part of your body, it'd be hard to achieve in any other way.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Yeah, and I see I'll take a broad view sort of lens on this for a second. I think I'm very in favor of anyone working in this problem space. So beyond PCI, I'm happy and excited and willing to support any way I can folks working on eye tracking systems, working on speech-to-text systems, working on head trackers or mouse sticks or quad sticks. And I haven't met many engineers and folks in the community that do exactly those things. And I think for the people we're trying to help, it doesn't matter what the complexity of the solution is as long as the problem is solved. And I want to emphasize that there can be many solutions out there that can help with these problems. And BCI is one of a collection of such solutions. So BCI in particular, I think, offers several advantages here. And I think the folks that recognize this immediately are usually the people who have spinocord injury or some form of paralysis. Usually you don't have to explain to them why this might be something that could be helpful. It's usually pretty self-evident. But for the rest of us folks that don't live with severe spinal cord injury.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“And these are deeply personal and very human problems. And what strikes me again and again when talking with these folks is that this is actually an engineering problem. This is a problem that with the right resources, with the right team, we can make a lot of progress on. And at the end of the day, I think that's a deeply inspiring message and something that makes me excited to get up every day.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“Some folks can mean independents to be able to work again, to be able to navigate a computer digitally, efficiently enough, to be able to get a job, to be able to support themselves, to be able to move out and ultimately be able to support themselves after their family maybe isn't there anymore to take care of them. And for some folks, it's as simple as just being able to respond to their kid in time before they run away or get interested in something else.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source
“First, just a thank you to all the people I've gotten a chance to speak with for sharing their stories with me. I don't think there's any world really in which I can share their stories as powerful ways they can. But just I think to summarize at a very high level, what I hear over and over again is that people with ALS or severe Supreme Cord injury in a place where they basically can't move physically anymore. Really, at the end of the day, are looking for independence, and that can mean different things for different people. For some folks, it can mean the ability just to be able to communicate again independently without needing to wear something on their face, without needing a caretaker to be able to put something in their mouth.”
2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source