YouSaid · the spoken record

Elon Musk

lines on the record
134
first
2024-08-02
most recent
2024-08-02
sittings or episodes
1
sources
podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

  1. “Right, right. So that is one drawback of the current approach. One other sort of case study here. So again, UX is how it works. And we think about that holistically from even the future detection level of what we detect in the brain to how we design the decoder, what we choose to decode, to then how it works once it's being used by the user. So another good example in sort of how it works once they're actually using the decoder, the output that's displayed on the screen is not just what the decoder says. It's also a function of what's going on on the screen. So we can understand, for example, that when you're trying to close a tab, that very small, stupid little X that's extremely tiny, which is hard to get precisely hit if you're dealing with sort of a noisy output of the decoder, we can understand that that is a small X you might be trying to hit and actually make it a bigger target for you, similar to how when you're typing on your phone, if you're used to like the iOS keyboard, for example, it actually adapts the target size of individual keys based on an underlying language model. So it'll actually understand if I'm typing, hey, I'm going to see”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  2. “The same is true with PCI scroll. So we had to spend some time to figure out what are the right nuances when you don't feel the screen under your fingertip anymore. What is the right sort of dynamic or what's the right amount of page give, if you will, when you push it to make it flow the right amount for the user to have a natural experience reading their book? And there's a million, I mean, I could tell you like there's so many little minutiae of how exactly that scroll works that we spent probably like a month getting right to make that feel extremely natural and easy for the user to navigate.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  3. “And the reason that feels so natural and intuitive is that when you move over to attach to it, it feels like magnetic. So you're sort of stuck onto it. And then it's one continuous action. You don't have to like switch your imagined movement. You sort of snap onto it and then you're good to go. You just immediately can start pulling the page down or pushing it up. And even once you get that right, there's so many little nuances of how this cruel behavior works to make it natural and intuitive. So one example is momentum. Like when you scroll a page with your fingers on the screen, you actually have some like flow. Like it doesn't just stop right when you lift your finger up.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  4. “And so we ended up building is this really brilliant feature. This is teammate named Bruce, who worked on this really brilliant work called Quickscroll. And QuickScroll basically looks at the screen and it identifies where on the screen are scroll bars. And it does this by deeply integrating with macOS to understand where are the scroll bars actively present on the screen using the sort of accessibility tree that's available to Mac OS apps. And we identified where those scrollbars are and we provided a BCI scrollbar. And the BCI scrollbar looks similar to a normal scroll bar, but it behaves very differently in that once you sort of move over to it, your cursor sort of morphs onto it. It sort of attaches or latches onto it. And then once you push up or down in the same way that you'd use a push to control the normal cursor, it actually moves the screen for you. So it's basically like remapping the velocity to a scroll action.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  5. “To read. And there's many ways to do scroll with a BCI. You can imagine different gestures, for example, the user could do that would move the page. But Scroll is a very fascinating control surface because it's a huge thing on the screen in front of you. So any sort of jitter in the model output, any sort of error in the model output causes like an earthquake on the screen. You really don't want to have your manga page that you're trying to read be shifted up and down a few pixels just because your scroll decoder is not completely accurate. And so this was an example where we had to figure out how”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  6. “Yeah. And maybe it's also worth calling out there are other alternative resistive technologies, but the particular situation Nolan's in, and this is not uncommon. And I think it's also not well understood by folks, is that he's relatively spastic, so he'll have muscle spasms from time to time. And so any assistive technology that requires him to be positioned directly in front of a camera, for example, an eye tracker or anything that requires him to put something in his mouth just as a no-go because he'll either be shifted out of frame when he has a spasm or if he has something in his mouth, it'll stab him in the face. If he spasms too hard. So these kind of considerations are important when thinking about what advantages a PCI has in someone's life. If it fits ergonomically into your life in a way that you can use it independently when your caretaker is not there, wherever you want to, either in the bed or in the chair, depending on your comfort level and your desire to have pressure sores, all these factors matter a lot in how good the solution is in that user's life. So one of these very fun examples is scroll. So again, Menga is something he wanted to be.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  7. “Right, yeah, it's basically, you can imagine it's a stylus that you hold between your teeth. It's basically a very long stylist.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  8. “Yeah, I mean, I'll give one concrete example. So he really wanted to be able to read manga. This is something that, I mean, It sounds like a simple thing, but it's actually a really big deal for him, and he couldn't do it with this mouse stick. It just wasn't accessible. You can't scroll with a mouse stick on his iPad on the website that he wanted to be able to use to read the newest manga. And”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  9. “UX is not something that you can always solve by just constant iterating on different things. Like sometimes you really need to step back and think globally, am I even in the right sort of minima to be chasing down for a solution? Like there's a lot of problems in which sort of fast iteration cycle is the predictor of how successful you will be. As a good example, like in an RL simulation, for example, the more frequently you get a reward, the faster you can progress. It's just an easier learning problem the more frequently you get feedback. But UX is not that way. I mean, users are actually quite often wrong about what the right solution is. And it requires a deep understanding of the technical system and what's possible combined with what the problem is you're trying to solve. Not just how the user expressed it, but what the true underlying problem is to actually get to the right place.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  10. “It also requires just to brag on the team a little bit. I work with a lot of exceptional people, and it requires the team being absolutely laser focused on the user and what will be the best for them. And it requires a level of commitment of, okay, this is what the user feedback was. I have all these meetings. We're going to skip that today and we're going to do this. That level of focus commitment is, I would say underappreciated in the world. And also, you know, you obviously have to have the talent to be able to execute on these things effectively. And yeah, we have that in loads.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  11. “And so that can be hundreds of different models that we would try in that day, like a lot of different things. Now, it's also worth noting that we update the application he uses quite frequently. I think sometimes up to like four or five times a day, we'll update his application with different features or bug fixes or feedback that he's given us. So he's been able to, he's a very articulate person who is part of the solution. He's not a complaining person. He says, hey, here's this thing that I've discovered is not optimal in my flow. Here's some ideas how to fix it. Let me know what your thoughts are. Let's figure out how to solve it. And it often happens that those things are addressed within a couple hours of him giving us this feedback. That's the kind of iteration cycle we'll have. And so sometimes at the beginning of the session, he'll give us feedback. And at the end of the session, he's giving us feedback on the next iteration of that process or that setup.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  12. “Yeah, a couple comments. So, one is probably worth trying to distinction between sort of research sessions where we're actively trying different things to understand what the best approach is versus sort of independent use where we wanted to have the ability to just go use a device, how anybody would want to use their MacBook. So what he's referring to is, I think, usually in the context of a research session where we're trying many, many different approaches to even unsupervised approaches like we talked about earlier to try to come up with better ways to estimate his true intention and more accurately decode it. And in those scenarios, I mean, we try in any given session, he'll sometimes work for like eight hours a day.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  13. “Just to comment on that too, we've probably tried like a thousand different ways to do various aspects. And so even just that, like help constraints sort of the beam search of different approaches that we could explore really helped accelerate for the next person, the set of things that we'll get to try on day one, how fast we hope to get them to useful control, how fast we can enable them to use it independently and to get value out of the system. So yeah, massive hats off to Noland and all the participants that came before him to make this technology a reality.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  14. “Yeah, the truth is, I don't know. I'm very excited to see with sort of the second participant that we implant what the journey is like for them, because we'll have learned a lot more. Potentially we can help them understand and explore that direction more quickly. This is something I didn't, you know, this wasn't me prompting Nolan to go try this. He was just exploring how to use his device and figured it out himself. But now that we know that that's a possibility, that maybe there's a way to, you know, for example, hint the user, don't try super hard during calibration. Just do something that feels natural or just directly control the cursor. Don't imagine explicit action. And from there, we should be able to hopefully understand how this is for somebody who has not experienced that before. Maybe that's the default mode of operation for them. You don't have to go through this intermediate phase of explicit motions.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  15. “Right, UX is how it works, and the ideal UX is one that the user doesn't have to think about what they need to do in order to get it done. It just doesn't.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  16. “And I cannot tell you what the difference between those two things are. I really truly cannot. He's tried to explain it to me before. I cannot give a first-person account of what that's like. But the exploitives that he uttered in that moment were enough to suggest that it was a very qualitatively different experience for him to just have direct neural control over a cursor.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  17. “So, this happened on a Tuesday. I remember this day very clearly because at some point during the day, it looked like he wasn't doing super well. It looked like the model wasn't performing super well. And he was like getting distracted. But he actually wasn't the case. What actually happened was he was trying something new where he was just controlling the cursor. So he wasn't imagining moving his hand anymore. He was just imagining, I don't know what it is, some like abstract intention to move the cursor on the screen.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  18. “Yes, yes, that's one way to do it. Maybe one nuance of when he's doing it, he can imagine many more things than we represent in that visual on the screen. So we show him sort of abstractly, here's a cursor, you figure out what works the best for you. And we obviously have hints about what will work best from that body mapping procedure. We know that this particular action we can represent well, but it's really up to him to go and explore and figure out what works the best.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  19. “He tried a whole bunch of stuff to explore the space of what is the most natural way for him to control the cursor that at the same time is easy for us to decode roll.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  20. “On his side and our side to figure out what's the most intuitive control for him. And the most intuitive control for him is sort of you have to find the set intersection of what do we have the signal to decode so we don't pick up every single neuron in the motor cortex, which means we don't have representation for every part of the body. So there may be some signals that we have better sort of decode performance on than others. For example, on his left hand, we have a lot of difficulty distinguishing his left ring finger from his left middle finger. But on his right hand, we have a good control and good modulation detected from the neurons we're able to record for his pinky and his thumb and his index finger. So you can imagine how these different subspaces of modulated activity intersect with what's the most intuitive for him. And this has evolved over time. Once we give him the ability to calibrate models on his own, he was able to go and explore various different ways to imagine and controlling the cursor. For example, he could imagine controlling the cursor by wiggling his wrists side to side or by moving his entire arm by one point to his feet.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  21. “And this is not us asking him to do that. I want to be clear. We're not saying, hey, you should play WebGrid tonight. We just gave him the game as part of our research. And he is able to play independently and practice whenever he wants. And he really pushes hard to push it. The technology is the absolute limit. And he views that as his job really to make us be the bottleneck. And boy, has he done that well? And so that's the first thing to acknowledge is that he was extremely motivated to make this work. I've also had the privilege to meet other clinical child participants from brain gain, other trials, and they very much share the same attitude of like they viewed this as their life's work to advance the technology as much as they can. And if that means selecting targets on the screen for four hours from 2 a.m. to 6 a.m., then so be it. And there's something extremely admirable about that that's worth calling out. Okay, so now how do you sort of get from where he started, which is no cursor control to APPS? So, I mean, when he started, there's a huge amount of learning to do.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  22. “Yeah. No, this is a great question. So, in my view, one of the primary reasons why Nolan's performance is so good is because of Noland. Noland is extremely focused and very energetic. He'll play WebGrid sometimes for like four hours in the middle of the night, like from 2 a.m. to 6 a.m. He'll be playing WebGrid just because he wants to push it to the limits of what he can do.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  23. “There's a very interesting journey ahead to get us to that same level of 10 BPS performance. It's not the case that sort of the tricks that got us from 4 to 6 BPS and then six to eight BPS are going to be the ones that get us from 8 to 10. And in my view, the core challenge here is really the labeling problem. It's how do you understand at a very, very fine resolution what the user is attempting to do. And yeah, I highly encourage folks in academia to work on this problem.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  24. “But the simplest, purest form is just blue targets jump on screen, blue means left click. That's the simplest form of the game. And the sort of prior records here in academic work and at Nerlink internally with sort of NHPs have all been matched or beaten by Noland with his Nerlink device. So sort of prior to Nerlink, the sort of world record for a human using device is somewhere between 4.2 to 4.6 BPS, depending on exactly what paper you read and how you interpret it. Nolan's current record is 8.5 BPS. And again, this sort of median neural link of performance is 10 BPS. So you can think of it roughly as he's 85% the level of control of a median neurinker using their cursor to select blue targets on the screen. And yeah, I think.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  25. “There's also different modes that you can configure this task. So the WebGrid task can be presented as just sort of a left click on the screen, or you could have targets that you just dwell over, or you could have targets that you left, right click on. You could have targets that are left, right-click, middle click, scrolling, clicking and dragging. You could do all sorts of things within this general framework.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  26. “Out of a grid, just like a software keyboard on the screen. And bits per second is a measure that's computed by taking the log of the number of targets on the screen. You can subtract one if you care to model a keyboard because you have to subtract one for the delete key on the keyboard. But log of the number of targets on the screen times the number of crux selections minus incorrect divided by some time window, for example, 60 seconds. And that's sort of the standard way to measure a cursor control task in academia. And all credit in the world goes to this great professor, Dr. Chenoy of Stanford, who came up with that task. And he's also one of my inspirations for being in the field. So all the credit in the world to him for coming up with a standardized metric to facilitate this kind of bragging rights that we have now to say that no one is the best in the world at this task with his BCI. It's very important for progress that you have standardized metrics so people can compare across different techniques and approaches. How old does this do? So yeah, big kudos to him and to all the team at Stanford. Yeah, so for Noland and for me playing this task.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  27. “Yeah, maybe I'll take one zoom out step there, which is just explaining why we care to measure this at all. So again, our goal is to provide the user the ability to control the computer as well as I can and hopefully better. And that means that they can do it at the same speed as what I can do. It means that they have access to all the same functionality that I have, including all those little details like command tab, command space, all this stuff. They need to be able to do it with the brain. And with the same level of reliability as what I can do with my muscles. And that's a high bar. And so we intend to measure and quantify every aspect of that to understand how we're progressing towards that goal. There's many ways to measure BPS, by the way. This isn't the only way, but we present the user aggreats, and basically we compute a score, which is dependent on how fast and accurate they can select, and then how small are the targets. And the more targets that are on the screen, the smaller they are, the more information you present per click. And so if you think about it from an information theory point of view, you can communicate across different information theoretic channels. And one such channel is a typing interface you could imagine.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  28. “It's also not the case. People think that, oh, Click is a binary signal. This must be super easy to decode. Well, yes, it is, but the bar is so much higher for it to become a useful thing for the user. And there's ways to solve this. I mean, you can sort of take the comp out approach of, well, let's just take five seconds to click. Let's take a huge window of time so we can be very confident about the answer. But again, world's best mouse. The world's best mouse doesn't take a second to click or 500 milliseconds to click. It takes five milliseconds to click or less. And so if you're aiming for that kind of high bar, then you really want to solve the underlying problem.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  29. “It's super annoying if you accidentally, like if you're a cursor, imagine if your cursor misclicked every once in a while. That's super obnoxious. And the worst part of it is usually when the user is trying to click, they're also holding still because they're over the target they want to hit and they're getting ready to click, which means that in the data sets that we build, on average is the case that sort of low speeds or desire to hold still is correlated with when the user is attempting to click.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  30. “So, if the user is trying to click at position A and they're currently at position B, they're trying to navigate over time to get between those two points. And as long as the output of the model is on average correct, they can sort of steer it through time with the user control loop in the mix that can get to the point they want to get to. The same is not true of a click. For a click, you're performing it almost instantly at a scale of neurons firing. And so you want to be very sure that that click is correct because a false click can be very destructive to the user. They might accidentally close the tab that they're trying to do something and lose all their progress. They might accidentally hit some send button on some text that is only like half composed and reads funny after. So there's different sort of cost functions associated with errors in this space. And part of the UX design is understanding how to build a solution that is when it's wrong, still useful to the end user.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  31. “Any machine learning system you build has some number of errors. And it matters how those errors translate to the downstream user experience. For example, if you're developing a search algorithm in your photos, if you search for your friend Joe and it pulls up a photo of your friend Josephine, maybe that's not a big deal because the cost of an error is not that high. In a different scenario where you're trying to detect insurance fraud or something like this and you're directly sending someone to court because of some machine learning model output, then the errors make a lot more sense to be careful about. You want to be very thoughtful about how those errors translate to downstream effects. The same is true in BCI. So for example, if you're building a model that's decoding a velocity output from the brain versus an output where you're trying to modulate the left click, for example, these have sort of different trade-offs of how precise you need to be before it becomes useful to the end user. For velocity, it's okay to be on average correct because the output of the model is integrated through time.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  32. “Think we're in the early stages of discovering those laws, so I wouldn't claim to have solved that problem yet. But there's definitely some things we've learned that make it easier for the user to get stuff done. And it's pretty straightforward when you verbalize it, but it takes a while to actually get to that point when you're in the process of debugging this stuff in the trenches. One of those things is that the...”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  33. “And you want a control surface that still makes it easy and intuitive for the user to understand the state of the system and how to achieve what they want to achieve. And ultimately, the end goal is that that UX completely fades in the background and becomes something that's so natural and intuitive that it's subconscious to the user. And they just should feel like they have basically direct control over the cursor, just does what they want it to do. They're not thinking about the implementation of how to make it do what they want it to do. It's just doing what they want it to do.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  34. “We refer to ourselves as a pick crew. He really is truly the F1 driver. And there's different control surfaces that different kinds of cars and airplanes provide the user. And we take a lot of inspiration from that when designing how the cursor should behave. And maybe one nuance of this is even details like when you move a mouse on a MacBook trackpad, the sort of response curve of how that input that you give the trackpad translates to cursor movement is different than how it works with a mouse. When you move on the trackpad, there's a different response function, different curve to how much a movement translates to input to the computer than when you do it physically with a mouse. And that's because somebody sat down a long time ago when they're designed the initial input systems to any computer and they thought through exactly how it feels to use these different systems. And now we're designing sort of the next generation of this input system to a computer, which is entirely done via the brain. And there's no perceptive feedback. Again, you don't feel the mouse in your hand. You don't feel the keys under your fingertips.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  35. “At the end of the day, it really is truly the case that UX is how the thing works. And so it's not just like what's showing on the screen. It's also what control surfaces does a decoder provide the user. We want them to feel like they're in the F1 car, not like some minivan, right? And that really truly is how we think about it. No one himself is an F1 fan”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  36. “So, this is actually an idea that comes out of academia. There are some prior work with sort of Braingate Clinical Child participants where they pioneered this idea of bias correction. The way we've done it, I think, is very privatized, very beautiful user experience where the user can essentially flash the cursor over to the side of the screen and it opens up a window where they can actually sort of adjust or tune exactly the bias of the cursor. So bias made for people who aren't familiar is just sort of what is the default motion of the cursor if you're imagining nothing. And it turns out that that's one of the first First, sort of qualia of the cursor control experience that's impacted by neural non-stationarity.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  37. “That Nolan has the ability to do today. So he can recalibrate the system at 2 a.m. in the middle of the night without his caretaker or parents or friends around to help push a button for him. The other important part of the solution is that when you have a good model calibrated, that you can continue using that without needing to recalibrate it. So how often he has to do this recalibration today depends really on his appetite for performance. We observe sort of a degradation through time of how well any individual model works. But this can be mitigated behaviorally by the user adapting their control strategy. It can also be mitigated through a combination of sort of software features that we provide to the user. For example, we let the user adjust exactly how fast the cursor is moving. We call that the gain, for example, the gain of how fast the cursor reacts to any given input intention. They can also adjust the smoothing, how smooth the output of that cursor intention actually is. They can also adjust the friction, which is how easy is it to stop and hold still. And all these software tools allow the user a great deal of flexibility.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  38. “Yeah. So this is a problem we've worked on both with NHP, non-human primates before our clinical trial and then also with Noland during the clinical trial. Maybe the first thing that's worth stating is what the goal is here. So the goal is really to enable the user to have a plug and play experience where I guess they don't have to plug anything in, but a play experience where they can use the device whenever they want to, however they want to. And that's really what we're aiming for. And so there can be a set of solutions that get to that state without considering this non-stationarity problem. So maybe the first solution here that's important is that they can recalibrate whenever they want. This is something that”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  39. “Maybe they just become more behaviorally engaged to begin with because the task is kind of boring when you don't have any feedback at all. And so there may be benefits to the user experience of showing something on the screen, even if it's not accurate just because it keeps the user motivated to try to increase that number or push it upwards.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  40. “So, you could imagine giving the user feedback on a screen, but it's difficult because at this point, you don't know what they're attempting to do. So what can you show them that would basically give them a signal of I'm doing this correctly or not correctly? So let's take this very specific example. Maybe your calibration task looks like you're trying to move the cursor a certain position offset. So your instructions to the user are, hey, the cursor is here. Now when the cursor disappears, a manager moving it 200 pixels from where it was to the right to be over this target. In that kind of scenario, you could imagine coming up with some sort of consistency metric that you could display to the user of, okay, I know what the spike train looks like on average when you do this action to the right. Maybe I can produce some sort of probabilistic estimate of how likely is that to be the action you took given the latest trial or trajectory that you imagined. And that could give the user some sort of feedback of how consistent are they across different trials. You could also imagine that if the user is prompted with that kind of consistency metric,”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  41. “Monkey is trying to go in a straight line to the target. It turns out that making those assumptions is actually more effective in producing a model than actually predicting the underlying hand movement.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  42. “So good that it's not even a question that you want it. And to build the world's best mouse, the superhuman version, you really need to nail that problem. And a couple maybe details of previous studies that we've done internally that I think are very interesting to understand when thinking about how to solve this problem. The first is that even when you have ground truth data of what the user is trying to do, and you can get this with an able-bodied monkey, a monkey that has a neural link device implanted and moving a mouse to control a computer, even with that ground truth data set. It turns out that the optimal thing to predict, to produce high performance BCI, is not just the direct control of the mouse. You can imagine building a data set of what's going on in the brain and what does the mouse exactly doing on the table. And it turns out that if you build the mapping from neural spikes to predict exactly what the mouse is doing, that model will perform worse than a model that is trained to predict sort of higher level assumptions about what the user might be trying to do. For example, assuming that the”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  43. “Another thing I want to mention and call out is that this problem doesn't need to be solved in order to give useful control to people. Even today with the solutions we have now and that academia has built up over decades, the level of control that can be given to a user today is quite useful. It doesn't need to be solved to get to that level of control. But again, I want to build the world's best mouse. I want to make it.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  44. “If you can do that, even if the ceiling is lower, you're going to be able to move faster because you have a tighter iteration loop debugging the problem. And in the open loop setting, there's not a feedback cycle to debug with the user in the loop. And so there's some reason to think that that should be an easier debugging problem. The other thing that's worth understanding is that even in a closed loop setting, there's no special sauce or magic of how to infer what the user is truly attempting to do. In the closed loop setting, although they're moving the cursor on the screen, they may be attempting something different than what your model is outputting. So what the model is outputting is not a signal that you can use to retrain if you want to be able to improve the model further. You still have this very complicated guesstimation or unsupervised problem of figuring out what is the true user intention underlying that signal. And so the open loop problem has the nice property of being easy to debug. And the second nice property of it has all the same information and content as the closed loop scenario.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  45. “Yeah, it's a good question. First and foremost, I would say this is not a solved problem. And for anyone who's listening in academia who works on BCIs, I would also say this is not a problem that's solved by simply scaling channel count. So this is, you know, maybe that can help and you can get sort of richer covariance structures that you can use to exploit when trying to come up with good labeling strategies. But if you're interested in problems that aren't going to be solved inherently by scaling channel account, this is one of them. Yeah, so how do you solve it? It's not a solved problem. That's the first thing I want to make sure it gets across. The second thing is any solution that involves closed loop is going to become a very difficult debugging problem. And one of my sort of general heuristics for choosing what prompts to tackle is that you want to choose the one that's going to be the easiest to debug.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  46. “Ways, or maybe just had some cork. If there's some part of the data distribution that didn't cover super well, and the user now figures out because they're a brilliant user like Nolan, they figure out the right sequence of imagined motions or the right angle they have to hold their hand at to get it to work. And they'll get it to work great. But then the next day they come back to their device and maybe they don't remember exactly all the tricks that they used the previous day. And so there's a complicated sort of feedback cycle here that can. Can emerge and can make it a very difficult debugging process.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  47. “An open loop calibration. They're being asked to perform something. Like, imagine if you sort of had your whole right arm numbed and you stuck it in a box and you couldn't see it. So you had no visual feedback and you had no appropriate feedback about what the position or activity of your arm was. And now you're asked, okay, given this thing on the screen that's moving from left to right, match that speed. And you basically can try your best to, you know, invoke whatever that imagined action is in your brain that's moving the cursor from left to right. But in any situation, you're going to be inaccurate and maybe inconsistent in how you do that task. And so that's sort of the fundamental challenge of Open Loop. The challenge with closed loop is that once the user is given a model and they're able to start moving the mouse on their own, they're going to very naturally adapt to that model. And that co-adaptation between the model learning what they're doing and the user learning how to use the model may not find you the best sort of global minima. It may be that your first model was noisy in some way.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  48. “Yeah, great question. There's a lot to unpack there. The first thing I would draw a distinction between a sort of open loop versus closed loop. So open loop, what I mean by that is the user is sort of going from zero to one. They have no model at all and they're trying to get to the place where they have some level of control at all. In that setup, you really need to have some task that gives the user a hint of what you want them to do such that you can build its mapping again from brain data to output. Then once they have a model, you could imagine them using that model and actually adapting to it and figuring out the right way to use it themselves and then retraining on that data to give you sort of a boost in performance. There's a lot of challenges associated with both of these techniques and we can sort of rabbit hole into both of them if you're interested. But the sort of challenge with the open loop task is that the user themselves doesn't get proriceptive feedback about what they're doing. They don't necessarily perceive themselves or feel the mouse under their hand when they're using an open, when they're trying to do”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  49. “And what really matters is how accurate are those assumptions. For example, you might say, hey, user, push upwards and follow the speed of this cursor. And your heuristic might be that they're trying to do it exactly what that cursor is trying to do. Another competing heuristic might be they're actually trying to go slightly faster at the beginning of the movement and slightly slower at the end. And those competing heuristics may or may not be accurate reflections of what the user is trying to do. Another version of the task might be, hey, user, imagine moving this cursor a fixed offset. So rather than follow the cursor, just try to move it exactly 200 pixels to the right. So here's the cursor, here's the target. Okay, cursor disappears, tried to move that now invisible cursor, 200 pixels to the right. And the assumption in that case would be that the user can actually modulate correctly that position offset. But that position offset assumption might be a weaker assumption and therefore potentially you can make it more accurate than these heuristics that are trying to guesstimate at each millisecond what the user is trying to do. So you can imagine different tasks that make different assumptions.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source

  50. “Correct. Although clean labels, I think maybe it's worth exploring what that exactly means. I think any given labeling strategy will have some number of assumptions it makes about what the user is attempting to do. Those assumptions can be formulated in a loss function.”

    2024-08-02 · Lex Fridman Podcast · #438 – Elon Musk: Neuralink and the Future of Humanity · IDENTIFIED FROM THE TRANSCRIPT · source