YouSaid · the spoken record

Illia Polosukhin

lines on the record: 46
first: 2023-09-15
most recent: 2023-09-15
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Yeah, but at the same time, we have every single large company doing their own hardware accelerator as well as a bunch of folks who are kind of spun out of those. And so we're going to have a market full of hardware accelerators, which are still optimized for transformers or at least similar structured architectures hitting the market this year and next year.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“And what changed that is pretty much mobile, right? And creation of IRM devices, RM chips that are kind of optimized for mobile and then came back into PCs, right? So yeah, so unless there's like a completely new form factor, which hard to predict, right? But also, that's a lot of investment to go from not just software, not just hardware, but like full stack, right? Innovation.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“And so it'll be really hard for them to be a viable business and kind of have the economies of scale that NVIDIA is having right now to kind of continue optimizing and building best state-of-the-art chips, right? So unless somebody's really investing in this, I think it will be more around what else we can do with carbon silicon, right? And kind of combinations of this. And then, I mean, maybe there's something new will come out.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“I mean, there might be another architecture that reasonably fits with the same solution. Of smaller compute chips, but they kind of all like in a grid and the data flows from one side to another. So the example there is on one side, it's like a really interesting architecture. You can build really cool things with this, but it doesn't fit transformers very well, right? Like you can do transformers with it, but it doesn't fit very well. you get like you know cost to to output ratio is not that interesting and so in comparison to you know you're just optimizing on gpus or using some of the new hardware accelerators and so this is where example like i mean and i'm not to speculate here on specific company but you know i wouldn't expect they will have like a ton of people lining up because like there is a ton of alternatives for transformers that come in and like somebody would need to like go in and develop a lot of new architectures that fit better this model”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Search in kind of this knowledge space means that probably like pushing more into that concept is more useful than doing more searches at inference time because like it means you already lost all the semantics if you're doing search at inference time.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“I mean, it's going to be like a combination, right? So I think we just need an architecture that had training time, it able to, so like, I mean, the simplest thing is like instead of outputting a token in the next, right, you can actually give it empty token, for example, for some period of time. And then when it says, okay, I'm ready, give it to output next token, right? And so this way you can train it to think more before outputting. And then at inference time, you can vary it, right? Like, hey, I'll give you more time to think, or like, no, you have no time to think. But then you can train it to like actually be able to dynamically to output. So again, this is like a very simple thing, but like you can keep expanding on this, you know, output it and then feed it back and like, is this the right answer, like, et cetera? So there's a few different models. But I think to Jakob's point, the fact that this model is doing a really effective.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“I need more time to think versus, you know, hey, I'm forcing you to output next token, even if you don't have an answer yet. So I think that parts we really need. And I think they kind of fit into the architecture, but just require more engineering and more different types of tasks as well for trading. I think the fact that we're just using a big language model is kind of interesting because this is not the task you would expect everything to be able to just predict next token. So like, you know, starting to obviously RLFH been already helpful, but like starting to like, hey, can you critique this answer? What would be the better answer, et cetera?”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Think there's definitely something around how do we get this model to have the capacity to let themselves think before outputting or kind of process more. And I think it's still within the transformer structure and it can be advanced, but I haven't seen anything that's really matches my intuition around us. But I think the simplicity of this architecture and indeed the amount of optimization that's going into this right now is just, it'll be really hard to match and kind of, you know, with enough exclusivity you can express any function. So like it's not, this is not a problem at this point of like, hey, we don't have an exclusivity, right? It's more around how do we compose a data set that's cleaner, better, or add some self-critique and understanding of like, is this content correct?”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Yeah. And it's like probably a shared environment as well where we probably will co-work on a business process and we'll share one display, but then we'll maybe fork it because I'm more interested in conversion and you're more interested in retention, for example. And so that's kind of the dynamism right now that also doesn't exist where we all look at the same JIRA task management. And I'm like, I don't really care about half of this stuff, right? But it's not a filter problem. It's like I want different information showed in a different way.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Exactly, yeah. And it's like adjusted for specific use case you need. And probably there's like a bunch of templates that is like, you know, fine-tuned for your specific problem. And this is possible right now.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“It will be a hybrid. In my imagination right now, at least, I expect you can describe a business process which is like, hey, when we have a new creative from marketing department, spin up a Twitter campaign and create me a dashboard that tracks the conversions on our product, right? And so what it does, it creates the pipeline of these things and then also creates a page where I can see normal user interface of analytics.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Components or even sort of describing with natural language some of the interfaces and business processes you want to have. So the way people will interact with kind of their business operations and all the tooling they need will start to change. And so I'm really excited about this space. And like we have one company that is kind of starting to build out some of the things in this space. And over next year, we'll see kind of that evolving.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“And finally, I'm really interested in kind of, as I mentioned, because we have now OpenLab and socialware, the kind of what I call future of SaaS. So I think a lot of between Web3 and AI, a lot of SaaS will actually start being replaced. Because right now what SaaS is is like one database with a specific UI for a specific problem. The database is the same between CRM, the hiring tool, marketing tool, even some of the project management tools. The database underlying is like not that different. And it's been just like the front end. And interconnecting all of this databases is like ton of work. It always breaks, right? But now you can have the database you own, right? So using kind of Web3 tech. And then you can build all of this front ends on top, either through kind of blockchain system, shared.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“So, we do actually have a project in this Web3 AI data marketplace that we are spinning out to focus on. Now they build a product, they have all the pieces. Now it's like ready to actually go to market and bring customers. I think the really interesting area is kind of partnering with existing ID already enabled or interested in Web3 teams who want to give access to more functionality to their users, right? We have, for example, Sweatcoin, which is a really good example of like it was a Web2 project that had 120 million installs that had a ton of people using it every day, kind of for a very specific use case, right? Kind of tracking their steps and maybe getting a discount on their next shoes. But now as it transformed into Web3, they're kind of opening up, right? And you can now participate in economic activity. Learn”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Like way more incentive to do this versus, let's say, if you're working on a contract, there's way more leeway usually if you're not doing your work, right? So there's like just way higher kind of self evaluation as well that happens. And so, I mean, there's a lot of pieces that needs to come together for this to be high quality. But again, it just opens up this marketplace and makes it effective. And it in a way removes a lot of the human part as well.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“On live stream marketplace, any student can join and do this, right? They don't need a contract with a specific company. They don't need to have the company in the local region to work with them. And students for coding, for example, are really interested in doing this because they usually don't have much money. And this is a way for them to practice their work anyway. As a task giver, you can actually specify the specific way you want the cross-validation to happen. And one of the things we've done, it's like honeypots, right, where you actually specify specific types of incorrect answers that people need to mark as incorrect. And otherwise, they actually lose the buy-in. And so there's actual very clear economic game theory where people have buy-ins. They lose them if they do poor quality of work. And so they have”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, so I think there's two parts. One is domain knowledge that generally it's hard to tap in into a specific centralized service, right? Because they need to kind of like for them to do payments, do all the things they need to set up a subsidiary in whatever country they have the workers, they need to train them, they need to hire them. Maybe it's contracts, but they need a lot of overhead that they do that. For example, developers. Let's imagine you're building a new, really cool developer platform which uses language funnels and you want to fine-tune on code, right? Well, the existing platforms, like them hiring a bunch of developers to actually do this, right? And, you know, if they're doing this full time, that's super complicated. Then kind of building out the validation tooling for how to cross validate that the work has been done.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“For everyone, and anybody everywhere can get paid at any time, like offering that both the people who doing this work want because they kind of are more protected actually. And it's fair game. And then the people who want to give tasks, they can actually get access to way larger workforce. They can specify specific parameters. They can price it at whatever level they want. That's going to be the kind of future of it.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I think decentralized kind of a Web3 marketplace is a more effective way to do this. And it kind of provides few interesting benefits. One of them is that it opens up kind of the market, right, where you don't need to set up a local office and kind of hire people and train them, et cetera. Like you can just open up global market. Anybody can join. And you have a very specific rules, right? That if they follow, they get paid, right? So I've used Mechanical Torque before, for example. And you can actually, as a client, you can just decline paying them. So people in Mechanical Turk, like the workers have very low kind of way to push back, if I say at the same time, they don't have any quality and knowledge assessment on the platform, right? So I think having quality knowledge and this kind of escrow model all embedded into one marketplace that opens up”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“You want large capacity that can scale with more usage? Tomorrow I have 10x more users. I want to be able to scale with that. And so this is where I think using some of this hardware that exists, as well as kind of leveraging maybe new methods of privacy and coordination that, again, crypto has like NPC, like multi-party computation, there's zero knowledge proofs, et cetera, like they can be leveraged to achieve that and have kind of secure decentralized inference. So I think that's way more realistic than training. And also way more needed.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“800 gigabit connect right between these GPUs. So maybe you have 100 megabit between this, usually not, and you need to replay and work around problems for decentralized. So I think decentralized training right now is still not as realistic, although there's some research people are trying. I think an inference is really interesting because we do need so much more compute for inference than we need for training, right? Like it's a very interesting economy of scale. You train once trained once and then everybody runs it everywhere. And so the inferences where I think there's a lot of interesting cases. One is you want it to be private, right? Right now, if you do an inference, you need to send it to some service and that service may or may not record it and both input and output. Second one is.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Like repurposing the same GPUs. Although, I mean, obviously, for smaller models, for some specific maybe inference things, there's maybe transition. There's a question of decentralized training in general, right? Like, hey, we have lots of GPUs everywhere. Can we train it? And the reality right now that the requirements on bandwidth, right?”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I mean, it's interesting because it sounds like such a no-brainer that, hey, let's grab those GPUs that, for example, Ethereum just moved from proof of work to proof of stake. Let's grab those and start using them. The challenge is the GPUs there are like not the ones that AI folks want to use, right? Kind of old AI is really zeroed in on how do I get A100s or H100s and the GPUs that folks used for Ethereum mining and similar is older ones that are not also focused on like floating point arithmetic, for example, as much. And so the challenge was more around people who did it, like Core Viva is probably a good example, right? They were mining company. It's more that they had a know-how how to build data centers and they can like get access to massive talk to NVIDIA and like get massive access to that.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“And you swipe their phone, and now you're able to impersonate them completely, right? So, yeah, so this is a real problem in having kind of authenticated passes required there to really establish it. And we have actually, like, the phones actually have so much already, like we have face ID and fingerprints. We have, you know, there's secure enclaves that sign things that are like haven't been hacked as far as I know. So there's a lot of the pieces are there. Now we just need like a product stack that actually pushes it to the user and like to the products.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“It's a huge consumer, but it's also like beyond that, is becoming like a real criminal problem, like criminals be able to use those tools now. And it's like the barrier of entry there is very low. And so this is where you really need the phone calls, all of this. You need more information identification and cryptography embedded into the system. Otherwise, it's completely going sideways, really quickly.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“And this goes into actually video calls and voice calls because right now somebody can call you on the phone and play a record, like generated audio of somebody they recorded 30 seconds off, right? And this can be very nefarious means, right?”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Like that, you want, and then market that. So, like, you can have all kinds of things now just exploding without any way of framing it on the user side. If does this have history? Is this coming from the right sources? Has it been validated, right? And so I think that's going to be a really important. I think the other side actually is law enforcement. And this is sadly already happening. People are using these tools now in very malicious ways right now. And law enforcement don't have really good ways to deal with this. And so I think everything from this on-camera signing, we need this now. Like they really have no way to kind of identify if the image was generated or not. And similarly, like for audio recordings and things like that, there needs to be kind of additional kind of levels of verification.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I think there will be probably next year will be very interesting in US because I think this will be a place where everybody will just take whatever their toys they have in Toolbox and do it even just for kicks, right? Even if it's not malicious, although some players will be malicious. And I think what we'll see everything from completely fake narrative candidates to like I would be very interested to see like a web page where you land and you know you log in and it literally generates specifically for this user based on their interest agenda for this candidate, right? So like hyper focused marketing for candidates based on like whose voter is, right? So things like that, like we'll have all those possible things where the media will kind of be flooded with like, you know, you can spin up new media right now and just generate content about your candidate.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Go over, like again, I think SSL is a really good example of something that's like it delivers value. It's clearly valuable, but it was such a uphill battle to get it there. And so I think until you have this critical mass of kind of website switched and browser support, it didn't become a default, right? So we kind of need the same here to happen. Like we'll need to have a critical mass of applications using identity and then we kind of seize it like in browsers or wallets or whatever like applications to hold it and then we'll see a transition function happen where like, hey, oh, you don't have it, like you should get it because it's actually easier and better to use it. And it gives you more financial freedom as well and more upside.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, while it has become an identity, right? And the reality is you're quote unquote private keys are your identity, but that's just too heart of a concept for people to actually work with, right? And so on NIR, we actually changed that. You have a properly named account, so like minus root.nir, which”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“It has happened to some extent, right? We have millions of people actually using blockchain right now and they're using it more for financial use cases. And that's their financial identity.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Communities that you are following. So that's probably like one important piece. And again, blockchain and cryptography is just like tools to enable that product experience. And then from there, we need similar things on the government level, right? When you file paperwork, when you file your identity, the fact that USSN is number that you give to everyone, which is supposed to be secret, is like, for example, ridiculous, right? So things like that is like all of this needs to improve and kind of upgrade to this new level where a massive amount of”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Identity and reputation is important. And so this is where kind of unchain identity becomes your kind of coalescence of all of the content and all of the interactions that you do. And then that links to kind of reputation in different communities and provides context for people who are watching for this content to be able to understand who is this person who's talking or where they're coming from and what are the information values they have. So I think it needs to be a kind of systematic approach and it'll start with pieces. I think one of the important pieces will be kind of a green lock similar to SSL transitioned on the content, right? As you go to YouTube, as you go to New York Times, you actually will see that like, hey, this content been signed by this party and this party is in some trust route or trust graph of”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“A set of primitives, right? Do we have cryptography to authenticate content and create a path, everything from, you know, you take a picture of his camera, some of them already half a secure enclave that can sign the image that's taken. And so as that image gets processed, we can actually propagate that information and have a proof that it came from specific time and place and then being processed by a specific set of filters. So that can give you like a anchor. Then you still need to know kind of who is publishing what, right? Like we're recording this podcast, you know, people listening to it. It could have been completely generated at this point, right? But if, for example, we all sign the final podcast and say, hey, yes, we've recorded it and this is the app content. Now when somebody is listening to it, they can check that indeed, hey, this content is signed by us. Now the question of us comes in, right? So this is where kind of.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Misinformation at scale, right? So, like we've kind of built, like a lot of our society has started building up tolerance to misinformation around TV and mass media, but we don't have a system and framework around dealing with it at scale. And that's what AI brings. Brings just scale to the same problem. And so this is where reputation, identity, and kind of systems around our social echoed operating system that powers our kind of communities is really important. How do all these pieces work together and how do they actually operate when there is malicious actors who potentially are able to en mass create very personalized misinformation or create fake political actor that is convincing every individual exactly in what they think that government should do to get elected? And this is where Web3 comes in as”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, so I think this is a very interesting topic. So I have this view that we need human alignment instead of AI alignment. So right now when we talk about, hey, we need to align AI with like human values. But the reality is that all the problems that exist, they all exist because of humans doing things and they've existed before. I actually like to use the Byzantine fault tolerance problem, right, which is basis for blockchain, but its roots are in history where there was people propagating misinformation and you were trying to figure out how to prevent misinformation in his army. So this is like a really old problem of misinformation and kind of like how to work around that. And so I think what we need to start doing is figuring out how do we build a society that is actually able to deal with kind of effective”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, I think DAOs is, and especially what happened with DAOs, there was a lot of people who were really excited about DAOs kind of as a concept. And so they put a lot of time running them. But it's actually a very not interesting job, right? It's like you onboard new members, you explain to them all the same thing. You answer to their questions. And so that's the part which you can already automate, right? You can have a Discord bot that is have all the context about the DAOs interactions and kind of onboard new people and gives them new tasks to start with and kind of coordinate them. So I think that will be the first place where this kind of starts showing up and as well because you have like payments kind of very like there and you don't have any social constraints that usually have in like regular organizations. A lot of people will revolt if you like tomorrow say, hey, by the way, your new boss is the say model. Yeah.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“The best at doing this specific sets of experiments, fund them for this amount of money, oversee their delivery and then kind of iterate. And if it thinks this lab is not doing a good job, Personal affiliations that people do have. I'm actually excited about some folks that are already building some examples of this in a simpler forms. But I think we'll see first organizations like this probably even this year where potentially with a simpler missions and kind of more straightforward like KPI metrics, but where kind of this information propagation and onboarding of people happens already through a kind of language model AI agent.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“I use this example. We want to continue making progress on solving cancer. And it's a very complex problem, right? There's a lot of specific subcancers that need research. And so all of this and coordinating people doing experiments, propagating information, recruiting people recruiting the candidates. All of this requires somebody to do this work and kind of organize the process and really set up a lot of pipeline and funding and all those things. And right now there's so much overhead around everything from how grant funding is allocated from the nonprofits that collect money for research, how experiments are set up, the information sharing, like all of those pieces are really kind of broken. And so you can actually have coordinated effort that is designed just to do that. And it can consume all this information and kind of specifically task, you know, who is the best person at doing the experiment, which lab is.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“Damn specific area of work and then kind of harnessing their creativity and putting it back together. I think that's a very interesting use case that kind of really melds blockchain and AI together.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“That this is the first time that a machine is able to communicate with people in the same way. There's no more need in an intermediate human that interprets data and then tells it to other people. Now machine can communicate directly to people. And so it can task them with work. It can provide them context. And so I really think one of the most interesting cases is organizations that are run completely by AI, right? where quote unquote CEO are always taken by AI agent who is tasked by community or board of directors or whatever is oversight governance is to hit specific KPIs and follow specific mission, they can even give specific feedback with training data when they don't think it's doing the right job. But what it does is like creates this kind of a new layer of management that potentially removes a lot of middle management right now, which is like transforming information and context for each individual person and giving”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“There's few levels of interesting intersections. I think the most obvious one that everybody talks about is various marketplaces for resources, right? Be that compute, model, or data, right? So data crowdsourcing. So those are pretty obvious, right? Web3 is really good at creating marketplaces, creating traceability, and providing an equitable place for everyone to participate. Now, the more interesting ones is where AI kind of agents, right, which we've seen like initial versions of, but obviously they're going to continue evolving. If you equip them with a blockchain account, right, they are now becoming an economic agent that is able to pay other people and pay other AIs to do work, right? And they can communicate, right? And I think one of the things that a lot of people who are, oh, like this language models are just the same advancements like as everything before, missing the point.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“So, the idea is that we want to kind of go up stack, right? We want kind of an environment where you can discover and use Web3 experiences, benefits from them and not need to think about the low level implementations and quote-unquote hardware that runs under it. So similarly how operating systems on your phone kind of abstracts out all the complexity of networking and payments and everything, you just use it and you have apps that developers can build. And so that's really what we're trying to achieve and kind of build this framework and platform for everybody to build their applications in Web3 and really deliver it to the user and to consumer.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“And so we started looking into blockchain just like to solve our own problem. AI kind of exponential explosion didn't happen at the time. And so we saw an opportunity of, we can actually build a blockchain that we would use to solve this first and focus on that while kind of waiting out the AI thing to really happen. And as you go into the blockchain rabbit hole, you realize there's a lot more that meets the eye.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“So we started this idea that we wanted to teach machines to code. We had transformers coming out. There was a lot of kind of really interesting push in 1617 around AI. And so our expectation was we kind of would write the exponential growth of AI, which has happened in this year, we thought it will happen in 1718. And so with that, we got a really interesting data set around language to code. More interestingly, we had a whole community of developers, mostly students, who were doing crowdsourcing for us. So we would give them code, they would write descriptions, we would give them descriptions, they would write code for them, write tests, like all kinds of tasks. And we actually faced a challenge of paying them because a lot of them were in China, in Eastern Europe, in kind of other countries where there's monetary control problems. People don't have bank accounts.”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT
“I worked on a team on natural language understanding focused on question answering and the state of the art at the time was LSTMs, recurrent neural networks, which you could not launch in production at all because they're too slow and take a fair bit of time to process as documents scale. So Jacob at the time was using attention for query similarity and he had this idea like using attention for encoder decoder type I kind of jumped into it and with a shish we're playing around with can we actually get it to train and understand the order of words and do translation just based on you know attention so yeah it was pretty cool to explore that and obviously grew into something very interesting and awesome”
2023-09-15 · No Priors · The Intersection of AI and Blockchain, with Transformers author and NEAR founder Illia Polosukhin · IDENTIFIED FROM THE TRANSCRIPT