YouSaid · the spoken record

Joshua Xu

lines on the record: 30
first: 2024-06-20
most recent: 2024-06-20
sittings or episodes: 1
sources: podcast

Every line below is reproduced as it was said and linked to the record it came from. Nothing here is summarised or generated. Directory · Search · Corrections

“Are a little bit over 40 people, but we are serving over 40,000 paying customers on the platform today. I think what's so interesting about our customers is that these are not the typical AI early adopters. These are mainstream companies from European manufacturers to small business to global nonprofits to Fortune 500 companies, which is the problem we are solving.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Would say some very similar, especially when we work on the camera software. So how do we know whether this parameter works better or the other one was better? And I think we can definitely come up with some very objective matches about, hey, lightning score, this is lightning score, this is resolution. But there's many things we figure out, hey, a better resolution, I mean, higher resolution doesn't mean it's a better image quality for customers. If you look at iPhone, it does not have the best resolution always compared to a lot of other phones, but it does produce the image that most people like about using iPhone to capture the image. And yeah, there's a very similar lesson out there we learned from in early days of Snap.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Mathematical standpoint, we kind of have to form a system, build a system around it, and be able to feed back those data into our model training so that we can continuously to improve.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Like a lot of other models, I think building video model, being able to integrate aesthetics into the AI models pretty hard. So video generation is not only about solving a mathematical problem, it's actually about creating something the customer loves and appreciates. So essentially a model with lower optimized cost function doesn't mean it actually produces a better visual outcome. So I guess that is the piece that making it really hard to evaluate, but also really important to deliver the last mile of the value for the customer. And generally, evaluation is also hot. We have to rely on in-product signal, for example, AB test, to know which model is actually better because only the customer can be the judger for that. And this process generates just not differentiable from”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, and one of the use cases we have seen from customers is pubs this group, they generate more than 100,000 videos, a thank you video to send to all the employees globally and localize into different languages, personalized with the name and what they like about it when they join the company and stuff like that. And historically, that is actually only delivered with one video, right? So they may be the CEO, the executive team hop on into a camera and we call something saying thank you for the 2023. But now that message and communication can really personalize at a very, very big scale.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Does not need to be the same. Let's say if maybe I like avocado, I should be watching an app with Coca-Cola and avocado and showing the new story about Coca-Cola with me and utilize something else you could be looking at something else. And this is not possible today because making videos expensive, but this could be very possible, let's say we can actually real-time generating the video ads that you like according to your user attribute that will potentially become a new format. You know, when we really look at today's video player, it corresponded to only one MP4 file. And it doesn't need to be like that. That video player can actually take in a lot of user attribute and generate something in real time to match what's the best way to deliver that content to customers.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“The technology has been always developing this chain. Two years from now, it would not be crazy to look at a lot of avatar generation asynchronization pipeline will become real-time streaming capable. And I also see the world is moving towards a way that we can probably generate the entire video in real time as well in the future. Let's say five years from now. I have an opinion like generative image is still image, but generative video is not a video. It is a new format. What I meant by that is, you know, when we really look at a video, we look at it as an MP4 file. So it is immutable. For example, if you and I are on Instagram, we probably get recommended by two different ads. But as long as we are recommended from the same business, we are looking at the same MP4 file.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“I look at it in two ways. One is that the real time application of the avatar, like even now it's possible, I think people can already experience that on HGN. We are making new updates that can make it even faster. So it can potentially become, let's say, the virtual AI SDR, virtual support that help to take customer calls or provide support.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“high quality content out there. So I want to give you one example is that I think a lot of AI generation is not only about obviously cost saving and time saving is one aspect of the value prop, but it's actually we are seeing a lot for customers using that to unlock new use cases and being able to do something they were not able to do it before, I think that's the key job in point of a lot of a business outcome today.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Know if people can generate very engaging and authentic video content, they will basically create more videos and use video more for their business to grow their business. And we live in a video first row where every business want to create more videos. I think the bottleneck today in the industries that video just are very expensive to make and it tastes like weeks or months to make a video. I think it would fundamentally change a lot of the way how people are thinking about how to grow the business, how to do the communication, how to do the marketing and sales. So I do think there's a huge possibility that we can create and generate a very high degree of personalized video, especially with the football avatar that being able to deliver a very dynamic and”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“First of all, we do not allow any political or election content on our platform today. HEN's policy strictly prohibits those creation of unauthorized content and we take abuse of the platform seriously. So we have our safety safeguard in very advanced user verification, include live video consent, dynamic verbal passcode and rapid human review in the back of all the advert being created on the platform. Chance and safety is critical to our business and we are actively partnering across the industry, continued developing the tools and best practice to combat misinformation in AI safety. We actually built the safety as part of the design if you look at a lot of creation process on HeGen and we base all this safety concern and safety on every single step of the creation process. As well.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“As well as the translation with ChatGPT and build a great experience around it and sort of like we are creating a whole new experience for localize their video and content.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“All the AI models had some sort of a limitation. And I think the key question is that in order to deliver a great product experience for the customer, is that how do we design a product around it so that we can try to avoid the limitation of the model, but help to amplify the strength of the model? And that's something that's really important to find the new area that unlock the new creation experience. One example would be when we look at the video translation technology is a whole new way to translate your content compared to traditional dubbing. We preserve the user, the natural voice and their facial expression. But if you look at really underlying the model, what enabled that video rendering, it is actually a lip sync model, right? But we kind of figure out a way to combine all this together together with the voice.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“I would say it's a combination. I will add one more thing that we need deeply understand the limitation around the model and try to find a connection between what is the customer looking for, what's capable with the technology.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Around that to assemble that final video together. We felt that this technical path is more capable to deliver the quality, the control and the consistency that the brand is looking for. Because, for example, there's some stuff we should probably should not try to generate. It's the logo and the fonts. That needs to be very accurate. And not to mention that we also need to be able to learn about, especially in the business context, we need to learn about the brand style, the color mapping, et cetera, from our customers. And I think the second approach would give us more flexibility and capability to build that system around it. And in fact, we actually see Sora as our partner because we are able to integrate that as one of the component generator and then feed that into our acquisition engine for the business application.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“We initially started Hen, we want to help the business solve the video creation problem. What does a business looking for? They're looking for quality, they're looking for control, they're looking for consistency. So when we try to look back, okay, this is not, how can we get there? What's the technical path to get us there? This is essentially probably potential two path. One is the test to image the SORA, where we try to generate the entire thing from end to end. And so you get the entire video at once. And the other approach is that what we believe in at Hajn is that we try to disassemble the whole video into different components. Lastly, we'll be A roll and B roll represent all different kinds of elements like voiceover music, transition, A-roll's being the avatar. And we try to tackle this component one by one, and then we build an orchestration engine.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“By actually getting the voice model and the video model training together so that it can sort of build a connection underlying the model as well. And that has been historically really, really hard because we have to train the TTS model on one hand and then feed the TTS model outcome into a video model. And it's really hard to build that connection with multimodal model training that's very possible.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“There's a three mode now, right? Text, voice, and video. So we work with OpenAI, ChatGPT on the test generation side. Obviously, it also serves the brain of the orchestration engine that we build internally. And we work with OpenAI and EventLab on the voice engine, but we build the entire video stack in-house, including after creation, video rendering, and VRO generation. I think over time, I think the whole technology trend has been moving towards a direction. A lot of all this thing will be chained together. The multimodal model, multimedia all get into one single model. One of the challenges I want to call out for the football generation is actually how do you actually connect that voice together with gesture motion. And that's actually something will be unlockable.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Help us to bring the avatar to bring the video into the next level of engagement and authentic. And that will help to unlock a lot of use cases in a broader case of marketing themselves.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“The way how I really look at it is that there is a spectrum of the quality requirement laying out on different use cases. Let's start from the left side of the spectrum. It's a learning development content, educational content. It's more like one to many broadcasting talking about educational changing content. The quality there is lower because the avatar can be more still, more professional. But if on the right side of the spectrum, we call the SID high-end marketing content, really dynamic. The one example would be the Screative. And people ship very, very dynamic content on ads because that can really help to improve the ROI for content making it more engaging. I think making the full body enabling that full body rendering would be able to”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“I think there are many things very exciting going on in our technology and product roadmap. I think particularly I'm very excited for the football generation of the avatar. Historically, all the urban technology has been focused on the upper body. So it's really hard to generate the gesture and the body motion, but a lot of academic research has proven that this is very possible now and we just need to basically take that into the last mile. And another thing I would say something I'm very excited about is the streaming avatar, especially with this on GPD40, really, really helped to improve the performance of the real-time interaction with text and voice. And HN avatar could become a videoization layer for all those applications.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Yeah, so I would say. Quality has always been the number one priority of the product and business and technology, I would say”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Know in this way, we can help customers to read it localize their content into local languages. And last but not least, people can also use Hen to personalize the video messaging at scale. So I think there's many, many very creative use cases on HGN today. We are a very horizontal platform. I would say one of my favorite use cases is probably the recent launch with McDonald. And McDonald launched a suite campaign where they can allow people to send a message to their family member in the different languages. When”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Categorize the use on H jet and do three create, localize, and personalize. People can The cast from a library from our avatar or create their own digital twin and just at a template or type the script and generate a video. This works the best for product explainer, how-to videos, lending development, and some sales enablement training content. We can also take the existing video that localize that into more than 175 different languages and dialect”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“I would say there's a many possibilities out there. I think what we are tackling the problem so far is the entry point of the content creation where all the content started with the camera. And then we would have people doing a lot of editing after that. We can clearly see a path where people can already assemble all this generative footage and apply the AI editing to assemble the final video. And again, if we push forward into technology, making the performance much better, I think we will be able to create experience like generative video in a streaming way. And that actually will be potentially replace a lot of the real-time conversation we have today, especially with the GBT4 and with all this multi-model real-time streaming technology all together.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“How we get started from trying to replace that piece of the process and making avata to replace the camera for the video production.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Initially, started the company, we tried to dissemble the whole video production process. There's really about camera and then editing. So camera is more about ARL, which represents the human spokesperson, the avatar piece. Editing is more about B-roll, adding different assets, voiceover, music, transition, animation, stuff like that. So entitling we just learn from customers that editing is not that expensive because they're pretty standard service, but cameras are super expensive and imagine, you know, it's a seal of a company he wants to recall something, we probably need to schedule that ahead of two weeks of time, we need to bring in the camera crew, have a studio to actually record it. And even for two minutes of footage, sometimes we need to record it for 20 minutes because people need to remember the script. And that's the piece that blocking a lot of business create new content.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Grew up my career in the whole mobile. Camera space where we walk on a lot of the software and technology to enable people feeling comfortable and making it easier for people to create content through the mobile camera. But there's still lots of people are not able to create quick content using the camera today. We felt that if we can replace the camera, that means we can remove the barrier for visual storytelling for visual content creation, and that will help us to step ahead in terms of the whole content creation space.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“Disney style filter. That was the first time I saw a computer can actually create and generate something that does not exist in the robe. I was just so fascinated by the technology back then and I had a feeling that would potentially change the way how people create the content. So Snapchat as a camera company and everybody create the content through the mobile camera. But we wanted to replace the camera because we think AI can create the content and AI could become the new camera. And that's how we get started with HGN and our mission is to making visual storytelling. Accessible to”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT
“That I was working in Snapchat for about six and a half years there. I started robotics at Carnegie Mellon and joined Snap back in 2014 there. I initially work on machine learning in Snapchat ads as ranking and recommendation, then I spent my last two years. Working on AI cameras. So Snap leveraged a lot of AI. To enhance the camera experience, if you look at 2018 Snapchat released the baby filter.”
2024-06-20 · No Priors · Can AI replace the camera? with Joshua Xu from HeyGen · IDENTIFIED FROM THE TRANSCRIPT