We immediately tried using super-prompts to get this language model to do things, and the result was very miserable,” he recalled. “There’s a demo from another company to use an LLM to go to MrBeast’s latest YouTube video and leave a comment. Yes, in theory, language models can do that. But it would cause you to have to literally watch your screen doing that step by step. And it takes roughly around two to three minutes to finish one task like that. We just don’t think that can convert into a good end user experience.” Their solution is the “large action model,” which is trained on hours and hours of actual users interacting with popular apps: “Spotify, Uber, Expedia, DoorDash, you name it. We have the top 800 highest frequency apps. Then we set up this neural symbolic network and ask this AI, which now we call large action model, to review those clips, but frame by frame.