When it comes to Chinese AI labs using distillation techniques to extract knowledge from frontier model makers, Y Combinator CEO Garry Tan is hoping regulators stay out of it. AI labs should perhaps play the same game. Distillation is when a model maker extensively prompts another model in order to learn how it works and reasons.
It is commonly, and legitimately, used by AI labs to help train new models. Anthropic this week released its second report alleging that Chinese labs are engaged in “illicit distillation attacks,” hiding their identities to distill without permission and relying on fraud and stolen credentials to do so. Anthropic CEO Dario Amodei had previously publicly called on U.S. regulators to crack down on distillation.
It’s notable that the commander of Silicon Valley’s prestigious and prolific startup accelerator doesn’t agree. To be clear, Tan isn’t advocating for American AI labs to use stolen credentials to distill. He wants them to be free to come in the front door.
In fact, his argument is twofold. He feels it’s an overreach for AI labs to dictate what their customers can do with the information their models share with them. He also notes that the proprietary AI labs didn’t ask permission when they vacuumed up as much human knowledge as they could to train their models.
They famously ingested plenty of copyrighted material without the permission of those intellectual property holders. Tan, who is himself such an avid AI user that once described himself as having cyber psychosis, wants to see a balance between open-weight AI labs and frontier labs. We want that to be fundable, and be a great business model ongoing,” he told CNBC.
It has the best AI researchers. It runs away with it and suddenly there’s one company that’s monolithic.
Extract — continue reading at the source.
