Michael Andrews has an interesting post about some of the credibility issues surrounding generative AI: phantom authorship, potential boobytraps, and countermeasures against them. A key insight is that most LLM knowledge isn’t from primary sources of facts, but rather from secondary sources’ simplified explanations of the primary sources. The latter are simply more numerous. So sometimes, a LLM’s knowledge lacks the depth you’d want.
If you’ve ever watched a television news report about something you’re an expert in, you already know how wary you should be of secondary sources.
Leave a Reply