Ask Me Anything
This post is more about writing than about LLMs, so I put it here; but I also wrote a brief cross-reference entry at my secret level.
How much money I make from writing has always been closely correlated with the kind of writing I do: advertising copy substantially; text books okayish; fiction minuscule; scientific papers nil. But all these types have one thing in common—you need ideas specific to type. A selling angle, a didactic scheme, a story idea, a hypothesis. For all these, creative ideas are hard to come by, and one of the reasons for that is that your brain comes up with ideas all the time, most of which alas disintegrate like vampires when exposed to daylight.
Thus, coming up with ideas for writing copy, fiction, primers, or papers is simultaneously the least of your problems and the worst of your problems. Which, I venture, applies for game ideas as well. Any idea merely puts you at a starting line in a colossal stadium where you don’t know anything yet about the track, the time, the requirements, or even the kind of discipline you’re going to compete in. You can’t patent or copyright ideas for a reason! What counts is the work you put into it. In my book The Ludotronics Game Design Methodology, I put forth the following definition of a creative idea:
A creative idea is the product of persistence, constraints, and an unfamiliar combination of familiar ideas, and the result should be original, interesting, and relevant.
Developing creative ideas is work. You have to sift the ideas your brain continually comes up with; put those ideas that seem to appear promising and feasible under harsh scrutiny; and then pick one and work on it until it ticks enough boxes to count as a creative idea. After that, sit down to work with it until you have the equivalent of a treatment—which, by the way, is copyrightable, pitchable, and sellable.
Now, ChatGPT, or large language models in general, can’t come up with an unfamiliar combination of familiar ideas, except if that combination already exists in the training data, which then wouldn’t be unfamiliar. Original, interesting, and relevant are characteristics intrinsic to world models LLMs don’t have, and the same is true for persistence. The only thing LLMs can compute are constraints, but that alone won’t suffice to generate a creative idea.
There are even more fundamental problems if you take the questions into account too. You can only ask questions you have sufficient knowledge to ask, and you won’t get to ideas via serendipity. Both of these, prior knowledge and serendipity, are hugely important. Research for an angle, a story, a scheme, or a hypothesis that actually earns its name will lead you into rabbit holes from which you return wiser and more seasoned and, hopefully, with a creative idea and a crackerjack plan.
I did quite a few tests with LLMs, and there’s one category of questions related to writing where they turn out to be actually useful: listing the specifics (and providing the keywords for further research) on questions of the “How does the office of a medical examiner look?” variety, i.e., topics I’m utterly unfamiliar with, whose facts are well-known, but would be a bother to collect. That’s fine. But asking for writing ideas or solutions instead, the answers to questions like, roughly, “How would my protagonist escape through an elevator shaft?” or “How can I refer to members of an alien species that have no biological sex or gender?” or “What kind of FTL technology would be compatible with our current understanding of the laws of physics?” are, while mostly not wrong, not helpful. All they can deliver are answers rich in well-known facts but poor in tricky details and interesting specifics. And here’s the catch: these answers do sound helpful and even impressive when, and only when, the person asking the questions has a) no, b) only superficial, or c) thoroughly half-baked prior knowledge about the underlying topic. And when that’s the case, they will remain blissfully unaware of any errors as a bonus.
Which doesn’t come down to LLMs making you dumber. But they don’t make you smarter either—i.e., more knowledgeable about a topic beyond easily digestible 101 introductions.
Of course, sometimes you have to come up with an idea fast, most often for copy, but also for writing competitions and the like. Or, you have writer’s block! But instead of turning to LLMs for help, you should turn to time-tested creative techniques. Or to curated writing prompts, with “curated” as the operating term. Sure, Heinlein is no longer around to help you out (the copy of his original letter, alas, has vanished from the Letters of Note site), but there are books full of writing prompts! For every genre you can imagine! However, make sure to check their release dates—you don’t want to waste your time and money on AI-generated book imitations woven of garbage and stolen goods.
If you have something valuable to add or some interesting point to discuss, I’ll be looking forward to meeting you at Mastodon!