📚️ How to Write Good Prompts - Using Spaced Repetition to Create Understanding

How to write good prompts: using spaced repetition to create understanding

Spaced repetition systems work only as well as the prompts you give them. And especially when new to these systems, you’re likely to give them mostly bad prompts. It often won’t even be clear which prompts are bad and why, much less how to improve them.

The most common mechanism of change for spaced repetition learning tasks is called retrieval practice. In brief: when you attempt to recall some knowledge from memory, the act of retrieval tends to reinforce those memories. You’ll forget that knowledge more slowly. With a few retrievals strategically spaced over time, you can effectively halt forgetting.

Retrieval is the key element which distinguishes this effective mode of practice from typical study habits. Simply reminding yourself of material (for instance by re-reading it) yields much weaker memory and problem-solving performance. The learning produced by retrieval is called the “testing effect” because it occurs when you explicitly test yourself, reaching within to recall some knowledge from the tangle of your mind. Such tests look like typical school exams, but in some sense they’re the opposite: retrieval practice is about testing your knowledge to produce learning, rather than to assess learning.

Retrieval practice prompts should be precise about what they’re asking for. Vague questions will elicit vague answers, which won’t reliably light the bulbs you’re targeting.

Retrieval practice prompts should produce consistent answers, lighting the same bulbs each time you perform the task. Otherwise, you may run afoul of an interference phenomenon called “retrieval-induced forgetting”. what you remember during practice is reinforced, but other related knowledge which you didn’t recall is actually inhibited. Now, there is a useful type of prompt which involves generating new answers with each repetition, but such prompts leverage a different theory of change. We’ll discuss them briefly later in this guide.

Retrieval practice prompts should be tractable. To avoid interference-driven churn and recurring annoyance in your review sessions, you should strive to write prompts which you can almost always answer correctly. This often means breaking the task down, or adding cues.

Retrieval practice prompts should be effortful. It’s important that the prompt actually involves retrieving the answer from memory. You shouldn’t be able to trivially infer the answer. Cues are helpful, as we’ll discuss later—just don’t “give the answer away.” In fact, effort appears to be an important factor in the effects of retrieval practice. That’s one motivation for spacing reviews out over time: if it’s too easy to recall the answer, retrieval practice has little effect.

Factual Knowledge

Such cues engage another another memory phenomenon cognitive scientists have explored experimentally: you make information easier to recall when you connect it to other memories. This process is called _elaborative encoding_See e.g. Bradshaw and Anderson, Elaborative Encoding as an Explanation of Levels of Processing (1982).. The members of an ingredient list can be difficult to relate to anything meaningful. In such cases, you can still leverage elaborative encoding by fabricating an association as a mnemonic device. Vivid associations work best, so it’s helpful to find relationships involving visuals, meaningful personal experiences, or emotions like humor and disgust.

Q. Typical chicken stock aromatics:

• onion

• ???

• celery

• garlic

• parsley

A. carrots (rhymes with “parrots”: picture a flock of parrots flying with carrots in their mouths, dropping them into a pot of stock)

Factual Knowledge

Prompts are cheaper than you probably imagine. An easy prompt will consume 10–30 seconds across the entire first year of practice, and much less in each subsequent year. Until you’ve internalized that observation, try to adopt this rule of thumb: write more prompts than feels natural.

Q. What’s the ratio of chicken bones to water in chicken stock?

A. A quart of water per pound of bones

Q. How much onion to use in chicken stock per pound of chicken bones?

A. Half an onion per pound

Q. How much carrots/celery to use in chicken stock per pound of chicken bones?

A. 1 carrot/celery rib per pound

Q. How much garlic to use in chicken stock per pound of chicken bones?

A. 2 smashed cloves per pound

Procedural

Q. At what speed should you heat a pot of ingredients for chicken stock?

A. Slowly.

Q. When making chicken stock, when should you lower the heat?

A. After the pot reaches a simmer.

Q. When making chicken stock, what should you do after the pot reaches a simmer?

A. Lower the temperature to a bare simmer.

Q. How long must chicken stock simmer?

A. 90m. Procedures can often be broken down into keywords like this. What are the important verbs, and when should you move between them? What are the key adjectives, adverbs, subjects, objects?

Conceptual Knowledge

Attributes and tendencies: What makes stock, stock? What’s always, sometimes, and never true of stock?

Q. How are stocks usually made?

A. Simmering flavorful ingredients in water.

Q. Why don’t stocks usually have a distinctive flavor?

A. To make them more versatile.

Similarities and differences: Knowing what stock is requires knowing what relates and distinguishes it from other adjacent concepts.

Q. How is stock different from soup broth?

A. Soup broth has a more complete flavor; stock isn’t meant to stand on its own.

Parts and wholes: What are some examples of stocks? Are there important “sub-concepts” of stocks? Is “stock” a part of some broader category? Visualize a Venn diagram, even if the edges are fuzzy.

Q. Name at least three examples of stock:

A. e.g. chicken, vegetable, mushroom, pork

Q. Stock is rarely served directly; it’s best thought of as a ??? (construction metaphor)

A. Building block.

Causes and effects: What does stock do? What causes it to do that? What doesn’t it do? When is it used?

Q. Why do restaurants use stock as a cooking medium instead of water? (name two reasons)

A. Adds flavor, improves texture.

Q. Stocks are a common base for… (name at least two)

A. e.g. sauces, soups, braises

Q. Restaurants often use stock as a cooking medium where home cooks might use ???.

A. Water

Open Lists

I like to think of open lists like tags—like the tags you might use in a system for digital bookmarks. My mental filing cabinet has a tag called “way to use chicken stock,” and I’ve fastened that tag to some notes about making purée soups.

When I encode this type of knowledge, I find three types of prompts consistently helpful. First I write prompts focused on each of the tagged items, linking from the instance to the tag. Then I might separately write prompts about the tag itself, perhaps inspired by patterns I notice in its instances. Finally, I often write a prompt which fuzzily links from the tag to its instances by asking for examples.

Saliance

Salience typically fades over time. If you don’t soon have a chance to connect that new idea to something meaningful in your life, you may stop noticing opportunities so readily. The dynamic seems similar to the problem of forgetting knowledge over time. So one valuable use for spaced repetition prompts is to keep ideas salient, top of mind, over longer periods of time. Gwern Branwen has pointed out** In private communication. that such prompts are effectively trying to extend the Baader-Meinhof phenomenon and control it for a purpose.

We’ve already written a few prompts which focus on salience:

Q. What should I ask myself if I notice I’m using water in savory cooking?

A. “Should I use stock instead?”

Q. What should I do with the carcass of a roast chicken?

A. Freeze it and make chicken stock

Q. Name a vegetable purée soup which you might try making with chicken stock (give an answer you haven’t given before)

A. e.g. potato, parsnip, celeriac, sunchoke, salsify, squash, carrot, pepper, lentil… The point of those prompts isn’t really to “know” those answers intellectually. It’s to cue certain ideas, which in turn may prompt new thoughts or create new behaviors. Viewed in this way, the point of repeating these prompts over time is to keep the relevant ideas salient until they have a chance to connect to something meaningful in your life. As economist Brad DeLong suggests, review sessions are surprisingly like a secular catechism.

Say you’re reading an article that seems interesting. Try setting yourself an accessible goal: on your first pass, aim to write a small number of prompts (say, 5-10) about whatever seems most important, meaningful, or useful.

I find that such goals change the way I read even casual texts. When first adopting spaced repetition practice, I felt like I “should” write prompts about everything. This made reading a chore. By contrast, it feels quite freeing to aim for just a few key prompts at a time.As Michael Nielsen notes, similar lightweight prompt-writing goals can enliven seminars, professional conversations, events, and so on. I read a notch more actively, noticing a tickle in the back of my mind: “Ooh, that’s a juicy bit! Let’s get that one!”

If the material is fairly simple, you may be able to write these prompts while you read. But for texts which are challenging or on an unfamiliar topic, it may be too disruptive to switch back and forth. In such cases it’s better to highlight or make note of the most important details. Then you can write prompts about your understanding of those details in a batch at the end or at a suitable stopping point. For these tougher topics, I find it’s best to focus initially on prompts about basic details you can build on: raw facts, terms, notation, etc.