We who have nothing to “wind string around” are lost in the wilderness. But those who deny this need are “burning our playhouse down.” If you put quotes around certain words it sounds more like a metaphor.

— John Linnell, 2009 interview with Rolling Stone

Take almost anything, heat it up, and it gets bigger. Heat it up enough, it melts and becomes a liquid. Heat it up even more, it becomes a gas, and takes up even more space. Or, cool it down, it contracts and becomes smaller again. 

The year is 1789. Antoine Lavoisier has just published his Traité Élémentaire de Chimie. Robert Kerr will soon translate it into English under the title Elements of Chemistry in a New Systematic Order containing All the Modern Discoveries, usually known as just Elements of Chemistry. 

The very first thing Lavoisier talks about in his book is this mystery about heat. “[It] was long ago fully established as a physical axiom, or universal proposition,” he begins, “that every body, whether solid or fluid, is augmented in all its dimensions by any increase of its sensible heat”. When things get hotter, they almost always get bigger. And when things get colder, they almost always shrink. “It is easy to perceive,” he says, “that the separation of particles by heat is a constant and general law of nature.” 

Lavoisier is riding a wave. About two hundred years earlier, Descartes had suggested that we throw out Aristotle’s way of thinking, where each kind of thing is imbued with its own special purpose, and instead bring back a very old idea from Epicurus, that everything is made out of tiny particles. 

The plan is to see if “let’s start by assuming it’s all particles” might be a better angle for learning about the world. So Lavoisier’s goal here is to try to describe heat in terms of some kind of interaction between different particles.

He makes the argument in two steps. First, Lavoisier says that there must be two forces: one force that pushes the particles of the object apart (which we see when the object heats up), and another force that pulls them together (which we see when the object cools down). “The particles of all bodies,” he says, “may be considered as subjected to the action of two opposite powers, the one repulsive, the other attractive, between which they remain in equilibrio.”

The force pushing the particles apart obviously has something to do with heat, but there must also be a force pushing the particles together. Otherwise, the separating power of heat would make the object fly entirely apart, and objects wouldn’t get smaller when heat was removed, things wouldn’t condense or freeze as they got cold. 

“Since the particles of bodies are thus continually impelled by heat to separate from each other,” he says, “they would have no connection between themselves; … there could be no solidity in nature, unless they were held together by some other power which tends to unite them, and, so to speak, to chain them together; which power, whatever be its cause, or manner of operation, we name Attraction.” Therefore, there is also a force pulling them together.

Ok, that was step one. In step two, Lavoisier takes those observations and proposes a model: 

It is difficult to comprehend these phenomena, without admitting them as the effects of a real and material substance, or very subtle fluid, which, insinuating itself between the particles of bodies, separates them from each other; and, even allowing the existence of this fluid to be hypothetical, we shall see in the sequel, that it explains the phenomena of nature in a very satisfactory manner.

Let’s step back and notice a few things about what he’s doing.

First: While he’s happy to speculate about an attractive force, Lavoisier is very careful. He doesn’t claim anything about the attractive force, does not even speculate about “its cause, or manner of operation”. He just notes that there appears to be some kind of force causing the “solidity in nature”, and discusses what we might call it. 

He does the same thing with the force that separates. Since it seems to be closely related to heat, he says we can call this hypothetical fluid “caloric” — “but there remains a more difficult attempt, which is, to give a just conception of the manner in which caloric acts upon other bodies.” 

We don’t know these fluids exist from seeing or touching them — we hypothesize them from making normal observations, and asking, what kind of thing could there be, invisible but out there in the world, that could cause these observations? “Since this subtle matter penetrates through the pores of all known substances,” he says, “since there are no vessels through which it cannot escape, and consequently, as there are none which are capable of retaining it, we can only come at the knowledge of its properties by effects which are fleeting, and difficultly ascertainable.”

And Lavoisier warns us against thinking we are doing anything more than speculating. “It is in these things which we neither see nor feel,” he says, “that it is especially necessary to guard against the extravagance of our imagination, which forever inclines to step beyond the bounds of truth, and is very difficulty restrained within the narrow line of facts.”

Second: In addition to speculating, Lavosier proposes a model.

But not just any model. Lavosier’s theory of heat is a physical model. He proposes that heat is a fluid with particles so small they can get in between the particles of any other body. And he proposes that these particles create a force that separates other particles from each other. The heat particles naturally seep inside the particles of other objects, because they are so small. And this leads to the expansion and contraction that was the observation we started with.

Lavoisier is proposing a model of entities and rules. In this case, the entities are particles. There are rules governing how the particles can interact: Heat particles emit a force that pushes apart other particles. Particles of the same body mutually attract. There may be more entities, and there will certainly be more rules, but that’s a start.

Third: Instead of something obscure, he starts by trying to explain existing, commonplace observations.

People often think that a theory should make new, testable predictions. This thought seems to come from falsificationism: if a theory gives us a prediction that has never been seen before, we can go out and try to falsify the theory. If the prediction stands, then the theory has some legs.

But this is putting the cart before the horse. The first thing you actually want is for a theory to make “testable” predictions about existing observations. If a new proposal cannot even account for the things we already know about, if the entities and rules don’t duplicate a single thing we see from the natural world, it is a poor theory indeed. 

It’s good if your model can do the fancy stuff, but first it should do the basic shit. A theory of weather doesn’t need to do much at first, but it should at least anticipate that water vapor makes clouds and that clouds make rain. It’s nice if your theory of gravity can account for the precession of the perihelion of Mercury, but it should first anticipate that the moon won’t fall into the earth, and that the earth attracts apples rather than repels them.

Fourth: His proposal is wrong! This model is not much like our modern understanding of heat at all. However, Lavoisier is entirely unconcerned. He makes it very clear that he doesn’t care whether or not this model is at all accurate in the entities: 

…strictly speaking, we are not obliged to suppose [caloric] to be a real substance; it being sufficient … that it be considered as the repulsive cause, whatever that may be, which separates the particles of matter from each other; so that we are still at liberty to investigate its effects in an abstract and mathematical manner.

People are sometimes very anxious about whether their models are right. But this anxiety is pointless. A scientific model doesn’t need to be right. It doesn’t even need to describe a real entity.

Lavoisier doesn’t care about whether the entities he describes are real; he cares about the fact that the entities he proposes 1) would create the phenomenon he’s trying to understand (things generally expand when they get hotter, and contract when they get colder) and 2) are specific enough that they can be investigated.

Lavoisier’s proposal involves entities that operate by simple rules. The rules give rise to phenomena about heat that match existing observations. That is all that is necessary, and Lavoisier is quite aware of this. “Even allowing the existence of this fluid to be hypothetical,” he says, “we shall see … that it explains the phenomena of nature in a very satisfactory manner.”

This is how scientific progress has always worked: Propose some entities and simple rules that govern them. See if they give rise to the things we see all the time. It’s hard to explain all of the things, so it’s unlikely that you’ll get this right on the first try. But does it explain any of the things?

If so, congratulations! You are on the right track. From here, you can tweak the rules and entities until they fit more and more of the commonly known phenomena. If you can do this, you are making progress. If at some point you can match most of the phenomena you see out in the world, you are golden.

If you can then go on to use the entities as a model to predict phenomena in an unknown set of circumstances, double congratulations. This is the hardest step of all, to make a called shot, to prove your model of rules and entities in unknown circumstances.

But first, you should prove it in known circumstances. If your theory of heat doesn’t even account for why things melt and evaporate, there’s no use in trying to make more exotic predictions. You need to start over. 

Superficial

Much of what passes for knowledge is superficial.

We mean “superficial” in the literal sense. When we call something superficial, we mean that it deals only with the surface appearances of a phenomenon, without making appeal or even speculating about what might be going on beneath the surface. 

There are two kinds of superficial knowledge: predictions and abstractions.

1. Predictions

Predictions are superficial because they only involve anticipating what will happen, and not why. 

If you ask an astronomer, “What is the sun?” and he replies, “I can tell you exactly when the sun will rise and set every day”… that’s cool, but this astronomer does not know what the sun is. That will still be true even if he can name all the stars, even if he can predict eclipses, even if he can prove his calculations are accurate to the sixth decimal point. 

Most forms of statistics suffer from this kind of superficiality. Any time anyone talks about correlations, they are being superficial in this way. “The closer we get to winter, the less time the sun spends in the sky.” Uh huh. And what is the sun, again? 

Sometimes it is ok to talk about things just in terms of their surface appearances. We didn’t say “don’t talk about correlations”. We said, “correlations are superficial”. But often we want to go deeper. When you want to go deeper, accept no substitutes!

Sometimes all you want to do is predict what will happen. If you’re an insurance company, you only care about getting your bets right — you need to have a good idea which homes will be destroyed by the flood, but you don’t need to understand why. You know that your business involves uncertainty, and these predictions are only estimates. If all you want to do is predict, that’s fine.

But in most cases, we want more than just prediction. If you’re a doctor choosing between two surgeries, you certainly would rather conduct the surgery with the 90% survival rate than the surgery with the 70% survival rate. But you’d ideally like to understand what’s actually going on. Even having chosen the surgery with better odds, what can you do to make sure your patient is in the 90% that survive, rather than the 10% that do not? What are the differences between these two groups? We aspire to do more than just rolling the dice.

Consider this for any other prediction. In the Asch conformity experiments, most participants conformed to the group. From this, we can predict that in similar situations, most people will also conform. But some people don’t conform. Why not? Prediction by itself can’t go any deeper.

Or education. Perhaps we can predict which students will do well in school. We predict that certain students will succeed. But some of these students don’t succeed, and some of the students we thought would be failures do succeed. Why? Prediction by itself can’t go any deeper.

There’s something a little easy to miss here, which is that having a really good model is one way to make really good predictions. However good your predictions are when you predict the future by benchmarking off the past, having a good model will make them even better. And, you will have some idea of what is actually going on. 

But people often take this lesson in reverse — they think that good predictions are a sign of a good understanding of the processes behind the thing being predicted. It can be easy to just look for good predictions, and think that’s the final measure of a theory. But in reality, you can often make very good predictions despite having no idea of what is actually happening under the hood. 

This is why you can operate a car or dishwasher, despite having no idea how they work. You know what will happen when you turn on your dishwasher, or shift your car into reverse. Your predictions are very good, nearly 100%. But you don’t know in a mechanical sense why your car moves backwards when you shift into reverse, or how your dishwasher knows how to shut off when it’s done.

If you want to fix a dishwasher that’s broken, or god forbid design a better one, you need to understand the inner guts of the beast, the mechanical nature of the machine that creates those superficial features that you know how to operate. You “know” how to operate the superficial nature of a TV, but how much do you understand of this:

Let’s take another different example. This Bosch dishwasher has only 6 buttons. Look how simple it is for any consumer to operate: 

But look how many parts there are inside. Why are some of the parts such weird shapes? How much of this do you understand? How much of it does the average operator understand: 

2. Abstractions

Successful models will always be expressed in terms of entities and rules. That might seem obvious — if you’re going to describe the world, of course you need to propose the units that populate it, and the rules that govern their behavior! 

But in fact, people almost never do this. Instead, they come up with descriptions that involve neither entities nor rules. These are called abstractions.

Abstractions group similar observations together into the same category. But this is superficial, because the classification is based on the surface-level attributes of the observations, not their nature. All crabs look similar, but as we’ve learned more about their inner nature, what we call DNA, we learned that some of these crabs are only superficially similar, that they came to their crab-like design from entirely different places. The same thing is true of trees.

We certainly cannot do without abstractions like “heat”, “depression”, “democracy”, “airplane”, and so on. Sometimes you do want to group together things based on their outward appearance. But these groups are superficial at best. Airplanes have some things in common abstractly, but open them up, and under the hood you will find that each of them functions in its own way. Democracies have things in common, but each has its own specific and mechanical system of votes, representation, offices, checks and balances, and so on. 

Imagine that your car breaks down and you bring it to a mechanic and he tells you, “Oh, your car has a case of broken-downness.” You’d know right away: this guy has no idea what he’s talking about. “Broken-downness” is an abstraction; it doesn’t refer to anything, and it’s not going to help you fix a car.

Instead, a good mechanic will describe your car’s problem in terms of entities and rules. “Your spark plugs are shot [ENTITIES], so they can’t make the pistons [ENTITIES] go up and down anymore [RULES].” 

It’s easy to see how ridiculous abstractions are when we’re talking about cars, but it can be surprisingly hard to notice them when we’re talking about science.

For instance, if you feel sad all the time, a psychologist will probably tell you that you have “depression.” But depression is an abstraction — it involves no theory of the entities or rules that cause you to feel sad. It’s exactly like saying that your car has “broken-downness.” Abstractions like this are basically useless for solving problems, so it’s not surprising that we aren’t very good at treating “depression.”

Abstractions are often so disassociated from reality that over time they stop existing entirely. We still use words like “heat”, “water”, and “air”, but we mean very different things by these words than the alchemists did. Medieval physicians thought of medicine in terms of four fluids mixing inside your body: blood, phlegm, yellow bile, and black bile. We still use many of those words today, but the “blood” you look at is not the blood of the humorists.

It’s possible that one day we’ll stop using the word “depression” at all. Some people find that idea crazy — depression is so common, so baked into our culture, that surely it’s going to stick around. But stuff like this happens all the time. In the 19th and 20th centuries, “neurasthenia” was a common diagnosis for people who felt sad, tired, and anxious. It used to be included in the big books of mental disorders, the Diagnostic and Statistical Manual (DSM) and the International Statistical Classification of Diseases and Related Health Problems (ICD). 

Now it isn’t. But that’s not because people stopped feeling sad, tired, and anxious — it’s because we stopped using “neurasthenia” as an abstraction to describe those experiences. Whatever people learned or wrote about neurasthenia is now useless except for historical study. That’s the thing about abstractions: they can hang around for a hundred years and then disappear, and we can be just as clueless about the true nature of the world as when we began. Don’t even get us started on Brain fag syndrome.

The DSM will never fully succeed because it’s stuck dealing with abstractions. One clue we’re still dealing with geocentric psychology here is that the DSM groups disorders by their symptoms rather than their causes, even though causes can vary widely for the same symptoms (e.g. insomnia can be biological, psychological, or your cat at 3 am).

Imagine doing this for physical diseases instead — if you get really good at measuring coughing, sneezing, aching, wheezing, etc. you may ultimately get pretty good at distinguishing between, say, colds and flus. But you’d have a pretty hard time distinguishing between flu and covid, and you’d have no chance of ever developing vaccines for them, because you have no concept of the systems that produce the symptoms.

Approaches like this, where you administer questionnaires and then try to squeeze statistics out of the responses, will always top out at that level. At best, you successfully group together certain clusters of people or behaviors on the basis of their superficial similarities. This can make us better at treating mental disorders, but not much better. 

If you don’t understand problems, it’s very unlikely you will solve them.

Abstractions are dangerous because they seduce you into thinking you know something. Medicine is especially bad at this. Take an abstraction, give it a Latin name, then say “because”, and it sounds like an explanation. You’ve got bad breath? That’s because you have halitosis, which means “bad breath”. This isn’t an explanation; it’s a tautology.

Will the treatment for one case of halitosis work on another case? Impossible to say. It certainly could. One reason things sometimes have the same surface appearance is because they were caused in the same way. But some people have halitosis because they never brush their teeth, some people have it because they have cancer, and other people have it because they have a rotting piece of fish stuck in their nose. Those causes will require different treatments.

Abstractions are certainly useful. But by themselves, abstractions are a dead end, because they don’t make specific claims. This is exemplified by flowchart thinking. You can draw boxes “A” and “B” and draw an arrow between them, but what is the specific claim made by this diagram? At most it seems to be that measures of A will be correlated with measures of B, and if the arrow is in one direction only, that changing measures of A will also change measures of B. 

That’s fine if this is the level of result you’re satisfied with, but it bears very little resemblance to the successes of the mature sciences. Chemistry’s successes don’t come from little flow charts going PROTON –> GOLD <—> MERCURY. If anything, that flowchart looks a lot more like alchemy. 

Abstractions can be useful starting points, but they’re bad ending points. For example, people noticed that snow melts in the sunlight and gold melts in a furnace. They noticed that hot water boils and that hot skin burns. It seemed like the same force was at work in all of these cases, so they called it “heat”.

The sensation of warmth, the force of sunlight, the similarities between melting and evaporation, are abstracted: “these go together so well that maybe they are one thing”. 

That’s only a starting point. Next you have to take the hypothesis seriously and try to build a model of the thing. What are the entities and rules behind all this warming, melting, and burning?

That’s what Lavoisier did: he came up with a model to try to account for these superficial similarities. Subsequent chemists proposed updates to the entities and the rules that did an even better job, and now we have a model that accounts for heat very well. We still call it “heat”, but because the model is a proposal about the underlying structure, it’s not superficial, so it’s not an abstraction. 

Game of Life

This is Conway’s Game of Life:

The universe of this game is an infinite two-dimensional grid of square cells. This means each cell has eight neighbors, i.e. the cells that are horizontally, vertically, and diagonally adjacent. 

The cells have only two properties — each cell is either alive or dead (indicated as black and white); and each cell has a location in the infinite two-dimensional grid. Time occurs in discrete steps and is also infinite. This is the full list of the entities in this world. 

At each step in time, the following rules are applied:

1. Any live cell with fewer than two live neighbors becomes dead.
2. Any live cell with two or three live neighbors stays alive.
3. Any live cell with more than three live neighbors becomes dead.
4. Any dead cell with exactly three live neighbors becomes a live cell.

This is the full list of the rules in this world.

All those parts, and no others, come together to create this world. You can try it for yourself here.

Despite being inspired by things like the growth of crystals, Conway’s Game of Life isn’t a model for any particular part of the natural world. However, it is an example of a set of simple entities, and simple rules about how those entities can interact, that gives rise to complex outcomes. 

This is the kind of model that has served as the foundation for our most successful sciences: a proposal for a set of entities, their features, and the rules by which they interact, that gives rise to the phenomena we observe. 

Instead of being a chain of abstractions, a flowchart that operates under vaguely implied rules, Conway’s Game of Life is a set of entities that interact in specific ways. And because it is so precise, it makes specific claims.

In principle, we can give you any starting state in the Game of Life, and you should be able to apply the rules to figure out what comes next. You can do that for as big of a starting state as you want, or for as many timesteps as you want. The only limit is the resources you are willing to invest. For example, see if you can figure out what happens to this figure in the next timestep:

Or if you want a more challenging example, try this one: 

There are, of course, an infinite number of these exercises. Feel free to try it at home. Draw a grid, color in some cells at random, and churn through these rules. Specific claims get made.

In comparison, take a look at this diagram. Wikipedia assures us that the diagram depicts “mental state in terms of challenge level and skill level, according to Csikszentmihalyi’s flow model”:

You might wonder what exactly is being claimed here. Yes, if you are medium challenged and low skilled, you are “worried”. But it’s not clear what that means outside of the context of these words.

This diagram is just mapping abstractions to abstractions. There is no proposal about the entities underlying those abstractions. What, specifically, might be going on when a person is medium skilled, or low challenged? LOW SKILL + HIGH CHALLENGE —> ANXIETY sounds like a scientific statement, but it isn’t. It’s like saying LOW CAR ACTIVITY + HIGH AMOUNTS OF WEIRD NOISES —> CAR BROKEN-DOWNNESS. Forget about such questions, what matters is that HIGH SKILL + HIGH CHALLENGE —> FLOW.

The Big Five is considered one of the best theories in psychology, and provides five dimensions for describing personality, dimensions like extraversion and openness. But the dimensions are only abstractions. The theory doesn’t make any claim about what constitutes being “high openness”, literally constitutes in the sense of what that factor is made up of. The claims are totally superficial. At most, the big five is justified by showing that its measures are predictive. This so-called theory is not scientific.

Modern scientists often claim that they are building models. However, these are usually statistical models. They are based on historical data and can be used to guess what the future will look like, assuming the future looks like the past. Statistical models predict relationships between abstract variables, but don’t attempt to model the processes that created the data. A linear regression is a “model” of the data, but no one really thinks that the data entered the world through a linear model like the one being used to estimate it.

This is made even more confusing because there is another totally different kind of “statistical model” found in fields like statistical physics. These are models in the sense that we mean. Despite involving the word “statistical”, they are nothing like a linear regression. Instead of looking backwards at historical data of abstract variables, models in statistical physics take hypothetical particles and step them forward, in an attempt to describe the collective behavior of complex systems from microscopic principles about how each particle behaves. These models are “statistical” only in the sense that they use probability to attempt to describe collective behavior in systems with many particles. 

We want a model that is a proposal for simple entities, their properties, and the rules that govern them, that can potentially give rise to the natural phenomena we’re interested in. The difference between the Game of Life and a genuine scientific model is simply that while the Game of Life is an artificial set of entities and rules that are true by fiat, answering to nothing at all about the real world, a scientific model is a proposal for a set of entities and rules that could be behind some natural phenomenon. All we have to do is see if they are a good match.

Particle Man

Physics first got its legs with a model that goes something like this. The world is made up of bodies that exist in three-dimensional space and one-dimensional time. The most important properties of bodies are their mass, velocity, and position. They interact according to Newton’s laws. There are also some forces, like gravity, though the idea of forces was very controversial at first. 

If you read Newton’s laws, you’ll see that these are the only entities he mentions. Bodies that have mass, speed/velocity, and a location in time according to space. Also there is a brief mention of forces.

Since this model was invented, things have gotten much more complicated. We now have electrical forces, Einstein changed the nature of the entities for space/time/mass, and there is all sorts of additional nonsense going on at the subatomic level.

We were able to get to this complicated model by starting with a simpler model that was partially right, a model that made specific claims about the entities and rules underlying the physical world, and therefore made at least somewhat specific predictions. These predictions were wrong enough to be useful, because they could be tested. Claims about the rules and entities could be challenged, and the models could be refined. They did more than simply daisy-chain together a series of abstractions. 

Coming up with the correct model on the first go is probably impossible. But coming up with a model that is specific enough to be wrong is our responsibility. Specific enough to be wrong means proposals about entities and rules, rather than superficial generalizations and claims about statistical relationships.

Like Lavoisier, we should be largely unconcerned as to whether these models are real or purely hypothetical. We should be more concerned about whether it “explains the phenomena of nature in a very satisfactory manner.” Remember that “we are not obliged to suppose this to be a real substance”!

As another example, consider different models of the atom.

Dalton was raised in a system where elements had been discovered by finding substances that could not be broken down into anything else. Hydrogen and oxygen were considered elements because water could be separated into both gases, but the gases themselves couldn’t be divided. So Dalton thought of atoms as indivisible. 

When electrons were discovered, we got a plum pudding model. When Rutherford found that atoms were mostly empty space, we got a model with a small nucleus and electrons in orbit. Emission spectra and other observations led to electron shells rather than orbits. None of these models were right, but they were mechanical and accounted for many observations.

The Nature of Science

Anyways, what is science?

Most people these days claim that the legitimacy of science comes from the fact that it’s empirical, that you’re going out and collecting data. You see this in phrases like, “ideas are tested by experiment”. As a result, people who do any kind of empirical work often insist they are doing science.

Testing ideas by experiment is essential — what else are you going to rely on, authority figures? But what kind of ideas are tested by experiment? Science can’t answer normative ideas, like “how should I raise my child?” or “what kind of hat is best?” It also can’t answer semantic ideas like “is a hot dog a sandwich?”

Some things are empirical but don’t seem very much like science at all. For example, imagine a study where we ask the question, “are red cars faster than blue cars?” You can definitely go out and get a set of red cars and a set of blue cars, race them under controlled conditions, and get an empirical answer to this question. But something about this seems very wrong — it isn’t the kind of thing we imagine when we think about science, and doesn’t seem likely to be very useful.

Similarly, you could try to get an empirical answer to the question, “who is the most popular musician?” There are many different ways you could try to measure this — record sales, awards, name recognition, etc. — and any approach you chose would be perfectly empirical. But again, this doesn’t really feel like the same thing that Maxwell and Newton and Curie were doing.

You could object to these studies on the grounds that the questions are moving targets. Certain musicians are very popular today, but someday a different musician will be more popular. Even if right now, across all cars, red cars are faster than blue cars, that may not be true in the future, may not always be true in the past. If you go far enough back in time, there weren’t any cars at all. 

You could also object that the results aren’t very stable, they can be easily altered. If we paint some of our red cars blue, if we spend some marketing dollars on one musician over another, the empirical answer to these questions could change. 

Both of these complaints are correct. But they identify symptoms, not causes. They reflect why the questions are nonsensical, but they’re not the source of the nonsense. 

Better to say, these studies are unscientific because they make no claim about the underlying entities.

We say that science is when metaphysical proposals about the nature of the entities that give rise to the world around us are tested empirically. In short, you propose entities and rules that can be tested, and then you test your proposal. Science does have to be empirical. But being empirical is not enough to make something science.

A good way to think of this is that we’re looking for a science that is not merely empirical, but mechanical, in the sense of getting at a mechanism. The ideal study tries to get a handle on proposals about the mechanics of some part of the natural world. And you can only get at the mechanics by making a proposal for entities and rules that might produce parts of the natural world that we observe. 

This isn’t always possible at first. When you hear there’s some hot new mold that cures infections, your first question should be plain and empirical — does it actually cure infections or not? The practical reason to firmly establish empirical results is to avoid dying of infections. But the scientific reason is so that you can come around and say, “now that we have established that this happens, let’s try to figure out why it happens.” Now you are back to mechanism.  

But you still have to be careful, because many things that people think are mechanisms are actually more abstractions. Psychology gets this wrong all the time. Let’s pick on the following diagram, which is theoretically a claim about mechanism, i.e. the mechanism by which your death/life IAT is correlated with some measure of depression. But “zest for life” isn’t a proposal for a mechanism, it’s just another abstraction. You need a specific proposal of what is happening mechanically for something to be a mechanism. 

Incidentally, this suggests that having a background in game design may give you a serious leg up as a theoretical scientist.

Game designers can’t be satisfied with abstractions. Their job is to invent mechanisms, to fill a world with entities and laws that make the gameplay they want to make possible, possible; the gameplay they don’t want impossible; and that help players have the intended experience. 

Compare this story from Richard Feynman: 

[My Father] was happy with me, I believe. Once, though, when I came back from MIT (I’d been there a few years), he said to me, “Now that you’ve become educated about these things, there’s one question I’ve always had that I’ve never understood very well.”

I asked him what it was.

He said, “I understand that when an atom makes a transition from one state to another, it emits a particle of light called a photon.”

“That’s right,” I said.

He says, “Is the photon in the atom ahead of time?”

“No, there’s no photon beforehand.”

“Well,” he says, “where does it come from, then? How does it come out?”

I tried to explain it to him—that photon numbers aren’t conserved; they’re just created by the motion of the electron—but I couldn’t explain it very well. I said, “It’s like the sound that I’m making now: it wasn’t in me before.” (It’s not like my little boy, who suddenly announced one day, when he was very young, that he could no longer say a certain word—the word turned out to be “cat”—because his “word bag” had run out of the word. There’s no word bag that makes you use up words as they come out; in the same sense, there’s no “photon bag” in an atom.)

He was not satisfied with me in that respect. I was never able to explain any of the things that he didn’t understand. So he was unsuccessful: he sent me to all these universities in order to find out those things, and he never did find out.

You can see why Feynman’s father found this frustrating. But to a game designer, nothing could be more trivial than to think that God designed things so that atoms spawn photons whenever the rules call for it. Where were the photons before? The question isn’t meaningful: “photons” is just a number in the video game engine, and when the rules say there should be new photons, that number goes up.

This is also why abstractions don’t work for science. Listening to someone explain a new board game is already one of the most frustrating experiences of all time. But imagine someone explaining the rules to you in abstractions rather than in mechanics.

In Settlers of Catan, the universe is an island consisting of 19 hexagonal tiles. Settlements can be built at the intersections of tiles, and tiles generate resources depending on their type. The game could be described abstractly. But this is not as useful as describing it mechanically:

MR. ABSTRACTIO: You can make a new settlement with resources. Maritime trade creates value. The player with the best economy wins. Okay, let’s play!

MR. MECHANICO: Building a new settlement requires a Brick, Lumber, Wool, and Grain card. A settlement or a city on a harbor can trade the resource type shown at 3:1 or 2:1 as indicated. You win by being the first to reach 10 victory points, and you earn victory points from settlements (1 point each), cities (2 points each), certain development cards (1 point each), having the longest road (2 points), and having the largest army (2 points).

Another source of unappreciated mechanical thinking is video game speedrunners. Game designers have a god’s-eye view of science, as they make the rules of a world from scratch; speedrunners are more like scientists and engineers, using experiments to infer the underlying rules of the world, and then exploiting the hell out of them. 

Many sciences like neuroscience and nutrition pretend to be model-building, but are actually just playing with abstractions. They appear to make claims about specific entities, but on closer inspection, the claims are just abstractions in a flowchart.

This can be hard to spot because many of these entities, like neurotransmitters or vitamins, really are specific entities in the chemical sense. But in neuroscience and nutrition these entities are often invoked only as abstractions, where they interact abstractly (e.g. more of X leads to more of Y) rather than mechanically. They tell you, “X upregulates Y.” How fascinating, what are the rules that lead to this as a consequence? 

As neuroscientist Erik Hoel puts it:

If you ask me how a car works, and I say “well right here is the engine, and there are the wheels, and the steering wheel, that’s inside,” and so on, you’d quickly come to the conclusion that I have no idea how a car actually works.

Explanations are often given in terms of abstractions. “Please doc, why am I depressed?” “Easy, son: Not enough dopamine.” If you’re like us, you’ve always found these “explanations” unsatisfying. This is because abstractions can’t make sense of things. They just push the explanatory burden on an abstract noun, and hope that you don’t look any deeper.

Explanations need to be in terms of something, and scientific explanations need to be in terms of a set of entities and their relationships. Why do sodium and chlorine form a salt? Because one of them has one extra electron in its outer shell, leading to a negative charge, while one has one missing electron in its outer shell, leading to a positive charge, and they form an ionic bond. This is why chlorine also readily forms a salt with potassium, etc. etc. The observed behavior is explainable in terms of the entities and their properties we’ve inferred over several hundred years of chemistry, interacting according to the rules we’ve inferred from the same. 

The fake version of this can be hard to spot. “Why am I depressed? Not enough dopamine” sounds a lot like “Why does my car not start? Not enough gasoline.” But the second one, at least implicitly, leads to a discussion of spark plugs, pistons, and fuel pumps acting according to simple rules, genuine mechanics’ mechanics. The first one promises such an implied mechanism but, in our understanding at least, does not deliver. 

This also dissolves one of our least-favorite discussions about psychology, whether or not there are “real truths in the social sciences”. There may or may not be real truths in the social sciences. But human behavior, and psychology more generally, is definitely the result of some entities under the hood behaving in some way, and we can definitely do more to characterize those entities and how they interact. 

There’s a common misunderstanding. We’ll use an example from our friend Dynomight, who says: 

Would you live longer if you ate less salt? How much longer? We can guess, but we don’t really know. To really be sure, we’d need to take two groups of people, get them to eat different amounts of salt, and then see how long they live.

This way of thinking follows a particular strict standard, namely “randomized controlled experiments are the only way to infer causality”. But this isn’t really how things have ever worked. This is pure extrapolation, not model-building. In contrast to the impressionistic research of inventing abstractions, you might call this brute-force empiricism.

Experiments are useful, but we can’t let them distract from the real goal of science, which is building models that work towards a mechanistic understanding of the natural world. 

To get to the moon, we didn’t build two groups of rockets and see which group made it to orbit. Instead, over centuries we painstakingly developed a mechanical understanding of physics, or at least a decent model of physics, that allowed us to make reasonable guesses about what kind(s) of rockets might work. There was a lot of testing involved, sure, but it didn’t look like a series of trials where we did head-to-head comparisons of hundreds of pairs of rocket designs, one pair at a time.

So to “get to the live longer”, we probably won’t build a low-salt and high-salt diet and fire them both at the moon. Instead we will, slowly, eventually, hopefully, develop a mechanical understanding of what salt does in the body, where things are likely to go well, and where they’re likely to go wrong. Then we will compare these models to observations over time, to confirm that the models are roughly correct and that things are going as anticipated, and we’ll correct the models as we learn more.

It won’t look like two groups of people eating broadly different diets in large groups. That is science done with mittens. There is a better way than losing all articulation and mashing together different conditions.

Astronomy may have forced us to do science the right way because it enforces a “look but don’t touch” approach. Newton didn’t run experiments where he tried the solar system one way and then tried it the other way. Instead he (and everyone else) looked, speculated, came up with models, and saw which models would naturally cause the action they had already seen in the heavens. None of the models were entirely right, but some of them were close, and some of them made interesting predictions. And in time, some of them got us to the moon.

Philosophy Time

These are the insights you need to make sense of the famously confusing but deeply insightful philosopher of science Thomas Kuhn.

One-paragraph background on Kuhn: Thomas Kuhn was a philosopher of science who introduced the concept of “paradigms”. According to Kuhn, each science (biology, chemistry, etc.) is built on a paradigm, and scientific progress is more than the slow accumulation of facts, it involves revolutions, where an old paradigm is tossed out and a new one installed as the new foundation. 

But even though it’s his biggest concept, Kuhn can be kind of vague about what a “paradigm” involves, and this has led to a lot of confusion. So let’s try to pin it down.

A paradigm is not just a shared set of assumptions or tools and techniques. If it were, any tennis club would have a paradigm. 

A paradigm is specifically a proposal (or rather, class of proposals) about the entities, properties, and relationships that give rise to some natural phenomenon.

Kuhn says: 

Effective research scarcely begins before a scientific community thinks it has acquired firm answers to questions like the following: What are the fundamental entities of which the universe is composed? How do these interact with each other and with the senses? What questions may legitimately be asked about such entities and what techniques employed in seeking solutions? At least in the mature sciences, answers (or full substitutes for answers) to questions like these are firmly embedded in the educational initiation that prepares and licenses the student for professional practice.

(The Structure of Scientific Revolutions, Chapter 1)

Why “a class of proposals” and not “a proposal”? Well, because the specifics are always very much up for debate, or at least subject to empirical scrutiny. Any particular proposal, with exact values and all questions pinned down, cannot be a paradigm. A paradigm is a general direction that includes some flexibility. 

For example, we may not know if the mass of a specific particle is 2 or 1 or 156 or 30,532 — but we do agree that things are made up of particles and that one of the things you can say about a particle is that it has some mass. 

There may even be disagreement about the limits of the proposal itself — can the mass of a particle be any real number, say 1.56, or is mass limited to the positive integers, like 2, 4, and 10? Can the mass of a particle be negative? But in general we have a basic agreement on what kind of thing we are looking for, i.e. the types of entities, their features, and their interactions. 

Kuhn gives an example based on Descartes’s corpuscularism. Descartes didn’t give a specific proposal about exactly what kinds of corpuscules there are, or exactly the rules by which they can interact. Instead, it was more of an open-ended suggestion: “hey guys, seems like a good model for physics would be something in the class of proposals where all things are made up of tiny particles”: 

After the appearance of Descartes’s immensely influential scientific writings, most physical scientists assumed that the universe was composed of microscopic corpuscles and that all natural phenomena could be explained in terms of corpuscular shape, size, motion, and interaction. That nest of commitments proved to be both metaphysical and methodological. As metaphysical, it told scientists what sorts of entities the universe did and did not contain: there was only shaped matter in motion. As methodological, it told them what ultimate laws and fundamental explanations must be like: laws must specify corpuscular motion and interaction, and explanation must reduce any given natural phenomenon to corpuscular action under these laws. More important still, the corpuscular conception of the universe told scientists what many of their research problems should be. For example, a chemist who, like Boyle, embraced the new philosophy gave particular attention to reactions that could be viewed as transmutations. 

(The Structure of Scientific Revolutions, Chapter 4)

Kuhn’s arguments definitely line up with one proposal: a book by the cyberneticist William Powers, called Behavior: The Control Of Perception. And the two men must have recognized at least some of this in each other, judging from the blurb that Kuhn wrote for Powers’s book: 

Powers’ manuscript, Behavior: The Control of Perception, is among the most exciting I have read in some time. The problems are of vast importance, and not only to psychologists; the achieved synthesis is thoroughly original; and the presentation is often convincing and almost invariably suggestive. I shall be watching with interest what happens to research in the directions to which Powers points.

And it’s worth considering what Powers says about models:

In physics both extrapolation and abstract generalization are used and misused, but the power of physical theories did not finally develop until physical models became central. A model in the sense I intend is a description of subsystems within the system being studied, each having its own properties and all—interacting together according to their individual properties—being responsible for observed appearances.

As you can see, this is another description of a model based on rules and entities.

The final concept to take away here is that these models are mechanistic. There’s a reason that Descartes was celebrated for his mechanical philosophy. When you assume the universe is akin to a gigantic clock, a real machine where the hands and numbers on the face are driven by the interaction of gears and levers below, your theories will be mechanical too. They will appeal to the interaction of gears and wires, rather than to abstract notions of what is happening on the clock face. (“The minute-hand has minute-force, and that’s why it moves faster than the hour-hand, which only has hour-force.”)

If a model is not mechanical in this way, if it does not speculate about the action of mechanisms beneath what is seen, it will be superficial. And it is not enough to speculate about things beneath. You can layer abstractions on abstractions (e.g. your anxiety is caused by low self-esteem). But you can’t design a watch without talking about individual pieces and how they will interact according to fixed rules.

A Third Direction for the Mind

Psychology is pre-paradigmatic. It’s not simply that we can’t agree on what entities make up the mind — it’s that there have been almost no proposals for these entities in the first place. There are almost no models, or even proposals for models, that could actually give rise to even a small fraction of the behavior we observe. A couple hundred years of psychology, and almost all we have to show for it are abstractions. 

But there are a few exceptions, proposals that really did try to build a model. 

The first major exception is Behaviorism. This was an attempt to explain all human and animal behavior in the terms of reward, punishment, stimulus, and muscle tension, according to the laws of association. If, after some stimulus, some muscle tension was followed by reward, there would be more of that muscle tension in the future following that stimulus; if followed by punishment, there would be less.

This ended up being a terrible way to do psychology, but it was admirable for being an attempt at describing the whole business in terms of a few simple entities and rules. It was precise enough to be wrong, rather than vague to the point of being unassailable, which has been the rule in most of psychology.

A more popular proposal is the idea of neural networks. While models based on this proposal can get pretty elaborate, at the most basic level the proposal is about a very small set of entities (neurons and connections) that function according to simple rules (e.g. backpropagation). And it’s hard to look at modern deep learning and large language models and not see that they create some behavior that resembles behaviors from humans and animals.

That said, it’s not clear how seriously to take neural networks as a model for the mind. Despite the claim of being “neural”, these models don’t resemble actual neurons all that much. And there’s a thornier problem, which is that neural networks are extremely good function approximators. You can train a neural network to approximate any function; which means that seeing a neural network approximate some function (even a human behavior like language) is not great evidence that the thing it is approximating is also the result of a neural network.

Finally, there is a proposal that the main entities of the mind are negative feedback loops, and that much or even all of psychology can be explained in terms of the action of these feedback loops when organized hierarchically. This proposal is known as cybernetics.

[Next: THERMOSTAT]