I.

The history of science is old, but most research methods are not. 

People have been doing research for thousands of years. But many of the methods we consider standard today — including questionnaires, blind and double-blind research, and the idea of control groups — were all invented after 1700. The first randomized controlled trial in medicine wasn’t published until 1948, and the term “evidence-based medicine” didn’t find its way into print until 1992. 

Research methods are still very new, probably we can sit down and invent some more. 

This is good news, because right now there are many problems that we have no idea how to solve. One area of particular mystery is human health. Doctors can do a lot for you with surgery, vaccines, and antibiotics, but outside of these interventions there remain many ailments that totally stump the system. 

A weird part of the postmodern experience is that many people feel kinda bad all the time, even if they aren’t “sick”. If you go to the doctor and you’re like “I’m feeling kinda bad”, they don’t know how to help you.

Being “actually sick” doesn’t get you much further. If anything it’s worse. Lots of people have mystery chronic illnesses, but when you go to doctors with one of these problems, they mostly just shrug at you. 

II. 

Alistair Kitchen began having stomach pain. It started out small, but over time grew to “an intensity of pain I didn’t know my body was capable of producing, a literally blinding sensation that shut down every sense in my body except the sensations of my stomach.” He says:

So, four years of this. In the third year, after an endoscopy and a series of scans had cleared me for anything “serious”, the advice given to me was, essentially, this:

Look, some people just have trouble with their stomachs. When they have trouble and we don’t know what is causing it, we just call it IBS. So you have IBS. Watch out for foods that might trigger you, and good luck.

Or take for example the experience of Elisabeth von Nostrand: 

I had a lot of conversations like the following:

Me (over 20 pages of medical history and 30 minutes of conversation): I can’t digest protein or fiber, when I try it feels like something died inside me. 

Them: Oh that’s no good, you need to eat so much protein and vitamins

Me: Yes! Exactly! That’s why I made an appointment with you, an expensive doctor I had to drive very far to get to. I’m so excited you see the problem and for the solution you’re definitely about to propose.

Them: What if you took a slab of protein and chewed it and swallowed it. But like a lot of that.

Me: Then I’d feel like something died inside me, and would still fail to absorb the nutrients which is the actual thing we want me to get from food.

Them: I can’t help you if you’re not willing to help yourself.

It’s an uncomfortably common story.

Having faced this system, many people end up taking their health into their own hands. This makes a lot of sense and we fully endorse it. But most people have no more success on their own than they do with doctors (though at least they’re not being condescended to). 

It seems like the average outcome is that you end up living with your mystery illness (or even just your mystery sense-of-mild-feeling-bad-all-the-time) for years. It either never goes away, or randomly goes away some day for no apparent reason.

III. 

We suspect that people do about as well on their own as they do with doctors because *no one* knows how to study individual issues. This is because our civilization has done a good job developing population-level research techniques, but a crummy job so far coming up with individual-level research techniques.

Our society has devoted a lot of time to doing research on large groups. We’ve come up with lots of ways of running studies on large samples, and lots of ways of thinking about it. We’d bet that 99% of the studies you’ve ever read are studies on groups.

In comparison, doing research on individuals is a very understudied and (dare we say) cutting-edge form of research. Scientists mostly haven’t developed techniques for it, because almost by definition, it isn’t the kind of thing they study. 

Possibly this is because doctors and researchers are more interested in population-level issues. After all, they are usually tasked with solving public health crises, tasked with curing common diseases, things that might affect millions of people. But individuals care more about, well, individuals.

Possibly this is because we started by focusing on the most common illnesses and are only now getting around to the rare ones. Common illnesses are best studied by looking at large groups, so we developed those techniques first, and are only now running up against their limitations.

Possibly it is all a question of computational power. The history of statistics is tainted, because statistics was invented before computers, and was designed within the limits of what a person can reasonably calculate by hand. Even up to the 1990s, consumer machines would take weeks to crunch the kinds of models that today you can run in 15 minutes on your phone. But now we can do more, and maybe that means we can do new things, things that weren’t possible before.

In any case, it must be possible to come up with protocols for such a thing. 

Individual-level research comes with certain advantages. The problem with population-level techniques is that the same treatment will always work better for some people than for others. If you give two people the same drug, it might work great for one of them and not work at all for the other, and your statistical modeling needs to take that into account. Individual-level research doesn’t have to worry about that! You are just looking at one person.

A technical way of describing this is that individual-level research always has high internal validity — the research question is “does this treatment protocol work for this person” and you always get a straight answer to that question. This comes at the cost of external validity — you have basically no idea whether your findings will generalize to any other person. That’s an ok tradeoff, because you are already choosing to study an individual, and because population-level techniques have questionable external validity to begin with.

We may also be able to use population-level techniques to guide individual-level research, and individual-level techniques to guide population-level research; there may be many ways in which they are complementary.

Individual-level techniques won’t be limited to studying chronic illness — you could also use them, for example, to make healthy people feel amazing more often, which would be pretty cool. People who already feel amazing all the time, you’re on your own. 

But chronic illness is a good place to start, because these illnesses are a drain on the lives of millions of people who are motivated to figure out a treatment, and population-level medicine isn’t cutting it.

This is a problem we have been mulling over and that we will be writing about over the next couple months. To start with, here are some simple distinctions that seem like they might come in handy: 

Testing vs. Finding Variables

When studying an individual, there are two main situations.

One situation is where you think you know some variables that are involved with your illness, and you want to test them. For example, you may suspect that caffeine makes you feel nauseous. You want to confirm this hypothesis or rule it out. You might also want to demonstrate to a high degree of certainty that caffeine is really a trigger for you, so you can write about it on the internet and other people with random nausea can benefit from your example, possibly by trying it for themselves.

The other situation is where you have no idea what is causing your illness. When you have no clear leads, you want techniques that can help you find variables that might plausibly be involved. This is a much harder problem, but it’s also much more important, because many people are chronically sick and have no idea what is causing their illness. 

This sucks, and it’s very tricky because there are approximately infinity variables in the world. But probably we can do better than “try variables at random” (even if some level of luck is inevitably involved), and we should see if we can come up with techniques for this situation.

Triggers vs. Deficiencies

Sometimes a chronic illness is caused by getting too much of something, like an allergen. These are generally known as triggers. Sometimes a chronic illness is caused by not getting enough of something, like a vitamin. We call these deficiencies. 

This distinction seems like it might be helpful, because the techniques for finding triggers may be very different from the techniques for finding deficiencies. (More on this in future posts.)

We should also keep in mind that chronic illnesses can be more complicated than simply getting too much or too little of something. Some chronic illnesses aren’t caused by any external variable — there’s always the possibility that you have a brain tumor or something, in which case there may be no triggers or deficiencies involved.

Ruling In vs. Ruling Out

Sometimes we will be in the fortunate position where we can get a lot of evidence that X causes Y. If we can conclusively pin things down and show that dairy causes your chronic nausea, that’s great. Now you can keep yourself from feeling sick all the time, and that’s enormously valuable. 

But sometimes we won’t be able to get evidence in favor of anything — we will only be able to disprove things. It’s important to remember that this is valuable too. Maybe you suspect that your nausea might be caused by dairy, caffeine, alcohol, or fatty foods. If you go through them one by one and rule them all out — nope it’s not dairy, it’s not caffeine, not alcohol, not fatty foods — that’s still good to know. 

It will be disappointing that you continue to feel nauseous all the time and you don’t know why. It may feel like a step backwards, because you’ve ruled out all your best guesses. But it’s still enormous progress. Disproving a hypothesis is valuable, and at least now you’ll be able to enjoy your milkshakes, Irish coffees, and beer-battered onion rings without fear.

The best thing research can offer you is a cure, but the second-best thing it can offer is some peace of mind.