Dark Matter: Now You See It, Now You Don’t

Two interesting claims about dark matter this week, and on the face of it, completely contradictory, but in fact, not obviously so. Before saying one word more, let me repeat my mantra — something that all physicists know but relatively few non-scientists appreciate — most claims of a radical new result turn out to be largely or completely wrong. This is not because physicists are stupid but because doing science at the forefront of knowledge involves using novel techniques that might have unknown pitfalls, and also because a single small mistake can create a fake effect (as we saw most recently with the OPERA neutrino speed measurement.)  And because nasty statistical accidents can play tricks on you.

Both claims that I’m about to describe use novel techniques, and their analyses have not been repeated by anyone else. At this point you should understand that both are tentative, and (based on the history of radical claims) the odds are against them. Both might be wrong. That said, both analyses look to me as though they’ve been reasonably well done, and if a mistake has been made, it will require someone far more expert in dark matter studies than I am to point it out.

So let me describe them in turn, to the best of my ability.

Now You See It

First, the claim (which I mentioned in yesterday’s post) of a possible signal of photons being produced with an energy of somewhere around 130 GeV in locations near (but not necessarily extremely near) the center of the Milky Way galaxy. I recently wrote an article on how you can get photons of a definite energy from dark matter annihilation, which I encourage you to read first if you haven’t already, since I’m going to assume you know what’s in it.

Christoph Weniger has put out a preprint (not yet peer-reviewed) in which he uses data collected by the Fermi Large Area Telescope (a satellite experiment that measures gamma-rays, i.e. high energy photons). He looks at their data in particular regions of the galaxy, of various sizes, that include its center. These are regions where he calculates (based on certain assumptions) that a photon signal from dark matter would most easily be detected. What he’s looking for are places where the signal from dark matter would be expected to be large and background from astrophysical processes would be relatively small.

Fig. 1: Taken from Weniger's paper. Left, the region in which photons are selected; the galactic center is at the middle of the picture and the white areas reach well into the galactic bulge (see Figure 3 below.) Right, two classes of data in black and purple dots show the number (not exactly, but let me skip this detail) of photons versus the energy of the photon. A notable excess above a smooth background appears somewhere around 129 GeV (marked with a blue vertical dashed line). You can ignore the black dashed line.

He looks in five such regions, and in three of them he finds that when he plots the number of photons versus the energy per photon, there is an excess for photons with energy roughly centered around 129 GeV. (The three regions aren’t independent of one another, so this isn’t three independent excesses but rather one excess viewed three ways.) I won’t show you all of the data (you can look at Figure 1 of his preprint for the whole thing) but in my own Figure 1 is an example plot, showing at left one region that he is looking in (including the galactic center and two bulbous regions above and below the center) and at right the data. You can see a bump around 130 GeV (the blue vertical dashed line is at 129; the black dots and the purple dots represent two data sets; the gray dashes represent a smooth fit to the data sets.) Weniger then looks at the region near 130 more carefully, as shown in Figure 2 (again there is a lot more information in the preprint.) The green dashes are a fit that is a smooth structureless curve, while the red dashes are a fit to a smooth curve plus the blue curve at the bottom, which is shaped the way a signal from dark matter would be expected to look. (See my article on how why this is what you expect.)  The blue curve is rather wide because no experiment is perfect, and the Fermi Telescope is no exception; the actual energies of the photons would be expected to form a very narrow peak were it not for these imperfections. Weniger’s claim is that the red curve fits the data (black points, with vertical uncertainty bars showing one standard deviation) much better than the green curve, indicating the presence of a signal.

Fig. 2: From Weniger's paper, a more precise analysis of the data, showing the number (really the number this time, unlike Figure 1 above) of photons observed with a given energy E. Top: the data is in black dots, green is the best fit to a smooth distribution, red is the best fit to a smooth distribution plus the blue peak centered around 129. Bottom: the difference between the data and the red fit, showing that the fit works pretty well statistically.

Now, one has to step back here. If you take any data that approximates a smooth curve, you will always find bumps and wiggles on it. That’s just basic statistics; the number of events will sometimes be a little higher and sometimes a little lower than you expect. The question is not if there’s a bump somewhere; it’s whether there is a statistically significant bump, one which is so large that it is unlikely to have occurred by accident. What Weniger claims (and I cannot easily verify) is that the bump he sees has a statistical significance of 3.3 standard deviations after taking into account, conservatively, the look-elsewhere effect. (Without the look-elsewhere effect the significance is 4.6 standard deviations.) Naively that’s a one in a thousand effect, big enough to take seriously — but such effects sometimes go away with more data, so it’s not completely convincing yet.

And meanwhile, the question remains whether he did his statistical calculation correctly. I’ve watched previous theorists claim 3 standard deviation effects and then be contradicted by expert experimentalists, who tend to understand the subtleties of statistical arguments better. So before we take this result too seriously, we need to hear what other experts, including those from the Fermi Telescope itself, have to say.

As Weniger himself emphasizes in his title, this is at best a tentative detection, and as he points out in his abstract, it will be several years before we have enough data to be sure whether this is a fluke or a real signal. So I am afraid that even if Weniger has identified a real effect, we may have no choice but to be patient for two to four years and hope nothing goes wrong with the Fermi satellite… unless someone can put up a competing experiment in the meantime, or find corroborating evidence in some other way.

Now You Don’t

The second paper in question (peer-reviewed; here’s a pdf) is by Moni Bidin, Carraro, Menendez and Smith. They claim that there is no sign of any dark matter within a region around the sun of about ten thousand or so light years in radius. [A light year is the distance light travels in a year; for comparison, it takes light less than three seconds to travel to the moon and back.] The Milky Way’s center, where dark matter is expected to be most abundant and where Weniger is looking for photons from dark matter annihilation, is about twenty-five thousand light years away from the Sun, so the region that this group is studying lies far from the center; see Figure 3.

Fig. 3: The Milky Way galaxy consists of stars and other ordinary matter (white) and is believed to lie within a large, complicated, and poorly understood `halo' of dark matter with many sub-haloes. The possible detection of photons from dark matter is coming from the central region of the galaxy marked in green. The claim that gravitational effects of dark matter are not being observed, suggesting dark matter is in less than expected quantities, is being made within the red region centered around the sun. Note that this is a sketch, and all locations and densities indicated are very approximate.

Their approach is to study the gravitational effects of nearby matter, as reflected by the motions of a small sample of the stars that lie within a few thousand light years of the sun.  [Sorry -- I still don't understand their method well enough to explain it to non-experts, but will fill in more details if and when I do.] . And they claim that the motions of those stars suggest that the only matter nearby is the ordinary matter we can see around us, with no dark matter in addition. They claim their results rule out almost all existing guesses/models for how the dark matter in the galaxy is distributed. This is shown in Figure 4. The result of the paper is the solid black line I have marked “Data”, with uncertainties marked with 1 and 3 standard deviation bands labeled “1 s.d.” and “3 s.d.”  The line marked VIS is their estimate for what the effect of all the VISible matter in the sun’s vicinity would be. The line marked MIN is their estimate for the effect of visible matter plus what they view as the MINimal expected amount of dark matter. The other lines represent the expected effects of dark matter distributed in halos of various different shapes proposed by various experts.

Fig. 4: Taken from Moni Biden et al., with my additional annotation. A quantity called surface density obtained from star data (black line marked "Data", with 1 and 3 standard deviation uncertainties in dotted and dashed lines) compared against an estimate (VIS) of what it would look like if only visible matter were present in the region relatively nearby to the sun (see Figure 3) and various models of how dark matter might be distributed in the galaxy (MIN being what the authors claim is the minimal option, SHM being a common assumption.) No uncertainty estimates on the grey lines are provided.

There are two questions you have to ask of such claims.

• First, did they do a correct analysis? Is the result that they present, and the uncertainty on that result, accurately obtained?
• Second, did they interpret their analysis correctly? Do they really rule out there being any significant amount of dark matter in the vicinity of the sun?

I can’t evaluate their particular method; it’s somewhat complicated and lies outside my expertise and intuition. (Nor do I know much about the star data that is used as an input to their method.) But it appears quite interesting and somewhat innovative. They authors present a long list of assumptions on which their technique rests, and argue that relaxing the assumptions doesn’t change the answer that much. We’ll have to wait for experts to tell us if they feel that the assumptions are too strong, or if there are hidden assumptions that are even more important and worrying than the ones that were mentioned. But with so many assumptions, it wouldn’t surprise me if the uncertainties are underestimated.

You might ask: does this paper suggest that there is no such thing as dark matter, and that we need to modify gravity to explain effects often attributed to dark matter? Well, you can’t jump to this conclusion, because the method used by these authors assumes that gravity isn’t modified. So if you modify gravity, you’d also potentially modify this result too. You’d need to redo their analysis carefully with your favorite modification of gravity before you could learn anything.

One thing I do know is that the distribution of dark matter in our galaxy is very poorly known. The models of dark matter that they claim to rule out have significant uncertainties, and these grey lines in Figure 4 really aren’t narrow lines — they are thick bands. For instance, dark matter may be very clumpy, not smooth, whereas all of the halos they consider are assumed to be reasonably smooth. They claim that the lines shown are the most conservative possible versions of any given model, but I suspect that experts in these models would disagree; we’ll have to ask. And there’s no way that we know the amount of visible matter in the sun’s general vicinity to such precision as the line marked VIS would suggest; that too should be a thick band. So I think we need to be very careful interpreting this graph. It doesn’t say “there’s no dark matter in the vicinity.” It says “there can’t be as much dark matter as the models were expecting.” And maybe that’s just a statement that I’ve heard before — was it at the start of this paragraph? — that the distribution of dark matter in our galaxy is very poorly known.  If the dark matter halo is sufficiently clumpy and complex, the current result may reflect things going on near the sun but might not reflect anything overly significant about the overall properties of the halo.

One type of experiment for which the amount of dark matter in the local neighborhood is very important is the search to directly detect the rare collisions of dark matter particles with atomic nuclei. [I've briefly described some of this effort here, but I need to write a longer article about it.] Obviously, the less dark matter there is near the Earth, the harder it is to find these particles. The authors of the paper go so far as to say that such efforts might be impossible because their results suggest the amount of dark matter around the Earth is “negligible”. Well.  Hmm.  Given what they’ve actually shown (Figure 4), the only thing they can actually say is that the amount of dark matter is somewhat smaller than people assume, but certainly not that it’s negligible. Even if the amount of dark matter were 20 percent of what people expected, that just would mean it would take longer to find any particular type of dark matter — but there are many possible types to look for. The real implication would be that what people currently think is known about dark matter particles would have to be somewhat weakened.  But people already know that the local density of dark matter is very uncertain, so the current result’s impact on what people think they know would be limited.

So while the method is interesting and their result very intriguing, I would worry at this point that they are going way too far with their interpretation of it. We’ll need to see the method verified, the assumptions tested, and the approach applied to larger groups of stars with resulting smaller uncertainties before we can be sure the result is as dramatic as the authors claim.

A sociological comment: Frankly, I would trust this result more if it hadn’t been for the strongly worded and almost inflammatory press release that accompanied this paper.  In my experience, when papers are made public at the same time as a press release, the more confident the press release sounds, the less likely the paper is correct. Why? Because when people are so confident in their result that they immediately put out a powerful statement for the press and public to read, well before their expert colleagues have had some time to consider, test and criticize the result, it says a lot about their personalities and scientific approach. Nothing humble about these folks! They are absolutely certain there’s no dark matter around here whatsoever, and that they couldn’t possibly have overlooked anything. Well, in my experience, that attitude often leads to subtle unconscious biases. It appears, from the way this paper and the press release are worded, that these authors really wanted to show there was no dark matter nearby… that they had a strong prior agenda. It’s always better not to have a strong bias, when you start a scientific analysis, as to what the result is going to be.

But even if this paper turns out not to be correct in its results or its interpretation, it is very interesting in that it shows that it is feasible to measure the local dark matter density using a method of this type. Surely the method can be improved and applied to larger data samples in the future. So it seems to me that the authors already deserve credit for some pioneering work that may herald an era of better and better measurements of this type, which might lead us eventually to a more convincing understanding of how dark matter is distributed across our galaxy — or perhaps even show that the current conventional view of dark matter’s role in the galaxy is mistaken.

And of course, if the result is correct, then it will be important to interpret it very carefully, and understand what it does and doesn’t imply.  I feel that the only thing this paper can be said to show is that there is relatively little dark matter in the region near the sun, but the galaxy is a very big place, the dark matter is expected to be distributed in a complex fashion, and most of the dark matter is expected to be near the center. See Figure 3.   Consequently I think we really don’t know that much yet.   But this is a research area worth keeping a close eye on.

And meanwhile, we’d also better seek a second opinion before we get overly excited.

Detected? Rejected?

So, has dark matter been detected at the center of the galaxy? Has dark matter been rejected in the sun’s galactic neighborhood? Claims by scientists like these have to be treated with respect — both of these groups are breaking new ground. They’ve done work that is challenging and interesting and innovative — and risky as a result. Either or both papers may be wrong; history shows that the odds are against them.  Much as we’d like to know today what their results mean, we have no choice but to be patient, for it will likely take a few years of additional measurements and scientific debate for nature’s true story to be revealed.