MilkyWay@home's volunteers rescue the crowdsourced search for dark matter

The MilkyWay@home Project lets more than 160,000 amateur space explorers create a virtual global supercomputer that’s mapping the Milky Way’s invisible dark matter and simulating the Galaxy’ collisions with other galaxies. Now America’s science funding crunch puts MilkyWay@home in jeopardy. Can the crowdsourced project’s volunteer community come to the rescue?

The Search for Dark Matter

MilkyWay@home explores one of the great mysteries of modern science: dark matter. When astronomers measured the motion of the Milky Way and other galaxies, their observations didn’t match their predictions. The only way to reconcile the models to reality is if there’s much more mass in the Universe than we can see. Dark matter is the prevailing explanation. Some kind of matter that has mass and yet doesn’t interact electromagnetically with visible matter dominates the Universe. Nobody can see it, but it’s there - making its presence felt by the gravitational forces it exerts. 

The Milky Way rips stars away from a doomed dwarf galaxy. The collision leaves the stars scattered in relatively dense streams in the thinly populated galactic halo.  Credit: Shane Reilly, MilkyWay@home

The Milky Way rips stars away from a doomed dwarf galaxy. The collision leaves the stars scattered in relatively dense streams in the thinly populated galactic halo. Credit: Shane Reilly, MilkyWay@home

Dr. Heidi Newberg, a professor at the Rensselaer Polytechnic Institute, decided to look at the way gravitational forces from the Milky Way’s dark matter shapes streams of stars in the halo of stars surrounding its spiral disk. A stream forms when dwarf galaxy collides with the Milky Way. Tidal forces rip stars away from the smaller galaxy, scattering them in streams that wind through the galactic halo. Mapping those streams, Newberg figured, would let her map the distribution of the Galaxy’s dark matter.

In this visualization of a MilkyWay@home simulation, the bright spiral in the center is the Milky Way. To the right is the Sagittarius Dwarf Elliptical Galaxy with its stream of stars wrapping around the Milky Way. The yellow fan represents the observed data from the Sloan Digital Sky Survey.  Credit: Shane Reilly, MilkyWay@home

In this visualization of a MilkyWay@home simulation, the bright spiral in the center is the Milky Way. To the right is the Sagittarius Dwarf Elliptical Galaxy with its stream of stars wrapping around the Milky Way. The yellow fan represents the observed data from the Sloan Digital Sky Survey. Credit: Shane Reilly, MilkyWay@home

Of course there’s a catch: the stars are too far away to measure their motion. To get around this, Newberg’s team creates a model of the Milky Way and uses a supercomputer to simulate the tidal streams circling the Galaxy. They compare the results to reality as observed by the Sloan Digital Sky Survey, tweak their assumptions, and runs the supercomputer again. And again. And again.

But that creates another problem: running so many simulations requires a lot of supercomputer time. The limited number of academic supercomputers are in high demand. Competition among researchers for the supercomputers’ time - and competition for the grants to fund it - is fierce. Without its own $100,000,000 supercomputer center, a project like MilkyWay@home wouldn’t happen - at least not through traditional approaches. 

Crowdsourcing a Dark Matter Search to Amateurs

Using the University of California Berkeley’s Boinc system, MilkyWay@home parcels the work over the Internet to a community of volunteers whose desktop computers process the data. A 2008 paper in the peer-reviewed Astrophysical Journal (DOI: 10.1086/589681) showed how the original network of 500 volunteers performed as well as RPI’s IBM BlueGene/L supercomputer - the 12th most powerful supercomputer in the world at the time. [The paper is a free download but you can also get the arXiv preprint.]

In a paper presented last year to the International Astronomical Union (DOI: 10.1017/S1743921313006273), Newberg’s team explained that their network had over 162,382 volunteers in 206 countries. About 25,000 people at any given time process the MilkyWay@home simulations, generating more than 500 TeraFlops of number-crunching power - at the time equivalent to the 45th most powerful stand-alone supercomputer in the world. [The paper is behind a paywall but you can download this PDF from RPI.]

Modern astrophysics increasingly depends on having researchers who understand software engineering and big data analytics in addition to the science. Newberg uses MilkyWay@home to give her students - both graduate and undergraduate - the skills they need to succeed.

“Most of my students are doing scientific research for the first time in their lives,” Newberg told me in an email interview, “and it does help to launch their science careers. One recent graduate is on his way to graduate school at Cambridge University, and another went to graduate school at the University of Wisconsin, Madison.”

The computational, communications, and other skills her students develop in their time with the project aren’t taught in the traditional classroom environment.

“MilkyWay@home has been important in developing the careers of future scientists, and also in developing skills that are used in industry,” Newberg said. “All of the students do well in the job market. One is working on the AMD GPU compiler, another took a job with a defense contractor, and another is at Google.”

For the bargain price of $150,000 a year from the National Science Foundation, MilkyWay@home is doing everything right. It engages the public in real science. It develops the skills of a new generation of scientists. It’s close to modeling the largest streams in the galactic halo. And it will produce even more sophisticated models as data from the European Space Agency’s Gaia mission becomes available.

If it survives. MilkyWay@home lost its grant last May.

Stormy Times for American Astrophysics

The National Science Foundation faces severe financial problems. When its astrophysics budget increased dramatically in the early 2000’s, the NSF committed to several next-generation observatories. But rosy budget assumptions vanished as Congress developed a new sense of austerity. It cut the NSF’s budget just as the new observatories entered their most expensive development stages. 

Rather than canceling projects needed for American leadership in astrophysics, the NSF will close existing observatories and cut the grant programs that fund 90% of the country’s astrophysics research. Scientists like Newberg must compete for a piece of a much smaller pie.

“When I started writing proposals fifteen years ago, the funding rate in NSF astronomy was 33%,” Newberg said in our interview. “It is now about 15%, and some of these are only partial funding to keep the project alive. Many well justified proposals from solid researchers are not funded. In this climate I suspect that funding gaps between three year grants [like the one MilkyWay@home faces] will become normal.”

The Power of Community

MilkyWay@home is just one of many projects that lost its funding, but it has something that other research projects don’t: community. The thousands of people around the world who donate their computer time to the project aren’t passive participants. They joined because they wanted to make a difference to a project that matters. The volunteers actively participate in the project’s message boards - resolving technical issues, helping other members improve their performance, and sharing in the search for dark matter. A MilkyWay@home member even contributed the code that lets the project harness desktop graphics processors - dramatically increasing the virtual supercomputer’s performance. The project’s future at stake, Newberg turned to the MilkyWay@home community.

“Without funding at least for students in summer 2015 it would be hard or impossible to keep up operations,” Newberg told me. “I thought very hard about what I am doing and whether I should continue in this research direction before I decided to ask the volunteers for financial support.”

Newberg sent a plea for help to the MilkyWay@home community and launched a fundraising campaign on the project’s website. By taking pay cuts and reducing expenses to a minimum, a $40,000 budget would carry MilkyWay@home through to the next round of grant awards. And the community responded - within 2 months their donations have already gotten the campaign more than halfway to the $40,000 goal.


You can help too. The fundraising campaign accepts donations through RPI (tax deductible for US citizens). You don’t have to be a volunteer to donate, but if you do join the dark matter search you get a virtual badge on your online profile to show how you supported the fundraising campaign.. Anyone - member or not - who donates more than $100 gets a MilkyWay@home t-shirt as a thank-you gift. 

Newberg’s team created MilkyWay@home to let the public help crunch numbers. But the community they developed may be the reason MilkyWay@home survives when more traditional research projects close for lack of funds.

“I don't know yet whether this will be a short-term bridge or a new normal,” Newberg said. “My hope is that I will continue to be able to write winning grant proposals and that federal funding will continue to support the majority of my research program. But we are also seeing that there is a trend… that government spending on basic research is decreasing while private funding is increasing.”

Newberg’s team is already revising their NSF proposal to resubmit for next year’s round of grant reviews. They are also exploring the possibility of finding support from foundations or corporate sponsors. In the meantime, they hope public donations will carry them through next summer.

“When we sent out the email,” Newberg recalled, “we did not know what kind of response it would generate. Here we are using their computers and we turn around and ask the volunteers for money to run our operations on top of that.” 

“I am very thankful that we are connected with such a great group of people.”


You can read my full interview with Dr. Newberg for more information about the MilkyWay@home's campaign to keep searching for the Galaxy's dark matter.

Disclosure: I have donated to MilkyWay@home's campaign, but nobody associated with the project knew of this before I published this article.