The diagnosis is in—bad memory knocked NASA’s aging Voyager 1 offline

Engineers have determined why NASA’s Voyager 1 probe has been transmitting gibberish for nearly five months, raising hopes of recovering humanity’s most distant spacecraft.

Voyager 1, traveling outbound some 15 billion miles (24 billion km) from Earth, started beaming unreadable data down to ground controllers on November 14. For nearly four months, NASA knew Voyager 1 was still alive—it continued to broadcast a steady signal—but could not decipher anything it was saying.

Confirming their hypothesis, engineers at NASA’s Jet Propulsion Laboratory (JPL) in California confirmed a small portion of corrupted memory caused the problem. The faulty memory bank is located in Voyager 1’s Flight Data System (FDS), one of three computers on the spacecraft. The FDS operates alongside a command-and-control central computer and another device overseeing attitude control and pointing.

The FDS duties include packaging Voyager 1’s science and engineering data for relay to Earth through the craft’s Telemetry Modulation Unit and radio transmitter. According to NASA, about 3 percent of the FDS memory has been corrupted, preventing the computer from carrying out normal operations.

Optimism growing

Suzanne Dodd, NASA’s project manager for the twin Voyager probes, told Ars in February that this was one of the most serious problems the mission has ever faced. That is saying something because Voyager 1 and 2 are NASA’s longest-lived spacecraft. They launched 16 days apart in 1977, and after flying by Jupiter and Saturn, Voyager 1 is flying farther from Earth than any spacecraft in history. Voyager 2 is trailing Voyager 1 by about 2.5 billion miles, although the probes are heading out of the Solar System in different directions.

Normally, engineers would try to diagnose a spacecraft malfunction by analyzing data it sent back to Earth. They couldn’t do that in this case because Voyager 1 has been transmitting data packages manifesting a repeating pattern of ones and zeros. Still, Voyager 1’s ground team identified the FDS as the likely source of the problem.

The Flight Data Subsystem was an innovation in computing when it was developed five decades ago. It was the first computer on a spacecraft to use volatile memory. Most of NASA’s missions operate with redundancy, so each Voyager spacecraft launched with two FDS computers. But the backup FDS on Voyager 1 failed in 1982.

Due to the Voyagers’ age, engineers had to reference paper documents, memos, and blueprints to help understand the spacecraft’s design details. After months of brainstorming and planning, teams at JPL uplinked a command in early March to prompt the spacecraft to send back a readout of the FDS memory.

The command worked, and Voyager.1 responded with a signal different from the code the spacecraft had been transmitting since November. After several weeks of meticulous examination of the new code, engineers pinpointed the locations of the bad memory.

“The team suspects that a single chip responsible for storing part of the affected portion of the FDS memory isn’t working,” NASA said in an update posted Thursday. “Engineers can’t determine with certainty what caused the issue. Two possibilities are that the chip could have been hit by an energetic particle from space or that it simply may have worn out after 46 years.”

Voyager 1’s distance from Earth complicates the troubleshooting effort. The one-way travel time for a radio signal to reach Voyager 1 from Earth is about 22.5 hours, meaning it takes roughly 45 hours for engineers on the ground to learn how the spacecraft responded to their commands.

NASA also must use its largest communications antennas to contact Voyager 1. These 230-foot-diameter (70-meter) antennas are in high demand by many other NASA spacecraft, so the Voyager team has to compete with other missions to secure time for troubleshooting. This means it will take time to get Voyager 1 back to normal operations.

“Although it may take weeks or months, engineers are optimistic they can find a way for the FDS to operate normally without the unusable memory hardware, which would enable Voyager 1 to begin returning science and engineering data again,” NASA said.