Weathering the Storm

Back in 2001, I had the pleasure to head the On-Board Software development team for FedSat, Australia's first Satellite for 30 years. The recent Solar Storms have caused all sorts of problems for many satellites, but FedSat is still humming along. In the days before the storms hit, a patched version of the software was uploaded to the satellite, ending in pass 4574 or thereabouts, when over a megabyte of data was transmitted to FedSat. Quite a bit more had been transmitted in the immediately previous passes. This corrected a minor problem with the transmission of program data to the Communications payload, and enabled the ground crew to successfully upload a new version of the software for the Comms payload experiment (theres quite a few separate computers on board, each prgrammable independently). I don't know what the problem was, but as the code upload worked perfectly for the other payloads, I suspect it was some duff parameters to do with where in the Comms payload's memory the data was supposed to be put.

Anyway, you can see a summary of the lastest data from Fedsat on the web. Battery voltage is still pretty good after nearly a year in orbit, better than expectations. You can see where the voltage drops due to the high power drain when communicating with the ground station. Our goal was 12 months of full functionality, another 2 years of almost-complete functionality, then maybe another 2 years of partial functionality. It looks as if we may get quite a bit longer.

I had an e-mail interview with a Journalist at the Age newspaper, and he did superb job of condensing my ramblings (and others) into a neat, tidy article. Here's a quote:

Only 58 centimetres square and weighing 50 kilograms, the tiny FedSat satellite is packed with five scientific experiments and all of the instruments required to communicate with Earth during its anticipated three-year life. At the heart of the satellite is a 10MHz ERC-32 processor - a SPARC-based 32-bit RISC processor developed for high-reliability space applications.

The ERC-32 sacrifices processing power for durability and reliability. It uses three chips to process a modest 10 million instructions per second and two million floating-point operations per second - less than 1 per cent of a Pentium 4's capabilities.

The pay-off is reliability: the ERC-32 uses concurrent error-detection to correct more than 95 per cent of errors.

Power-hungry microprocessors such as the Pentium 4, which runs a standard office PC bought off the shelf today, would be an intolerable burden on the solar-powered satellite. The ERC-32 consumes less than 2.25 watts at 5.5 volts.

Designed to survive extreme radiation bursts from solar flares, the ERC-32 can tolerate radiation doses up to 50,000 rad. This is 100 times the lethal dose for humans.

Low-Earth-orbit is "a cruel place to put a computer", says software engineer Alan Brain, who is responsible for FedSat's data-handling system.

It will orbit at about 803 kilometres above the Earth's surface and will circle the planet every 100 minutes.

"The radiation will cause random bit-flips and can even fry components," Brain says. "The vacuum boils the volatile gasses out of normal chips, making them useless and coating everything nearby with conductive gunk. In the Earth's shadow, temperatures make Antarctica look balmy, and in the sun's glare it's hotter than the Simpson Desert. On the way up, the vibration of the rocket would shake most normal circuit boards to pieces."

Spaceflight avionics software development is not for the faint-hearted either.

"The question for software developers is not, 'Are you paranoid?', the question is, 'Are you paranoid enough?' " Brain says. "Every software module, every function, procedure or method has to assume that information coming in may have been spoilt by a malfunction and be prepared for the worst. The system must be ductile - bending, not breaking - when things go wrong. In space no one can press Control/Alt/Delete."

A team of Australian programmers developed FedSat's onboard software, building on work done in Britain. It is written in Ada-95, a programming language designed for embedded systems and safety-critical software. All it has to work with is 16MB of RAM, 2MB of flash memory for storing the program, a 128K boot PROM (programmable read only memory) and 320MB of DRAM in place of a hard disk that would never survive the launch process. All essential data is stored in three physically different locations.

Along with controlling the satellite, this software must interface with the satellite's five experimental payloads. These are designed to study UHF and Ka-band transmission characteristics and coding methods, the Earth's magnetic field, Global Positioning System applications, high-performance computing and the stars.

Along with power restrictions, the main constraint on FedSat's designers is its limited contact with Earth.

The FedSat ground station will control the satellite via an S-Band bidirectional radio link using a dish on the roof of the Signal Processing Research Institute building at the University of South Australia's Mawson Lakes campus.

Owing to FedSat's orbit, the ground station will only be able to communicate with the satellite during two 20-minute periods each day.

As such, a large component of the software's work is logging data to be downloaded and storing commands to be executed while the satellite is out of contact.

Pretty Cruel is right. As John Hohl told the BBC :

Commenting on the solar events of the past few days John Kohl of the Harvard-Smithsonian Center for Astrophysics in the US, said: "It's like the Earth is looking right down the barrel of a giant gun pointed at us by the Sun...and it's taken two big shots at us."

"The Sun is really churned up. The timing of two very large X-class flares aimed directly at the Earth, occurring one right after another, is unprecedented.

"I have not seen anything like it in my entire career as a solar physicist. The probability of this happening is so low that it is a statistical anomaly."

And as I said, "The Question is not 'Are you paranoid?' but 'Are you paranoid enough?'. <gloat>I beefed up the memory error-checking and automatic correction way beyond the spec, just in case, and because with the design I'd come up with it was actually easier to go a bit overboard than restrict the checking to something less aggressive.</gloat>