Unfortunately, we can’t just make them all the largest size the instrument is capable of taking, because our camera will get too hot. If it overheats, the instrument will shut itself off in order to prevent any damage to the electronics. So we have to be careful, and only plan images that won’t overheat HiRISE. In order to predict those temperatures, we use a tool called HiTemp (of course!). Here’s what it looks like (click on the image to see a bigger version).

This program reads in our planning files, and then models the temperatures of two key spots on the focal plane of the camera. It’s our job to make sure we don’t go above the dotted red line – this gives us a comfortable buffer below the scary solid red line. That’s when HiRISE would shut off, or safe. We know from experience by now that this is a big pain in the neck – a lot of work is required to get us back up & running, and we miss observations while we’re turned off. So we watch our HiTemp plots!

We had the difficult detective job of figuring out what went wrong. It was clear early on that the instrument overheated, but we couldn’t figure out why. Our tool that predicts the temperatures (”HiTemp”) didn’t predict anything that hot. We didn’t take a really large image, which would heat us up (at least, nothing bigger than normal! ). The local operations team worked with the health & safety people, the spacecraft engineers at LMA, and some of the software developers at Ball Aerospace that originally designed HiRISE. Together we all investigated the problem.

We studied the telemetry (information from the spacecraft), the details of the commands that were sent to the instrument, and we re-modeled the temperatures and memory use. The problem was complicated by several other unusual events that occurred around the same time: First, the memory on board the spacecraft (the “Solid State Recorder”, or SSR) had filled up because one of the dishes of the Deep Space Network was broken. This meant we couldn’t send data back to Earth, so it piled up in the memory until it overflowed. Second, HiRISE’s “keep-alive counter” was withheld. This is a steady heartbeat HiRISE sends to MRO that indicates HiRISE is still running. After a certain number of heartbeats are missed, MRO will safe HiRISE. Also around the same time, there were some errors in the spacecraft’s software. The timing was also mysterious: HiRISE safed about 15 minutes after an image. This is a long time afterwards – the image should have been completely done within just a few minutes. Instead, the temperature sensors showed that we continued to heat up for 15 minutes!

Finally, after a day of research, we found an answer. What happened was this: First HiRISE did all the setup steps to take an image (set the number of lines, etc.). One of these steps turns on the CCDs (sensors) in the camera. Then, right before it was about to actually expose the image, it found out that the memory was full. Since there wasn’t enough room in memory for the data, it didn’t take the image. However, everything was left turned on! So with everything powered on, we continued to heat up until we reached the limits we have set to protect the instrument. This withheld the keep-alive counter, and HiRISE safed. So in fact, the instrument worked exactly as it should have, in order to keep itself out of danger. It was just an unexpected response to this unusual situation.

With the help of the LMA engineers, we were able to power HiRISE back on the following day and start imaging again very quickly. Thankfully, we were up & running in time for another very special observation that I’ll be writing about soon….

That’s GuyMac on the left, HiCommander (me!) in the middle, and HiKu on the right.

HiKu is part of the operations staff on the uplink side of things. He’s on the team that does the targeting and the planning. What was he doing on a Saturday? Same thing he does every day: targeting and planning.

GuyMac and I are part of the software development team—we write the programs the ops team uses to do their job. GuyMac works primarily for the downlink group. He spent the day on a program called HiVali, which will be used by the downlink ops team to make sure a given HiRISE image accomplished its goal.

I work for the uplink group. I spent the day working on a program with the second greatest HiRISE software name: the HOGG. That’s the HiRISE Observation Generation GUI. The “the” is an important part of its name, by they way. You don’t use HOGG to generate HiRISE camera parameters. You use the HOGG.

A lot of our HiRISE tools have funny names; the most common way to name a piece of software here is to get a one-word description of its function and then add “Hi” to it. The planning software? HiPlan. The commanding software? HiCommand. The validation software? HiVali. The camera temperature modeler? HiTemp.

I like “the HOGG” for three reasons. One, who doesn’t like hearing their peers use such a ridiculous word to talk about something serious? Two, it breaks the HiRISE naming convention. And three? I get to correct people and point out that the “the” is part of the name. Still, it’s only my second favorite HiRISE software name. My favorite?

That’d be the HiRISE photometry predictor.

HIPHOP.

I’ll write more about these tools in the coming weeks.