It appears that a recent Java patch causes problems launching the IAS Viewer client and other Java-based software launched via Java Web Start. The update changed the location of the Java Web Start application so that the system opens the downloaded JNLP file as a text file, most likely with something called Dashcode. One of our system administrators found a solution on an Apple support discussion archive. You should only have to do this once to fix the problem:

1. Find any *.jnlp file in the Finder. These may be on your Desktop, or in a download folder, depending on how your web browser is configured.
2. Highlight the file by clicking on it, then select ‘Get Info’ from the File menu.
3. In the Open with: section, click on the popup menu and select ‘Other…’.
4. In the file chooser window that pops up, under Devices, pick the hard disk icon that corresponds to the name of your system hard disk (probably has the same name as your computer).
5. From there, select the System folder, then Library, then CoreServices, and scroll down to find the Java Web Start application, select it and click the ‘Add’ button. (Note, the location of Java Web Start application may differ on your system.)
6. Back in the Get Info window, click the button that says change all to apply this change to all of your JNLP files, then close the Get Info window.

When something tries to open a .jnlp file now, it should be properly handled by Java Web Start, launching the corresponding application.

Thanks to one of our twitter followers, @doug_ellison, for pointing out that many of you are having this problem!

Please note, we offer this for informative purposes, and you should make changes at your own discretion.

I finally got around to trying it out, and it’s very easy to set up following his instructions. It allows you to see the footprints of acquired HiRISE images on a larger context map, and the Google [Planet] interface is really easy to use. Clicking on a red H footprint gives you a short description of the image, and a link right to our image release page, where you can browse or download the image products. CTX footprints are available, too. If I’m understanding this right, these KML files pull all currently released data from the PDS, so whenever we release data, the new stuff is automatically included.

The basemaps aren’t in 3-D (yet – maybe someday?!), so the perspective view isn’t much use, but you can kind of trick yourself into thinking it looks 3-D with the shaded relief maps. You can “fly” over the planet, zooming in & out, which is really fun.

I had trouble trying to get two basemaps visible at once (colorized MOLA elevation over the greyscale MDIM). With just one basemap, though, it works just fine, and it’s very fast (this probably depends a lot on your internet connection).

One really nice thing about the Google interface is when there are two overlapping footprints (which all of our stereo images are), clicking on them expands the choices and allows you to pick one or the other. Other tools I’ve used don’t handle this as nicely, and sometimes it’s impossible to select the “bottom” one.

Nice job, Ross & Google!

My first pre-coffee reaction was to panic, but then I read it more closely and realized what day it was. Turns out the HiCommander had snuck some “special” updates into a recent release of HiPlan.

The HiWall was also displaying a certain department head’s “face on Mars” all day.

Also, our team is really growing!

You’ll notice a lot of new faces compared to a few years ago!

We spent the first day updating the team on our operations here at HiROC. Then for two days, almost everyone on the science team presented new results from recent HiRISE data, and explained their plans for future observing and analysis. We saw some really great images! A lot of people are doing “quick & dirty” three-dimensional analgyphs with our stereo data (like these), so we got to wear our red/blue glasses a lot. (We don’t have things calibrated & automated to the point where we can do this “correctly” yet, so these are still mostly preliminary, hand-processed products.)

Today, most of the team is participating in software training. Since we only have a day, we’re just doing short demos of the planning tools (HiPlan – see previous HiBlog entry for a recent update) and some of the analysis tools (like ISIS and IDL/ENVI)

For operations, it’s been wonderful to see the fruits of our efforts — we go to a lot of trouble to acquire an image, but once it’s acquired, we tend to lose track of it, because we’re planning the next images! Seeing the science that comes out of the data not only helps us understand strategies and be more intelligent in our targeting and planning, it’s also just really cool! I think keeping the “big picture” in mind is important in keeping us motivated and excited about what we’re doing.

The processing pipelines have been introduced in earlier entries. Thanks to the efforts of HiRISE developers (mostly before my time with the project) these have provided a very solid foundation for our automated ground data system. There has been very little need for trouble-shooting or fine-tuning of the core software.

One issue that did come up earlier in PSP however was a strange failure that happened periodically, though not predictably. If you are a programmer, there is nothing so dreadful as a bizarre, non-repeatable bug… not counting Monday morning meetings, of course.

This particular bug happened when a new observation came down. Each processing pipeline needs to make a new directory, a new folder in the file system where it can store a log of its work. Starting with the transition phase, most of our pipelines have two or more instances running in parallel on a small Linux cluster, each working on part of the observation.

The bug was that, occasionally, the directory creation would fail. Someone from downlink would have to investigate and take corrective action. Or, if the way forward was unclear, to call in somebody from the software development crew.

In this case, the problems seemed to happen downstream from “my” pipelines: I was blissfully unaware that it could effect HiDog and EDRgen as well.

Then on Thursday, Rich emails to say that the workarounds put in place by two other developers seemed to have helped their pipelines, could I add them to my code as well?

A little shocked, I replied (no doubt too haughtily) that I’d rather understand the problem than apply some hasty workarounds. It had all the classic hallmarks of a multi-threaded problem: notoriously hard to repeat & difficult to debug, but always when two processes were going after the same thing.

The particular bit of code—a subroutine shared by each pipeline to create a log directory—looked absolutely fine on first inspection. It uses the perl language and invokes a function called mkdir. That really should be no problem, I thought. Perhaps if there were a permissions problem. But this succeeded more often than not.

So that evening, I decided to try a little test: a small program that I’d run simultaneously in two windows. Each one would try to make the same directory, whose name would contain the current time in seconds. Since the mkdir operation should take much less than a second, there should be frequent “collisions”—if that was really what the problem was.

As a developer, you grow accustomed to the fact that almost nothing works right on the first try; you have to take small steps and constantly check yourself. So I first tried a test in just a single window. It looked like this:

% while 1
? perl -e 'print "mkdir failed!" unless(mkdir $ARGV[0]);' `date '+%S'`
? end

That should succeed every time, I thought, then I’ll move on to trying it in two terminal windows at the same time. But lo and behold, I pressed return and started getting many failure messages!

In the blinding flash of the obvious, I realized what I should have known: that mkdir returns false, indicating it failed to make the directory, because it already been created. My test script was interfering with itself, succeeding when it was a new second and failing every other time.

This explained the bug. It was what is called a race condition. The condition in this case was the creation of a directory. Two pipelines were racing to create the same directory. Occasionally they were close enough that they were calling the same subroutine, and both bypassed its initial check that the directory already existed. One got to the mkdir step a fraction of a second ahead of the other, probably due to a slightly heavier load on the machine, or other variables.

Compounding the problem, the workarounds that had been added in several previous versions (retrying up to five times with random delays) failed to check properly. A single check after mkdir would remove the race condition. Testing this (in parallel) works as expected.

Or a simpler way is to call the Linux mkdir command with an option so that it does what I (and perhaps the other developers) thought it should: to return true (success) if the directory exists or was created.

Tracking down a mystery like this is a fun part of the job. But only if it happens infrequently.

The uplink team is constantly looking where to point the camera next. There is a program which is in beta testing now called HiWeb which allows scientists and other people to input suggestions. The Uplink team reviews the suggestions in the database, assigns a priority to each of these suggestions, and then finds when we can point the camera at the part. They also make sure a certain percentage of the upcoming pictures are assigned to look for a Phoenix landing spot, as this is a high priority item at the moment. They are still learning exactly how to best command the camera, and are constantly sharpening their skills.

The downlink team is making sure operations run smoothly at HiROC. They are verifying that the processing has taken place, make sure that the images have been calibrated correctly, that there are no image processing artifacts on the images we are about to release. If there is any artifacts created from processing the image, the source of the problem is identified and fixed, and then the image is reprocessed. While previously we have sent images to the public that had some small processing artifacts during the post-MOI and Transition imaging, we currently are waiting until the images have been completely validated. The downlink team is also taking a quick look at each image that comes down, and making sure there isn’t something unexpected, for example, haze at Mars, lots of saturated pixels, etc. If any such problems are found, they notify the uplink team, to ensure that we don’t have continuing problems. These problems are very rare, but on occasion happen, due to the changing nature of Mars.

During and after the validation process, the images are reviewed by several of the science team members of HiRISE. Things of special nature are noted, and these images receive captions. The others are slated for a more general release. Due to the large size of the HiRISE images, it is almost impossible to search every square inch of the pictures by any one or even a small group of people. I’m sure many of you have noticed this with just the images which have been released, there are many more which are still being validated which have yet to be released.

The Systems team is responsible for making sure that the HiROC computers are all working in top shape. They are quick to find problems when they arise and fix them so that it does not affect the flow of data here. They are preparing servers for two upcoming services that HiROC will provide, HiWeb, which was mentioned previously, and a JPIP server, which will allow for the effective distribution of JPEG 2000 images.

The software team is writing software that will make people’s lives easier. Some are working with the HiPlan suite of tools, which is used to plan upcoming images, to make it even easier to use for the uplink team. Some are working on HiVali, the validation software, which is used to make it easier to verify that an image is ready to release to the public, quickly finding problems with the image. Some are working on HiView, a program which will allow distribution of images over the JPIP protocol to the general public. Still others are working on getting HiWeb ready for public release.

Let me also talk a bit about a few upcoming products mentioned in this entry. HiView will allow you to download only the parts of a HiRISE image that you find most interesting. It will work great, even for those who have slow internet connections. I personally have tested this with a connection rate of 1kBytes/sec, and it works reasonably well even at that slow speed. It will allow the user to save the parts of the image they find the most interesting to their hard drive for future study (HiView will require a constant internet connection to download the image)

Another upcoming product is HiWeb. HiWeb will allow any user (Yes, that’s you!) to suggest future targets to image with the HiRISE camera. Preference is given to targets of scientific interest. The suggestions are given a priority, and placed in a database to be targeted depending on the orbit of MRO and the allocated bandwidth.

So, that’s what’s happening at HiROC these days. In short, we are all very busy, but very much enjoying our work. I personally can’t remember a time that I’ve had as much fun working as these last few months have been. And surely the best is yet to come!

1. The image is taken by the HiRISE camera, and is stored in up to 28 channels, two for each of the 14 CCD arrays of the camera. Each channel covers about half of the image. Of the 14 CCDs, 10 are red CCDs, two are blue-green, and two are near-infrared. The color CCDs are aligned with the center red CCDs.
2. The image is placed inside a buffer on MRO, awaiting transmission to Earth, along with science data from the other instruments on MRO.
3. The image is received in packets by the Deep Space Network (DSN).
4. After 4 hours of collecting data at the DSN, the Jet Propulsion Laboratory (JPL) puts the packets together for what is known as a “quick look”. The entire image generally has not yet been received by this point in time, but it is enough of the image that it can be processed to take a quick look at it. Subsequently, JPL puts together all of the data it has received every 4 hours and makes it available to the computers at HiROC.
5. After the files have been put together by JPL, then one of the computers at HiROC looks and sees that there is data on the JPL server and copies the data to our system at HiROC. This is the start of what is known as the pipeline, the system of programs at HiROC which process the images. This usually happens either via a direct connection to JPL (slower), or through the Internet 2(Faster, but sometimes can be bogged down).
6. The images are put together into a viewable format, using the minimum processing possible, and create what’s known as an EDR, or Experimental Data Record. This is done without calibration, stitching together the channels, or any other processing, aside from putting the image together. For an image which uses all 14 CCDs, there will be 28 EDRs. These generally speaking are of mainly scientific interest, but they will be released to the general public via the Planetary Database System (PDS). They will be in the standard PDS format.
7. After the EDRs have been created, they are converted to another format for ISIS. ISIS, the Integrated Software for Imagers and Spectrometers is a suite of tools used for processing images for most interplanetary missions, that was developed by the United States Geological Society (USGS). Most of the tools that we use at HiROC for processing our images are written for ISIS files.
8. After the ISIS files have been created, they are calibrated via a program called HiCal. This reduces the inherent noise of the camera to be more consistent with what is being photographed. All digital cameras create some level of noise, and while HiRISE is an extremely good instrument, it still generates a low level of noise.
9. After the individual channels are calibrated, then they proceed to a program called HiStitch, which puts the two channels of the same CCD together. As they are a part of the same CCD, this requires little processing.
10. Next, after each CCD been stitched together, the full CCD images run through a program called HiccdStitch. This program puts the different ccds together, making a mosaic for each color band. This requires some processing, as the ccds slightly overlap, and it can sometimes be difficult to match the different arrays exactly.
11. If the image has not been completely received, then at this point, the pipeline stops, until JPL has received the entire image, or if there are a few confirmed gaps in the image which we haven’t been able to recover. Transmission over the vast distance between Earth and Mars is not easy, and even the best systems have some small error.
12. After the image has been completely stitched together, then the image is geometrically projected. To understand this, realize that the images that HiRISE takes are flat, while Mars is actually round. Geometrical Projection alters the image so that the image points in compass directions, while correcting any distortions that are created by the ellipsoidal shape of Mars. With the geometrical projection images and the right software tools, such as qview for ISIS, the exact distance can be found between two point on the image. In order for this to happen, we must wait for information to be gathered on the exact position of the spacecraft. This is done by the nagivational team, based off of the downlink frequency. This takes two weeks after the picture has been taken, so Geometric Projection might take a while. This is the longest wait point of the operation. An image can be released from predicted information, however, most images will wait for the correct SPICE kernels to be calculated, in order to get the best information. If an image is geometrically projected from predicted information, it will be calculated with the correct info after it has been received.
13. The images are then validated by a team of students known as the HiRISE Validators. They check to make sure that everything in the pipeline worked perfectly, see if there are any gaps in the images, and other similar tasks. If they notice a problem, they contact the HiRISE Operators, who will take steps to resolve the problems, which may include passing part or all of the image through the pipeline again, or tweaking the software to make it work perfectly.
14. The image is converted to a format that the general public can use. Currently that format is JPG, or TIFF, but eventually we will use JPEG 2000.
15. After all of this, the science team members of HiRISE will look at an image to see if there is anything noteworthy. If there is, it is given a caption, and perhaps a press release. If not, it will be posted on the HiRISE website. They are also posted on the MRO website, and occasionally on others.

This process may take as long as a week or two to complete, depending on the load of MRO, scheduling concerns, load at HiROC, etc. The first image took about 9 hours to be completely processed after it was taken by HiRISE. The Victoria Crater picture, taken during a much busier time on MRO, took about 36 hours to make its way to our hands. This was in part due to the larger size of the image, as well as the cache of images already awaiting transmission on MRO to earth. The captions for the images taken during Transition imaging took anywhere from a few hours to a few weeks to write, and this will likely continue to hold. We at HiROC want to release the images we take as fast as possible to the public, and we are doing everything we can to realize this goal. Several shortcuts were taken during the Transistion imaging phase that allowed for images to be released quicker. For Primary Science Phase, this will take a bit longer because these shortcuts will not be taken, but we expect that we will release most images within two weeks after them being taken, shortly after we have finished receiving, processing, and captioning the image.

There are some variations to this process, for example, the Victoria Crater picture was released in a press conference jointly with the Mars Exploration Rovers (MER) team. Also, color images require extensive calibration and take a lot more time. However, this is the general idea. Currently the entire system, except for writing the captions and adding the images to our website, is essentially completely automatic for receiving and processing HiRISE images, due to years of preparation by the HiTECH and HiOPS teams.

Wayne Sydney (LMA) and Roy Gladden (JPL) taught me how to talk to our spacecraft. Ira Becker (BATC), Rick Battistelli (BATC), and Steve Tarr (BATC) taught me how to talk to our instrument. Peter Xayprayseuth (JPL) bends so far over backwards for us he appears to be standing upright. He and Curt Eggemeyer (JPL) make sure what I’m saying to the spacecraft isn’t complete gibberish. Mark Johnson (LMA) checked my math. Finally, Michael Weiss-Malik (ASU), Eric Engle (ASU), Saadat Anwar (ASU), and Noel Gorelick (ASU) built a fantastic piece of software for me to hook into for our observation planning.

There are dozens more, of course, but these are the ones that directly apply to me, and without them, none of this would have happened. So thanks, guys. If I were to wear a hat, it would be off to you.