It appears that a recent Java patch causes problems launching the IAS Viewer client and other Java-based software launched via Java Web Start. The update changed the location of the Java Web Start application so that the system opens the downloaded JNLP file as a text file, most likely with something called Dashcode. One of our system administrators found a solution on an Apple support discussion archive. You should only have to do this once to fix the problem:

1. Find any *.jnlp file in the Finder. These may be on your Desktop, or in a download folder, depending on how your web browser is configured.
2. Highlight the file by clicking on it, then select ‘Get Info’ from the File menu.
3. In the Open with: section, click on the popup menu and select ‘Other…’.
4. In the file chooser window that pops up, under Devices, pick the hard disk icon that corresponds to the name of your system hard disk (probably has the same name as your computer).
5. From there, select the System folder, then Library, then CoreServices, and scroll down to find the Java Web Start application, select it and click the ‘Add’ button. (Note, the location of Java Web Start application may differ on your system.)
6. Back in the Get Info window, click the button that says change all to apply this change to all of your JNLP files, then close the Get Info window.

When something tries to open a .jnlp file now, it should be properly handled by Java Web Start, launching the corresponding application.

Thanks to one of our twitter followers, @doug_ellison, for pointing out that many of you are having this problem!

Please note, we offer this for informative purposes, and you should make changes at your own discretion.

One important feature enhancement is that the auto-stretch preference is now “sticky.” You can disable auto-stretch once and for all, as indicated in the screenshots below.

With this change, you can load our images and see them in the original stretch, while still having the ability to re-stretch within higher-resolution areas.

For example, such software could display the actual coordinates on Mars of the pixels you are looking at, allow you to measure features directly in physical units, or stitch together images based on their absolute location on the planet. A number of GIS (Geographical Information Systems) applications use GeoTIFF; many on our science team have been waiting patiently for this feature to be rolled out.

We have already begun to produce RDRs with GeoTIFF, and they will start appearing in our weekly releases. At some point, a major reprocessing effort will be underway to bring this feature (and others) to all of our pre-existing products.

This brings up the topic of versioning: namely, how to tell which version of a HiRISE product you are working with.

Every HiRISE product has a line in the PDS label with the DATA_SET_ID. Every released HiRISE product to date shows that it has a DATA_SET_ID version of 1.0. The GeoTIFF RDR’s have a DATA_SET_ID showing version 1.1. This 1.1 version of our RDR processing also contains the updated color stretches described in a previous post. Later this year, that version number will likely be bumped up again when our improved calibration algorithms are put into production.

In addition, every HiRISE PDS product also has a line in the label with the PRODUCT_VERSION_ID. And every released HiRISE product to date has a PRODUCT_VERSION_ID of 1. When we make a new version of a released product, the version number will be incremented. If that new version is then released, it will replace the older version. We will only replace products with newer versions after a process of validation.

We’re halfway through a major update to our production pipelines that allows us to create, store, and reprocess these newer versions without effecting the released versions. This version number will only increase by integer amounts, and will never “skip”.

Our HiPlan developer, the HiCommander, had a little surprise for us in our planning software, like last year. I was forced to click “Authorize” in order to use HiPlan, and now I owe HiRISE $620. Hopefully that’s a tax-deductible business expense.

On another topic, a number of us have signed up for Google’s new Mars expedition. It seems like the next logical career move after MRO’s mission is complete. In fact, we’re scouting potential sites for Virgle City right now. That’s assuming their references to the first “manned journey to Mars” are not to be taken literally, and that women will be considered as potential colonists. (ahem.)

A few days ago I realized that after more than a year, I was still in the manual stage of monitoring data quality and not making good use of existing tools to help streamline the process. All that cutting and pasting was beginning to get ridiculous, even more so during a period of high data rates and an increasing number of observations.

When we receive a product with gaps, our automated processing pipelines usually process it just fine. Sometimes, however, there are pipeline failures. I keep a list of these failures and the actions I have taken to try to correct the problem. This generally means I have to manually processing the file or repairing its metadata header. Once the channel is repaired and/or recovered, then I can feed it back into the remaining processing pipelines.

These gapped products include “_G” in their file name. In a spectacular “Duh!” moment, I realized that rather than hunting in my browser through a list of observations that might have gapped product names, and then copying and pasting any I find into Excel, I could instead just perform a command line search in Terminal for all raw data products with “_G” in their file name.

     % ls -1 */*_G*.DAT

Why didn’t I do this over a year ago!? I am embarrassed to say I have no idea.

This only provides a list of products with gaps. Missing channel products usually arrive as gapped channel products 48 hours later (due to the nature of the automated processes that handle new MRO data at JPL.) In my spreadsheet, I scan through the missing channels and determine if a gapped file has in fact arrived. If it has been longer than 48 hours I consider the data lost for good and I “force” whatever observation data we have received through the pipelines. Of course, sometimes lost data is found or reprocessed at JPL long after the 48 hours has expired, and very rarely these new data will arrive. This requires reprocessing of the entire observation so that this newly found data can be added in.

A long time ago, our database specialist wrote a Perl script to create a daily list of missing and partial observations. For whatever reason, I stopped using this tool and then promptly forgot about it. Instead I started looking through the list of observations in my browser, clicking on those that were not complete, and manually figuring out what channels were missing. Making use of this missing EDR’s tool is so much easier and faster. Again, “Duh!”

The output from these quick searches for gapped and missing products requires a little bit of tweaking to make it look nice in Excel, but then a quick sort merges the two list into observation ID order. In no time at all, I have my list of observation products to follow up on for the day. My copying, pasting, and mouse clicking madness has been vastly reduced.

Every day I see the very worst data, generally caused by data transmission problems, but these data make up only a small percentage of all HiRISE data. To try to quantify this, I counted up the number of channels I have reported with problems (they have gaps, were missing, were somehow corrupted, etc.) and then divided this number by the total number of raw data files we have received (these are transformed into EDRs after being downloaded to Tucson.) As of this morning, I have listed 5392 channels with problems, and we have received 160,858 raw channel files. This is roughly 3% of our data, although most of these have some useful data in them. Even an observation with gaps or missing a channel or two is of potential use to a scientist. If too much data has been lost, then our targeting specialists might command HiRISE to try again in a later orbit.

Over time, we have developed many procedures for dealing with all sorts of problems. Now that I have sped up my daily data quality monitoring, I will have more time to improve these procedures, partially automate as much as I can, and provide suggestions to the software developers about tools that would make my job ever more efficient.

Note: My calculation of problematic data above is not rigorous and only a rough estimate. One flaw: I am counting missing channels in the numerator but not in the denominator.

We use the IAS Viewer ourselves in most cases. A prerequisite is having Java installed already on your computer; I’m pretty sure that both Microsoft Vista and Apple’s OS X do that by default, and most older versions of those operating systems do too. You can check by going to java.com and clicking on the “Do I Have Java?” link.

I have tested the IAS Viewer on a 2001-era computer (an iMac DV) with a low-speed wireless connection. Surprisingly, it worked about as good as on our work machines (dual or quad-core Macs with gigs of memory and ultra-fast Ethernet connections to nearby servers). With much older PC’s or via dial-up it may not be usable, I expect. But the bottom line is, you do not need to have the latest and greatest in computing technology to fill your screen with a steady source of high-res HiRISE pixels.

Early in the mission, our partners at NASA Ames put together a site using a Flash applet called Zoomify. They still maintain this site; however, it takes time and effort for them to keep up with our releases. Zoomify uses “tile pyramids”, or multiple copies of image data at each zoom resolution. So not only must the data be transferred, it must then be rendered into many tiles, occupying slightly more space than the original data. For that reason, they convert to JPEG, which eliminates some of the highest resolution information. Still, it may be faster because the lower resolutions are pre-rendered and the highest resolution has been decreased. Flash, like the IAS Viewer, is supported on Windows, Mac, and (x86 flavors of) Linux.

Because it is closed-source software, we can’t make improvements to the IAS Viewer. Or, if we did, we could not redistribute them under the terms of our license agreement. But here are some features I would like to see it have someday:

• Press page-up (etc) for navigating one screenful at a time
• The ability to open a URL to a particular region at a particular resolution
• The ability to copy the URL for the view you are looking at
• Voice interface (like the zoom scene in Blade Runner)!

The middle two items would, I think, eliminate the need to email multi-megabyte screenshots around for the purposes of scientific discussion.

I finally got around to trying it out, and it’s very easy to set up following his instructions. It allows you to see the footprints of acquired HiRISE images on a larger context map, and the Google [Planet] interface is really easy to use. Clicking on a red H footprint gives you a short description of the image, and a link right to our image release page, where you can browse or download the image products. CTX footprints are available, too. If I’m understanding this right, these KML files pull all currently released data from the PDS, so whenever we release data, the new stuff is automatically included.

The basemaps aren’t in 3-D (yet – maybe someday?!), so the perspective view isn’t much use, but you can kind of trick yourself into thinking it looks 3-D with the shaded relief maps. You can “fly” over the planet, zooming in & out, which is really fun.

I had trouble trying to get two basemaps visible at once (colorized MOLA elevation over the greyscale MDIM). With just one basemap, though, it works just fine, and it’s very fast (this probably depends a lot on your internet connection).

One really nice thing about the Google interface is when there are two overlapping footprints (which all of our stereo images are), clicking on them expands the choices and allows you to pick one or the other. Other tools I’ve used don’t handle this as nicely, and sometimes it’s impossible to select the “bottom” one.

Nice job, Ross & Google!

Unfortunately, we can’t just make them all the largest size the instrument is capable of taking, because our camera will get too hot. If it overheats, the instrument will shut itself off in order to prevent any damage to the electronics. So we have to be careful, and only plan images that won’t overheat HiRISE. In order to predict those temperatures, we use a tool called HiTemp (of course!). Here’s what it looks like (click on the image to see a bigger version).

This program reads in our planning files, and then models the temperatures of two key spots on the focal plane of the camera. It’s our job to make sure we don’t go above the dotted red line – this gives us a comfortable buffer below the scary solid red line. That’s when HiRISE would shut off, or safe. We know from experience by now that this is a big pain in the neck – a lot of work is required to get us back up & running, and we miss observations while we’re turned off. So we watch our HiTemp plots!

DRA performs what image processing folks call a “stretch.” A stretch takes some range of pixel values from the file and maps it onto a new range for the screen. To take an example, consider an image that appears over-exposed: much of the information is in the upper range of pixel values and you will have trouble distinquishing any detail. If the over-exposed pixels are not completely saturated (i.e. they don’t all have the maximum value) then a stretch that reduces brightness can reveal this otherwise hidden detail.

HiRISE has a very high signal-to-noise ratio, and our targeting specialists do a very good job choosing camera settings (which they do individually for each and every image) so completely saturated pixels are very rare.

But this also means that a stretch that works well over the entire image (a global stretch) may not be the best, the optimal stretch, for any one sub-image area that you are viewing. This is where the Auto DRA function in IAS becomes critical.

The button (shown below) is located on the right-hand side of the toolbar. A single click will do a stretch based only on the pixels you are viewing. This can bring out detail in shadow–amazingly, there is enough ambient light scattering around in the thin atmosphere to illuminate those scenes (and HiRISE is sensitive enough to pick enough of it up). It can also bring out detail in bright areas of over-exposure. For the color images in particular this can make things look a whole lot better.

Another factor plays a part in this. By default, the IAS viewer performs a global DRA when the image is loaded. As seen in the screenshot below, there are areas in our image that can skew the stretch. The large red rectangle is an area where the red CCDs start imaging before the blue-green. The IRB images often will have a cyan region where one of the IR CCDs was too noisy. We have elected to keep these areas in our images.

When in a sub-area, hit the Auto DRA button and the image should be drastically improved, as you can see in this final screenshot.

DRA early and often!

My first pre-coffee reaction was to panic, but then I read it more closely and realized what day it was. Turns out the HiCommander had snuck some “special” updates into a recent release of HiPlan.

The HiWall was also displaying a certain department head’s “face on Mars” all day.