On Monday morning, I started up HiPlan, our image planning and targeting tool (see this post for more about HiPlan), and this unexpected window popped up:
My first pre-coffee reaction was to panic, but then I read it more closely and realized what day it was. 
Turns out the HiCommander had snuck some “special” updates into a recent release of HiPlan.
The HiWall was also displaying a certain department head’s “face on Mars” all day. 
To give you some idea as to the size of a HiRISE image, I’ve put together a few screenshots from our planning software, HiPlan. These images compare the HiRISE footprint with that of the the THEMIS instrument aboard Mars Odyssey. These screenshots are shown below as thumbnails; click on a thumbnail to see the screenshot at its actual resolution.
I should note that these are not planned to be actual HiRISE images; I was using HiPlan in test mode while working on the display of the individual HiRISE CCD footprints.
Take a look at this screenshot. It covers a small region of Mars roughly one degree across and slightly less than a degree tall:
The HiRISE project has developed a fairly significant amount of software. I’ve been privileged to play a part in that development, which continues even as we get deeper into the primary mission. So, rather than space science or operations, this post will discuss one of the nittier, grittier aspects of our work.
The processing pipelines have been introduced in earlier entries. Thanks to the efforts of HiRISE developers (mostly before my time with the project) these have provided a very solid foundation for our automated ground data system. There has been very little need for trouble-shooting or fine-tuning of the core software.
One issue that did come up earlier in PSP however was a strange failure that happened periodically, though not predictably. If you are a programmer, there is nothing so dreadful as a bizarre, non-repeatable bug… not counting Monday morning meetings, of course.
I thought I’d offer a few more words as to what is done with images at HiROC. Validation has been mentioned in the blog, and I’d like to explain a bit more about that. I’ve been involved in writing the primary validation tool, HiVali, and I will be the primary student validator for the next month. (The regular student validators are from out of state, and are going home for the Christmas holidays. I’m from around here, and offered my services to look at pretty pictures from Mars all day;-))
To conclude our exploration of the pipelines that take raw channel files and create a beautiful, unmapped mosaic, let me introduce the Stitch pipelines: HiStitch and HiccdStitch.
The HiStitch pipeline combines the matching HiCal products for the same CCD into one more-or-less lined up CCD cube file. HiccdStitch combines these HiStitch cubes into RED, IR, and BG mosaics.
Both pipelines take some time, as overlapping pixels are accounted for and brought together. After these mosaics are created, additional steps create smaller jpeg files for easier viewing, and full-sized jpeg2000 files. We use these jpeg2000 files for validating our images.
There are later pipelines, but we first validate the HiccdStitch products: Did the previous pipelines work correctly? Did the uplink team command the camera correctly? Is there haze or clouds obscuring our view of the surface?
If everything looks good, and we have received the correct reconstructed SPICE ephemeris data, then the geometry pipelines are invoked. These pipelines project the images mathematically to a model of Mars and add geometry data to the images so that each pixel becomes a point on Mars with latitude and longitude coordinates.
Well, it’s been a while since we’ve been posting a lot, so I thought I’d just give you guys some kind of an idea as to what we’re doing these days.
The uplink team is constantly looking where to point the camera next. There is a program which is in beta testing now called HiWeb which allows scientists and other people to input suggestions. The Uplink team reviews the suggestions in the database, assigns a priority to each of these suggestions, and then finds when we can point the camera at the part. They also make sure a certain percentage of the upcoming pictures are assigned to look for a Phoenix landing spot, as this is a high priority item at the moment. They are still learning exactly how to best command the camera, and are constantly sharpening their skills.
The downlink team is making sure operations run smoothly at HiROC. They are verifying that the processing has taken place, make sure that the images have been calibrated correctly, that there are no image processing artifacts on the images we are about to release. If there is any artifacts created from processing the image, the source of the problem is identified and fixed, and then the image is reprocessed. While previously we have sent images to the public that had some small processing artifacts during the post-MOI and Transition imaging, we currently are waiting until the images have been completely validated. The downlink team is also taking a quick look at each image that comes down, and making sure there isn’t something unexpected, for example, haze at Mars, lots of saturated pixels, etc. If any such problems are found, they notify the uplink team, to ensure that we don’t have continuing problems. These problems are very rare, but on occasion happen, due to the changing nature of Mars. (more…)
Twelve delightful images for your viewing pleasure are now available. I’m also starting a page answering JPEG-2000 questions, keep sending them in!
The raw HiRISE image data needs to be cleaned up, and the HiCal pipeline is where this work is done. Now that the raw image data has been converted to a *.cub file format, ISIS tools can be used. These include hiclean, hipical, and hidestripe.
Hiclean does just what is says. Noise introduced into the image data by spacecraft electronics is corrected. Noise can show up as vertical and horizontal lines in the raw image and other periodic manifestations.
Hipical is a newer tool that performs calibration on the image data. For example, flatfield and gain corrections are performed by hipical. Hipical will be upgraded as we learn more about our instrument in its environment around Mars.
Hidestripe corrects a known striping pattern in HiRISE images.
We use other tools to collect even more statistical data about the newly calibrated image data. The HiCal pipeline will continue to be upgraded as our software matures. New statistics will be collected while corrections are added or improved.
After cleanup has been completed and a new *.hical.cub channel product created, HiCal creates a variety of jpeg browse and thumbnail images. The cleaned up channels are large, and for quick previews, these smaller jpegs come in handy.
Finally, HiCal lets the next pipeline – HiStitch – know that cleaned up channels are ready to be stitched together into CCD products.
Below is an example of raw data, prior to going through the HiCal pipeline. This image sample was taken from TRA_000873_1780; “Victoria Crater” at Meridiani Planum.
Below is the same image sample after going through the HiCal pipeline (notice that the bright vertical line in the center and the faint vertical lines throughout the image have been correctly removed by HiCal):
Here we are over two weeks into PSP, and we haven’t even finished releasing our TRA images!
Actually, today we got 31 TRA images out the door, just about completing that set. In addition, the previous 32 grayscale images have been reprocessed with our improved geometry—no more jagged edges on the reprojected “Red Mosaics”.
Furthermore, the full-res images are now available as lossless JPEG-2000 (JP2) files. At HiROC we are using the ExpressView application from a company called LizardTech… a subsidiary of some Japanese company (that has no affiliation to us FWIW).
ExpressView is available for Mac and Windows. Their download page describes it only as a browser plug-in, but the installer contains an application as well.
ExpressView provides progressive rendering (though you still have to have the entire file first—we are considering moving to a streaming model using the JPIP protocol but there are even fewer clients available). It should also reduce the amount of memory (RAM) needed to view our largest images.
There are a few options available to Linux users, but nothing we have tried is as fast or as feature-rich. In any case, let us know what works or doesn’t work for you.
All of these images are available at the usual location.
A system for streamlining the process of editing captions, highlighting cut-out areas and pushing out web pages is in the works. We will get caught up.
In the meantime, look for the first set of PSP releases by this time next week.
P.S. Happy Thanksgiving.
After a channel of raw data has been downloaded and converted into an *.IMG file, we need one more conversion before cleanup of the image can begin.
The EDR_Stats pipeline creates a *.cub file from the *.IMG file. These cube files are the file type used in ISIS 3.0, an image processing software package provided for planetary science missions by the United States Geological Survey (USGS). This package contains an entire suite of useful tools, many of which are used by our pipelines.
During the creation of a cube, a variety of statistics are gathered. For example, the number of gaps, saturated pixels, calibration pixels, and other pixels are counted. Image mean, standard deviation, and other statistics are also calculated. EDR_Stats takes these results and uploads them to our database. The resulting cube is archived in our storage directory.
The final EDR_Stats pipeline step lets the next pipeline – HiCal – know that an image channel cube file is ready for calibration processing. Let the cleanup of image data begin!
