You might remember that we were planning on releasing HiRISE images to the public on a monthly basis. That plan was delayed by issues with our processing software, hardware and other events. A productive summer working on these issues culminated last week with one of our larger releases of Mars images!  Here are some statistics about our September 2009 release, which includes the images the HiRISE camera took of the Martian surface between Mars Reconnaissance Orbiter (MRO) orbits 12,600 to 14,199, or roughly April 4 through August 6, 2009:

• 2,996 RDRs, 1 TB
• 42,370 EDRs, 1 TB
• 34,481 RDR Extras, 1.6 TB
• 83,784 EDR Extras, 0.02 TB
• 636 Anaglyphs, 0.01 TB

Totals for this release: 163,631 image products, 3.6 TB

This brings our total released product numbers and data volume to:

• 22,676 RDRs, 12 TB
• 317,120 EDRs, 10.4 TB
• 192,270 RDR Extras, 15.3 TB
• 612,769 EDR Extras, 0.1 TB
• 2,892 Anaglyphs, 0.5 TB

Total: 1,148,363 images, 37.5 TB

In summary, we released nearly 1500 observations, most of those with both black & white and color RDR products. Several newer observations matched up with older observations from a slightly different angle of the same location on the surface, resulting in 636 awesome new anaglyphs. The RDRs are the fully processed, geometrically projected products best for scientific inquiry. If you really want to, though, anyone can download and process HiRISE data from scratch. You can do this using ISIS software, which is publicly available for free download. See the ISIS Web site for download information, processing instructions, and tutorials.

Starting this week, I will be looking over the observations taken August 6 through August 26 before MRO went into safe mode and make sure they are ready for release. We plan to release these images in early October. We are also in the process of reprocessing those Extended Science Phase mission images prior to all the latest processing pipeline fixes and updates.  Once we are satisfied with that data set, we will release them to the public and then start reprocessing the images from the Primary Science Phase…a major project that should keep me and the rest of Downlink busy for several months!

GeoTIFF is an industry standard for embedding geographic information in images. Beginning soon, HiRISE RDRs will include GeoTIFF info in the Jpeg-2000 files. All of the information about the image will continue to be in the RDR label (.LBL plain text file), but with this additional info in the JP2, image viewing software that supports GeoTIFF will be able to take advantage of it.

For example, such software could display the actual coordinates on Mars of the pixels you are looking at, allow you to measure features directly in physical units, or stitch together images based on their absolute location on the planet. A number of GIS (Geographical Information Systems) applications use GeoTIFF; many on our science team have been waiting patiently for this feature to be rolled out.

We have already begun to produce RDRs with GeoTIFF, and they will start appearing in our weekly releases. At some point, a major reprocessing effort will be underway to bring this feature (and others) to all of our pre-existing products.

This brings up the topic of versioning: namely, how to tell which version of a HiRISE product you are working with.

(more…)

Today, our software group provided a set of major updates to our downlink operations team. It was the first major update in many months. One of the most anticipated features is smarter “stretch” algorithm for our color products (RDR Extras). As discussed in a previous post, a stretch (in image processing terms), is a mapping between one range of pixel values and another. In our case, it provides our viewers with a better-looking image up-front, with less need to adjust parameters in display software such as IAS (though this is still often very helpful when zoomed in). As always, the full range of original data is preserved in the RDR JP2.

Our former algorithm for the NOMAP and Quicklook products said that the pixel values above the brightest 0.1% and below the darkest 0.1% would be mapped to the extreme values, with a linear fit in between. For a majority of images, this was a good choice that showed excellent contrast but prevented too much saturation.

However, 0.1% (a thousandth) of a two Gigapixel image is still two million pixels. So if there were a particularly bright spot, like a rocky outcrop amid a field of dunes, or a particularly dark spot, like a cavern opening in a plain of boulders, then all the saturation would occur in that one area, washing it out completely, and lowering contrast everywhere else in the image. So the algorithm needed to be more adaptable. After a good deal of experimentation, the algorithm we settled on looks at the brightest and darkest pixels in a thumbnail version of the image, and uses those values for the extremes, instead of the values at 0.1%. We shrink a copy of each color band to 1/11th the original scale. Pixel values in the original below the darkest in the thumbnail are mapped to pure black, while pixel values above the highest are mapped to pure white. The stretched bands are then merged to make the color image. Hence, a bright or dark spot smaller than 1/11th x 1/11th of the image size will no longer dominate the stretch.

What this ultimately means is, our RDR Extras now show more detail in areas that would be completely washed out by the old algorithm.

For example, in this ‘cave’ image, the left is from the original RGB.NOMAP.JP2, while the right is the same product using the new algorithm. As you can see, previously you could not tell if there was a floor to the hole or if it sloped away to greater depths.

The new algorithm is used strictly for the JP2’s; the browse and thumb are already scaled down enough that it would not make a substantial difference with them. The new algorithm went into effect today; coincidentally we just started orbit 8000. Images with the new stretch will likely appear in upcoming weekly releases and we plan to reprocess everything with this change (and improved calibration) during the summer.

(more…)

Starting with the 10/10 release, color images are included for the first time. We’ll describe how we process these in the days and weeks to come. But what I’d like to do first is give a brief description of all our product types as they currently are available. You’ve no doubt noticed a mind-boggling array of new options on our product pages. They now include what we call our “NOMAP” products; NOMAP means that they are not map-projected. In other words, not rotated to the direction of north, not mapped to a coordinate system, and not scaled to any particular geometric resolution.

I’ve prepared this ugly table that outlines each of the products now available (excluding the raw EDRs). So reading the columns from left to right: there are three types of “NOMAP” products, two types of lossy “QLOOK” (Quicklook) RDRs, and two types of lossless RDRs.

With that as a reference, now I’ll try to define everything more precisely.

“NOMAP”

Non map-projected product. Always lossy compressed for smaller size and quicker viewing. These are not formal Planetary Data System products; they’re “special”, meaning there is no PDS label and no Software Interface Specification describing them. Available for IRB, RGB and RED.

RDR

Reduced Data Record: reduced in the sense of refined or processed, not raw data. Formal PDS products with accompanying labels and a detailed SIS document describing their format and processing steps. Available both in lossless and quicklook formats for both RED & COLOR.

“QLOOK”

Quicklook: a special product that is a lossy compressed version of the RDR. In a normal RDR, all of the original data is retained. But with a quicklook, some of the highest resolution detail is discarded to make for quicker viewing.

RED

The image obtained by the red-filtered CCDs. It will be over the full swath width, typically data from all ten red CCDs. Covers the visible wavelength band from 550 to 850 nanometers.

IR

Infrared. Covers the near-IR wavelengths from 800-1000 nanometers.

BG

Blue-Green, visible wavelengths from 400-600 nm.

COLOR

A color RDR. It contains data from the IR, BG and center RED ccds. Typically this will be a skinny strip (”center swath”) inside a skinny strip, or as I like to say, the bacon-strip effect.

IRB

An enhanced color NOMAP. It has the same color bands as the RDR: IR, RED and BG.

RGB

An enhanced color NOMAP. It contains only data from the RED and BG. The blue is derived from the difference between the RED and BG. The color bands are RED, BG and the synthetic blue.

EDR

Experiment Data Record, a formal PDS product that is raw uncompressed data with a label header.

Note: we will be working towards making all of these products available for all prior releases.

Spacecraft missions are complicated endeavors that result in a wealth of scientific and engineering data. Long after the mission has ended, these data can be extremely useful for later study and discovery. With so many missions over so many years, how can later generations find and make use of these data?

The solution for many NASA missions has been the development of the centralized Planetary Data System (PDS). The PDS is several things: a collection of websites, a search capability, an archive, a database, a learning tool, etc. The PDS Imaging Node is located at http://pds-imaging.jpl.nasa.gov/ and acts as “the curator of NASA’s primary digital image collections from past, present and future planetary missions.” These missions include Voyager, Galileo, Cassini, and many more. Now the Mars Reconnaissance Orbiter (MRO) has been added to the list, with the HiRISE team releasing our first several months of image data.

• MRO PDS page: http://pds-imaging.jpl.nasa.gov/Missions/MRO_mission.html
• MRO Product Search page: http://pds-imaging.jpl.nasa.gov/search/index.jsp
• HiRISE Volume: http://hirise-pds.lpl.arizona.edu/PDS/

What we have released is an archive of the HiRISE Experiment Data Records (EDRs) and Reduced Data Records (RDRs). EDRs are in the *.IMG file format and represent individual CCD channels (remember, there are 14 CCDs in the HiRISE camera and two channels per CCD, for a total of 28 channels). These EDRs are cleaned up, calibrated, stitched together, and mapped to Mars’ geometry, resulting in the RDR products. RDRs are in the *.JP2 and *.LBL formats. JPEG2000 is the technology that enables us to offer our gigantic images to the scientific community and the public in a timely and efficient manner. An observation’s image data are in the *.JP2 file and its meta data are in the detached *.LBL files. To view these products, JPEG2000 compatible software is required (see our site for a list of offerings).

While we have been trying to release up to five captioned images a week for the past few months, the PDS release represents several hundred images, most of them without captions. You can find them using the PDS search capabilities, and you can also find them on the new HiRISE site, unveiled today to coincide with this first PDS release. The redesigned site focuses on the images while providing, hopefully, a more user-friendly interface:

• HiRISE Site: http://hirise.lpl.arizona.edu/
• “About Our Redesign”: http://hirise.lpl.arizona.edu/profil.php
• Images released to the PDS: http://hirise.lpl.arizona.edu/pds_release.php

As word gets out about the new site and the PDS release, you may experience some site slowness. Please be patient, and thank you for your interest!