For starters, our Extended Science Phase (ESP) images are now consistent, making use of all the latest and greatest fixes and improvements to our processing pipelines. These updated products are out now. The older Primary Science Phase (PSP) observations are still reprocessing; we should have them ready for an upcoming monthly release.

Next up, we have a fantastic new product called a Digital Elevation Model. Learn more about these products in an upcoming blog post!

Finally, several of our observations have been improved with updated descriptions.  For example, an image might have been taken of a crater that did not have an official name at that time. After the crater receives an official name, we try to go back and update the description for observations of that crater. Recently we had time to do that for a lot of observations. Unfortunately, the software used to update the EDR and RDR labels inadvertently corrupted the first few image data lines. Our attentive Targeting Specialists and Student Validators spotted the minor differences between reprocessed images and older versions. We have corrected the description update tools and reprocessed the EDRs from the 745 observations affected by this problem. Anyone who has downloaded HiRISE EDRs prior to the December 2009 release should check the list below to ensure any EDRs being worked with are not on this list. If the EDRs you have are from the observations on the list, you can check the “creation date” in their label. If the creation date is before November 4, 2009 then you should download the latest version. If the creation date is after November 4, 2009 then the EDR is good. The latest version of all of the EDRs for the following observations are now available on the PDS and on our website.

TRA_000823_1720 TRA_000827_2640 TRA_000828_2495 TRA_000836_1740 TRA_000841_1300 TRA_000841_2460 TRA_000844_2145 TRA_000846_2475 TRA_000847_2055 TRA_000848_2890 TRA_000849_1675 TRA_000856_2500 TRA_000860_2585 TRA_000861_1530 TRA_000861_2940 TRA_000863_2640 TRA_000865_2615 TRA_000869_1605 TRA_000870_1995 TRA_000873_1415 TRA_000873_1780 TRA_000873_2015 TRA_000876_1715 TRA_000878_1410 TRA_000881_1750 TRA_000881_2475 TRA_000882_2705 TRA_000894_2475 TRA_000896_2220 PSP_001348_1770 PSP_001349_1310 PSP_001351_1715 PSP_001368_1400 PSP_001389_2225 PSP_001390_2660 PSP_001401_1850 PSP_001417_1720 PSP_001438_2160 PSP_001454_2670 PSP_001481_2410 PSP_001493_2235 PSP_001507_1400 PSP_001510_1690 PSP_001528_2210 PSP_001530_2670 PSP_001552_1410 PSP_001563_2660 PSP_001580_2630 PSP_001582_2730 PSP_001584_2670 PSP_001608_2560 PSP_001616_2670 PSP_001630_2730 PSP_001636_2760 PSP_001640_2640 PSP_001652_2640 PSP_001660_1950 PSP_001678_1770 PSP_001686_1785 PSP_001686_2665 PSP_001691_1320 PSP_001697_1390 PSP_001730_2660 PSP_001768_2160 PSP_001782_1655 PSP_001794_2615 PSP_001800_2640 PSP_001810_1825 PSP_001820_1985 PSP_001823_1320 PSP_001871_1965 PSP_001876_1850 PSP_001892_1715 PSP_001930_1835 PSP_001935_1115 PSP_001938_1405 PSP_001948_2140 PSP_001964_2275 PSP_001968_1325 PSP_002002_1390 PSP_002014_1415 PSP_002021_1850 PSP_002022_1455 PSP_002041_1075 PSP_002068_1510 PSP_002100_1840 PSP_002108_1445 PSP_002108_2090 PSP_002128_2110 PSP_002146_1335 PSP_002172_1410 PSP_002179_1330 PSP_002186_2110 PSP_002193_1670 PSP_002199_1400 PSP_002202_2250 PSP_002204_1655 PSP_002214_2210 PSP_002238_2140 PSP_002281_1830 PSP_002281_2115 PSP_002283_1335 PSP_002285_1560 PSP_002286_1785 PSP_002287_2010 PSP_002289_1725 PSP_002290_1585 PSP_002291_1335 PSP_002292_1055 PSP_002292_1490 PSP_002292_1875 PSP_002293_1450 PSP_002294_1805 PSP_002315_2030 PSP_002317_1445 PSP_002317_2420 PSP_002319_1460 PSP_002320_1415 PSP_002320_2360 PSP_002321_1405 PSP_002321_2015 PSP_002324_1815 PSP_002326_1590 PSP_002327_1720 PSP_002327_2205 PSP_002328_2080 PSP_002329_1505 PSP_002330_1385 PSP_002332_1455 PSP_002336_2305 PSP_002337_1255 PSP_002337_1805 PSP_002338_1105 PSP_002338_2285 PSP_002340_1735 PSP_002343_1735 PSP_002344_1645 PSP_002344_2065 PSP_002345_1095 PSP_002345_1770 PSP_002347_1915 PSP_002351_1105 PSP_002351_2050 PSP_002354_1360 PSP_002354_1680 PSP_002358_1840 PSP_002359_1670 PSP_002359_2020 PSP_002360_1415 PSP_002361_1990 PSP_002365_1460 PSP_002368_1275 PSP_002369_1420 PSP_002370_1415 PSP_002371_1295 PSP_002371_1890 PSP_002372_1980 PSP_002373_2090 PSP_002379_1275 PSP_002379_1755 PSP_002380_0985 PSP_002380_1935 PSP_002381_1390 PSP_002385_1450 PSP_002387_1510 PSP_002387_1985 PSP_002390_1320 PSP_002390_1840 PSP_002391_1995 PSP_002396_1900 PSP_002397_1080 PSP_002397_1880 PSP_002398_1805 PSP_002398_2060 PSP_002399_1025 PSP_002401_2245 PSP_002403_1820 PSP_002406_1130 PSP_002406_1745 PSP_002411_1400 PSP_002411_1820 PSP_002411_2190 PSP_002413_1410 PSP_002419_1675 PSP_002420_2040 PSP_002423_1335 PSP_002425_1425 PSP_002425_2150 PSP_002427_1470 PSP_002429_1800 PSP_002431_1835 PSP_002432_1730 PSP_002435_1755 PSP_002437_1055 PSP_002437_1875 PSP_002439_2265 PSP_002444_1430 PSP_002444_1535 PSP_002446_1700 PSP_002449_1420 PSP_002449_1860 PSP_002451_1770 PSP_002456_1855 PSP_002457_1310 PSP_002457_1665 PSP_002457_2195 PSP_002459_0965 PSP_002459_1715 PSP_002461_1545 PSP_002461_1975 PSP_002464_1745 PSP_002468_1340 PSP_002472_1810 PSP_002474_1075 PSP_002478_1365 PSP_002478_1770 PSP_002482_1255 PSP_002484_1725 PSP_002486_1860 PSP_002492_1410 PSP_002494_0980 PSP_002496_2295 PSP_002498_1735 PSP_002502_1115 PSP_002508_1800 PSP_002510_1460 PSP_002514_1420 PSP_002522_1880 PSP_002524_2320 PSP_002526_2485 PSP_002528_1375 PSP_002528_1960 PSP_002532_0935 PSP_002532_2020 PSP_002533_0935 PSP_002536_1755 PSP_002536_2195 PSP_002538_1720 PSP_002539_2490 PSP_002542_1080 PSP_002542_1765 PSP_002544_1025 PSP_002544_1540 PSP_002545_1430 PSP_002548_1255 PSP_002550_1725 PSP_002551_1700 PSP_002560_1335 PSP_002567_0955 PSP_002568_2090 PSP_002570_1950 PSP_002572_1520 PSP_002574_1865 PSP_002576_0930 PSP_002580_2035 PSP_002582_1055 PSP_002584_0935 PSP_002586_1880 PSP_002590_1765 PSP_002590_2320 PSP_002592_2030 PSP_002594_1920 PSP_002600_0955 PSP_002602_0930 PSP_002602_1875 PSP_002605_2080 PSP_002617_0930 PSP_002617_1690 PSP_002618_1920 PSP_002620_1410 PSP_002622_0945 PSP_002622_1820 PSP_002628_1760 PSP_002630_1695 PSP_002632_2175 PSP_002634_1725 PSP_002640_1325 PSP_002641_2025 PSP_002642_1715 PSP_002646_2035 PSP_002648_1880 PSP_002651_0930 PSP_002652_0930 PSP_002655_1280 PSP_002655_2035 PSP_002664_0930 PSP_002668_1875 PSP_002671_1790 PSP_002675_0945 PSP_002676_1415 PSP_002677_1695 PSP_002679_2245 PSP_002680_1825 PSP_002682_1510 PSP_002686_1410 PSP_002689_1025 PSP_002689_1540 PSP_002692_0930 PSP_002692_2295 PSP_002694_1985 PSP_002696_1745 PSP_002702_0935 PSP_002703_1920 PSP_002706_1870 PSP_002710_1370 PSP_002710_2040 PSP_002711_1550 PSP_002712_1925 PSP_002718_0950 PSP_002720_1585 PSP_002721_1530 PSP_002721_2210 PSP_002722_1665 PSP_002728_1645 PSP_002731_1210 PSP_002733_1880 PSP_002736_2075 PSP_002737_1985 PSP_002743_1985 PSP_002747_2030 PSP_002752_0930 PSP_002752_1405 PSP_002752_1800 PSP_002754_1005 PSP_002756_1830 PSP_002758_0945 PSP_002759_0855 PSP_002759_1815 PSP_002761_1220 PSP_002762_0955 PSP_002764_1800 PSP_002766_2285 PSP_002770_0940 PSP_002772_1205 PSP_002776_0940 PSP_002776_2025 PSP_002782_2230 PSP_002786_1090 PSP_002786_1580 PSP_002788_2010 PSP_002793_1885 PSP_002794_1755 PSP_002800_0955 PSP_002804_0930 PSP_002806_1870 PSP_002809_1965 PSP_002810_0950 PSP_002812_1330 PSP_002812_1855 PSP_002814_1665 PSP_002814_2055 PSP_002816_1840 PSP_002820_0945 PSP_002820_1860 PSP_002822_0955 PSP_002822_1830 PSP_002824_1355 PSP_002824_2295 PSP_002828_0940 PSP_002828_1700 PSP_002832_1770 PSP_002833_1005 PSP_002838_1090 PSP_002839_1825 PSP_002840_1855 PSP_002841_1740 PSP_002844_0925 PSP_002846_1040 PSP_002846_1390 PSP_002848_1330 PSP_002848_2245 PSP_002850_0935 PSP_002853_1720 PSP_002858_1990 PSP_002860_1650 PSP_002860_2095 PSP_002862_1195 PSP_002862_1715 PSP_002866_1280 PSP_002866_2225 PSP_002868_0985 PSP_002871_1425 PSP_002874_1665 PSP_002876_0935 PSP_002878_1880 PSP_002882_0940 PSP_002884_1395 PSP_002885_1080 PSP_002888_1400 PSP_002888_2025 PSP_002890_2205 PSP_002892_1760 PSP_002894_0945 PSP_002899_1100 PSP_002900_1025 PSP_002902_2205 PSP_002904_1255 PSP_002908_0865 PSP_002908_2200 PSP_002909_2000 PSP_002910_0930 PSP_002912_1510 PSP_002912_2075 PSP_002914_1275 PSP_002917_2175 PSP_002918_2055 PSP_002922_0945 PSP_002922_1725 PSP_002924_1120 PSP_002928_1035 PSP_002932_1445 PSP_002936_0955 PSP_002938_1890 PSP_002940_0935 PSP_002940_1740 PSP_002942_0935 PSP_002944_1880 PSP_002946_1765 PSP_002948_0935 PSP_003062_0930 PSP_003063_1095 PSP_003063_1525 PSP_003063_2050 PSP_003064_0815 PSP_003068_0955 PSP_003070_1055 PSP_003070_1400 PSP_003074_0945 PSP_003075_0930 PSP_003075_2020 PSP_003077_1530 PSP_003078_2320 PSP_003083_1890 PSP_003084_2085 PSP_003086_2015 PSP_003087_0930 PSP_003090_1895 PSP_003092_0985 PSP_003092_1715 PSP_003093_2485 PSP_003094_1430 PSP_003096_1375 PSP_003096_1870 PSP_003098_1075 PSP_003101_2065 PSP_003102_1320 PSP_003104_1655 PSP_003104_2155 PSP_003106_1190 PSP_003110_1005 PSP_003112_1445 PSP_003113_0940 PSP_003114_0930 PSP_003118_0925 PSP_003118_1690 PSP_003125_1665 PSP_003125_2250 PSP_003141_1330 PSP_003151_0935 PSP_003155_1820 PSP_003158_0985 PSP_003177_2275 PSP_003179_1830 PSP_003213_0955 PSP_003234_2210 PSP_003237_0985 PSP_003246_0950 PSP_003246_2180 PSP_003249_0925 PSP_003270_1405 PSP_003285_0935 PSP_003306_1045 PSP_003306_1510 PSP_003312_1810 PSP_003314_1825 PSP_003326_1255 PSP_003360_1845 PSP_003363_1690 PSP_003368_1755 PSP_003373_1955 PSP_003378_2145 PSP_003379_1835 PSP_003381_1540 PSP_003382_1715 PSP_003388_1590 PSP_003392_1825 PSP_003398_1910 PSP_003418_1865 PSP_003442_1985 PSP_003448_0985 PSP_003496_0935 PSP_003517_1395 PSP_003520_1010 PSP_003523_0935 PSP_003560_1900 PSP_003575_0935 PSP_003576_1745 PSP_003593_0985 PSP_003603_1325 PSP_003611_1970 PSP_003641_0935 PSP_003660_2155 PSP_003669_1325 PSP_003728_0930 PSP_003776_2020 PSP_003790_1755 PSP_003798_1985 PSP_003803_0940 PSP_003814_1325 PSP_003832_0945 PSP_003880_1325 PSP_003907_1780 PSP_003937_0945 PSP_003940_1955 PSP_003944_0940 PSP_003984_1720 PSP_004009_1355 PSP_004044_1165 PSP_004060_1440 PSP_004074_1080 PSP_004091_1325 PSP_004130_1325 PSP_004206_1075 PSP_004219_1155 PSP_004244_1970 PSP_004249_1255 PSP_004299_1985 PSP_004313_1760 PSP_004340_1235 PSP_004341_1330 PSP_004421_1850 PSP_004434_1885 PSP_004510_1990 PSP_004673_0935 PSP_004709_0995 PSP_004739_0935 PSP_004765_0940 PSP_004777_1825 PSP_004778_0945 PSP_004799_1550 PSP_004822_1160 PSP_004842_1325 PSP_004869_1325 PSP_004976_0940 PSP_004989_0945 PSP_005017_1160 PSP_005063_1070 PSP_005095_0935 PSP_005234_1555 PSP_005251_1325 PSP_005357_1840 PSP_005374_1095 PSP_005392_1885 PSP_005397_1270 PSP_005405_1645 PSP_005408_1235 PSP_005550_1440 PSP_005593_2005 PSP_005600_1160 PSP_005616_1440 PSP_005647_1815 PSP_005649_1710 PSP_005658_1760 PSP_005766_1850 PSP_005779_1775 PSP_005816_1750 PSP_005842_1970 PSP_005883_2040 PSP_005890_1160 PSP_005947_2250 PSP_005988_1860 PSP_006003_1320 PSP_006126_0945 PSP_006135_2025 PSP_006141_1500 PSP_006143_1745 PSP_006145_1855 PSP_006146_1055 PSP_006161_1250 PSP_006163_1345 PSP_006171_1085 PSP_006191_2045 PSP_006233_1195 PSP_006239_1715 PSP_006288_1740 PSP_006572_1665 PSP_006602_1560 PSP_006609_1330 PSP_006651_1665 PSP_006679_1680 PSP_006688_2140 PSP_006715_1320 PSP_006726_1830 PSP_006813_1565 PSP_006820_1760 PSP_006821_1270 PSP_006891_1260 PSP_006899_1330 PSP_006908_2215 PSP_006910_1975 PSP_006926_1320 PSP_006945_1490 PSP_006965_1330 PSP_006973_1740 PSP_006978_2140 PSP_007029_1775 PSP_007045_2060 PSP_007082_1830 PSP_007135_1400 PSP_007183_1115 PSP_007228_1630 PSP_007231_1345 PSP_007266_1970 PSP_007420_0120 PSP_007525_1565 PSP_007559_2070 PSP_007583_1830 PSP_007715_2530 PSP_007719_1710 PSP_007746_2035 PSP_007767_1970 PSP_007796_2195 PSP_007804_1970 PSP_007811_1350 PSP_007849_1775 PSP_007949_1970 PSP_007961_2530 PSP_007964_1860 PSP_008088_1670 PSP_008131_2615 PSP_008161_2505 PSP_008205_1880 PSP_008416_2585 PSP_008448_1495 PSP_008665_2040 PSP_008703_1985 PSP_008738_1495 PSP_008741_2505 PSP_008755_2160 PSP_008846_2220 PSP_008871_1880 PSP_008883_1490 PSP_008914_2005 PSP_008930_1880 PSP_008932_1645 PSP_008978_1950 PSP_009008_1915 PSP_009020_1825 PSP_009028_1495 PSP_009069_1650 PSP_009089_1550 PSP_009268_1995 PSP_009270_1970 PSP_009324_1830 PSP_009408_2640 PSP_009447_1730 PSP_009648_2650 PSP_009758_2030 PSP_009901_1440 PSP_009928_1600 PSP_010038_1500 PSP_010154_1825 PSP_010193_1970 PSP_010228_1490 PSP_010364_1835 PSP_010368_2225 PSP_010500_1575 PSP_010524_1655 PSP_010584_1500 PSP_010679_2205 PSP_010710_1320 PSP_010725_1345 PSP_010749_1325 ESP_011327_1980 ESP_011391_1440 ESP_011403_0945 ESP_011404_0945 ESP_011509_1380 ESP_011522_1045 ESP_011527_1325 ESP_011536_1440 ESP_011538_1970 ESP_011600_1120 ESP_011630_1985 ESP_011714_1345 ESP_011718_1500 ESP_011727_1490 ESP_011732_1595 ESP_011738_1325 ESP_011749_1000 ESP_011931_0945 ESP_011954_0930 ESP_011958_1845 ESP_012024_1440 ESP_012206_1390 ESP_012215_0935 ESP_012272_1390 ESP_012294_0935 ESP_012307_0940 ESP_012320_1530 ESP_012326_2125 ESP_012333_0945 ESP_012361_1685 ESP_012398_1775 ESP_012411_1320 ESP_012479_1710 ESP_012504_1410 ESP_012559_0945 ESP_012588_0930 ESP_012628_1390 ESP_012641_1495 ESP_012708_0855 ESP_012767_1320 ESP_012852_1495 ESP_012863_0820 ESP_012869_1860 ESP_012912_1320 ESP_012965_1770 ESP_013058_1550 ESP_013092_1630 ESP_013103_1315 ESP_013110_1325 ESP_013123_1330 ESP_013189_1330 ESP_013217_0970 ESP_013243_0945 ESP_013268_1330 ESP_013272_1090 ESP_013274_1495 ESP_013283_0935 ESP_013287_1160 ESP_013302_1095 ESP_013334_1320 ESP_013514_1630 ESP_013543_1060 ESP_013677_1835 ESP_013692_0935 ESP_013726_1475 ESP_013751_1810 ESP_013775_1490 ESP_013954_1780 ESP_014081_1440 ESP_014147_1440 ESP_014151_1325 ESP_014329_1435

• 2,996 RDRs, 1 TB
• 42,370 EDRs, 1 TB
• 34,481 RDR Extras, 1.6 TB
• 83,784 EDR Extras, 0.02 TB
• 636 Anaglyphs, 0.01 TB

Totals for this release: 163,631 image products, 3.6 TB

This brings our total released product numbers and data volume to:

• 22,676 RDRs, 12 TB
• 317,120 EDRs, 10.4 TB
• 192,270 RDR Extras, 15.3 TB
• 612,769 EDR Extras, 0.1 TB
• 2,892 Anaglyphs, 0.5 TB

Total: 1,148,363 images, 37.5 TB

In summary, we released nearly 1500 observations, most of those with both black & white and color RDR products. Several newer observations matched up with older observations from a slightly different angle of the same location on the surface, resulting in 636 awesome new anaglyphs. The RDRs are the fully processed, geometrically projected products best for scientific inquiry. If you really want to, though, anyone can download and process HiRISE data from scratch. You can do this using ISIS software, which is publicly available for free download. See the ISIS Web site for download information, processing instructions, and tutorials.

Starting this week, I will be looking over the observations taken August 6 through August 26 before MRO went into safe mode and make sure they are ready for release. We plan to release these images in early October. We are also in the process of reprocessing those Extended Science Phase mission images prior to all the latest processing pipeline fixes and updates.  Once we are satisfied with that data set, we will release them to the public and then start reprocessing the images from the Primary Science Phase…a major project that should keep me and the rest of Downlink busy for several months!

Consider the May PDS release our practice run. Thank you for your patience…monthly releases of HiRISE images are indeed coming, but not quite as soon as we expected.

In these days leading up to the beginning of ESP the downlink team is checking our processing software and tools to make sure they are ready for the observation ID prefix change from PSP_XXXXXX_XXXX to ESP_XXXXXX_XXXX. The automated processing pipelines are ready to go.  My own validation and reporting Perl scripts make use of modules that are mission phase aware and pattern match file extensions instead of observation ID prefixes, so I think I am set. Any minor tools we miss can be easily updated as necessary.

With improvements to our tools and new procedures, we can sometimes recover image data previously stuck in the original raw data files. During this quiet period, I had an opportunity to go back to old observations, some from early in the Primary Science Phase (PSP), and recover errant channels that failed our processing software at the time.   This resulted in a few new channels of image data that we will include in a later reprocessing of our images.  For example, PSP_001746_1515 was originally processed without the RED1_0 channel. You can see this channel gap if you click on the observation’s “Full image (grayscale, non-map projected)” link.  An improved mosaic will include most of RED1_0, albeit with a small image data gap near one end.  Why did we not create this channel before?  Sometimes a data gap occurs between channels, obliterating the second channel’s science header.  The software at Jet Propulsion Laboratory’s RSDS that splits our image data into separate raw data channel files cannot figure out where to split the image data correctly within this gap and so image data from two or more channels are trapped in one raw data file.  When this jumbo raw data file hits our EDRgen processing pipeline here in Tucson, Arizona, the tool called HiRISE_Observation became confused when it discovered image data from another channel near the end of the file, and the error it flags halts the EDRgen processing pipeline.

We now have improved tools and procedures for dealing with this problem.  By running HiRISE_Observation outside of the EDRgen pipeline, it will successfully produce an EDR for the first channel before it complains about the second channel’s image data.  While we would love to get at that second (or third, or later!) channel’s image data, right now we recoverthe data that is easiest to reach.  Someday, we might have a tool that will recognize a missing science channel header, reconstruct this header, and then fit the header in correctly between the first channel and the additional channels stuck in one raw data file.

When will you see the new PSP_001746_1515 products, newly improved with the recovered RED1_0 channel?  Hopefully in a few months.  While I have recovered this EDR (as well as EDRs from other observations), we would like to wait to reprocess the entire observation until we have even better image calibration built into our processing pipelines.  It makes little sense for me to reprocess this observation now when we are just going to be reprocessing it again sometime early next year. Once all of our improvements are in place in the upgraded processing pipelines, we will reprocess ALL of our observations, a huge undertaking.

Next week we will be back to our normal downlink activities: making sure the processing pipelines behave, validating new observations, providing daily reports to the rest of the team, and processing color and RDR mosaics.  We will also add in routine creation and validation of anaglyphs and preparation for our next Planetary Data System (PDS) release in March.  The quiet time during solar conjuction was nice, but we are anxious to see new images from Mars!

And now, for all of those curious minds out there, a brief overview of the HiGlyph Pipeline:

• Anaglyphs are created in a three-step process. The first step is to take the two images of the stereo pair and map project them. This helps the pipeline determine which image will be the left image and which will be the right image in the anaglyph.

• The second step takes the two images and looks to see if there are any improvements that can be done on them before putting them together. If there are not, the images move on. Often, due to the difference in viewing angle, the two images do not have a 100% overlap. Thus, to make the image a bit neater and easier to see, we trim off the excess portion of the image (the parts that do not overlap) and then assemble them so the left image is the red and the right image is the blue/green.

• The third and final step of this image processing is simply to prepare the images you see here and to update our catalog.

Seems complicated, right? Well luckily we have wonderful programmers that create these intricate programs. All I have to do is create a list of these images and run them through this pipeline. What really makes my job interesting is the validation process!

I have had the pleasure of being able to look at all 362 of the anaglyphs we have released today. But, you might ask, aside from looking super cool in 3-D glasses , what does it take to validate these anaglyphs? Well, at the beginning of this process the student validators and I got to ask that very same question. Since HiRISE has never had software to create images like this before, we played lab rat and came up with an entirely new technique for validation.

• You may notice that when not wearing the 3-D glasses there is a bit of a horizontal shift in the anaglyph. This shift is good because it is what allows us to see the image in 3-D. But, since the map projection of this process is not always spot on, we sometimes wind up with a vertical shift too. This is bad! Since most of us do not have googly eyes, this makes the image very difficult to see. With our validation process, we have to spot this out and fix it so that you do not have to strain your eyes (well… not too much at least ) in order to see the anaglyph.

Well, with that said, I leave you to your regularly scheduled HiRISE browsing! Enjoy!

Here are a few examples of cool images, which were previously unreleased:

• PSP_008248_2640, Polygons and spots on defrosting dunes (right)
• PSP_008269_1395, crazy weird stuff in Hellas Planitia (be sure to look at the whole browse image on this one!)
• PSP_008322_1865, Multiple generations of slope streaks on a crater in Arabia Terra (left)
• PSP_008343_1430, Gullies on mesas in Gorgonum Chaos

I’ve only looked through the first few pages in the release. I know there are a lot more amazing images in there, so if you’re browsing through the images, post some of your favorites below!

For example, such software could display the actual coordinates on Mars of the pixels you are looking at, allow you to measure features directly in physical units, or stitch together images based on their absolute location on the planet. A number of GIS (Geographical Information Systems) applications use GeoTIFF; many on our science team have been waiting patiently for this feature to be rolled out.

We have already begun to produce RDRs with GeoTIFF, and they will start appearing in our weekly releases. At some point, a major reprocessing effort will be underway to bring this feature (and others) to all of our pre-existing products.

This brings up the topic of versioning: namely, how to tell which version of a HiRISE product you are working with.

Every HiRISE product has a line in the PDS label with the DATA_SET_ID. Every released HiRISE product to date shows that it has a DATA_SET_ID version of 1.0. The GeoTIFF RDR’s have a DATA_SET_ID showing version 1.1. This 1.1 version of our RDR processing also contains the updated color stretches described in a previous post. Later this year, that version number will likely be bumped up again when our improved calibration algorithms are put into production.

In addition, every HiRISE PDS product also has a line in the label with the PRODUCT_VERSION_ID. And every released HiRISE product to date has a PRODUCT_VERSION_ID of 1. When we make a new version of a released product, the version number will be incremented. If that new version is then released, it will replace the older version. We will only replace products with newer versions after a process of validation.

We’re halfway through a major update to our production pipelines that allows us to create, store, and reprocess these newer versions without effecting the released versions. This version number will only increase by integer amounts, and will never “skip”.

Our former algorithm for the NOMAP and Quicklook products said that the pixel values above the brightest 0.1% and below the darkest 0.1% would be mapped to the extreme values, with a linear fit in between. For a majority of images, this was a good choice that showed excellent contrast but prevented too much saturation.

However, 0.1% (a thousandth) of a two Gigapixel image is still two million pixels. So if there were a particularly bright spot, like a rocky outcrop amid a field of dunes, or a particularly dark spot, like a cavern opening in a plain of boulders, then all the saturation would occur in that one area, washing it out completely, and lowering contrast everywhere else in the image. So the algorithm needed to be more adaptable. After a good deal of experimentation, the algorithm we settled on looks at the brightest and darkest pixels in a thumbnail version of the image, and uses those values for the extremes, instead of the values at 0.1%. We shrink a copy of each color band to 1/11th the original scale. Pixel values in the original below the darkest in the thumbnail are mapped to pure black, while pixel values above the highest are mapped to pure white. The stretched bands are then merged to make the color image. Hence, a bright or dark spot smaller than 1/11th x 1/11th of the image size will no longer dominate the stretch.

What this ultimately means is, our RDR Extras now show more detail in areas that would be completely washed out by the old algorithm.

For example, in this ‘cave’ image, the left is from the original RGB.NOMAP.JP2, while the right is the same product using the new algorithm. As you can see, previously you could not tell if there was a floor to the hole or if it sloped away to greater depths.

The new algorithm is used strictly for the JP2’s; the browse and thumb are already scaled down enough that it would not make a substantial difference with them. The new algorithm went into effect today; coincidentally we just started orbit 8000. Images with the new stretch will likely appear in upcoming weekly releases and we plan to reprocess everything with this change (and improved calibration) during the summer.

Here is a more specific description of our old and new stretch algorithms.

Old Algorithm

Determine the min and max DN value corresponding to the following cumulative
percents into the histogram.

For the full RDRs, the stretch is linear and the output is 10-bit and
preserves the special pixel values: 0 = NULL, 1 = Low Representation Saturation, 2 = Low Instrument Saturation , 1022 = High Instrument Saturation, 1023 = High Representation Saturation.

For everything else, the stretch is linear, 8-bits per band and each band
is done independently. Special pixels are not preserved.

New Algorithm

The JPG products (browse and thumb) are be made with the original algorithm, with the following exception.

• The Color JPGs use the same stretch as the IRB JPGs. (Otherwise, the black area around the rotated image change the stretch).

The JP2 products are stretched as described above, with the following changes:

• The min and max for the stretch are determined not from a cumulative percentage
  but from the min and max DN taken by reducing the image or band by a factor of

  11 * ( binning / binning of red band )

  . The reduction averages pixels.

Notes

1. Due to PDS format restrictions, the 3 bands of the color RDR (IR-RED-BG) are not
   stretched independently.
2. The ISIS applications hirdrgen and isis2raw are used for stretching. The latter for
   RDR quicklooks and NOMAPs.

A few days ago I realized that after more than a year, I was still in the manual stage of monitoring data quality and not making good use of existing tools to help streamline the process. All that cutting and pasting was beginning to get ridiculous, even more so during a period of high data rates and an increasing number of observations.

When we receive a product with gaps, our automated processing pipelines usually process it just fine. Sometimes, however, there are pipeline failures. I keep a list of these failures and the actions I have taken to try to correct the problem. This generally means I have to manually processing the file or repairing its metadata header. Once the channel is repaired and/or recovered, then I can feed it back into the remaining processing pipelines.

These gapped products include “_G” in their file name. In a spectacular “Duh!” moment, I realized that rather than hunting in my browser through a list of observations that might have gapped product names, and then copying and pasting any I find into Excel, I could instead just perform a command line search in Terminal for all raw data products with “_G” in their file name.

     % ls -1 */*_G*.DAT

Why didn’t I do this over a year ago!? I am embarrassed to say I have no idea.

This only provides a list of products with gaps. Missing channel products usually arrive as gapped channel products 48 hours later (due to the nature of the automated processes that handle new MRO data at JPL.) In my spreadsheet, I scan through the missing channels and determine if a gapped file has in fact arrived. If it has been longer than 48 hours I consider the data lost for good and I “force” whatever observation data we have received through the pipelines. Of course, sometimes lost data is found or reprocessed at JPL long after the 48 hours has expired, and very rarely these new data will arrive. This requires reprocessing of the entire observation so that this newly found data can be added in.

A long time ago, our database specialist wrote a Perl script to create a daily list of missing and partial observations. For whatever reason, I stopped using this tool and then promptly forgot about it. Instead I started looking through the list of observations in my browser, clicking on those that were not complete, and manually figuring out what channels were missing. Making use of this missing EDR’s tool is so much easier and faster. Again, “Duh!”

The output from these quick searches for gapped and missing products requires a little bit of tweaking to make it look nice in Excel, but then a quick sort merges the two list into observation ID order. In no time at all, I have my list of observation products to follow up on for the day. My copying, pasting, and mouse clicking madness has been vastly reduced.

Every day I see the very worst data, generally caused by data transmission problems, but these data make up only a small percentage of all HiRISE data. To try to quantify this, I counted up the number of channels I have reported with problems (they have gaps, were missing, were somehow corrupted, etc.) and then divided this number by the total number of raw data files we have received (these are transformed into EDRs after being downloaded to Tucson.) As of this morning, I have listed 5392 channels with problems, and we have received 160,858 raw channel files. This is roughly 3% of our data, although most of these have some useful data in them. Even an observation with gaps or missing a channel or two is of potential use to a scientist. If too much data has been lost, then our targeting specialists might command HiRISE to try again in a later orbit.

Over time, we have developed many procedures for dealing with all sorts of problems. Now that I have sped up my daily data quality monitoring, I will have more time to improve these procedures, partially automate as much as I can, and provide suggestions to the software developers about tools that would make my job ever more efficient.

Note: My calculation of problematic data above is not rigorous and only a rough estimate. One flaw: I am counting missing channels in the numerator but not in the denominator.

This process seems simple in concept, but in practice it is quite complicated. There are three factors to account for:

• Relative timing
• Pixel binning
• Spacecraft jitter

First of all, HiRISE nearly always uses different resolutions for each color. For instance, RED might be at a scale of 25 cm/pixel (bin1) while IR and BG are at 1 meter/pixel (bin4). This “binning” minimizes the amount of data that has to be sent back to Earth, which is the most important constraint that HiRISE needs to deal with. Another reason for binning color is to improve the signal-to-noise ratio (SNR). This means that you can get a better signal by combining pixels at the expense of spatial resolution. In order to get the binned IR and BG images to line up properly, they must be enlarged to match the dimensions of the RED image. For example: RED is bin1, IR and BG are bin2. The RED image will be 2000 pixels wide by 40,000 pixels long (for example), the corresponding IR and BG images will be 1000 pixels wide by 20,000 pixels long. So the first step is to make the dimensions of all three images match.

Now the relative timing is easy to take care of. The start time of all images is a known quantity, and does not change from image to image. We know exactly when the BG detector starts imaging, followed by the RED detector, followed by the IR detector. So the beginning of each image is offset by a fixed amount. Once the images are shifted by this fixed offset (accounting for binning), they will be approximately lined up.

This brings us to the third factor in coregistering images — spacecraft jitter. Because HiRISE is imaging at such a high spatial resolution and at great speed, tiny motions of the MRO spacecraft cause slight variations in where the surface features appear in each of the three color detectors. Imagine that HiRISE is taking a picture of a 1m sized boulder on Mars. If the rock shows up in line 100 of the RED image, and we have already accounted for the relative offsets of the detectors and for binning, then the rock should also show up in line 100 of the BG image. But say we look and it is actually in line 99. Now when we try to stack the two images, the objects in them won’t line up exactly. Our color processing software corrects for this by holding the RED image fixed, and adjusting the corresponding BG and IR images to match it precisely. This is not a perfect process, but most of the time it works extremely well.

Producing the color HiRISE products is not a trivial process. But it is to a point where the processing is automated so new data is released without delay. Enjoy the colorful view!