Planetary scientists used to keep new data from the spacecraft explorers of the solar system within the mission team for a lengthy period of time so they could make all the cool initial discoveries.
Only later would the mission’s data sets be archived on the public Planetary Data System (PDS). Once archived, these data could then be used by the scientific community and public for further research and discovery.
Dr. Alfred McEwen, HiRISE principal investigator, decided early on that this incredibly powerful instrument should be “The People’s Camera”. This meant, among other things, that we would endeavor to make the data returned by HiRISE available to the scientific community and public as quickly as possible. We have PDS release requirements, but our goal has always been to beat those requirements. To do so, we needed to develop automated software pipelines to take the raw data and turn them into useful calibrated and geometrically mapped products. We also needed to develop the right PDS release tools, train a talented group of operations staff to validate the data and fix problems, and develop a website to effectively and beautifully showcase HiRISE images.
We now believe we have reached the point to be able to support a monthly release of recent HiRISE images to the public! This week we released the observations HiRISE took of Mars between orbits 11,600 and 12,599, or between January 16 and April 04, 2009. This makes us the first mission to release a data set to the PDS so quickly! Here are the statistics for this release, including the number of each product type released and their respective data volumes (EDRs are the individual uncalibrated image channels and RDRs are the calibrated, mosaicked, and geometrically-projected observations):
- 1,179 RDRs, 520 GB
- 16,861 EDRs, 459 GB
- 13,512 RDR Extras, 788 GB
- 33,152 EDR Extras, 7 GB
- 342 Anaglyphs, 51 GB
Totals for this release: 64,704 image products, 1.7 TB
This brings our total released product numbers and data volume to:
- 19,667 RDRs, 11 TB
- 278,807 EDRs, 9.5 TB
- 166,816 RDR Extras, 13.7 TB
- 529,095 EDR Extras, 0.1 TB
- 2,892 Anaglyphs, 0.5 TB
Total: 993,277 images, 34 TB
Those are various products for about 9998 Mars observations, and another reason why it makes no sense to hoard our data; there is too much of it and too few of us! The team scientists have plenty to do and there are plenty of discoveries to be made, old hypotheses to update, and new mysteries to solve. The operations staff are now hard at work getting observations from orbits 12,600 through 12,999, or between April 04 and May 5, 2009, ready for the June PDS release. This involves making sure each observation has been processed by our software pipelines correctly, fixing any problems, and checking and double checking that the relevant image products are ready for release. Sometimes we have to manually force an observation through the pipelines because some of its channels were lost during transmission to the Earth, or we might stumble across an observation we somehow forgot to send on to the color pipelines after it had been calibrated. There are spreadsheets to maintain, lists of problematic observations to keep (see the ERRATA.TXT file), and a variety of other tasks that need to be completed before the latest data set is ready for release.
Over the next few months we will see how this goes! It is a lot of work, but our desire for you to see these beautiful images of Mars as quickly as possible is strong. No promises, but we will also explore releasing completed observations even faster!