Over 9400 of these observations have been released to the Planetary Data System as JP2 files (over 9000 of these have color). As of today, 365 observations have been received after the end of the last PDS delivery, and are in the queue for the next one.

A handful of observations have data gaps in critical portions of the raw files (namely, science channel headers), or other severe problems that prevent processing from proceeding. We’re working on improved tools and procedures to alleviate the large amount of manual work to process those images; there’s a very good chance that the usable data can ultimately be recovered and released.

The rest (243 to be exact) are ‘CAL’ or calibration observations, used to verify settings on HiRISE, or obtain data that helps us remove noise and other imaging artifacts in our automated processing. They’re normally taken while on the night side of Mars with ’stim’ lamps on to provide a known, fixed light source.

Take a look at the plot below, which shows the cumulative number of observations received (in salmon) and the cumulative number released (as RDR’s, in mauve) as a function of time.

• The big steps are PDS data releases. The first was six months after beginning the primary operations in November 2006.
• There’s a missing step around December of 2007. That’s because we released a large number of color images, getting caught up with our color processing in lieu of a standard EDR/RDR release.
• The horizontal steps are not completely horizontal. The slope is our rate of weekly featured image releases.
• As noted earlier, just about everything is released, the totals are converging, the lag is 365 observations, or about a month at our current rate.
• The totals are a little less than 10,000; that’s the difference made by the CAL observations and the handful that currently can’t be easily recovered.
• There’s a ‘hump’ of received observations in late 2008, that’s the high data rate period (when Earth and Mars were closest).
• There are a few flat steps where no observations are received. One is the month-long solar conjunction period centered on Nov-Dec 2008, the safe modes of February 2007 and February 2008, etc.

We’re doing about ten observations per day at the moment. During the high data rate period it was nearly double that.

Finally, while we’re getting all nerdy with numbers, let’s look at the words too. Namely, the words used by our science theme leads to describe each observation. Here a word-frequency list, the top 100 words with the number of times it has been used.

3291 in
2777 crater
2502 of
1105 and
1050 polar
876 sample
874 layered
839 region
779 deposits
673 on
660 north
632 possible
572 terrain
552 gullies
547 with
539 chasma
534 south
418 site
404 the
388 planitia
369 vallis
360 dunes
357 terra
353 light
342 toned
331 fossae
326 near
323 impact
318 landing
314 mons
312 floor
299 small
296 central
289 exposure
287 northern
283 plains
280 planum
264 latitude
260 valles
250 fresh
238 image
238 ejecta
234 wall
229 at
215 layers
213 dark
211 flow
196 west
193 monitoring
193 hellas
190 for
190 basin
188 moc
183 layering
182 deposit
175 valley
174 rim
173 large
171 high
168 lander
167 phoenix
166 southern
164 peak
162 surface
159 well
152 between
150 seen
144 ice
143 elysium
141 east
139 material
138 unnamed
138 msl
136 channel
135 pit
134 scarp
134 rover
134 a
128 olympus
128 meridiani
120 ridge
119 seasonal
119 olivine
119 cap
117 arabia
115 lava
115 features
114 landforms
109 residual
107 exposed
106 chaos
105 utopia
103 western
103 contact
102 looking
101 nili
100 area
99 kilometer
99 candor
97 slope

On the technical side, we’ve been working with the spacecraft engineers on this sequence for a while now. Any special activity like this requires a lot of hand-built commands, which have to be carefully reviewed and tested before we send them to MRO. We had to slew away from Mars, towards the Earth. Then we moved HiRISE’s focus mechanism a little, so we’d be in better focus to view the Earth & Moon. The spacecraft then slowly rotated so the Earth would pass through our field of view. Then it rewinded (rewound? ) & repeated the same thing, so we could take a second image that would be exposed correctly for the Moon, which is much darker. The focus mechanism was moved back to our perfect Mars focus, and then we resumed imaging Mars normally.

Due to the downlink outage I referred to in the previous post, we’ve had to scramble to protect these data. Many thanks to a certain prophetically-named Targeting Specialist for working so hard to cancel (and un-cancel) other images in order to make sure we get these down safely. After all this work, we really want to see the pictures! It won’t be for a while, though. We have to wait for the data to be downlinked and processed, which could take several days. Since these image are so different from our regular Mars images, our regular smooth pipelines are useless. Instead, everything has to be done by hand by our diligent downlink team. Fingers crossed until then!

UPDATE: This image has finally been released! http://hirise.lpl.arizona.edu/earthmoon.php The color processing had to be done by hand, and this just wasn’t as high of a priority as getting more Mars images, improving software, and preparing for our big PDS releases.

One by one, we’re introducing new uplink personnel to the complex system of uplink planning. In fact, each cycle so far has been planned by a new Targeting Specialist, so we’re still in the midst of a lot of training. Once we each get through one cycle, I think we’ll notice a big improvement in efficiency. Starting with cycle 007, new CIPPs (Co-Investigators of the Pay Period — the science team members who help plan observations for a given cycle) will also be visiting HiROC for a few weeks while they learn about the planning process. So the “cycle” of training continues, with Targeting Specialists training each other, and then passing that knowledge on to new CIPPs. We expect this to persist through PSP, since there are ten Co-Investigators, plus their post-doctoral researchers and students who may get involved.

Special activities like off-planet observations provide a little variety, a change from the cycle of planning and the constant cascade of downlinked data. Some of these approaching activities include:

• Stellar calibration — 11 January
• Updating the LUTs (Look-Up Tables) we use to convert from 16-bit to 8-bit data on board the spacecraft — mid-January
• Deimos calibration — late January

We’re currently developing and testing the command sequences for these events.

There are three stages to validation: quick look validation, in-depth validation, and geometric validation. Most of the validation is done by undergraduate students in a variety of departments from the University of Arizona. If they notice something odd, they flag it and notify the full-time operations team, who do a more detailed analysis. Currently there are two students who work solely with validation, and me. I volunteer when their work load becomes heavy (normally I work as a programmer for different software needed at HiROC).

The first stage is to let the staff at HiROC know quickly if there is any problem in commanding, or if there is haze in the field of view. Low resolution “browse” images that have come down in the last 24 hours are examined by someone (usually either myself or RichardLeis) to see if there is anything obviously wrong. If there is suspicion, a flag is raised and then other people will take a look at the images. This quick look helps prevent any commanding issues from continuing, and also helps us avoid taking more pictures in areas with some kind of atmospheric distortions (”haze”). This could include dust storms, clouds, melting ice caps in the polar regions, etc.

The second stage of more in-depth analysis involves a tool called HiVali. This tool allows one to quickly take a look at an image in more detail, and see if there are any kinds of problems with it. It reports statistics of pixels to see if there is saturation or low contrast. It checks to see if there are any gaps in the image, and other kinds of routine image checks. One part of this process, the part that takes the most time, is the visual validation, where a human physically looks at every inch of the picture in high resolution to see if there is anything odd. These are usually things which can be fixed in the calibration processing.

The third stage is also a quick look, which is done after the image has been geometrically projected. If something is found that is strange in this stage, then the geometric processing is re-done to correct the error. If the image is in good condition, then many of the intermediary files are marked for deletion. Note that the EDRs and the final products are always kept, but there are several stages to the pipeline, as readers of this blog will know.

Sometime during this process, science team members also look at the image, to find if there is anything of special interest in the image. If there is, a caption is written, and it is prepared for the next batch of images to be released. If not, the image may be released without a special caption. All images will be released within 6 months of being taken. Once everything is perfected at HiROC, this release time will be reduced greatly, perhaps even to a few weeks. Currently, though, we are still working on the process of getting the images released.

The uplink team is constantly looking where to point the camera next. There is a program which is in beta testing now called HiWeb which allows scientists and other people to input suggestions. The Uplink team reviews the suggestions in the database, assigns a priority to each of these suggestions, and then finds when we can point the camera at the part. They also make sure a certain percentage of the upcoming pictures are assigned to look for a Phoenix landing spot, as this is a high priority item at the moment. They are still learning exactly how to best command the camera, and are constantly sharpening their skills.

The downlink team is making sure operations run smoothly at HiROC. They are verifying that the processing has taken place, make sure that the images have been calibrated correctly, that there are no image processing artifacts on the images we are about to release. If there is any artifacts created from processing the image, the source of the problem is identified and fixed, and then the image is reprocessed. While previously we have sent images to the public that had some small processing artifacts during the post-MOI and Transition imaging, we currently are waiting until the images have been completely validated. The downlink team is also taking a quick look at each image that comes down, and making sure there isn’t something unexpected, for example, haze at Mars, lots of saturated pixels, etc. If any such problems are found, they notify the uplink team, to ensure that we don’t have continuing problems. These problems are very rare, but on occasion happen, due to the changing nature of Mars.

During and after the validation process, the images are reviewed by several of the science team members of HiRISE. Things of special nature are noted, and these images receive captions. The others are slated for a more general release. Due to the large size of the HiRISE images, it is almost impossible to search every square inch of the pictures by any one or even a small group of people. I’m sure many of you have noticed this with just the images which have been released, there are many more which are still being validated which have yet to be released.

The Systems team is responsible for making sure that the HiROC computers are all working in top shape. They are quick to find problems when they arise and fix them so that it does not affect the flow of data here. They are preparing servers for two upcoming services that HiROC will provide, HiWeb, which was mentioned previously, and a JPIP server, which will allow for the effective distribution of JPEG 2000 images.

The software team is writing software that will make people’s lives easier. Some are working with the HiPlan suite of tools, which is used to plan upcoming images, to make it even easier to use for the uplink team. Some are working on HiVali, the validation software, which is used to make it easier to verify that an image is ready to release to the public, quickly finding problems with the image. Some are working on HiView, a program which will allow distribution of images over the JPIP protocol to the general public. Still others are working on getting HiWeb ready for public release.

Let me also talk a bit about a few upcoming products mentioned in this entry. HiView will allow you to download only the parts of a HiRISE image that you find most interesting. It will work great, even for those who have slow internet connections. I personally have tested this with a connection rate of 1kBytes/sec, and it works reasonably well even at that slow speed. It will allow the user to save the parts of the image they find the most interesting to their hard drive for future study (HiView will require a constant internet connection to download the image)

Another upcoming product is HiWeb. HiWeb will allow any user (Yes, that’s you!) to suggest future targets to image with the HiRISE camera. Preference is given to targets of scientific interest. The suggestions are given a priority, and placed in a database to be targeted depending on the orbit of MRO and the allocated bandwidth.

So, that’s what’s happening at HiROC these days. In short, we are all very busy, but very much enjoying our work. I personally can’t remember a time that I’ve had as much fun working as these last few months have been. And surely the best is yet to come!

Hiclean does just what is says. Noise introduced into the image data by spacecraft electronics is corrected. Noise can show up as vertical and horizontal lines in the raw image and other periodic manifestations.

Hipical is a newer tool that performs calibration on the image data. For example, flatfield and gain corrections are performed by hipical. Hipical will be upgraded as we learn more about our instrument in its environment around Mars.

Hidestripe corrects a known striping pattern in HiRISE images.

We use other tools to collect even more statistical data about the newly calibrated image data. The HiCal pipeline will continue to be upgraded as our software matures. New statistics will be collected while corrections are added or improved.

After cleanup has been completed and a new *.hical.cub channel product created, HiCal creates a variety of jpeg browse and thumbnail images. The cleaned up channels are large, and for quick previews, these smaller jpegs come in handy.

Finally, HiCal lets the next pipeline – HiStitch – know that cleaned up channels are ready to be stitched together into CCD products.

Below is an example of raw data, prior to going through the HiCal pipeline. This image sample was taken from TRA_000873_1780; “Victoria Crater” at Meridiani Planum.

Below is the same image sample after going through the HiCal pipeline (notice that the bright vertical line in the center and the faint vertical lines throughout the image have been correctly removed by HiCal):

The EDR_Stats pipeline creates a *.cub file from the *.IMG file. These cube files are the file type used in ISIS 3.0, an image processing software package provided for planetary science missions by the United States Geological Survey (USGS). This package contains an entire suite of useful tools, many of which are used by our pipelines.

During the creation of a cube, a variety of statistics are gathered. For example, the number of gaps, saturated pixels, calibration pixels, and other pixels are counted. Image mean, standard deviation, and other statistics are also calculated. EDR_Stats takes these results and uploads them to our database. The resulting cube is archived in our storage directory.

The final EDR_Stats pipeline step lets the next pipeline – HiCal – know that an image channel cube file is ready for calibration processing. Let the cleanup of image data begin!

This is an example of a HiRISE image pre-calibration. Notice the vertical streaks across the image. Harder to see is something known as dark current, which appears with any camera if a black picture is taken with it. There is also variations of pixel sensitivity, all of which must be accounted for. This doesn’t include the special calibrations needed for color images, but there are some as well.

This is the same image in roughly the same spot, taken after the calibrations have been done. Note that there is no longer any vertical lines, at least, none that are obvious. That is one of the things that is corrected during calibration. There are several other things which are done as well, but they aren’t as visible, but the image as a whole has been made more scientifically useful. There is always the risk of cutting into the useful part of an image, but HiRISE has several advantages in this respect. HiRISE has a high SNR, or Signal to Noise Ratio. SNR is exactly what it sounds, the ratio of the signal (In our case the light reflected from the Martian suface) to the noise (Which can be anything from Thermal noise, Cosmic Background Radiation, etc) With HiRISE’s large mirror, and pushbroom CCD arrays, it is able to achieve what hasn’t been achievable in other cameras. What this means is that even relatively dark surfaces are still visible to a degree with HiRISE, which has not been the case with most previous imagers.

Calibration images are taken to assist in this process. They are mostly taken by pointing HiRISE at the dark side of Mars, with various temperatures, recording what image results. There are a variety of what’s known as Stim lamps to give an approximation of white light as well, which also assist with calibration. The results from these images (Which are all very small) are placed into a database to know the correct way to calibrate each image. All of these are done so that everyone can see Mars the way that it was meant to be seen.

To tell the truth, some of us are experiencing a little bit of a lull. We’re in conjunction right now — this means that the Sun is directly between Earth and Mars, so we can’t communicate with the spacecraft. (Here’s a link with a few diagrams to illustrate this.)

The HiRISE instrument is turned off, and we’re not taking any images. However, there are other activities going on at HiROC….

We start imaging again on November 8, and a few of us are already planning for that. PSP, the Primary Science Phase, is divided into two-week cycles. The first cycle is rm001, the second rm002, etc. Each cycle has a Targeting Specialist assigned to it (this one isn’t me, or I wouldn’t have time to write this!). The Targeting Specialist works with a member of the science team, the “Co-Investigator of the Pay Period,” (”pay period” because of the two-week cycle) or CIPP. The CIPP helps to choose scientifically important targets, and the Targeting Specialist does the scheduling and commanding. They work together on coordinating with other teams, choosing camera parameters, etc. There are a lot of details that need to be worked out!

We’re also building commands for a few special calibrations that will occur during the first cycle. On November 9th, we’re going to take a series of flat field images for calibration purposes. For these, the spacecraft will yaw (rotate around the normal axis) 90 degrees. Then when we take an image, it will smear across the whole field of view, giving us as close as we can get to a uniformly bright image. Once we average this over a lot of observations, we will divide our images of Mars by this as one of the calibration processing steps.

We will also be participating in a Deimos observation on November 13th. The calibration is actually set up for CRISM, and we’re just riding along, so the viewing conditions are not ideal for HiRISE. So don’t expect a fantastic HiRISE observation of Deimos! Instead, we’re taking this opportunity to measure stray light. Stray light is the extra scattered light that gets into the camera’s optics. We’re pretty confident that not much stray light gets into the excellent optics of the HiRISE camera, but we want to make sure.

Of course, the work never really stops, so we’re all busy with other things, as well — updating procedures and software, training new people, and trying to get ourselves organized and prepared. We want to be ready for the onslaught of images that will start in a few weeks and continue for (at least) the next two years!

NASA story: During Solar Conjunction, Mars Spacecraft Will Be on Autopilot