SUNDAY, 02.22.09

0827: Abort
Be advised that Mary Ann, the talent scheduled for tomorrow’s dive/film/recovery, suffered a reset abort this evening at around midnight, recovered at 0100. It gives all indications of being a software abort. As well, there are some building ground faults when she gets down to depth so they are troubleshooting that issue. It’s doubtful anything will get done about the software before she goes back in — although we’re working on that, trying to reach the software engineer (even so, he will need some time with the data).

The troubleshooting on the ground faults has led to a potential path: they think there may be a solenoid going bad. They are replacing that and going to do some testing. Current launch time is roughly around 9 a.m. I will know more when you guys get up. I’m not sure if you will want to wait until tomorrow to try for another recovery film op (which would then be around the same time as today’s planned event, 0800, Monday, Feb 23), or go back to the idea we had initially about sacrificing survey time for a little mini-mission on the surface today before the launch.

I did ask Greg Packard if a mini-mission on the surface would be helpful to the troubleshooting process and he said ‘yes, somewhat,’ but we both agree the benefit would not outweigh the loss of survey time. Our current plan is not to sacrifice any survey time and to launch for depth at 0900. Ginger is not expected up until 2200 this evening, so she’s still out of the dive loop.

So, dive ops are very likely canceled for today. Touch base with me when you get up and we’ll see how things are going. We can also discuss the plan going forward but it looks like you guys might be there for another day.

Ted Waitt Personal Log Entry
The biggest news from yesterday was adding two search boxes closer to Howland on the search grid based on our chopper flight re-creations. It’s all very challenging due to excessive geology on the bottom and a government-imposed 12-mile setback. I think we have a high-level plan, and Greg Packard and Andy Sherrell will refine it and give me a final later today. We had to scrub our dive ops today due to a vehicle abort, so it’ll be a quiet Sunday, hopefully.

0843: Day 7 (Yesterday, February 21)
Hello all, it seems these updates are now coming in to report on the day before. Sorry about that, perhaps I’ll get caught up eventually (or play on through to the other side).

Starting with the bad news, we had another vehicle abort today. After two successful missions, one subsequent to the software fix, Mary Ann had her first reset abort during this operation. Her previous difficulties had been sonar problems. It’s been Ginger that was giving us difficulties with the software aborts (which is an issue with the EdgeTech sonar communicating with the REMUS operating system, causing resets that abort the vehicle). However, it looks like Mary Ann has caught the virus. She was halfway through her survey of box 4a and suffered a reset, pretty much the same as the ones Ginger was having the other day. Greg tells me it has the same sequence of telltales we usually get when we have one of these aborts.

I am unhappy to point out it seems Mary Ann may be having something of a depth-related issue as well. Yesterday during her mission (Day 7), we were getting ground fault indications that got a bit worse as the mission progressed. While this generally will not lead to an abort, it is an indication of a problem. Greg tells me the information gleaned from the log file is very informative and should help him track it down eventually. He does not think it represents an intrusion of water, but perhaps a connector, some potting going bad or a pinched wire. While he works on the software issue he is doing double duty trying to track this down.

The good news is that John Duchesney, the Hydroid software engineer who wrote the new code, should be available today to assist with the software issues. While we are disappointed, I would say confidence remains high we will track down a fix.

Ginger was recovered last night at roughly 1950 after a successful mission. Ted and some of the crew from his ship were aboard for a picnic on the back deck, and he sat in on the first pass-through of the data. No target worthy of re-imaging presented itself and Ginger was sent back down to survey box 4b at 2315. She is currently on the bottom and working away.

Other news of the day includes the following:

The launch of Mary Ann at 1030 was done in concert with helicopter and film operations. I am told by the film team they got some great footage. They have now been on Ted’s ship for two days and spent last night there as well. We are still hoping to get some dive footage of the vehicles and, as we must dive from there due to restrictions on the RVSJ, they have just been staying aboard Ted’s ship until we get her done. Joe Lepore is over there with them and he will act as safety officer for the dive ops when they finally take place. He is also going to look at the ROV, a pre-check for some work Ted wants to do later this spring. Today, I received his report this is complete and all is looking well with the ROV.

Captain Gunther invited Ted and his crew for dinner last night on our ship. Saturday is ‘Steak Night’ on the back deck here on the RVSJ. They did join us. The film crew came along, as well as most of Ted’s crew. Captain Passmore and Joe stayed aboard Ted’s ship to cover the watches. It was a very pleasant evening and I think it was certainly good for the morale of both crews; I know it was for ours. There was a vehicle recovery right around that time so the crew was able to observe that. Afterward, I gave the kids the nickel tour. Ted was able to sit and speak with Andy about the search boxes (they are adding four in the central high probability zone closer to the island, to the east of the current grid), and he also sat and watched the first return of sonar data from the mission.

Also today we recovered two Deep Ocean Transponders (DOTs) from the first four search boxes and have deployed them upstream. We also added two more to the mix for a total of six in the water. We are at this moment surveying them in while maintenance on Mary Ann continues.

Ginger’s current schedule makes her a poor candidate for dive ops. She was recovered at 2128 and went back in at 2315 last night, a very nice turnaround time. She is expected back sometime this evening, probably around 2100-2130.

Mary Ann was scheduled to be filmed when she came to the surface later this morning, probably around 8 a.m., and plans were made for the dive teams and Ted to be on station for this. However, this will now have to be postponed. Ted made it quite clear he did not want to affect operations to accommodate this footage, not with the losses we have already suffered. We are going to do this only when the opportunity arises, which very likely means a delay of another day. There is the possibility the maintenance of Mary Ann taking place right now will keep her on the surface long enough for them to do this work, but I sincerely hope not. I want to get her working.

I just heard from Greg that while sifting through the log files of Mary Ann’s last job, he thinks he may have found the culprit for the ground fault issues — potentially a bad solenoid. He is going to test it on the bench and the fix is a quick one. The software issue will likely not get resolved in time to upload before we get everything else buttoned up, so we’ll likely launch her and take our chances. Those reset aborts are so random, it’s worth trying. As I mentioned earlier, Mary Ann has not yet had one during this operation (and I’m sure Greg and his team are looking at the possibility the software ‘fix’ caused this one; they are some pretty sharp cats). I will update again later this morning.

Also, I will begin including the daily report on our progress that Andy makes up at the end of each day. Today’s report will include the last three and then you’ll be caught up.

1146: From Ted Waitt, RE: ABORT
We should schedule interacting with Andy Sherrell, Greg Packard and Lee Frey when we have two things:

1. A fresh set of data to review; and,
2. The finalization of the two additional boxes east of the main grid.

Good discussion there. Some was captured last night, but it ties with what we saw from the chopper and the changes we made to the search area because of it.

1426: From Ian Kellett, RE: ABORT
We just chatted with Joe and outlined the plan. Our priority is to get a two-shot of Ted and the AUV. Ted can be on snorkel, and as long as we have the AUV and Ted in frame, we are in business. It seemed to us that putting in 15 feet ahead of the AUV immediately after weight launch should achieve this objective. Our current plan is to keep it as simple as possible with both Ted and Stephani Gordon on snorkel.

1603: To Dive Team Aboard Ted’s Ship, RE: ABORT
Hey gents, I just realized I had been thinking about Ginger’s recovery tonight strictly in terms of dive ops. You could do the film interview then with Greg, Lee and Andy easily. Actually, the evenings are probably better than mornings for that because it’s much closer to the changeover time. Cheers. Andy and Lee are on shift now.

Plan for the next 48 is to recover a DOT in the next 20 minutes (you can help look for it, listen to VHF 80 for the play-by-play, it’s fun!). Ginger will be up at 10ish tonight, re-deploy around 0100. Mary Ann up tomorrow morning at 0630 and re-deploy around 0930. Ginger back up around 2300 - midnight, re-deploy around 0200 or 0300. Mary Ann up Tuesday morning around 0830, re-deploy around 1130 noon and is back up Wed at 1030.

My brain literally just stopped working. Too many numbers and too few letters. We have five DOTs in the water after this recovery (A, B, C, D and A2 on boxes 5a, 5b, 6a, 6b, 7a, 7b, 8a, 8b) and have no plans to deploy more any time soon. I’ll be here for 10, then out.