Waitt Institute

The Waitt Institute implements innovative technologies in the field through collaborations with world-renowned scientific institutions, synthesizing global expertise and accelerating groundbreaking research. More »

Replacement Parts

Details on how Ginger was damaged, the difficulty of securing parts and the logistics to get them delivered to the remote search grid…

TUESDAY, 02.24.09

0422: To Mike Purcell, RE: Part Replacements
I personally skiffed the parts requested by the other company over to Ted Waitt yesterday, and they will be following a reverse path to you if things work out.

As I think I mentioned yesterday (it’s been hectic), it would be best to hand-carry those parts to San Diego I think. We could overnight them, but I think given the time crunch Ted has us under, that would be unwise. We could courier them, but that would be more expensive than simply putting them in someone’s hands ASAP and getting them on a plane for San Diego. Do you think one of your people could fill that role? Please keep me closely posted on our progress today. I am online now and will continue to be in order to expedite anything necessary until the day is done. Thanks, shipmate.

0457: To Steve Nielsen (Ted Waitt’s Pilot), RE: Parts
Hey man, here’s what I got now. The guy at Woods Hole (WHOI)is going to finalize the deal, which is looking pretty good although not yet confirmed. When they get it, I plan to have them run that and as many other spares as we can get together to San Diego to you guys. Then you haul to the jet in Honolulu and then Tarawa. Do we have someone on the ground in Honolulu? If not, I’ve got a guy on Molokai who I can have run over there.

We also need a couple gallons of denatured alcohol, which the courier from the East Coast will not be able to bring with him. If we do not have a guy to get that together on the ground in Honolulu, I will get my guy over there to have it when you arrive and hand it off.

Ted Waitt is en route Tarawa and will hopefully be there when you arrive. ETA is Wednesday morning I think, checking that now. We need to do everything as fast as we can. I would be checking into getting over to Honolulu tonight or first thing Wednesday morning. More to follow.

0737: From Mike Purcell RE: G’Morning
Dessner, I think I have the first two approvals to get the thruster housing and awaiting the third. I need to make one part to replace the foam. I will work on courier. If I get approval today, it may be on the way to San Diego tomorrow.

Mike Purcell, Director of the AUV Operations Group at Woods Hole Oceanographic Institute, was invaluable in his support of the mission working logistics from his lab at WHOI.

Woods Hole (WHOI) Team Coordinator Mike Purcell provides invaluable support.

0739: Response To Mike Purcell
Dude, nice. There’s no way we can get it on a plane today? Also, can’t we just get someone from Woods Hole to fly it over? Don’t you want to go spend the night in a nice hotel in La Jolla? LOL. Cool, keep me posted. Thanks man, and nice work

0947: Update
Roughly 32 hours ago, at 2030 local, Ginger was leaving the bottom after successfully completing a survey of box 4b. She reached the surface at 2116 and the ship lined up for an approach. The day shift was nearing the end of their watch, Mark Dennett was the watch leader and Steve Murphy was the deck boss. Captain George Gunther was driving the vessel, which was in a ‘limited machinery state.’ In essence, this means that the ship had only two out of three generators running, which allows the use of only one thruster at any given time, stern or bow, and that thruster will not be at full capability.

There was only one main engine online and that was the starboard, limiting us to the use of that screw. It was decided to use only thrusters during recoveries with the understanding by all that in winds or seas of any significance the engine might be required to assist the ship staying on station. I have been told there was also an agreement not to use the starboard screw during recoveries (although I cannot personally confirm this). The ship’s policy is a minimum of 15 minutes to bring a generator online and an hour to bring a main engine online.

As Ginger came amidships, the order was given by Steve to release the recovery float and this command was remotely sent to the vehicle, but the float failed to deploy. This is usually caused by the float recovery line being too loosely packed into the box below the float, putting pressure on the float release and keeping it from actuating. In the past when this has happened, we have sidled the ship up next to the AUV and tapped the float with a boat hook; it usually pops right off, the boat hook is used to grab the float recovery line and it is then attached to the LARS recovery line.

The order is given from the deck to the pilot to make way to stream the vehicle behind the ship for retrieval into the LARS. In the past when the float deployment failed, the entire process had a tendency to work its way down the side of the ship. The air-powered grappling cannon located just aft of the ship’s side derrick crane is the starting line for getting hooked up to the AUV for recovery. The ship always has some forward motion when the float is released and after the grapple is fired over the line, the entire process normally walks aft down the starboard rail.

When working down the starboard side of the ship during a recovery, quarters can be tight.  There is only room for one person between the RVSJ A-frame and the rail.

There is room for only one person between the RVSJ A-frame and the starboard rail during an AUV recovery.

When the float failed to deploy, the recovery went about as expected: the ship slid over next to the vehicle and the crew tapped the float with a boat hook, and the float released. The LARS recovery line was attached to the float line and the radio call was given that the vehicle was attached to the boat and there was some separation between them. Then a call went out that the distance was closing; the wind was pushing the ship in the direction of the vehicle. What happened next is still being debated, but one thing is certain: the starboard screw was engaged, the ship crabbed to starboard and the vehicle was sucked under the ship (or the ship passed over it), was struck by a propeller and then squirted back out from under the stern. The ship then gained way and the vehicle was recovered.

The damage to the vehicle is extreme, but not irreparable. The RVSJ propeller made one strike to the vehicle just forward of its propeller and severely damaged the thruster unit assembly housing, which is part of the overall tail assembly on the aft section of the vehicle (fin control box and motor controller being the other main components of the tail assembly). When the vehicle was back on deck and still in the LARS, it was immediately determined the oil-compensated housing of the thruster unit assembly was hemorrhaging oil. The team attached absorbent pads on the wound with electrical tape and tried to assess the damage. Within 20 minutes, the thruster assembly had been removed and the damage inventoried.

This was Greg Packard’s report to Mike Purcell at WHOI regarding the damage: “During recovery, the tail of the vehicle went under the hull and made contact with the turning ship’s screw. The hit was obvious in the movement of the vehicle and the noise associated with it. Recovery saw the top rudder fin bent over, the tail flotation cone gone, a prop slice in the thruster housing and damage to the prop and spinner retainer. Oil was bleeding from the motor housing. The dissection began. The motor housing was sliced through. Shaft was taken out of true. Internal motor retainer plate bent and popped some of its retaining screws. Rudder fin assembly jammed all the way back to the motor block assembly. Fin control housing appears okay. End plate and motor controller housing appears okay (still digging in that far to check the electronics). We removed the main flotation quadrants from the frame to inspect the strong back for cracks. The frame’s integrity is okay.”

Right as this was happening, the science team was going over the results from a codend tow that they had made just before the recovery. They were  very thrilled and fortunate to find, alive in their net, a small angler fish. I have enclosed a picture of that here. It is just one of the craziest things I have ever seen. Knowing almost immediately that we had just suffered a major hit to the expedition, I contacted Ted Waitt’s ship to get the photographers, who were aboard his ship, over to the RVSJ. I thought to try and salvage anything we could from the day. Footage of the incident aftermath and the little fish — that would almost certainly die within the hour — would be about all we could hope to achieve that would have some benefit to our program. This was done. Ted may have initially been considering joining them, but the sea state made their load up onto an RVSJ skiff very dicey. Plus, I was told Ted decided his presence looking over the shoulders of a team that was already freaked might be counterproductive. The film team arrived just as we feared the fish had expired, they roused it with a pair of chopsticks to a state of activity that I am told made some excellent footage. Small consolation.

Mary Ann was recovered that morning at 0719. Her mission was successful and she brought in data from the last half of 4a and the first half of 5a.

The team then met in the dry lab to further assess the damage, and it was decided the housing was beyond repair. There was very likely damage to some of the internal components of the thruster assembly and although some of them may still be usable, they’re being assessed to see if they remain within their tolerances. Immediate action was taken to contact personnel at Woods Hole and Hydroid in order to see if it would be possible to obtain a thruster assembly from one of the other AUV systems around the world. We also began making preparations to obtain the components necessary to repair the vehicle, primarily a new housing, which would have to be machined out of a 12″ x 9″ block of titanium.

The possibility of constructing this out of stainless steel was explored.  Titanium has a lead time, but more importantly it would take four days to machine the housing of titanium due to the slow speed of the lathe on such a hard material. Stainless steel can be substituted as the housing is an oil-compensated space, and we can adjust for the weight difference. Harbor Branch did not have sufficient stainless and volunteered to make one out of aluminum which was rejected by the Woods Hole tech on board. I’m still looking into having a stainless housing turned up in Samoa,  and Woods Hole is already turning one up that we will have access to if the replacement falls through (if the loan goes through, it goes to the construction of a replacement for them).

Mary Ann was then launched at 1119 to survey 5b. As the day progressed, there was progress made on the replacement front. Mike Purcell informed us the possibility of a thruster assembly loan was better than 50/50, although arrangements would have to be made to provide enough spare parts from our inventory for their replacement to be constructed at Hydroid. Later in the day, we gave our assurance that our operations would be concluded at the end of May so we would have the capability to provide them with their original unit should new construction prove to be an issue.

Around the same time the initial meetings about the replacement were taking place, Ted Waitt came aboard the RVSJ to see the damage, take a look at photos of the angler fish and the data that had come up with Ginger (which was in good condition despite the incident). He then re-boarded his ship and began making way toward Tarawa so he could take delivery of the part as soon as possible and run it back out to us (unhappily reminiscent of the situation in ‘06 when he also ran a part that been delivered by his jet to Tarawa).

As Ted was transiting to Tarawa, we were informed by Mike Purcell of a requirement for the replacement, that we provide parts for the construction of a new replacement thruster assembly and the cutoff date for that delivery. The ship turned back toward us to take possession of those parts. Ted then informed me of his plan to leave the region before the dates outlined and the boat turned back toward Tarawa. Further discussion aboard the RVSJ indicated it would be best for us to get those parts to Ted now for delivery, that this would make their acquiescence to the arrangement much more likely. Rather than turn Ted around (again) we ensured that Mary Ann was nearly at depth, brought our tower up and made best speed (after a half hour warming up the other main) to Ted’s ship. When we were in visual range, we launched our skiff and a deckhand from the RVSJ ran me six or seven miles to close with the ship and hand off the parts to Ted. We returned to RVSJ, headed back to our grounds, arrived at 1845 and successfully re-established contact with Mary Ann — still below on survey.

Since then, we have conducted another codend tow, concluding at 0330 with minor success — a couple critters. Scheduled recovery of Mary Ann is at 0930.

Some observations of what went wrong:

There is culpability on both the ship and the AUV team leaders, including me. On our end, it’s clear our procedures did not adequately address a failed float deployment. We failed to recognize we should have developed a separate procedure for this event instead of going to an immediate action scenario where we simply do whatever it takes to get that float to pop.

[It's hard to plan for everything. As an example: we had a grappling line snarl on the vehicle the other day that made it impossible to properly attach the LARS recovery line to the float recovery line in a way that would allow us retrieve the vehicle without fouling the prop. The grappling line was draped across the top of the vehicle and tangled in the GPS antennae. We used a fishing pole and line to snag the float recovery line and it worked out. A quick fix.]

Backing down the float

This is what's supposed to happen when the vehicle receives the command to release its float.

Normally, the recovery float is popped when the vehicle comes along side amidships. Then, we back down the AUV to get separation to fire the grappling hook across the vehicle float line and the ship’s momentum takes the vehicle past us as we connect to the LARS line. This way, everything streams out past the stern and we get back underway, stream the vehicle behind the boat and recover. When the float has not deployed in the past, we have simply worked down the side of the boat as we crabbed over next to the vehicle and poked at it with boat hooks to get the float to pop, all the while walking down the starboard rail. This clearly was an exceptionally poor practice.

The new procedure will be to pass by the vehicle, in the event of a failed float deployment, and bring the boat around for another pass. We need to maneuver the boat so the vehicle comes alongside amidships — where there is an open working space under the derrick crane used for CTD (Connectivity, Temperature and Depth) casts — pop the float and hook up the LARS recovery line to the float recovery line. Then we crab the ship away from the vehicle and, when there is sufficient distance, make way and stream it behind the vessel for recovery. All this work will now take place well forward of the gun and thus the ships screws. Hindsight.

I do feel the ship shares in our responsibility, even though this reflects on us and our failure to realize how one situation has the potential to negatively impact another. The ship is always trying to conserve fuel and this is the reason for working in a limited machinery state at times. We have been frustrated by this, sometimes wanting to run fast to get a Deep Ocean Transponder (DOT) deployment or recovery and then get back into our primary box to re-establish communication with a vehicle. We have been working with the ship’s crew, trying to develop communication protocols that give them enough notice to bring equipment up to readiness for our efforts and we’ve been experiencing reluctance at times.

The fix is simple. I have left instructions with the deck crews that if all three generators and both engines are not online and ready for full use, we will not recover a vehicle. Rather we will wait until the equipment is at full readiness. This is nowhere near as important during deployments. I am speaking with the captain this morning about this, and I will ensure they get as much notice as they need to meet this requirement. I am also informing all involved that should they fail to meet these needs, we will lose time and this will certainly lead to a reprimand. I will be in the front of the line of those receiving that reprimand, so rest assured that I will do all in my power to make sure this happens even if I have to personally go to the engineers an hour before every recovery and watch them turn this equipment on.

I have asked Lee Frey for any potential impacts to our fuel consumption on site and our ability to remain on station. I feel most strongly that we need full capability when recovering our vehicles and I am surprised that the ship did not feel that way as well. As well, the use of the starboard screw makes no sense whatsoever. I have been told the agreement was made to never use the starboard screw during a recovery when the vehicle always runs down the starboard side of the ship.

We are now waiting to hear from Mike Purcell on the decision to loan us the part, along with the final terms of the loan. If this falls apart, we will take possession of the housing being turned up at Woods Hole ASAP, and reconstruct the unit assembly when it arrives out here. I am still trying to see if Samoa has the stock and capability to machine this part, and I am also looking into having it run out on a charter from Pago. I spoke to the shipyard owner yesterday on both questions and he was checking on the stock and boat. We will speak again in two hours. I expect to hear from Purcell any minute.

Ted is steaming toward Tarawa with an ETA of Wednesday morning, according to Captain Passmore. If the part becomes available, I hope to have it in our hands by noonish EST. My recommendation (and what I have been planning toward) is to have it hand carried to San Diego and placed in the hands of the pilots who will fly it to Honolulu. They will then fly Ted’s plane to Tarawa and hand it over to him. Best case scenario in my mind is to have Ginger back in the water by Saturday.

Latest: Just heard from Purcell and he thinks tomorrow is the most realistic time to receive and send on the parts. After all the terms are lined out, he needs one last approval and then the time to fabricate some foam to adjust for the slight discrepancy between their configuration and ours. I think that slides the estimates above back by a day. If this fails, I think we lose two more days while the titanium housing is machined. So, back in the water Sunday if we get the agreement in place, Wednesday (best case) if not.

That’s it for now

Image Gallery