Boeing 787 toast

inetdog · February 2013

Re: Boeing 787 toast

westbranch wrote: »

Makes one ask why they are connected in the first place. It's supposed to be a redundant system isn't it, if the other one failed??

No, according to the wiring diagrams supplied by Boeing on their spin control site anyway.
They are used for totally separate purposes, driving a different set of loads. There is no provision for either one to back up the other. Any interconnection which results in one battery powering the designed loads of the other would have to be the result of an unintended cross connection or relay/switching failure.

Neither battery is capable of powering the entire aircraft's electrical system, even for a short time. When the forward battery powers anything, the non-critical loads are supposed to be disconnected from the power bus. When the aft battery powers anything, it is either the APU starter or a specific lighting system for ground towing, I believe. And I think that the charging power for the aft battery comes entirely from the APU, except when external power is connected.

westbranch · February 2013

Re: Boeing 787 toast

Hmmm, lack of a redundant system(s) does not make me very safe about flying in a 787...

inetdog · February 2013

Re: Boeing 787 toast

westbranch wrote: »

Hmmm, lack of a redundant system(s) does not make me very safe about flying in a 787...

The battery power is the redundant system! Ahead of it comes two generators per engine and two generators on the APU. Any one of the six will provide enough power for a safe landing. Followed by the Ram Air Turbine (RAT).
The APU itself can be started/restarted by any one of the main engine generators, or the RAT or the APU starting battery. No need for two batteries for each use. The primary power to the DC busses fed by each battery is rectifiers from any one of the other sources. The batteries are isolated from that bus by a diode isolator to prevent reverse current and power steering relays which keep the battery disconnected from its DC bus unless all of the other sources fail.

waynefromnscanada · February 2013

Re: Boeing 787 toast

Very interesting power supply information you've provided Inetdog! Thanks a lot for that, really adds to my understanding. Can't wait for these problems to be resolved, I'd LOVE to fly on it

westbranch · February 2013

Re: Boeing 787 toast

Same comment as Wayne, thanks.

nsaspook · February 2013

Re: Boeing 787 toast

waynefromnscanada wrote: »

Good News????????????
Boeing 787 Dreamliner's failed battery was wired incorrectly, Japan says!
Japan's transport safety board said in a report that the battery for the aircraft's auxiliary power unit was incorrectly connected to the main battery that overheated.
On Wednesday Air India became the latest airline to say Boeing had told it it is hopeful of getting the Dreamliner back in service soon.
http://www.guardian.co.uk/business/2013/feb/20/boeing-dreamliner-failed-battery-wired
Flickering of the plane's tail and wing lights after it landed and the fact the main battery was switched off led the investigators to conclude there was an abnormal current traveling from the auxiliary power unit due to miswiring. http://www.komonews.com/news/boeing/Investigators-say-battery-on-Boeing-787-was-wired-incorrectly-192027601.html

I think there is a slight translation problem in the JTSB findings of a separate problem. The wiring problem is not related to the battery fire but it caused a problem when the failed battery was taken off-line and the other good battery cross-connected into the power buss. The failed battery circuit was not completely isolated from the electrical system as it should have been. The 'protective valve' translates to a diode isolation circuit.

waynefromnscanada · February 2013

Re: Boeing 787 toast

True, but it points to a possible cause for ONE of the two serious problems.

nsaspook · February 2013

Re: Boeing 787 toast

http://www.flightglobal.com/news/articles/leahy-questions-787s-heavy-reliance-on-electrical-power-382535/
http://www.reliabilitynow.net/2013/02/boeing-dreamliner-root-cause-and-asset.html

http://www.boeing.com/787-media-resource/

inetdog · February 2013

Re: Boeing 787 toast

waynefromnscanada wrote: »

Very interesting power supply information...

You can find a ridiculously large amount of information, opinion, and theories here: http://www.pprune.org/tech-log/505695-787-batteries-chargers-39.html (Close to 800 posts are running....)
There are other threads there on the subject too, but this seems to be the most active and interesting one.
The latest sub-topic is the possibility of isolator failure or other wiring system mishap leading to unintended loads on and/or unintended charging current to a battery. (Based on the ANL incident observations.)

nsaspook · February 2013

Re: Boeing 787 toast

The battery monitor is internal to each battery box (there is a in/out current monitor, individual cell voltage monitors and a internal power shutoff contactor) and would log energy to and from each battery if there was excess charging or loading from each battery due to a miss-wire or isolation failure. I'm sure the first thing they did was to pull the logs and look for abnormal trends happening before the fire. Nothing released so far has pointed in that direction. The battery ground cable being blown seems to be from a cell shorting to the battery frame (the frame is isolated from the internal cells) http://www.pprune.org/tech-log/505695-787-batteries-chargers-39.html#post7705605

787%20batteryparts.jpg

inetdog · February 2013

Re: Boeing 787 toast

nsaspook wrote: »

I'm sure the first thing they did was to pull the logs and look for abnormal trends happening before the fire.

From the discussion in the same thread, it appears that either the per-cell voltages were not logged at all or else the most recent data (at least the whole last flight) was lost when the BMU circuit boards in the battery case got fried. The released data referred to the whole-battery voltage history only. :-(
Likewise, it appears from photos and released data that there was only one temperature sensor, mounted at the top of the cell array in a location which may have had good thermal contact with at most two cells or else was deliberately isolated from the cells. I think this is the big green glob shown in the middle of the wiring harness. (Roughly 30 individual wires for 8 cells limits the number of current balancing and voltage sensing wires they could have used. Several of the wires for each cell appear to be electrically connected to roughly the same place)
It is still not clear whether the design sent BMU data over the data connector to the charger for logging, or kept it only locally in the battery case to be looked at if needed.

waynefromnscanada · February 2013

Re: Boeing 787 toast

As of 22 Feb:
" the Japanese ministry said this week it had found the ANA jet’s auxiliary power unit had been erroneously wired to the main battery that overheated."
Regarding other 787 problems:
"An oil leak was caused by an improper paint job that led to a switch not working properly, while inadequate taping led to cracks in cockpit glass, and a faulty part led to braking problems, according to the Transport Ministry’s investigation."
http://www.azcentral.com/business/consumer/free/20130222japan-identifies-some-boeing-problems.html

nsaspook · February 2013

Re: Boeing 787 toast

inetdog wrote: »

From the discussion in the same thread, it appears that either the per-cell voltages were not logged at all or else the most recent data (at least the whole last flight) was lost when the BMU circuit boards in the battery case got fried. The released data referred to the whole-battery voltage history only. :-(
Likewise, it appears from photos and released data that there was only one temperature sensor, mounted at the top of the cell array in a location which may have had good thermal contact with at most two cells or else was deliberately isolated from the cells. I think this is the big green glob shown in the middle of the wiring harness. (Roughly 30 individual wires for 8 cells limits the number of current balancing and voltage sensing wires they could have used. Several of the wires for each cell appear to be electrically connected to roughly the same place)
It is still not clear whether the design sent BMU data over the data connector to the charger for logging, or kept it only locally in the battery case to be looked at if needed.

Keeping it only on the BMU board that's inside a possible burning battery would seem to be very poor engineering when using cells with a long history of catching fire. Critical process monitors/controllers are usually physically isolated in a seperate enclosure with a bulk-head for wiring and control lines. I can see not sending bulk routine data to the recorder but you would think that some BMS data should be archived on the recorder to research a problem with a nuked battery or charger.

http://www.securaplane.com/products/power-conversion

Securaplane battery chargers store every fault including battery over-temperature, cell unbalance, defective temperature sensors, defective charger/battery connection and GMT time/date of fault period. Our chargers possess extensive diagnostics such as charger microprocessor status and permanent memory of faults with readout to the integrated 8-character alphanumeric display.

http://www.securaplane.com/download/document-links/cat_view/4-main-ship-battery-chargers
Wiring: http://www.safran-group.com/IMG/pdf/EN_mag2_29-31-2.pdf

inetdog · February 2013

Re: Boeing 787 toast

nsaspook wrote: »

Securaplane battery chargers store every fault including battery over-temperature, cell unbalance, defective temperature sensors, defective charger/battery connection and GMT time/date of fault period. Our chargers possess extensive diagnostics such as charger microprocessor status and permanent memory of faults with readout to the integrated 8-character alphanumeric display.

Storing the fault, which could be just a simple time stamped event reported by the BMS, is a lot different from storing the per-cell data which might might have triggered a fault indication from the BMS. I hope that your estimate of their engineering judgement is correct, but I still have not seen a single published voltage for an individual cell during this incident.

nsaspook · February 2013

Re: Boeing 787 toast

inetdog wrote: »

Storing the fault, which could be just a simple time stamped event reported by the BMS, is a lot different from storing the per-cell data which might might have triggered a fault indication from the BMS. I hope that your estimate of their engineering judgement is correct, but I still have not seen a single published voltage for an individual cell during this incident.

This is assuming they used standard protocols for the BMS battery communication.

I think you are correct that they may have only reported the primary 'Smart Battery Data' commands. http://sbs-forum.org/specs/sbdat110.pdf Appendix A
http://elektrotanya.com/files/forum/2013/02/sbc110.pdf

I can't find information on what chip was inside the Boeing battery but they normally use the 'optional' codes to report individual cell voltage data. Due to the nature of the battery being monitored I would hope that a similar chip was used on the Boeing system.
http://www.ti.com/lit/ug/sluu330b/sluu330b.pdf Appendix A.2 CellVoltage

inetdog · February 2013

Re: Boeing 787 toast

inetdog wrote: »

Storing the fault, which could be just a simple time stamped event reported by the BMS, is a lot different from storing the per-cell data which might might have triggered a fault indication from the BMS. I hope that your estimate of their engineering judgement is correct, but I still have not seen a single published voltage for an individual cell during this incident.

Well, according to some recent statements by Boeing called out at http://www.pprune.org/tech-log/505695-787-batteries-chargers-41.html, post #802, it appears that the existing design not only does not monitor individual cell temperature, it does not even send individual battery voltage information from the BMS to the separate charger/controller.

Boeing also said it plans to develop a new battery design that will measure
the temperature and any voltage changes in individual cells.

Coupling that with the early description, which I had discounted as inaccurate but now am willing to believe, about the battery charger, and I think we have a real engineering CF.

It stated something to the effect that by looking at the cell voltage over time the charger would predict the time an individual cell (or maybe the whole battery???) would reach End of Charge instead of closely monitoring the cell voltages for the subtle EOC signature.

PS: Meanwhile,

The interim fix includes a heavy-duty titanium or steel containment box around the battery cells, and high-pressure evacuation tubes that, in the event of a battery fire, would vent any gases directly to the outside of the jet.

solar_dave · February 2013

Re: Boeing 787 toast

Sounds like they should talk to GM about battery management.

inetdog · February 2013

Re: Boeing 787 toast

solar_dave wrote: »

Sounds like they should talk to GM about battery management.

Elon Monk of Tesla also offered his help. :-)

westbranch · February 2013

Re: Boeing 787 toast

Or the Concorde people? they had a pretty good (battery) record I believe, other than random parts falling off...oooops!

inetdog · February 2013

Re: Boeing 787 toast

westbranch wrote: »

...other than random parts falling off...oooops!

It's biggest and saddest problem resulted from running over a random part which fell off a different type of plane.

nsaspook · March 2013

Re: Boeing 787 toast

NTSB Interim Factual Report
http://www.ntsb.gov/investigations/2013/boeing_787/interim_report_B787_3-7-13.pdf

When smoke is detected, the avionics cooling function is designed to exhaust smoke overboard through fans in the cooling ducts and changing supply valve positions (and the use of differential pressure if the airplane is in flight). During this incident, the supply valves (which are electrically driven) lost electrical power after the APU shut down because the APU was the only source of electrical power being used at the time.

westbranch · March 2013

Re: Boeing 787 toast

inetdog wrote: »

The battery power is the redundant system! Ahead of it comes two generators per engine and two generators on the APU. Any one of the six will provide enough power for a safe landing. Followed by the Ram Air Turbine (RAT).
The APU itself can be started/restarted by any one of the main engine generators, or the RAT or the APU starting battery. No need for two batteries for each use. The primary power to the DC busses fed by each battery is rectifiers from any one of the other sources. The batteries are isolated from that bus by a diode isolator to prevent reverse current and power steering relays which keep the battery disconnected from its DC bus unless all of the other sources fail.

When smoke is detected, the avionics cooling function is designed to exhaust smoke overboard through fans in the cooling ducts and changing supply valve positions (and the use of differential pressure if the airplane is in flight). During this incident, the supply valves (which are electrically driven) lost electrical power after the APU shut down because the APU was the only source of electrical power being used at the time.

Hmm doesn't appear to have worked as designed if power to the exhaust fans got cut...???

inetdog · March 2013

Re: Boeing 787 toast

westbranch wrote: »

Hmm doesn't appear to have worked as designed if power to the exhaust fans got cut...???

From the Interim Report:

The APU controller (discussed in section 1.6.5) monitors the parameters that are needed to operate the APU. The APU controller is powered by the APU battery bus, which receives its power from the APU battery. If the APU battery fails, then the APU battery bus will no longer receive power, and the APU will shut down.

The ventilation fans in question are probably powered by multiple sources, including the total of 4 engine driven generators, and during flight the fans would therefore likely work OK.
The problem here is definitely that when the battery is both the only power source for the fans and the battery is also the source of the smoke (or perhaps there is NO battery power for the fans) you have an unsurmountable logical problem.
It is much less likely that the ventilation would have failed had the plane been in flight, especially as there would, at least in the new design, have been some ventilation from cabin pressurization and low pressure on the outside of the flame thrower -- I mean vent duct.

This seems even worse design to me though:

Firefighters reported that removing the battery was difficult because a metal kick shield installed in front of the battery prevented them from accessing the battery’s quarter-turn quick disconnect knob. Also, the quick disconnect knob could not be turned because it was charred and melted.

!!!:

The stainless steel sleeve and signal wires had damage consistent with excessive electrical current where they attached to the connectors at each end; at the battery case, the damage was also consistent with fire.

Excessive current on the signaling wires to the battery charger? Maybe just ground current in the signal grounds and the shield because of internal short circuit inside the battery case?

Only two temperature sensors in the box (thermistors), possibly one for each of two BMU modules? The 8 wires from each BMU were described as voltage sensing wires, but may have doubled as balancing current conductors. If the latter, how did they coordinate balancing between the two BMUs?

FDR data showed that, at 1021:01, a 1-volt decrease from the designed voltage of the APU battery (32 volts) was recorded. Three seconds later, the data showed a change
in current flow to 44 to 45 amperes into the battery. The battery voltage continued to decrease, and, at 1021:08, the current flow returned to 3 amperes out of the battery. At 1021:30, the battery voltage decreased to 28 volts, and the APU shut down 7 seconds later.30 Table 2 shows selected events recorded before and after the APU shutdown. The FDR did not record any data indicating that the APU battery voltage had exceeded 32 volts.

Not that it necessarily contributed to the event, but the initial sudden 1 volt anomaly did not cause the charger to shut down! This continued for 8 minutes. The steady decrease in battery voltage while charging also was not seen as significant!! The charge rate indicates that for whatever reason (probably the routine APU startup 15 minutes earlier) the battery was still substantially discharged at the time of the event. It was also being rapidly recharged after the substantial load of the APU start. No temperature information seems to have been recorded (?)

I think that the current flow reported is from the hall effect sensor, positioned like a shunt, in the battery itself and so represents net charging current to the battery independent of what the charge controller may have been delivering to the bus at the same time.

Aft EAFR stopped recording. Forward EAFR continued recording for about 9 minutes
58 seconds.

When the battery went offline and the APU spun down, there was no longer a source of power to the rear EAFR (Flight recorder).

OOPS:

The BMU main circuit card and sub-circuit card do not contain nonvolatile memory (NVM), and none of the BMU data are recorded on the FDR.

And, the last of my stream of consciousness commentary for now:

The battery shall be designed to prevent spilling flammable fluid, a hazardous event with occurrence with a probability of less than 10^-9.

Much has been made in the press with the fact that two battery events have occurred within the first 50,000 flight hours that should have had a probability of 1 in 10,000,000 hours or less. If you think that the evidence of either of the two events supports that flaming electrolyte was in fact discharged, we have now hit the 1 in 1,000,000,000 condition too!

nsaspook · March 2013

Re: Boeing 787 toast

The BMU cards were completely fried but they had no local NVM storage so history data was not lost and none of the data from them is archived to the FDR by way of the BCU so there is a huge gaping hole of missing data that might provide some answer to had happened inside the battery before it caused the APU to start logging errors.There is a critical need for an battery electrical systems data recorder for this airplane because these batteries are a ticking time-bomb without a root cause solution.

Detailed data report: "Airworthiness Factual Report of Group Chairman"
http://dms.ntsb.gov/pubdms/search/document.cfm?docID=388319&docketID=54251&mkey=85973

http://dms.ntsb.gov/pubdms/search/hitlist.cfm?docketID=54251&CFID=344434&CFTOKEN=65836259

Rolls Royce can download and monitor a complete flight profile from it's engines but we don't have local storage for simple battery data?
http://innovationnow.raeng.org.uk/innovations/default.aspx?item=15

The Rolls-Royce Engine Health Monitoring Unit is an extremely complex software set that takes signals from dozens of sensors around the engine and transmits the data via satellite while the aircraft is in flight. The company’s service engineers are therefore alerted to potential issues early, in advance of them causing an operational problem to the airline. If repairs are necessary, the company can have a field team standing by on the ground by the time the plane lands. In this way Rolls-Royce is saving the customer down time, minimizing disruption, keeping the engine in service, and keeping the passengers flying.

waynefromnscanada · March 2013

Re: Boeing 787 toast

Very interesting report.
Dreamliner's Battery Woes A Deja Vu Moment For Aviation Industry

http://www.npr.org/player/v2/mediaPlayer.html?action=1&t=1&islist=false&id=173559204&m=173559183

inetdog · March 2013

Re: Boeing 787 toast

waynefromnscanada wrote: »

Very interesting report.
Dreamliner's Battery Woes A Deja Vu Moment For Aviation Industry

http://www.npr.org/player/v2/mediaPlayer.html?action=1&t=1&islist=false&id=173559204&m=173559183

Very much to the point. There were properties of then current NiCd battery design that were not well enough known in terms of use in an aircraft environment. Boeing tried to jump the learning curve gap with limited experiments and lots of statistical info about LiCo under different conditions than those in the plane. They missed their landing.

I wonder is someone on this forum is going to propose NiFe as the solution. :-)

Cariboocoot · March 2013

Re: Boeing 787 toast

inetdog wrote: »

I wonder is someone on this forum is going to propose NiFe as the solution. :-)

Sure!
Of course then the passengers will be limited to carry-on luggage alone in order to have enough room for the battery bank.

From now on I will only fly on a dependable aircraft like a Ford Tri-Motor.

nsaspook · March 2013

Re: Boeing 787 toast

The FAA has given the OK for Boeing to prove their (fix?) will work.

http://www.faa.gov/news/press_releases/news_story.cfm?newsId=14394

“This comprehensive series of tests will show us whether the proposed battery improvements will work as designed,” said Transportation Secretary Ray LaHood. “We won’t allow the plane to return to service unless we’re satisfied that the new design ensures the safety of the aircraft and its passengers.”

The battery system improvements include a redesign of the internal battery components to minimize initiation of a short circuit within the battery, better insulation of the cells and the addition of a new containment and venting system.
...
The FAA will approve the redesign only if the company successfully completes all required tests and analysis to demonstrate the new design complies with FAA requirements. The FAA’s January 16, 2013 airworthiness directive, which required operators to temporarily cease 787 operations, is still in effect, and the FAA is continuing its comprehensive review of the 787 design, production and manufacturing process.

waynefromnscanada · March 2013

Re: Boeing 787 toast

nsaspook wrote: »

The FAA has given the OK for Boeing to prove their (fix?) will work.

Great news! Hopefully all works out well. This plane needs to get back in the air.

inetdog · March 2013

Re: Boeing 787 toast

waynefromnscanada wrote: »

Great news! Hopefully all works out well. This plane needs to get back in the air.

Great news for Boeing and the economy of the US Pacific Northwest, etc.
But really sad news for good engineering principles and maybe not even such good news for Boeing.

If the proposed fix does everything that Boeing claims it will do, it will allow a 737 to proceed on to its destination (or an alternate) with a smoldering battery safely contained in its SuperBox (TM).
It will do absolutely nothing to prevent the battery failures (of still unknown origin) that caused both fires.
And it may not allow the 787 to be used for over-water flights, since the safely smoldering battery will not be able to provide the backup battery functions that it was designed for. In particular with the APU battery defunct, the APU cannot be used. And with the main battery defunct, there is no backup for some of the controls and the wheel brakes should the engines fail.

I think the most disturbing thing so far, but probably predictable, was that when the NTSB x-rayed the working and believed normal Main battery of the Boston JAL plane, they found that some individual cells had distorted wrinkled and creased electrodes, bulging walls, and other abnormalities. The implication is that the Main battery was headed toward a failure similar to the APU battery but had not gotten there yet. But that battery was still showing normal according to the on-board monitoring.

Either the manufacturer had a serious QC problem or else there is something in the 787 environment which was not expected when the battery was designed in.

What is also not clear is whether the current tests will involve only the SuperBox, or whether they will also involve some significant changes to the battery which will probably take longer to complete.

PS: Still unclear is whether the redesign will require Boeing to meet the current Li-Ion battery safety criteria, which the 787 was grandfathered out of and which it would not be expected to pass.

Boeing 787 toast

Comments

Categories