Standard engineering practice or junk science? Part 3
By Richard Schulte
Schulte & Associates, Evanston, Ill.
The first article in this series discussed the fires that destroyed the McFrugal's Warehouse in New Orleans on March 21, 1996, and Dr. Craig Beyler's use of the Large Eddy Simulation (LES) fire model in expert testimony in the subsequent litigation. The second article discussed Hughes Associates, Inc.'s (HAI) use of the Fire Dynamics Simulator (FDS) in a research study of the concept of the use of "ganged" smoke/heat vent operation in buildings protected by standard spray sprinklers. This article will continue the discussion of HAI's study of the concept of "ganged" roof vent operation, but first, a brief discussion on the general use of fire modeling in building design (and also litigation) is in order for background purposes.
An article addressing the use of computer fire models, titled "Reliability of Computer Fire Models in Fire Safety Design," written by Dr. Alan N. Beard of the Civil Engineering Section of the School of the Built Environment at Heriot-Watt University located in Edinburgh, Scotland, appeared in the April 2008 issue of Industrial Fire Journal. The following are a few excerpts from Dr. Beard's article:
"This concern covers all kinds of models, including computational fluid dynamics [CFD] models. Concerns centre around the degree to which such models may or may not have the potential to represent the real world reasonably accurately and the ways in which such models may be used and results interpreted."
"It is crucial, therefore, to conduct a priori comparisons with well instrumented experimental tests, but very few indeed have ever been performed."
"A similar 'round-robin' a priori study has just been carried out by Edinburgh University in collaboration with Strathclyde Fire Brigade, centred on the Dalmarnock fire tests. The results were presented at a meeting in Edinburgh in November 2007. In these tests, a fire was started on a sofa in a two-bedroom flat in Dalmarnock, Glasgow. ... The big question was, as with the CIB study, how would the predictions by model users compare with each other and with experimental results? Ten model user teams took part, eight using the same CFD model and two using a zone model. ... As a general rule, the predictions were not at all good; there was generally a wide scatter amongst the predictions by users and, also, predictions usually compared poorly with experimental results."
"The basic message was clear: a predicted result from a model cannot be assumed to be accurate; i.e. to reflect the real world. Further, consistency cannot be assumed; i.e. that a given model will consistently over-predict or consistently under-predict."
"Whether or not a model may be reliably used as part of fire safety decision-making depends not only upon the conceptual and numerical assumptions in the model itself but also upon how it is used and how the results are interpreted. Using models as part of decision-making may be dangerous."
"A 'knowledgeable user' must be capable of using an acceptable methodology to apply a particular model to a particular case in a comprehensive and exhaustive way, making all assumptions and procedures explicit, and interpreting results in a justifiable way."
Dr. Beard's statements excerpted above are reinforced by a draft document titled, "Fire Dynamics Simulator (Version 5) Verification & Validation Guide (Volume 1: Verification)" dated May 30, 2007. Excerpts from this document include the following:
“This guide is based in part on the "Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models," ASTM E 1355 [1]. ASTM E 1355 defines model evaluation as 'the process of quantifying the accuracy of chosen results from a model when applied for a specific use.' The model evaluation process consists of two main components: verification and validation. Verification is a process to check the correctness of the solution of the governing equations. Verification does not imply that the governing equations are appropriate, only that the equations are being solved correctly. Validation is a process to determine the appropriateness of the governing equations as a mathematical model of the physical phenomena of interest. Typically, validation involves comparing model results with experimental measurement. Differences that cannot be explained in terms of numerical errors in the model or uncertainty in the measurements are attributed to the assumptions and simplifications of the physical model."
"Evaluation is critical to establishing both the acceptable uses and limitations of a model. Throughout its development, FDS has undergone various forms of evaluation, both at NIST and beyond. This guide provides a survey of work conducted to date to evaluate FDS."
Dr. Beard's statements are even further reinforced by the "Fire Dynamics Simulator (Version 5) Technical Reference Guide" (NIST Special Publication 1018-5) dated October 1, 2007. Excerpts from this document include the following:
"Sufficient evaluation of any model is necessary to ensure that users can judge the adequacy of its technical basis, appropriateness of its use, and confidence level of its predictions. This document provides the theoretical basis for the Fire Dynamics Simulator (FDS), following the general framework set forth in the "Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models," ASTM E 1355 [3]."
"The U.S. Department of Commerce makes no warranty, expressed or implied, to users of the Fire Dynamics Simulator (FDS) and accepts no responsibility for its use. Users of FDS assume sole responsibility under Federal law for determining the appropriateness of its use in any particular application, for any conclusions drawn from the results of its use and for any actions taken or not taken as a result of analysis performed using these tools."
"The major assumptions of the model, for example, the large eddy simulation technique and the mixture fraction combustion model, have undergone a roughly 40-year development and are now documented in popular introductory textbooks. More specific sub-models, like the sprinkler spray routine or the various pyrolysis models, have yet to be developed to this extent. As a consequence, all documents produced by NIST staff are required to go through an internal editorial review and approval process. This process is designed to ensure compliance with the technical requirements, policy and editorial quality required by NIST. The technical review includes a critical evaluation of the technical content and methodology, statistical treatment of data, uncertainty analysis, use of appropriate reference data and units, and bibliographic references."
"Any user of the numerical model [FDS] must be aware of the assumptions and approximations being employed. There are two issues for any potential user to consider before embarking on calculations. First, for both real and simulated fires, the growth of the fire is very sensitive to the thermal properties (conductivity, specific heat, density, burning rate, etc.) of the surrounding materials. Second, even if all the material properties are known, the physical phenomena of interest may not be simulated, due to limitations in the model algorithms or numerical grid. Except for those few materials that have been studied to date at NIST, the user must supply the thermal properties of the materials, and then validate the performance of the model with experiments to ensure that the model has the necessary physics included. Only then can the model be expected to predict the outcome of fire scenarios that are similar to those that have actually been tested."
An example of how the Building and Fire Research Laboratory (BFRL) of the National Institute of Standards and Technology (NIST) utilizes experiments, in combination with the FDS, in order to verify the accuracy of the model's predictions is illustrated in the following excerpts from the report of NIST's investigation of the fire at the Station nightclub. (This fire occurred on February 20, 2003, in West Warwick, Rhode Island.)
"Predicting sprinkler activation and suppression and the influence of fire fighting activities on the spread of the fire is another aspect of the problem that can be done today [June 2005] only at the grossest level of precision."
"Computer simulation has been demonstrated to be credible, when properly applied, as a tool to help fill in critical details of a fire incident and to demonstrate the value of alternative building designs and fire safety measures [1-14]."
"The comparison of the sprinkler activation times from the sprinklered mockup experiment and the FDS [Version 4] simulation of that experiment are given in Table 5-4. In FDS, the activation time of the first sprinkler was the result of adjusting the RTI in the simulation until the times were similar. The RTI that provided the best match, 16 m1/2 s1/2 (32.6 ft1/2 s1/2), was used as the RTI for the remaining sprinklers in both the mockup and the full nightclub simulation. The order of sprinkler activation and the number of sprinklers activated were the same in the simulation and the experiment. The times to activation differed by no more than 6 seconds."
"Both the experiment and the computer model demonstrate that the sprinklers would prevent flashover and considerably mitigate the hazard from the fire. However, the degree to which the fire is controlled is different between the experiment and the model, since the simulation has more flame spread along the edges of the alcove ceiling after activation of the sprinklers."
"The temperature, heat flux, and oxygen volume fraction comparisons show reasonable agreement between the experiments and the model in terms of both trends and range. Again, some differences were caused by the increased burning after the start of suppression in the shielded areas, but that phenomenon has been documented in other experiments as well [22] . Tenability limits were predicted never to be exceeded using FDS, consistent with what was observed in the full-scale mockup experiments. The lower portion of Table 5.3 compares the extreme values of heat flux, temperature and oxygen volume fraction predicted in the simulation to the measured values."
"The full-scale mockup results were used to compare against the FDS simulations to validate the implementation of the data in the model and to determine the model's capabilities for this fire incident. Further, the sprinklered mockup results were used to develop a means to model the sprinkler in the full nightclub simulation."
"While the ability of FDS to predict fire suppression is simplified and cannot capture all of the physics involved in the process, FDS is able to predict the trends in reasonable agreement with the measured temperatures, heat fluxes and oxygen volume fractions."
With the above background, a discussion of the HAI study on the concept of "ganged" operation of roof vents in sprinklered buildings is now in order. The following are excerpts from the HAI's report titled, "Analysis of the Performance of Ganged Operation of Smoke and Heat Vents with Sprinklers and Draft Curtains" dated February 18, 2008:
"The gang operation concept involves opening all the [smoke/heat] vents within the coverage area of the sprinkler system in which the fire originates one minute after the first sprinkler has operated." (page 11)
"A numerical study was undertaken to explore the performance of gang operated smoke and heat vents, working with and without draft curtains, in an 7,430 m2 (80,000 sq. ft.) sprinklered warehouse." (page 11)
"Comparison of sprinkler operations between vented and unvented cases clearly shows that the operation of sprinklers was not affected by smoke and heat vents or by smoke and heat vents with draft curtains. The time to first sprinkler operation, the number of sprinkler operations and the pattern of operation were not impacted by the venting system." (page 11)
"The operation of the smoke and heat vent system had no effect on the operation of sprinklers and as such maintained the operational effectiveness of the sprinkler system while improving the conditions within the building in support of fire department operations." (page 12)
"The Fire Dynamics Simulator, version 4 (FDS4), was used to perform the field calculations. FDS4 is a three-dimensional large eddy simulation CFD program developed at the National Institute of Standards and Technology's (NIST's) Building and Fire Research Laboratory (BFRL) [McGrattan & Forney, 2004; McGrattan, 2005]." (page 14)
"The main concern of this investigation is the activation of sprinklers. Sprinkler activation typically occurs after the ceiling jet engulfs a sprinkler head[sprinkler] ... Hence, based on the cooling effects of the sprinkler sprays and the spacing of the heads [sprinklers] , it is not expected that radiative heat transfer will be a contributing factor in sprinkler activation." (page 26)
"For a variety of reasons, comparisons with the commodity tests [in McGrattan, Hamins, & Stroup, 1998] are not appropriate. For example, in the current set of simulations, the heat release rate is well defined. It was not measured for the commodity tests [in McGrattan, Hamins, & Stroup, 1998]. Without this information, it is difficult to determine which experiments best correspond with the sixteen simulations available in this report. Some of the NFPARF [NFPRF] tests, such as P-3, are not applicable at all because fuel was placed directly underneath the draft curtain. Nonetheless, the averages of the first sprinkler activations will be compared. For tests P-1- P-5 [in McGrattan, Hamins, & Stroup, 1998], the average is 70.6 s. For Runs 1- 9, the average is 71 s. The difference is 14%. The comparisons presented in the section are of suitably low percent differences to conclude that the validation exercise is a success. (page 31)
[Note: The average activating time for the first sprinkler to activate in Tests P-1 through P-5 (NFPRF sponsored research, 1997/1998) is actually 82.0 seconds, not 70.6 seconds as indicated above. The average activating time for the first sprinkler to activate in HAI model runs 1 through 9 is 70.2 seconds, not 70.6 seconds. Based upon this, the actual average activating times for the first operating sprinkler in Tests P-1 through P-5 is 16.8 percent greater than predicted in Model Runs 1-9.]
"With smoke and heat vents installed, Figure 22 shows that eighteen sprinklers operated as compared to the 19 sprinklers that operated in Run #4 (i.e. no dramatic change). As before, most sprinklers operated before 135 s. The final two sprinkler operations lagged their unvented counterparts in Figure 20 by no more than 40 s. The sprinkler activation map in Figure 23 shows that the ring-by ring activation pattern was not affected by the presence of the vents." (page 44)
"The simulations performed in this study were based upon the experimental work of McGrattan et al [1998]. The results with respect to sprinkler operation are consistent with that work both in terms of the time to first sprinkler operation and the total number of sprinklers operating. The simulations clearly span the range of performance that can be expected in practice with sprinkler operations numbering from four up to the number of sprinklers in the design area." (page 91)
Discussion
The Hughes Associates Inc.'s study of the concept of the "ganged" operation of smoke/ heat vents for single story buildings protected by standard spray sprinklers addresses two issues. The first issue is whether or not the "ganged" operation of roof vents will improve the environmental conditions within the building under fire conditions (assuming that the fire is controlled by the sprinkler system and that the capability of the sprinkler system is not diminished by the opening of the vents). The second issue addressed is whether or not the "ganged" operation of roof vents will have an adverse affect on the ability of the sprinkler system to control the fire.
Common sense tells us that opening roof vents during a fire will allow both heat and smoke contained in the building to vent, thus improving the environment within the building. Obviously, the sooner the vents are opened and the more vents that open, the greater the improvement of the environment within the building (again, assuming that the capability of a sprinkler system is maintained). In other words, the conclusion that the "ganged" operation of roof vents will improve the conditions within a building during a fire is not really any great revelation.
With regard to the second issue, the Hughes' study indicates that "the main concern of this investigation is the activation of sprinklers." Well, not exactly. The main concern of the study should have been the impact on the activation times of the sprinklers when multiple vents are opened simultaneously. (The study assumes that 30 vents that are 4 feet by 8 feet in dimension open simultaneously, roughly 60 seconds after the water flow indicator in the sprinkler system is activated. The report indicates that the vents are opened 127 seconds after ignition.)
The Executive Summary of the report indicates that "the operation of sprinklers was not affected by smoke and heat vents or by smoke and heat vents with draft curtains" and further indicates that the criteria utilized to make this determination was "the time to first sprinkler operation, the number of sprinkler operations and the pattern of operation." Since the "ganged" opening of the roof vents only occurs after the operation of the first sprinkler, "the time to first sprinkler operation" obviously would not be affected by the opening of the roof vents. Hence, to include this point in the criteria used to determine whether or not the capability of a sprinkler system is adversely affected is meaningless. Of course, the engineers at Hughes should know this. Including "the time to first sprinkler operation" in this list appears to be an attempt to simply "fluff up" the length of the list of criteria or, perhaps, it's just an indication of the authors' lack of respect for the intellect of the readers of their report, or maybe both.
While, certainly, the number of sprinkler operations and the pattern of sprinkler operations are important criteria to determine whether or not a sprinkler system will perform its intended function, what has been omitted is that the importance of these two factors is time-dependent. Obviously, if the activation of sprinklers that should operate in order to control a fire is delayed by venting, the capability of the sprinkler system to control a fire could be degraded, depending upon which sprinkler activations are delayed and upon the length of the delay before operation.
Prior to proceeding any further with this discussion, it needs to be acknowledged that venting definitely has an impact on the activation time of sprinklers, regardless of how many times Dr. Craig Beyler makes statements to the contrary. Repetition of an inaccurate statement ad infinitum does not make the statement true. Including this statement in a research paper (authored coincidentally by Beyler and Cooper) also does not make this statement true, regardless of how many times Beyler references his own paper as an independent work in another of his papers. A very simple experiment can confirm that this is indeed the case.
Given a fire of some specific size and a sprinkler located at the ceiling at some specific horizontal and vertical distance from the fire, we can determine the activating time of the sprinkler experimentally. If we then do a second experiment replicating the first experiment in all respects, except with an open vent in the ceiling between the fire and the sprinkler, there should be little doubt in anybody's mind that the open vent will impact the activation time of the sprinkler. One would expect that the delay in the activation time of the sprinkler would depend upon the size of the vent, the orientation of the vent with respect to the fire and the sprinkler and the horizontal distance of the vent away from the sprinkler.
The fact that an open vent would impact sprinkler operation time in this experiment is so obvious that it's not really necessary to conduct the experiment. However, predicting the activation time of the sprinkler in these experiments, utilizing the FDS and then conducting the experiments to determine whether or not the FDS can actually predict the delay in sprinkler activation time, would be of interest. In fact, conducting this very simple experiment would be one test of the capability of the FDS to actually predict the impact of venting on sprinkler activation time.
It should be noted that the interference with the activation of sprinklers by an open vent has already been clearly demonstrated in Test P-3 of the large-scale tests conducted in the National Fire Protection Research Foundation research on the interaction between sprinklers, vents and draft curtains in 1997/1998. (See page 47 of the NIST report on this research for an illustration that shows in plan view the location of the ignition point, the activation times of the various sprinklers and the orientation of the roof vent that opened, with respect to both the fire and sprinklers. The report is referred to as NISTIR 6196-1 and is dated September 1998.) In this test, a fire was ignited in the storage array near a draft curtain. The two sprinklers closest to the point of ignition activated 67 seconds and 72 seconds after ignition, while an automatic roof vent with one of its edges located roughly 5 feet from the fire opened 4 minutes and 11 seconds after ignition.
A total of six sprinklers on the side of the draft curtain opposite the fire operated, while sprinklers located downstream of the open vent on the "fire side" of the draft curtain failed to activate. Why would sprinklers located the same distance from the fire as operating sprinklers (with a draft curtain in between the fire and operating sprinklers) fail to operate? The obvious answer to this question is the open vent between the fire and the sprinklers. Now that we have established beyond any shadow of a doubt that open vents in close proximity to the fire may indeed delay the operation of sprinklers and may actually prevent sprinklers from opening, hopefully, Dr. Beyler will stop telling us that open vents do not have any significant impact on sprinkler operation.
The activation time of sprinklers varies with the size of the fire, the ceiling height, the location of the sprinkler with respect to the ceiling, the construction of the ceiling, the temperature rating of the sprinkler, the sprinkler response-time index (RTI), the type of sprinkler escutcheon provided (i.e. recessed or concealed) and, in storage occupancies, whether water droplets from operating sprinklers are deposited on the operating mechanism of sprinklers which have yet to operate. There are a few more variables that also affect sprinkler operation time, but this list is already long enough.
As a general rule of thumb, the quicker standard spray sprinklers activate, the better control of a fire is achieved by the sprinkler system, all other things being equal. (A good illustration of this point is that NFPA 13 permits a reduction in assumed sprinkler-operating area (the number of sprinklers assumed to operate) where quick response sprinklers are provided instead of standard response sprinklers.) The reason for this is that the heat release rate (HRR) generated by a fire is typically an exponential function of time(t). The sooner sprinklers operate, the smaller the fire that the sprinklers have to control.
[Note: The equation HRR = atn is often used to describe a fire mathematically. In this equation, "n" is typically a whole number greater than one and a (alpha) is a constant that varies with the type of fire that will develop, i.e. HRR = at2. In other words, if the exponent "n" in this equation is two, doubling the time results in quadrupling the heat release rate of the fire. That's one reason why firefighters are in such a hurry to get to a fire and why quick response sprinklers are able to gain control of a fire utilizing less water than standard response sprinklers.]
[Note: In some storage occupancies, high temperature (standard response) sprinklers will perform better than ordinary temperature (standard response) sprinklers. The reason for this is that fewer high temperature sprinklers operate than ordinary temperature sprinklers. In other words, the total water flow required to control some fires will be less when high temperature sprinklers, rather than ordinary temperature sprinklers, are installed. (Just another illustration of why large-scale testing is still of use.)]
Conclusion
The "Verification & Validation Guide" for Version 5 of the Fire Dynamics Simulator, as well as the "Technical Reference Guide" for the FDS, both published by NIST, specifically indicate that the use of the Fire Dynamics Simulator should be evaluated by conducting experiments to confirm that the model is actually capable of predicting the data of interest in model runs. The NIST investigation report on the fire at The Station nightclub is a clear illustration that NIST intends the FDS to be utilized, along with experiments to test and document the capabilities of the FDS.
A review of the Hughes Associates Inc.'s study of the "ganged" operation of roof vents (in buildings protected by standard spray sprinklers) indicates that Hughes apparently conducted no experiments to evaluate their use of the FDS to predict the activation times of multiple sprinklers (as many as 22 sprinklers). The Hughes' study relies solely on data from the NFPRF research in 1997/1998 to validate the predictions from the FDS.
Can the FDS actually be used to predict the activation time of multiple sprinklers with any reasonable degree of accuracy? Quite frankly, it's my opinion that no one knows the answer to that question for sure; however, if I where a gambling man, I'd be betting against the FDS model's predictions. Given our present state of knowledge and computing power, the interaction between sprinkler water spray and fire is just too complex for us to model with any degree of assurance that the model's predictions are precise enough to rely upon.
To quote Dr. Beard once again, "using models as part of decision-making may be dangerous." It's my opinion that relying upon the HAI study of the concept of "ganged" operation of roof vents as presented in their report dated February 18, 2008, might well be "dangerous," for building designers and for the manufacturers of vents as well.
With more study, researchers may well be able to demonstrate that the "ganged" roof vent operation concept has merit, but, at least at this point in time, HAI has not exercised the prerequisite standard of care necessary to document that the use of this concept will not result in anything but failure of the sprinkler protection and charred ruins.
We know that standard spray sprinklers are capable of providing protection for buildings without roof vents. We also know that individually activated automatic roof vents don't work well in buildings protected by sprinklers, if the vents work at all. What we don't know is whether or not "ganged" roof vent operation is the "fix" for the problem of using roof vents in sprinklered buildings. Until the manufacturers of vents can provide conclusive proof that "ganged" roof vent operation does not do more harm (to the capabilities of the sprinklers to control a fire) than good, my recommendation would be to stick with the sprinklers and let the vent manufacturers keep their vents.
It's my understanding that, if you turn smoke/heat vents over, they make really lovely flowerpots. Other than that, I can't think of any use for them, at least in sprinklered buildings.
Richard Schulte is a 1976 graduate of the fire protection engineering program at the Illinois Institute of Technology. After working in various positions within the fire protection field, he formed Schulte & Associates in 1988. His consulting experience includes work on the Sears Tower and numerous other notable structures. He has also acted as an expert witness in the litigation involving the fire at the New Orleans Distribution Center. He can be contacted by sending e-mail to rschulte@plumbingengineer.com.