Frameworks for Disaster Research and Evaluation
This series of papers represents work done by Birnbaum, Daily, O’Rourke, and Loretti to define and illustrate guidelines for the study of disaster events. The Framework papers are published with the realization that there are other researchers and scholarly groups who take issue with some of the concepts presented. Prehospital and Disaster Medicine presents the papers without prejudice and with the intent to provide a platform for discussion and exploration of disaster research methodology.
Readers of the Journal are encouraged to review the Framework reports and submit comments that either supports the Framework concepts or that present competing knowledge of appropriate methods for disaster research and evaluation. Particularly encouraged is submission of papers that explore the logic and scientific basis used to develop the Frameworks series. In addition, a particularly important issue is the establishment of a common vocabulary with precise definitions of terms commonly used in disaster research.
The study of disasters is challenging due to the complex, unpredictable nature of disaster events. Traditional health and medical science methods that compare a controlled intervention within a controlled study population is not a method that is common or effectively applied to disaster research. The development of research methods for prehospital care, cardiac arrest resuscitation, and trauma serve as examples of fields that have formed Frameworks that allow for research, evaluation, and understanding outcomes for complex, unpredictable processes.
As an example, in prehospital care, much of the early work in developing a foundation for research revolved around precisely defining time frames and sequences common to the field. With defining response, treatment, and transport times, as well as the organizational structure of a prehospital delivery system, one is able to conduct research that allows for comparison among similar prehospital systems of interventions and system designs. The Utstein method for reporting cardiac arrest research data provides a mechanism in which the uncontrolled event of cardiac arrest can be studied. The Utstein model for reporting data allows for evaluation of the impact of interventions on study populations in comparison to other populations. Early in the study of trauma interventions, it became necessary to develop scoring systems that could be used to predict trauma victim outcome.
This need for scoring the severity of trauma and correlating it to outcome led to the development of the Revised Trauma Score and Injury Severity Score systems which allow researchers and system evaluators to compare data and outcomes among multiple trauma care systems. None of the Frameworks for research and evaluation in prehospital care, cardiac arrest, or trauma were developed without controversy. Important, though, is that the research Frameworks allowed for advancing the science and literature of prehospital care, cardiac arrest management, and trauma care.
Unfortunately, disaster health and medical research has lagged other similar fields in developing a Framework for comparative study. Rather, disaster research has lingered with observational and descriptive methodology for decades. Current disaster research is in a stage in which hypotheses are being firmly generated, but where there is a lack of comparability of study results. Good quantitative and qualitative observation research is valuable and lays a foundation for more advanced research in a field, but comparison of outcomes qualifies a research field as being well developed. As with the other unpredictable, complex fields discussed above, only with a common Framework for study can comparisons of outcomes be attained in a manner that will advance the science of disaster health and medical research.
With the objective of opening discussion and scientific development for disaster health and medical research, the Framework reports of Birnbaum, Daily, O’Rourke, and Loretti are being presented in Prehospital and Disaster Medicine. All who have an interest in disaster research are invited to provide comment regarding the Frameworks, either in the form of letters to the Editor or by submission of special reports. As with any science-related issue, publication of concepts and logic for discussion and open comment in the literature will lead to support, refinement, or refute.
SStratton. Frameworks for disaster research and evaluation. Prehosp Disaster Med. 2015; 27(5):547.
Part I: An Overview
The ultimate goals of conducting disaster research are to obtain information to: (1) decrease risks that a hazard will produce a disaster; (2) decrease the mortality associated with disasters; (3) decrease the morbidity associated with disasters; and (4) enhance recovery of the affected community and decrease the risks that a hazard will produce a disaster. Two principal, but inter-related, branches of disaster research are: (1) Epidemiological; and (2) Interventional. Epidemiological research explores the relationships and occurrences that comprise a disaster from a particular event. Interventional research involves evaluations of interventions, whether they are directed at relief, recovery, hazard mitigation, capacity building, or performance.
In response to the need for the discipline of Disaster Health to build its science on data that are generalizeable and comparable, a Disaster Logic Model (DLM) and a set of five Frameworks have been developed to structure the information and research of the health aspects of disasters. These Frameworks consist of the: (1) Conceptual; (2) Temporal; (3) Societal; (4) Relief/Recovery; and (5) Risk-Reduction Frameworks. The Frameworks provide a standardized format for studying and comparing the epidemiology of disasters, and with the addition of the DLM, for evaluating the interventions (responses) provided prior to, during, and following a disaster, especially as they relate to the health status of the people affected by, or at-risk for, a disaster. Critical to all five Frameworks is the inclusion of standardized definitions of the terms.
The Conceptual Framework describes the progression of a hazard that becomes an event, which causes structural damage, which, in turn, results in compromised, decreased, or losses of function(s) (functional damage) that, in turn, produce needs that lead to an emergency or a disaster. The Framework incorporates a cascade of risks that lead from the presence of a hazard to the development of a disaster. Risk is the likelihood that each of the steps leading from a hazard to a disaster will take place, as well as the probabilities of consequences of each of the elements in the Conceptual Framework. The Temporal Framework describes this chronological progression as phases in order of their appearance in time; some may occur concurrently.
In order to study and compare the effects of an event on the complex amalgam that constitutes a community, the essential functions of a community have been deconstructed into 13 Societal Systems that comprise the Societal Framework. These diverse, but inter-related, Societal Systems interface with each other through a 14th System, Coordination and Control. The DLM can be used to identify the effects, costs, outcomes, and impacts of any intervention. Both the Relief/Recovery and Risk-Reduction Frameworks are based on the DLM.
The Relief/Recovery Framework provides the structure necessary to systematically evaluate the processes involved in interventions provided during the Relief or Recovery phases of a disaster. The Risk-Reduction Framework details the processes involved in interventions aimed at mitigating the risk that a hazard will produce a destructive event, and/or in capacity building to augment the resilience of a community to the consequences of such an event.
Part II: The Disaster Health Conceptual Framework Revisited
A Conceptual Framework upon which the study of disasters can be organized is essential for understanding the epidemiology of disasters, as well as the interventions/responses undertaken. Application of the structure provided by the Conceptual Framework should facilitate the development of the science of Disaster Health. This Framework is based on deconstructions of the commonly used Disaster Management Cycle. The Conceptual Framework incorporates the steps that occur as a hazard progresses to a disaster. It describes an event that results from the changes in the release of energy from a hazard that may cause Structural Damages that in turn, may result in Functional Damages (decreases in levels of function) that produce needs (goods and services required).
These needs can be met by the goods and services that are available during normal, day-to-day operations of the community, or the resources that are contained within the community’s Response Capacity (ie, an Emergency), or by goods and services provided from outside of the affected area (outside response capacities). Whenever the Local Response Capacity is unable to meet the needs, and the Response Capacities from areas outside of the affected community are required, a disaster occurs. All responses, whether in the Relief or Recovery phases of a disaster, are interventions that use the goods, services, and resources contained in the Response Capacity (local or outside).
Responses may be directed at preventing/mitigating further deterioration in levels of functions (damage control, deaths, injuries, diseases, morbidity, and secondary events) in the affected population and filling the gaps in available services created by Structural Damages (compromise in available goods, services, and/or resources; ie, Relief Responses), or may be directed toward returning the affected community and its components to the pre-event functional state (ie, Recovery Responses). Hazard Mitigation includes interventions designed to decrease the likelihood that a hazard will cause an event, and should an event occur, that the amount of energy released will be reduced. Capacity Building consists of all interventions undertaken before an event occurs in order to increase the resilience of the community to an event related to a hazard that exists in an area-at-risk.
Resilience is the combination of the Absorbing, Buffering, and Response Capacities of a community-at-risk, and is enhanced through Capacity-Building efforts. A disaster constitutes a failure of resilience.
Part III: Framework for the Temporal Phases of Disasters
Each of the elements described in the Conceptual Framework for disasters has a temporal designation; each has a beginning and end time. The Temporal Framework defines these elements as phases that are based on characteristics rather than on absolute times. The six temporal phases include the: (1) Pre-event; (2) Event; (3) Structural Damage; (4) Functional Damage (changes in levels of functions of the Societal Systems); (5) Relief; and (6) Recovery phases. Development is not a phase of a disaster.
The use of the Temporal Framework in studying and reporting disasters allows comparisons to be made between similar phases of different disasters, regardless of the hazard involved and/or the community impacted. For research and evaluation purposes, assessments, plans, and interventions must be described in relation to the appropriate temporal phase.
Part IV: Framework for Societal Structures: the Societal Systems
For the purposes of research and/or evaluation, a community/society is organized into 13 Societal Systems under the umbrella of an overall Coordination and Control System. This organization facilitates descriptions of a community/society or a component of a community for assessment at any designated time across the Temporal Phases of a disaster. Such assessments provide a picture of the functional status of one or more Systems that comprise a community.
Since no system operates in isolation from the other systems, information of the concomitant status of several Societal Systems is crucial to gaining a complete understanding of compromised functions, as well as the effects and side effects of any intervention directed at restoring the functional state of the affected community or risk-reduction interventions of a community-at-risk.
The 13 Societal Systems include: (1) Public Health; (2) Medical Care; (3) Water and Sanitation; (4) Shelter and Clothing; (5) Food and Nutrition; (6) Energy Supply; (7) Public Works and Engineering; (8) Social Structures; (9) Logistics and Transportation; (10) Security; (11) Communications; (12) Economy; and (13) Education. Many functions and sub-functions of the Systems overlap, or share some common sub-functions with other systems. For the purposes of research/evaluation, it is necessary to assign functions and sub-functions to only one of the Societal Systems.
Part V: Epidemiological Disaster Research
Studies of the health aspect of disasters focus either on the epidemiology of disasters to define the causes and the progression from a hazard to a disaster, or the evaluations of interventions provided during any phase of a disaster. Epidemiological disaster research studies are undertaken for the purposes of: (1) understanding the mechanisms by which hazards evolve into a disaster; (2) determining ways to mitigate the risk(s) that a specific hazard will progress into a disaster; (3) predicting the likely damages and needs of the population-at-risk for an event; and (4) identifying potential measures to increase the resilience of a community to future events.
Epidemiological disaster research utilizes the Conceptual, Temporal, and Societal Frameworks to define what occurs when a hazard manifests as an event that causes a disaster. The findings from such studies should suggest interventions that could augment the absorbing, buffering, or/and response capacities to lessen the probability of similar damages occurring from the next event. Ultimately, the use of these Frameworks in studying the health aspects of a disaster will help define what to expect in a specific setting and the standards and best practices upon which education, training, competencies, performance, and professionalization will be built.
Part VI: Interventional Research and the Disaster Logic Model
Disaster-related interventions are actions or responses undertaken during any phase of a disaster to change the current status of an affected community or a Societal System. Interventional disaster research aims to evaluate the results of such interventions in order to develop standards and best practices in Disaster Health that can be applied to disaster risk reduction. Considering interventions as production functions (transformation processes) structures the analyses and cataloguing of interventions/responses that are implemented prior to, during, or following a disaster or other emergency.
Since currently it is not possible to do randomized, controlled studies of disasters, in order to validate the derived standards and best practices, the results of the studies must be compared and synthesized with results from other studies (ie, systematic reviews). Such reviews will be facilitated by the selected studies being structured using accepted frameworks. A logic model is a graphic representation of the transformation processes of a program [project] that shows the intended relationships between investments and results. Logic models are used to describe a program and its theory of change, and they provide a method for the analyzing and evaluating interventions.
The Disaster Logic Model (DLM) is an adaptation of a logic model used for the evaluation of educational programs and provides the structure required for the analysis of disaster-related interventions. It incorporates a(n): definition of the current functional status of a community or Societal System, identification of needs, definition of goals, selection of objectives, implementation of the intervention(s), and evaluation of the effects, outcomes, costs, and impacts of the interventions. It is useful for determining the value of an intervention and it also provides the structure for analyzing the processes used in providing the intervention according to the Relief/Recovery and Risk-Reduction Frameworks.
Part VII: The Relief/Recovery Framework
The principal goal of research relative to disasters is to decrease the risk that a hazard will result in a disaster. Disaster studies pursue two distinct directions: (1) epidemiological (non-interventional); and (2) interventional. Both interventional and non-interventional studies require data/information obtained from assessments of function. Non-interventional studies examine the epidemiology of disasters. Interventional studies evaluate specific interventions/responses in terms of their effectiveness in meeting their respective objectives, their contribution to the overarching goal, other effects created, their respective costs, and the efficiency with which they achieved their objectives. The results of interventional studies should contribute to evidence that will be used to inform the decisions used to define standards of care and best practices for a given setting based on these standards.
Interventional studies are based on the Disaster Logic Model (DLM) and are used to change or maintain levels of function (LOFs). Relief and Recovery interventional studies seek to determine the effects, outcomes, impacts, costs, and value of the intervention provided after the onset of a damaging event. The Relief/Recovery Framework provides the structure needed to systematically study the processes involved in providing relief or recovery interventions that result in a new LOF for a given Societal System and/or its component functions.
It consists of the following transformational processes (steps): (1) identification of the functional state prior to the onset of the event (pre-event); (2) assessments of the current functional state; (3) comparison of the current functional state with the pre-event state and with the results of the last assessment; (4) needs identification; (5) strategic planning, including establishing the overall strategic goal(s), objectives, and priorities for interventions; (6) identification of options for interventions; (7) selection of the most appropriate intervention(s); (8) operational planning; (9) implementation of the intervention(s); (10) assessments of the effects and changes in LOFs resulting from the intervention(s); (11) determination of the costs of providing the intervention; (12) determination of the current functional status; (13) synthesis of the findings with current evidence to define the benefits and value of the intervention to the affected population; and (14) codification of the findings into new evidence. Each of these steps in the Framework is a production function that facilitates evaluation, and the outputs of the transformation process establish the current state for the next step in the process. The evidence obtained is integrated into augmenting the respective Response Capacities of a community-at-risk. The ultimate impact of enhanced Response Capacity is determined by studying the epidemiology of the next event.
Part VIII: Risk, Risk Reduction, Risk Management, and Capacity Building
There is a cascade of risks associated with a hazard evolving into a disaster that consists of the risk that: (1) a hazard will produce an event; (2) an event will cause structural damage; (3) structural damage will create functional damages and needs; (4) needs will create an emergency (require use of the local response capacity); and (5) the needs will overwhelm the local response capacity and result in a disaster (ie, the need for outside assistance). Each step along the continuum/cascade can be characterized by its probability of occurrence and the probability of possible consequences of its occurrence, and each risk is dependent upon the preceding occurrence in the progression from a hazard to a disaster.
Risk-reduction measures are interventions (actions) that can be implemented to: (1) decrease the risk that a hazard will manifest as an event; (2) decrease the amounts of structural and functional damages that will result from the event; and/or (3) increase the ability to cope with the damage and respond to the needs that result from an event. Capacity building increases the level of resilience by augmenting the absorbing and/or buffering and/or response capacities of a community-at-risk. Risks for some hazards vary by the context in which they exist and by the Societal System(s) involved.
Part IX: Risk-Reduction Framework
A disaster is a failure of resilience to an event. Mitigating the risks that a hazard will progress into a destructive event, or increasing the resilience of a society-at-risk, requires careful analysis, planning, and execution. The Disaster Logic Model (DLM) is used to define the value (effects, costs, and outcome(s)), impacts, and benefits of interventions directed at risk reduction. A Risk-Reduction Framework, based on the DLM, details the processes involved in hazard mitigation and/or capacity-building interventions to augment the resilience of a community or to decrease the risk that a secondary event will develop.
This Framework provides the structure to systematically undertake and evaluate risk-reduction interventions. It applies to all interventions aimed at hazard mitigation and/or increasing the absorbing, buffering, or response capacities of a community-at-risk for a primary or secondary event that could result in a disaster. The Framework utilizes the structure provided by the DLM and consists of 14 steps: (1) hazards and risks identification; (2) historical perspectives and predictions; (3) selection of hazard(s) to address; (4) selection of appropriate indicators; (5) identification of current resilience standards and benchmarks; (6) assessment of the current resilience status; (7) identification of resilience needs; (8) strategic planning; (9) selection of an appropriate intervention; (10) operational planning; (11) implementation; (12) assessments of outputs; (13) synthesis; and (14) feedback. Each of these steps is a transformation process that is described in detail. Emphasis is placed on the role of Coordination and Control during planning, implementation of risk-reduction/capacity building interventions, and evaluation.
