Digital Management: Revalidation of E-Psychological Instrument for Flood Victims Version-II

Measuring the psychopathology symptoms among flood victims is an important step for intervention and treatment. However, there is a gap of a valid, reliable and an efficient instrument to measure flood victims' mental distress in Malaysia. This study aims to revalidate e-Psychological Instrument for Flood Victims Version II (e-PIFV-II) by using expert judgment method to prove content-related evidence. The e-PIFV-II is a digital self-report inventory that has 56 items with 4 dimension scales namely stress, anxiety, depression and trauma. Two studies have been done to revalidate the instrument. Results showed that there were very strong content coefficient validity for each subscale of the instrument. In study II the coefficient values for stress and trauma subscale of the e-PIFV-II were increased. The coefficient value of stress was 0.8-0.9, anxiety was 0.9, depression was 1.0, trauma was 0.9-1 and overall was 0.9. This study supports the theoretical framework and provides practical implication in the field of digital flood victims management, psychological testing and clinical psychology.


Introduction
Natural disasters have an immediate impact on human lives and often result in the destruction of the physical, biological, social, environmental and economic of the affected people, thereby having a longer-term impact on their health, well-being and survival (World Health Organization, 2017). In fact, floods are the most common type of global natural disasters responsible for almost 53,000 deaths in the last decade (Alderman et al., 2012;Doocy et al., 2013). Furthermore, floods usually have a high mortality rate in less developed countries (Kundzewicz and Kundzewicz, 2005). The National Disaster Management Agency of Malaysia reported that, from 2011 to 2016, floods are the highest natural disasters occur in Malaysia National Disaster Management Agency (NADMA) (2016). In 2014, the severe floods that hit several states on the east coast of Peninsular Malaysia, Kelantan, Terengganu and Pahang have caused physical damage, loss of property and mental distress to the victims (Ministry of Health Malaysia, 2015). Besides, The National Health and Morbidity Survey found that 1 in 3 Malaysian people experiences mental distress (Minister of Health Malaysia, 2016). Therefore, mental distress in Malaysia needs to be addressed wisely and effectively. One of the important target group of mental distress is flood victims because they expose to stressful and traumatic situation and flood takes place frequently and involves many victims simultaneously throughout the country. Due to the current alarming mental distress and the need of an effective flood management system in Malaysia (Hussain et al., 2014). It is a crucial need to find ways how to identify the symptoms of mental distress proactively and efficiently prior to any intervention and treatment.
In the fourth industrial revolution, digital technology provides new means of measuring mental distress (Christopher et al., 2017). Psychological testing does not have the choice of standing still. Technology has become a fixture in our daily life and psychological testing must also keep up with changing times. Electronic and digital options in daily personal and professional lives are quickly becoming the preferred way for people to function. In fact, today's adolescents are considered digital natives as they were born into a technological world. Now, we live in the new reality, with digital, electronic, online and computerized world. Computers are now being used for test administration, item scoring, transformation of scores, test interpretation and data storage. In fact, digital have become a powerful asset for psychological testing and assessment and will likely continue to enhance and advance our work in the field (Butcher et al., 2004;Butcher, 2013). Digital psychological testing can improve both the administration and interpretation of testing questionnaires which, until recently, have been largely in pencil-andpaper format and manually scored. Many questionnaires are now available in digital form and can be completed via a website or app. This allows them to be automatically scored and interpreted with reference to established norms, and in some instances the scores can be transmitted directly to the clinician, the user and the user's clinical record (Sireci and Faulkner-Bond, 2014).
Most of the instruments for measuring stress, anxiety, depression and trauma originally have been developed in overseas and there is a lack of such instruments that are designed in the context of developing countries. Research has also suggested for not blindly import measures used in another culture without adaptation (Goh et al., 2017) as well as taking into consideration the uniqueness and diversity of human being (Morris et al., 2016). A very few of such instruments have gone through the process of adaptations (Musa et al., 2017). Most of the instruments also cover limited constructs in a single instrument and have been administered on paper and pencil basis, which are less efficient to be used in the context of flood victims especially during flood Alshagga et al., 2015;Chan et al., 2016;Cohen and Williamsom, 1988;Ezzati et al., 2013;Ho et al., 2007;Lee, 2012;Musa et al., 2017;Mustaffa et al., 2014;Nasir et al., 2012;O'Connor et al., 2010;Othman et al., 2016;Otto et al., 2006;Pallant and Tennant, 2007;Sauer et al., 2013;Shoeb et al., 2007;Sipon et al., 2014;Sipon et al., 2015;Whittle et al., 2012;Yusoff, 2014;Zigmond and Snaith, 1983). As the e-Psychological Instrument for Flood Victims Version II (e-PIFV-II) is a newly developed digital instrument (Nazilah et al., 2018a;2018b), the present study aims to revalidate the e-PIFV-II by using expert judgment method to prove content-related evidence.
Psychological testing is a process of measuring psychology related variables by means of devices or procedures designed to obtain a sample of behavior. It is typically to obtain some gauge, usually in numerical in nature, with regard to an ability or attribute. Testing may be individual or group in nature. After test administration the tester will add up the number of correct answers of certain types of responses. The tester is not the key to the process and may be substituted for another tester without especially affecting the evaluation. Testing also typically requires technician-like skills in terms of administering and scoring a test as well as in interpreting a test result. Besides, testing yields a test score or series of test scores (Cohen et al., 2013;Crocker, 1986;Gregory & Robert, 2004).
The process of developing an instrument basically occurs in five stages. Instrument conceptualization, construction, tryout, item analysis and revision. Instrument conceptualization occurs when the idea for an instrument is conceived. Test construction occurs when writing test items, rewriting or revising items, formatting items, setting score rules. Once a preliminary form of an instrument is developed, it is administered to a representative sample of test takers under conditions that simulate the conditions that the final version of the instrument will be administered. The data from the tryout will be collected and test takers' performance on the test as a whole and on each item will be analyzed. Statistical procedure such as item analysis are employed to make judgments about which items are good, which items need to be revised and which items should be discarded. Item analysis may include analyses of item reliability, item validity and item discrimination (Cohen et al., 2013;Crocker, 1986;Gregory & Robert, 2004).
The unified conceptualization of validity with regards to content-related evidence has been stated in the Standards for Educational and Psychological Testing. The development of the Test Standards began when the American Psychological Association (APA) published a formal proposal (Technical Recommendations for Psychological Tests and Diagnostic Techniques: A Preliminary Proposal) in 1952 on the standards to be used in the development, use, and interpretation of measurement psychological instruments. The proposal led to the publication of the first standards in 1954, the Technical Recommendations for Psychological Tests and Diagnostic Techniques. In the document, validity was classified into content, predictive, concurrent, and construct (Chan, 2014). In fact, content-related evidence of validity is a central concern during instrument development. Expert professional judgment should play an integral part in developing the definition of what is to be measured, such as describing the universe of content and generating or selecting the content sample, thus, inferences about content are linked to instrument construction as well as to establish evidence of validity after an instrument has been developed.
Previously, a study has been done to validate the content-related validity of e-PIFV-II using expert judgment method. Based on 2 x 2 content coefficient validity, the results showed that there were a strong content-related coefficient validity for each subscale of the e-PIFV-II. The coefficient value of stress was 0.8, anxiety was 0.9, depression was 1.0, trauma was 0.9 and overall was 0.9. Figure (1) is the conceptual framework for the present study.

Construct Symptoms
The conceptual framework explains the uniqueness of human beings and the nature of stress, depression, anxiety and trauma. Human being is a unique organism that consists of physical, mind and overt behavior. The mind includes cognitive, emotion, value and motives. Research found that stress, depression, anxiety and trauma responses also include biological, mental and behavioral reactions. The nature and reactions of the four constructs are outlined as follows:

Stress
Stress is a negative emotional experience or physiological response that is produced following exposure to life circumstances that are threatening or represent harm or loss to the individual or those things valued by the individual. These events are commonly referred to as stressors. Stressors exist on a continuum and may take from a variety sources (Reyes and Elhai, 2008). It is as internal process that occurs as people try to adjust to events and situations, especially those that they perceive to be beyond their coping capacity. Stress promotes survival because it forces people to adapt to rapidly changing environmental conditions. This type of adaptive stress is known as eustress. However, when the stress is too powerful and beyond people's capacity, it would cause negative effect to them. Such this maladaptive stress is sometimes referred to as distress. Stress reactions involve the physical and psychological responses that occur in the face of stressors (Taylor, 2002). Hans Selye found that physical reactions to stressors include an initial alarm reaction, followed by resistance and exhaustion. These three stage pattern of responses triggered by the effort to adapt to stressors is known as The General Adaptation Syndrome (GAS). The model has been very influential, but it has also been criticized for underestimating the role of psychological factors in stress, such as changes in emotions, thoughts and behaviors. These criticisms led to the development of psychobiological models which emphasized the importance of psychological as well as biological stress responses (Ganzel et al., 2010;Lazarus and Folkman, 1984).

Depression
Depression is a persistent, pervasive disorder that causes a low, sad, hopeless mood, inability to take interest or find pleasure in almost anything. Other mental, emotional, physical and behavioral symptoms occur as well, and the net effect of all these symptoms is serious distress or significant problems in daily life (Andrews, 2010). Depression differs from simple grief or mourning which is a proper emotional response to the loss of love persons or objects. Biology as well as psychological mechanisms is important in producing and maintaining depressive symptoms. The biochemical cause appears to be the defective regulation of the release of neurotransmitters in the brain, particularly norepinephrine and serotonin. Reduced quantities or activities of these chemicals in the brain is thought to cause the depressed mood in some sufferers (Encyclopedia of Britannica, 2018). A person who is depressed usually experiences several symptoms such as feelings of sadness, hopelessness, pessimism, lowered self-esteem, a decrease or loss of ability to take pleasure in ordinary activities, reduced energy and vitality, slowness of thought or action, loss of appetite and disturbed sleep or insomnia.

Anxiety
Anxiety is an emotion characterized by feelings of tension, worried thoughts and physical changes like increased blood pressure. People with anxiety disorders usually have recurring intrusive thoughts or concerns. They may avoid certain situations out of worry. They may also have physical symptoms such as sweating, trembling, dizziness or a rapid heartbeat (Kazdin, 2000). Anxiety also includes future-oriented negative thoughts such as "What if I look foolish in front of the audience?" or "What if he doesn't like me?" It is when the anxiety is out of proportion to the event that we say a person may suffer from an anxiety disorder (Reyes and Elhai, 2008). Anxiety also occurs when having a feeling of dread, fear, or apprehension, often with no clear justification. Anxiety is distinguished from fear because the latter arises in response to a clear and actual danger, such as one affecting a person's physical safety. Anxiety, by contrast, arises in response to apparently innocuous situations or is the product of subjective, internal emotional conflicts, the causes of which may not be clear to the person himself. Some anxieties inevitably arise in the course of daily life and are considered normal. But persistent, intense, chronic, or recurring anxiety not justified in response to real-life stresses is usually regarded as a sign of an emotional disorder (Encyclopedia of Britannica, 2018).

Trauma
Trauma is defined as events that are emotionally shocking or horrifying, which threaten or actually involve death or a violation of bodily integrity (such as sexual violation or torture) or that render the affected person helpless to prevent or stop the resultant psychological and physical harm (Reyes and Elhai, 2008). Besides, trauma has been a sudden, potentially deadly experience, often leaving lasting, troubling memories (Figley, 2012).

Methods
This study was designed to revalidate a newly developed of e-IPFV-II by using expert judgment method to prove content-related evidence. The purpose of e-PIFV-II is to measure digitally stress, anxiety, depression and trauma among flood victims from age 12 years to late adulthood. The score is used to prescribe the level of stress, anxiety, depression and trauma symptoms. The instrument uses 3 points Likert scales; yes, no and not sure (Yes = 3 marks, No = 0 marks, not sure = 1 mark). The e-IPFV-II has 56 items and each subscale has 14 items. The instrument covers four domains; physical, cognitive, emotional and behavioral reactions. The e-PIFV-II takes 10-15 minutes to be completed. This study adopt similar models used in previous studies. The models emphasis on test conceptualization, test construction, test tryout, item analysis and test revision (Cohen et al., 2013;Crocker, 1986;Gregory & Robert, 2004). There were three phases in this study:-

Phase I
Two experts were involved (a clinical psychologist and a psychiatrist). An informal consent was obtained from each expert before a formal letter of appointment sent to them. Then, a letter of appointment was sent to both experts together with a standardized rating form. The form contains a list of items based on each construct, rating column with two options (appropriate or inappropriate) and a column for comments and suggestions.

Phase II
The two experts reviewed the instrument privately and took two weeks to return the completed forms.

Phase III
Data that retrieved from feedback forms were counted and then put in the 2 x 2 contingency table as figure (2).

Figure-2. 2 x 2 contingency table
After completing the 2 x 2 contingency table, then index for content coefficient validity was calculated using the following formula: D Content coefficient validity = (A + B + C + D)

Results and Discussion
Statistical analysis was done to obtain content coefficients of the instrument. The content validity coefficients showed that e-IPFV-II had a very strong content validity. The coefficient value for stress and trauma subscale were increased compared to the previous study (Table 1). Realizing that diagnosis in clinical settings need to the holistic and multi-method approach in identifying mental distress (Cohen et al., 2013), therefore e-IPFV-II is only considered as a prescreening tool of four types of mental distress symptoms especially for the flood victims. The e-IPFV-II score is aimed to categorize people with less or more symptoms of stress, anxiety, depression or trauma. Even though e-IPFV-II may not be sufficient tool to diagnose an individual's mental distress, at least it can help to provide early inputs to professionals more objectively, easier, faster and cover a wide range of ages.

Conclusion
The present study shows that the coefficient value of e-IPFV-II content validity is increased. The instrument also supports the existing theories or models of stress, depression, anxiety and trauma that the nature and reactions of all constructs encompass both physical and psychological aspects (Andrews, 2010;Encyclopedia of Britannica, 2018;Figley, 2012;Ganzel et al., 2010;Kazdin, 2000;Lazarus and Folkman, 1984;Reyes and Elhai, 2008;Taylor, 2002). Therefore, it is important to take note that flood victims are not only expose to physical risk but also cognitive and emotional risk. The government, organization, volunteers and experts should provide assistance which cover biological needs as well as cognitive and emotional supports.
The e-PIFV-II is planned as a research output and becomes an effective social innovation tool to prescreen flood victims' mental distress. It is also aimed to offer an alternative to solve the Malaysian mental distress by producing more objective, effective, sustainable, contextual, general and faster tool to cater mental distress among flood victims. This instrument will be a very helpful tool as it can help to save energy, time, expenses and experts limitation such as counsellors, clinical psychologists and psychiatrist to serve in the ground. In addition, it can help the public to address, perceive and report mental distress more objectively, especially in the situation where professional teams are not available or before the affected flood victims can be reached by them.
A short version of this instrument is highly recommended to screen flood victims' mental distress especially to be used during a very traumatic flood events. The researchers also find that digital instrument is a good approach to draw attention of adolescent group as well as to get their cooperation. This group has very positive attitude toward digital application. Whereas for other groups such as the adult and old people their attitude toward digital instrument are neutral. However, they prefer to get help from the instrument administrator to operate the application for them. But for those who are not familiar with digital tools or applications, they refuse to operate the instrument by themselves and ask the instrument administrator to operate for them. It shows that the attitude gap towards the digital instrument between the groups is not only related to the generation basis but more on their familiarity with the digital applications.
As e-IPFV-II is a newly developed instrument, we suggest to replicate this study in order to confirm the findings. In this study, we examine the validity evidence based on test content. American Educational Research Association, American Psychological Association and National Council on Measurement in Education define validity as the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. With respect to evidence, they specify five sources of evidence that might be used in evaluating a proposed interpretation of test scores for particular purposes. These five sources are validity evidence based on test content, response processes, internal structure, relations to other variables and testing consequences. Therefore, there is no singular source of evidence sufficient to support a validity claim. Construct validity is the central component in validation work, encompasses five sources of evidence to the validation of the interpretation and use of the score of an instrument.
Further studies involving flood victims from diverse backgrounds, cultures, environments and flood severity as well as a large sample and populations need to be done to establish generalization. Theoretically this instrument is suitable for measuring stress, anxiety and depression for flood victims as well as the others. Therefore, this instrument can be widely used in various settings. However, the reliability of the instrument must be tested before it can be claimed as a reliable instrument for other particular group of clients. Meanwhile, the trauma subscale is only suitable for flood victims because all items in this subscale specifically focus on traumatic flood events. This instrument is also limited to be used for children aged 12 years to elderly as to understand most of the items in this instrument requires a particular stage of cognitive maturity that is formal operational stage (Bernstein et al., 2012).