An Assessment Instrument of Product versus Process Writing Instruction: A Rasch Measurement Analysis

Recent researchers continuously stress on the importance of writing instruction in language learning. Therefore, there is an immense need to have an appropriate instrument to measure the writing instruction and the instrument has to be fully tested for validity. Therefore, we developed and validated an instrument using Rasch Measurement Model to assess two prominent approaches (product approach versus process approach) used in writing instruction contemporarily. We tested the instrument by computing its infit, outfit, item reliability, item separation, item difficulty and item discrimination. The results demonstrated that the instrument showed adequate estimates of reliability in assessing both the approaches used by teachers in ESL classrooms. The raw variance result showed that the accuracy of the items of product approach (85.7%) and process approach (63.7%) were excellent and good. The values of item reliability and item separation for product approach were 0.98 and 6.33, while process approach showed the item reliability and separation values of 0.65 and 1.37 respectively. The results showed that product approach is being dominantly practised that did not enhance students‟ writing skills in the long run. To address this issue, we propose a new framework for writing instruction as a guide to ESL teachers.


Introduction
Nowadays, Rasch model analysis is widely applied for examining and validating the psychometrics of instruments in different fields. As a measurement tool it provides more accuracy to the measuring instrument (Linacre, 2002). Rasch measurement model effectively helps in producing more reliable instruments (Noor et al., 2010). However, Rasch model is not common in teaching writing research. This study aimed at exploring the reliability and validity of instrument for assessing two prominent approaches used in writing instructions in Malaysian English for second language (ESL) classrooms which were the product and process writing approaches. The study also aimed to propose an effective writing framework to be used in writing instruction for Malaysian ESL teachers.
Studies have demonstrated the importance of writing instruction for students" writing achievement and success. However, there is also enough evidence that ineffective writing instruction negatively affects students" writing particularly at early stages. It may be one of the crucial responsible factors contributing to writing problems in Malaysian ESL classroom context (Tan and Richardson, 2006). The major aim of this study was to assess the application of product versus process writing approaches by Malaysian ESL teachers. Although, writing in English as a foreign language has always been a challenging experience for Malaysian students (Foo, 2007;Kamimura, 2010;Zheng, 1999), however, teaching writing is also found by the teachers to be a difficult job (Suriyanti and Yaacob, 2016). Many research studies have reported about the barriers in ESL writing such as learners" lack of skills about sentence structure, limited word choice and cultural influence (Ghabool et al., 2012;Kamimura, 2010;Wong et al., 2009). Besides these factors, the teacher"s approach adopted to teach writing is one of the major factors that affects the outcome of the teaching itself.
the traditional approach where they rely heavily on textbooks (Akinwamide, 2012;Alnufale and Grenfell, 2012;Foo, 2007;Wong et al., 2009;Zeng, 2010). Previous studies showed that teachers used multiple approaches in teaching writing. According to Badger and White (2000), the product and process writing approaches have dominated the writing arena for more than two decades in the ESL classrooms. Before that, the language teachers were more focused on product approaches or the finished product where they expected the students to produce an error free and coherent text (Nunan, 1999).
Over the years, product and process writing approaches have been frequently used by ESL teachers as the writing instruction. By practicing the product approach, the teachers focus on grammatical corrections and lexical pattern. They emphasize on accuracy in writing then the content (Suriyanti and Yaacob, 2016). Under this approach the aim of writing task is to produce a final product which is free of error (Harmer, 2002). The teacher often demands for grammatical corrections and always focuses on language structure (Leki, 1990). So this type of approach basically invites the learners to imitate, to copy and to transform models given by the teachers.
Interestingly, product approach is used in those countries where English is taught as a second language such as Singapore, Indonesia, China, Thailand, Malaysia and others (Ariyanti, 2010;Foo, 2007;Somsak, 2008;Zeng, 2010). These studies revealed that in all these countries the writing teachers showed interest in the final product of writing by students with a lot of emphasis on coherence and error free text. The ultimate result of the product approach is that learners become inactive, passive and dependent on teachers and the model itself. Hence, researchers have consistently reported that this approach neglects the important processes involved in writing activity (Raimes, 1983;Tribble, 1996). This approach does not seem to answer questions such as, how students write, how they generate ideas and how they produce a piece of writing.
On the contrary, in the process approach, the main priority is on how the writing process evolves where the students receive guidance about writing, generating ideas, drafting writing and editing their own work. The proponents of process approach emphasize on promoting the creativity of the writers rather the final product alone. They pay attention towards developing practices rather than imitating models given by teachers which was the practice of product approach (Tribble, 1996). According to Hedge (2015) process writing approach pays attention to the recursive nature of writing like getting ideas together, planning the outline, making notes and preparing drafts, revising and finally, getting ready to publish the work. Process writing allows students to choose the topics they want to write about. However, they do get some guidance from their teachers and there is no fear of errors in writing because the main aim is not to avoid grammatical errors but rather to write freely. Research has demonstrated that students who practise process approach show much better performance because this approach focuses on creativity and not the final product (Raimes, 1983;Tribble, 1996).
Effective writing instruction is characterized by prewriting, drafting, revising, editing and publishing (Grabe and Kaplan, 2014). Skilled writers monitor and regulate their own writing. Younger ESL classroom learners need a clear writing instruction (Gilbert and Graham, 2010). In another study, Çavdar and Doe (2012) stated that it becomes difficult to bring improvement in students" writing skills until instructors design effective writing instruction to teach writing. Effectively designed writing practices provide better prospects for developing ESL learners" writing skills (Johns, 2011). A teacher"s approach in teaching writing involves structuring lessons with suitable activities in order to achieve the teaching and learning objectives. According to Badger and White (2000), the teaching approaches refer to practices used by educators to teach writing. Students are provided with tools to so that they can take responsibility for their own learning. Ferris (2010) suggested that in order to plan effective writing, the teachers need a reliable guidance that elaborates on the frequently practised strategies and approaches as well as the knowledge on the effects of the approaches on students as far as the short term and long term effects are concerned. In actual terms, the writing instruction of Malaysian ESL teachers is considered crucial to comprehend especially when students need writing skills at tertiary level and real work situations (Chow, 2007). Skilful writing is necessary for both academic and professional success. Among many approaches studied, the product and process approaches are found to be predominantly employed in the ESL writing instruction (Graham and Sandmel, 2011;Subramaniam and Khan, 2016). However, these writing approaches practised by ESL writing instructors in the primary and secondary school level in Malaysian classrooms have gone through several paradigm shifts (Musa et al., 2012). The controversy of which approach will work the best to teach writing skills continues between product approach and process approach. Thus, it is crucial to analyse the characteristics of these approaches in order to know how the actual development of writing instruction can be better facilitated in the ESL classrooms. This study aims at proposing a new effective writing instruction framework for ESL teachers in the Malaysian language teaching and learning context.
Numerous studies have investigated the issues of the challenges faced by teachers in teaching writing in ESL classrooms (Hasan and Akhand, 2010;Nicol, 2010;Shafie et al., 2010). In this regard, much attention has been paid towards students" perceptions and needs of language learning specially related to writing and less attention has been paid towards the effectiveness of using the selected writing approaches in teaching writing (Ghabool et al., 2012;Nordin, 2017). In addition, very limited studies are available on using the validated instrument for assessing writing instruction especially related to the application of product versus process writing approaches. Previous studies have not investigated into this area thoroughly. Hence, more research is needed in this area.
In this study we attempted to develop an instrument to investigate the application of product versus process writing approaches in the Malaysian classroom context. The two objectives of this study were: 1. to validate an instrument to assess the product versus process writing approaches in Malaysian ESL classrooms and 2. to identify the dominant approach used in writing instructions among the teachers teaching in the primary ESL classrooms.

Method
This explorative study was conducted to develop and validate an instrument for assessing product and process writing approaches in the Malaysian ESL classroom context. For this purpose, first an instrument was developed and validated to observe the teachers" writing practices in the ESL classrooms. The participants of this study were teachers of a Teacher Education Institution in Malaysia. Purposive sampling method was used in selecting the participants. A group of ten teachers were identified based on saturation point to participate in the research.
Classroom observations were carried out in order to gain knowledge about the teachers" teaching practices including the teaching strategies and the types of writing activities assigned to students. This method was selected as it provides immediate data about human behavior in real context (Musa et al., 2012).
In this study, non-participatory classroom observation was the main method employed to gather a wide range of information from the participants as they carried out the writing activities for the second language (L2) learners. The classroom observations of the teachers" teachings were video-taped. The skill of video-taping is important in order to conduct "observational research" though the researcher can "see" the actions (Fischl and Fisher, 2007). For data collection, an Observation Checklist (OC) was used to collect data from the teachers. A Rasch Measurement Model was used for validation of the OC (Fischl and Fisher, 2007). The constructs of the checklist were tested using Rasch Model Measurement to confirm the item reliability and person reliability. The selected approach and the teaching strategies adopted by the teachers were recorded in the OC and later quantified to study the common practice among the teachers in teaching writing. The scores were calculated to obtain the frequency of the approaches and teaching strategies used. The practised approach that the teachers employed dominantly in the classroom was identified and the strategies of teaching writing were studied. Data from the OC and the recordings of the classroom observations were the main source of information to identify the product versus process writing practices adopted by the teachers to teach writing in the selected ESL classrooms.

Validation of Instrument
The Rasch Measurement Model was also used to validate the OC. The analysis was conducted separately for the two approaches to enhance the reliability of the findings of each approach. The OC was prepared by taking note of the important strategies and criteria of the PD and PC. The researcher marked on the OC based on the strategies and approaches practised by the teachers to teach writing. The OC was tailored to examine various aspects of writing instructions in the classroom specially to study the current and dominant teaching strategies adopted by the teachers. The constructs of the OC were further tested using Rasch Measurement Model to measure its instrument validity and person validity. The OC contained the strategies involved in the two selected approaches (PD and PC). According to Steel (2004), PD consists of four strategies (familiarisation, controlled writing, guided writing and free writing) and PC comprises eight strategies (brainstorming, planning, mind mapping, first draft, peer feedback, editing, final draft and evaluation and feedback).
The strategies of each approach was analysed by preparing two constructs respectively in order to cover all the strategies well without missing out any details. Therefore, PD had eight items whereas PC had 16 items in the OC respectively. All the items were placed carefully in the OC in a scattered manner to avoid stereotyping when keying in the score. A 5-point Likert scale was used as a guide to gauge the amount and degree of the strategies selected by the participants of the study in their writing lessons. Based on the score and frequency of the usage of the selected strategies, the dominant approach was identified. The approaches were measured based on the item and person reliability (Fischl and Fisher, 2007). In addition, the analysis on dimensionality assured the validity of the items used in the study. Table 1 shows the summary of the eight measured items. The item separation based on Rasch Measurement Model analysis was 6.33. Fisher further says that the item separation is excellent if the value is above 5. The item separation of this study was high as it highlighted that there were six different levels of ability of teachers that had been measured using the items of the OC. Thus, the items were utilised to examine the practice of product approach among the teachers to teach writing in the classroom from various angles to enhance the reliability of the study. The values of infit and outfit mean square of the study for item reliability was found between 0.97 and 0.98 which was within the acceptable range (Fischl and Fisher, 2007). Rasch Measurement Model also provides person reliability. Ten teachers involved as participants of the study were tested using Rasch Model. The reading of alpha Cronbach showed 0.86 which proved that the study is reliable. Table 2 shows the summary of ten measured person. Based on Rasch Measurement Model analysis, the value of the person reliability was 0.81 which was considered good. Therefore, the teachers who were the participants of the study would produce an identical result for an alternate observation. The person separation reported was 2.04. This value was fair as the sample was small. Moreover, this was a case study and purposive sampling method was used in the selection of the teachers as participant of the study. The teachers were selected based on their level of teaching. Therefore, the levels of variance among the teachers were rather small. However, the values of infit and outfit mean square of the study for person reliability was between 0.92 and 0.97 which was within the acceptable range. High reliability does not necessarily contribute to high validity. Rasch Measurement Model analysis on dimensionality can assure the validity of the items used. The validity can be referred to the raw variance of the items. The result of raw variance showed that the accuracy of the items of PD used in the OC was 85.7% which was excellent as shown in Table 3. In order to identify the reliability of PC, the constructs that modeled the strategies of PC in OC were tested using Rasch Measurement Model. The analysis was done on 16 items (Item 9 to Item 24) from the OC that featured the strategies of PC. The Rasch Measurement Model analysis on dimensionality can assure the validity of the items used. The validity can be referred to the raw variance explanation by measures of the items constructed. The result of raw variance showed that the accuracy of the items used in this approach was 63.7% which was good. Hence the items were considered valid to be used in the study. This is shown in Table 4. Rasch Measurement Model analysis measures item reliability and item separation. Table 5 shows the summary of 16 measured items of PC. It presents the values of item reliability and item separation which are 0.65 and 1.37 respectively. The results showed that the item reliability and person separation were low for PC. Apart from that, Table 5 shows that the infit and outfit z-standardised value of the study were 0.1 and 0, which were within the acceptable range (Fischl and Fisher, 2007). The analysis proved that teacher trainees rarely adopted the strategies of PC to teach writing even though the dimensionality of the items was considered valid. The result of raw variant explained by measure showed that the accuracy of the items of PC is good based on the OC. Rasch Measurement Model also analyses person reliability. Ten teachers who were the participants of the study were tested using Rasch Measurement Model on the ability of using PC to teach writing. Table 6 shows the summary of ten measured persons.  Table 6 presents the value of person reliability for PC which is 0.54 and person separation 1.09 respectively. Both the values were considered poor (Fischl and Fisher, 2007). Therefore, it was obvious that the teachers were not able to employ the strategies of PC. However, Table 6 presents the value of alpha Cronbach which is 0.69. This value proved that the study was considered reliable. So it was confirmed that the teacher trainees involved in the study were very reliable for the study. The values of infit and outfit z-standardised value of the study for person reliability were 0.2 and -0.1 which were within the acceptable range. Hence, the OC is valid for the study. The result of the analysis helped to achieve the first objective of the study.

The "3P" Writing Stages: An Alternative Framework in Writing Instruction
All the ten teachers were found to prepare writing lessons based on five teaching stages in teaching writing: set induction, presentation, practice, product and closure. Apparently, the teachers focused on writing activities during the "3P" Stages: Presentation, Practice and Production stages. The set induction and closure stages were used to get the students" attention towards the lesson. These two stages were also useful to recapitulate the lesson of the day (Swales and Feak, 2004). The classroom observation revealed that the real writing lesson took place during the "3P" Stages. Thus, data from the "3P" Stages were analysed thoroughly to investigate the ESL writing practices in identifying the preferred writing approach used by the teachers. The strategies of the PD and PC employed by the teachers in the "3P" Stages were categorised and analysed separately to make sure that none of the writing activities was neglected from the study. The categories of the analysis conducted is highlighted as a framework in Figure 1. Data from the OC assisted the analysis of the practices of the three modes: Product (PD), Process (PC) and Mix (PD and PC). The practice of the four strategies of PD and the eight strategies of PC were investigated throughout the "3P" stages of each writing lessons conducted the teachers. The strategies were identified and marked in the OC based on the items respectively. of A descriptive analysis was conducted to find out the preferred strategies practised by the teachers in various stages. The strategies portrayed the approach selected by them in teaching writing in ESL classrooms. Table 7 shows the frequency of approaches practised by the ten teachers during the "3P" stages in detail.  Table 7 shows the final results of the data analysis. First, the findings from OC show that the majority of the teachers incorporated familiarisation strategy of PD during the presentation stage. Second, the findings show that all the ten teacher trainees incorporated controlled writing strategy of PD in getting their students to practise the emphasised language component of the writing lesson during the practice stage. Third, the final results based on OC revealed that all the ten teacher trainees used guided writing and free writing strategies of PD dominantly in getting their students to produce and submit the final written task. Thus, it was confirmed that majority of the teacher trainees favoured to apply PD during Part 1 and Part 2 of the production stage. In conclusion, the findings show that the overall dominant approach practiced by the teachers to teach writing was the PD. Therefore, the second objective of this study is achieved.
The findings of this study confirms that all the selected teachers dominantly practised PD in teaching writing in various teaching stages in the ESL primary classrooms. However, there were several issues to be discussed based on the findings and earlier research done on the practice of teaching writing approach employed. The, previous researchers indicated that Malaysian students faced much constraints in the area of language learning especially in writing skills due to the ineffective instructional methods used (Mukundan et al., 2013) recommended PC writing approach as a way to overcome this situation (Al-Sawalha, 2014;Lo, 1996;Wirawati et al., 2013). It was obvious that the literature favoured PC but the teacher preferred to practice PD for the convenience of the teaching purposes. This result also supports the findings of previous studies (Maarof et al., 2011;Morita, 2004), that teachers in Malaysian schools preferably used PD in teaching writing due to various pedagogical factors that affect their teaching decisions.

Conclusion and Future Direction
The validation of the OC on the basis of Rasch Measurement Model testing was important a vital instrument used to investigate the approaches practised by the teachers to teach writing. The results of this study also supported the findings of previous studies that Malaysian ESL teachers used PD dominantly in teaching writing (Thulasi et al., 2015). The analysis further showed that PC was not practised by the teachers in abundance even though the constructs were validated by Rasch Measurement Model due to several factors. In conclusion, the Rasch model analysis helped in enriching the understanding of the application of product versus process writing approaches by the teachers. The summary scores colected based on the OC and the analysis using Rasch model showed that the teachers dominantly use product writing approach. The findings of this study provided evidicence that OC can be used in writing studies as a reliable instrument for deeper undertsanding of the writing instruction in other context by the language researchers and practitioners in ESL classrooms.
Finally, based on the results of this study we recommend the "3P" Stages writing framework for ESL teachers in teaching writing. This structure was found to be helpful in staggering the planning of the writing lessons more systematically. In this way, the teachers could envisage the overall writing lesson as well as the details of each stages with the type of activities selected for the class. Therefore, teachers would have room to include the type of approach and strategies they want to employ, be it fully PD, PC or a mixture of PD and PC.
The findings of the study revealed that the teachers preferred to use PD more dominantly than the PC for their own convenience. Research showed that PD can be useful in teaching writing especially for beginners (Macbeth, 2010). However, the PC had to be practised to teach students to become creative and independent writers in the long run. Therefore, we argue that PC has the potential to guide the writing teachers in helping students to produce better drafts. Thus, we recommend that teachers utilise this framework as a platform to plan and execute PC in teaching writing. Teachers may practice the PC strategies during the writing activities where the students would go through the three stages of writing as mentioned above. Teachers could plan to employ the strategies of PC respectively to make writing lessons more meaningful and useful for the students in the long run.