A Multidimensional Analysis of Teacher Preparation in Texas

Despite a healthy production of teachers, teacher attrition is a significant concern facing school administrators across the state of Texas. This study sought to determine the extent to which questions on the Exit and Principal Surveys reflect three sets of standards which guide educator preparation in Texas: Association of Teacher Educators (ATE) Standards for Teacher Educators, Texas Administrative Code (TAC) Educator Preparation Program Curriculum Standards, and Pedagogy and Professional Responsibilities (PPR) Exam Standards. This analysis provides important information about the validity of survey questions as a measure of standards compliance for educator preparation in Texas and also sought to determine if there is a difference between teacher candidates’ Exit Survey evaluation of preparation and principals’ Principal Survey evaluation of first-year teachers’ preparation. Findings indicated a clear disconnect. Recommendations are provided as contributions for future discussion on much needed educator preparation program standards reform.


Introduction
In 2018, educator preparation programs across the state of Texas prepared and certified 87,680 new teachers in areas including elementary and bilingual education, career and technical education, computer science, foreign languages, special education, and the four core areas: mathematics, science, social studies, and English (Texas Education Agency, 2019). Despite a healthy production of teachers, teacher attrition is a significant concern facing school administrators (Carver-Thomas and Darling-Hammond, 2019; Sass et al., 2011). In Texas, attrition of all teachers was markedly higher for the smallest school districts than for the largest. Further, attrition of beginning teachers was more than twice as high as for veteran teachers (Ramsay, 2018). Research finds, however, that these realities are not novel as shortages in core subject areas date as far back as the 1930s (Sutcher et al., 2019). Moreover, the problem is not so much production as it is retention. In fact, almost one-third of new teachers leave the field within the first three years and almost half of all new teachers leave the field after the first five years (Darling-Hammond, 2003;Sutcher et al., 2019). Ingersoll et al. (2014), contend that the linkage between quality teacher preparation and teacher retention is real. The researchers' analysis found that above other characteristics such as the type of educator preparation program, there were significant linkages between differences in substance, quality, and design of teacher preparation and the degree to which teachers leave their assignments. Thus, contrary to popular assumptions, teacher shortages are not so much a result of too few teachers but of poorly trained or inadequately prepared teachers who leave the profession long before retirement (DeAngelis et al., 2013;Ingersoll et al., 2014).
According to Furlong et al. (2013), there is an assumption in educator preparation program standards development that the standards established will sufficiently parallel content standards established for K-12 classrooms. The effects of this assumption are damaging as, without a thorough investigation to ensure alignment, they create the implied disconnect between theory and practice evidenced in research. Matched with the critical attention of various education stakeholders such as parents, legislators, and employers, these trepidations justify a thorough evaluation of educator preparation program standards and performance. Unfortunately teacher preparation exists in a policy shadow of other education goals and priorities. Policy which fails to prioritize teacher preparation as an important component to facilitate student achievement perpetuates the disconnect between theory and practice evidenced in research. Teacher preparation is too important to dismiss as secondary to the goals of education. Instead it is a vital piece of the conversation on school and learning improvement strategies (Tobery-Nystrom, 2011).
Educator preparation programs in Texas are held to an extensive accountability system established by the Texas Education Agency (TEA) and State Board for Educator Certification (SBEC) (19 Tex Adm Code Ch, §229.4,, 2019). Chapter 229 of the Texas Administrative Code (TAC) describes the Accountability System for Educator Preparation Programs (ASEP) (19 Tex Adm Code Ch, §229.4,, 2019). Rule 229.4 provides details as to the process of accreditation, outlining requirements such as certification exam pass rates, first-year teacher appraisals performed by hiring administrators, and the documentation of frequency, duration, and quality of field-based supervision for firstyear teachers (19 Tex Adm Code Ch, §229.4,, 2019).
One component of the accountability system includes the utilization of surveys designed to measure educator preparation program effectiveness. The Educator Preparation Program Candidate Exit Survey (Exit Survey) is completed by teacher candidates upon completion of an educator preparation program (Texas Education Agency, 2019). Questions on the survey ask teacher candidates to rate the level of preparation provided by their educator preparation programs. Respondents complete this survey as a part of the application process for a teaching certificate and therefore have not yet served as a teacher of record. However, evaluation of their preparation may be based on field-based and clinical teaching experiences required in educator preparation program training.
The Teacher Preparation Effectiveness Survey: First-Year Teachers (Principal Survey) survey completed by supervising principals is administered through the TEA Educator Certification Online System (ECOS) late in the spring semester of each academic year (Texas Education Agency, 2017). The Principal Survey is completed and submitted by principals supervising first-year teachers or by a designee such as an assistant principal or department chair. Questions on the Principal Survey mirror the Exit Survey and ask the respondent to rate the level of preparation of first-year teachers based on their performance during their first year in the classroom. Surveys are only completed for first-year teachers who have been on the campus for five months or longer. Information about the educator preparation program from which the first-year teacher emerged is pre-populated by the TEA (Texas Education Agency, 2017).
The purpose of this study was two-fold. First, this study sought to determine the extent to which survey questions on the Exit Survey and Principal Survey reflect three sets of standards which guide teacher preparation in Texas: Association for Teacher Educators (ATE) Standards for Teacher Educators, TAC Educator Preparation Program Curriculum Standards, and Pedagogy and Professional Responsibilities (PPR) Exam Standards; (19 Tex Adm Code Ch 228, 2019); Pedagogy and Professional Responsibilities Standards (EC-12), 2015). As an accountability tool, survey questions must align with standards established for teacher preparation. This analysis provides important information about the validity of survey questions as a measure of standards compliance.
Second, this study sought to determine if there is a difference between teacher candidates' Exit Survey evaluation of preparation and principals' Principal Survey evaluation of first-year teachers' preparation in the areas of classroom environment, instruction, students with disabilities, limited English proficient students, technology integration, and use of technology with data. Results of this analysis contribute to research on the theory vs. practice disconnect and other potential deficiencies in teacher preparation. Moreover, this study contributes to the discussion on minimum proposed standards in teacher preparation. Findings may provide important feedback for policymakers regarding the legitimacy of surveys as an indicator of teacher effectiveness and as a guide for educator preparation program improvement.

Educator Preparation Program Curriculum Standards
It is the position of the Texas Education Agency that in order to be prepared for teaching assignments in Texas classrooms, teacher candidates as future educators must be well prepared according to high standards (Educator Preparation Home, 2015). Accordingly, the agency in cooperation with SBEC has developed educator preparation programs standards outlined in Title 19 Part 7 Chapter 228 of the TAC (19 Tex Adm Code Ch, §229.4,, 2019). This section of the code has seen several revisions. Initially adopted in 1999, educator preparation program standards were revised in 2003, 2008, in 2014 after the adoption of Chapter 149, in 2016 upon legislation requiring instruction in the detection of dyslexia, mental health, substance abuse, and youth suicide was added, and most recently in 2018 upon a revision to require the incorporation of new technology standards into educator preparation program curriculum (19 Tex Adm Code Ch, §229.4,, 2019).
In January 2014, the Commissioner of the Texas Education Agency proposed the Commissioner's Rules Concerning Educator Standards, Chapter 149 Subchapter AA Teacher Standards. Originally designed to guide the development of a new teacher evaluation system in Texas, the standards also articulate performance standards with the goals of improving instructional practices, student learning, and achievement (13_11 Proposed New 19 TAC 149.1001. Chapter 149 was accepted and became effective in March 2014. The TEA and SBEC later proposed revisions to educator preparation program standards outlined in Chapter 228 to adhere to these new guidelines. Drafted with intentions to reflect the TEKS, and now also considering the standards outlined in Chapter 149, the code now specifies that in order to foster teacher effectiveness, educator preparation program curriculum should be built using scientifically based research. Moreover, requirements listed in the administrative code prescribe that coursework and training should be sustained, rigorous, interactive, student-focused, and performance-based (19 Tex Adm Code Ch, §229.4,, 2019). All educator preparation programs in Texas are required to provide training and preparation in the following areas:  Reading instruction, including instruction that improves student content-area literacy  The code of ethics and standard practices for Texas educators, pursuant to Chapter 247 of the TAC  The skills and competencies captured in the Texas teacher standards (found in Chapter 149) which include:  Instructional planning and delivery  Knowledge of students and student learning  Content knowledge and expertise  Learning environment  Data-driven practice  Professional practices and responsibilities  Instruction in detection and education of students with dyslexia  Instruction in detection of students with mental or emotional disorders  Instruction in digital learning, including a digital literacy evaluation followed by a prescribed digital learning curriculum

Inconsistencies in Educator Preparation Programs Design
Across the nation, there are approximately 1,400 colleges and universities which house educator preparation programs. Countless other alternative certification program options are also approved to certify teachers (Zeichner and Bier, 2013). In Texas alone, as of the 2017-2018 academic year accreditation ratings report, there are 134 educator preparation programs operating in the state of Texas as either traditional undergraduate programs or alternative certification programs (Texas Education Agency, 2019). Under economic pressures and in an effort to reduce barriers to the teaching profession, alternative certification programs offer a fast-track route to teacher certification (Darling-Hammond, 2006). In the process of reducing barriers and decreasing costs generally associated with training and coursework, educator preparation programs find flexibility in program development and admission requirements Henry et al. (2014). This flexibility creates inconsistencies in teacher training giving cause for concern as to the quality of preparation teachers receive. In fact, across the nation, there are over 1100 different licensure exams for teacher certification and even with the same tests, different standards for passing scores in different states (Greenberg et al., 2013).

Policy Complications
In Texas, all educator preparation programs regardless of classification as traditional undergraduate or alternative are required to follow the curriculum guidelines established in Chapters 227 and 228 of the TAC Title 19 Part 7 (19 Tex Adm Code Ch 228, 2019).
These requirements for educator preparation programs are established as the minimum standards for teacher preparation. Programs however may require coursework and training in addition to the standards established, and may also outline admission requirements that go above those listed. Chapter 228 mandates programs to require at least thirty clock hours of in-field observation prior to the clinical teaching period and at least twelve weeks of full days in the classroom as a student teacher. However, some programs in the state incorporate into their program design up to eighty clock hours or more of classroom observation and anywhere from fifteen weeks to one year of clinical teaching experience.

Teacher Competency Assessment
The Texas Examinations of Educator Standards (TExES) tests are criterion-referenced examinations designed to measure a candidate's content and pedagogical knowledge in relation to an established criterion rather than to the performance of other candidates. All of the tests in the TExES program contain multiple-choice questions. Some tests also have additional types of questions, such as oral or written responses based on the nature of the tested competencies (Texas Educator Certification Examination Program, 2019). These timed examinations require a passing scaled score of 240 out of a possible 300 points.
To qualify for teacher certification candidates must pass a minimum of two TExES examinations: the content area exam which matches the certificate field and the grade level authorization sought and the Pedagogy and Professional Responsibilities (PPR) written for grades Early Childhood (EC) -12 (19 Tex Adm Code Ch 230, 2019). If the candidate is pursuing a language-based certificate (i.e., Spanish, Bilingual, etc.) he/she must also pass a language proficiency test to qualify for initial certification.

Educator Preparation Program Accountability
In 2009, Senate Bill 174 was introduced in the 81 st Texas Legislative session (19 Tex Adm Code Ch, §229.4,, 2019). The bill called for more accountability measures of higher education institutions in the state, specifically those acting as educator preparation programs, and was used as a framework to rebuild Chapter 229 of the TAC Title 19 Part 7. Chapter 229 describes the Accountability System for Educator Preparation Programs (ASEP), the annual accreditation process for educator preparation programs in Texas. Rule 229.4 provides details as to the process of determination, outlining requirements such as pass rates (the percent of students who pass certification exams) recognized as Standard I, first-year teacher appraisals performed by hiring administrators (Standard II), the performance of beginning teachers (Standard III), and the documentation of frequency, duration, and quality of fieldbased supervision of first-year teachers (Standard IV) (19 Tex Adm Code Ch, §229.4,, 2019).
To accomplish the goals of Standard II and the Consumer Information Website, the Educator Preparation Program Candidate Exit Survey (Exit Survey) was developed. Teacher candidates complete the Exit Survey upon completion of their educator preparation program at the point of application for a Standard Classroom Teacher Certificate. Survey questions are grouped into the following categories:

2.5a. Classroom Environment
Definitions provided on the Exit Survey explain that classroom environment is a close relationship between the teacher and students that is characterized by polite, respectful, warm, and caring interactions that reflect an understanding of the students' cultural and developmental differences (Texas Education Agency, 2019). Facilitated through strategic utilization and of physical space and planning of instructional goals (Klahr, 2014;Roskos and Neuman, 2011), appropriate teacher-student relationships can nurture a stronger sense of job satisfaction for teachers, and a higher level of engagement from students (Wubbels and Brekelmans, 2012).

2.5b. Instruction
The fundamental purpose of instruction is to facilitate learning which is measured by outcomes or objectives (Caffarella and Daffron, 2013). On the Exit and Principal Surveys, instruction is defined as that which integrates critical thinking, inquiry, problem solving, differentiation, formative assessment, modeling, self-reflection, and standards alignment (Texas Education Agency, 2019).

2.5c. Students With Disabilities
A child is considered a student with disabilities if s/he has a physical, cognitive, behavioral, or other related impairment (19 Tex Adm Code Ch, §89.1001, 2001. The Texas Education Code specifies that students qualified for special education services include those with diagnosed autism, visual or auditory impairments, emotional disturbances such as oppositional defiant disorder, intellectual disability, orthopedic or other health-related impairment, learning disabilities, traumatic brain injury, or other non-categorical disabilities (19 Tex Educ Code Ch, §89.1040.

2.5d. Limited English Proficient Students
In the state of Texas, limited English proficient students are those whose primary language is other than English and whose English language skills are such that the student has difficulty performing ordinary classwork in English (19 Tex Educ Code Ch, §29.052,, 2011). These students are also known as English Language Learners. On various state reports they may also be identified as English as a Second Language or Bilingual Education students. Citing continuously increasing rates of immigration as a factor, the number of students classified as limited English proficient has grown exponentially each year (Chin et al., 2013).

2.5e. Technology Integration
On the Exit and Principal Surveys, questions related to technology integration pertain to integration of technology-based classroom learning opportunities to support student learning (Texas Education Agency, 2019). Technology integration encompasses not only the practice of incorporating technological resources into instruction and delivery, but in planning, management and assessment as well Al Musawi (2011). In fact, questions on the Exit and Principal Surveys distinguish between an evaluation of preparation in integrating available technology effectively into curricula and instruction and an evaluation of preparation to use available technology with data to increase student achievement (Texas Education Agency, 2019).

2.5f. Use of Technology With Data
In response to increasingly available technologies and growing standards of accountability, data-driven instruction is an instructional planning concept which incorporates the collection, analysis of, and response to student performance data as an approach to improve student learning (Datnow and Hubbard, 2015). On the Principal and Exit Surveys, questions on use of technology with data focus on the teachers' ability to analyze student performance data then respond by designing instruction based on that information.

Methodology
Following approval from the Institutional Review Board (IRB), data was collected from the TEA via a Public Information Request (PIR) as to survey responses on the Exit Survey and Principal Survey for the years 2011-2013, and responses on the Principal Survey for First-Year Teachers for the years 2011-2013. The PIR specified that survey response data should be deidentified to maintain respondent confidentiality. The rationale for choosing only the 2011-2013 years was due to the pilot year being excluded from analysis and to ensure that data collected is finalized and verified. All existing survey response data collected from the TEA was randomized for analysis.
In this study, there were two factors or independent variables tested: Exit Survey responses by teacher candidates and Principal Survey responses by principals. There were six dependent variables in this study: preparation in the areas classroom environment (1), instruction (2), students with disabilities (3), limited English proficient students (4), technology integration (5), and use of technology with data (6). To complete the analysis using these dependent variables, survey questions were grouped according to each category. Likert scale survey responses were assigned values as follows: well-prepared =1, sufficiently prepared= 2, not sufficiently prepared = 3, and not at all prepared = 4. Survey responses were totaled for interval scores then included in the analysis.
For the first portion of the study the researcher identified key words in each survey question that were determined to capture the targeted competency evaluated. To enable analysis of descriptive statistics, the number of instances in which each question reflected a standard was tallied. Survey response data provided by the TEA in response to the PIR resulted in 50,126 survey responses between both groups over the two-year collection period. A random sample of all Principal and Exit Survey responses submitted in the years 2011-2013 was included in this analysis. The sample included a total of 400 surveys: 100 Principal Surveys from 2011-2012, 100 Principal Surveys from 2012-2013, 100 Exit Surveys from 2012-2012, and 100 Exit Surveys from 2012-2013. Survey response options were listed and reported by the TEA as follows: wellprepared =1, sufficiently prepared= 2, not sufficiently prepared = 3, and not at all prepared = 4. Thus, a higher score indicates low evaluation of preparation.
A one-way multivariate analysis of variance (MANOVA) was conducted to determine if there was a difference between teacher candidates' Exit Survey evaluation of preparation and principals' Principal Survey evaluation of first-year teachers' preparation in the areas of classroom environment, instruction, students with disabilities, limited English proficient students, technology integration, and use of technology with data.

Results and Discussion
The results of this analysis, shown in Table 2, indicate that questions on the Exit and Principal Survey reflect PPR examination standards most (M= 3.15,Median=2.77,SD= 11.13). This set of standards outlines in detail the knowledge and skills required of teachers on various levels of teacher competencies. As such, proficiency standards may more closely address the expectations of classroom readiness and performance questions listed on the Exit and Principal Surveys.
TAC Educator Preparation Program Curriculum Standards guide educator preparation programs as the standards for teacher preparation curriculum and training. Results of this analysis shown in Table 1 suggest that this set of standards is minimally reflected in questions on the Exit and Principal Surveys (M= 1.15, Median=1.00, SD= .667). Similarly, ATE Standards for Teacher Educators are minimally reflected in survey questions (M= 1.06, Median=1.00, SD= .827). The results suggest that PPR standards and questions on the Exit and Principal Surveys are better related in how they capture classroom experience and performance expectations. TAC and ATE standards, which guide educator preparation program curriculum and delivery, are minimally reflected, giving rise to the concern of a disconnect between theory and practice.  Table 2 illustrates descriptive statistics results for each category of questions on the surveys compared to the three sets of standards. Here again, TAC (M= 1.00, Median=1.00, SD= .707) and ATE (M= .60, Median=1.00, SD= .548) standards for educator preparation programs and teacher educators fail to connect with practice to the extent measured on Exit and Principal Surveys. Technology standards were quite well reflected in survey questions, though mostly regarding data-driven instructional and assessment design generated by use of technology with data (M= 4.75,Median=4.50,SD= 2.50). Although growing trends in public schools call for greater attention to special populations and differentiated learning, the results of this analysis indicate that the three sets of standards do not address or assess these classroom and student needs to the extent necessary. Survey questions regarding students with disabilities were found to only minimally reflect in ATE (M= 1.43,Median=2.00,SD= .976),TAC (M= 1.57,Median=1.00,SD= .787) and PPR exam standards (M= 1.29,Median=0.00,SD= 1.70). Standards regarding limited English proficient students were even less well reflected in survey questions: ATE (M= 1.20, Median=2.00, SD= 1.10), TAC (M= 1.20,Median=1.00,SD= .447),and PPR exam (M= .80,Median=0.00,SD= 1.304).
The results for the second portion of the study indicate that Principal Survey evaluation of first-year teachers' preparation in the area of classroom environment (M= 2.54, SD= 1.023, N=200) was rated lower than teacher candidates' Exit Survey evaluation of preparation in the area of classroom environment (M= 1.27,SD= .414,N=200). Principal Survey evaluation of first-year teachers' preparation in the area of instruction (M= 3.24,SD= .575,N=200) was also lower than teacher candidates' Exit Survey evaluation of preparation (M= 1.27,SD= .394,N=200). Likewise, Principal Survey evaluation of first-year teachers' preparation in the areas of students with disabilities (M= 2.62, SD= .901, N=200), English language learners (M= 2.61, SD= .905, N=200), technology integration (M= 2.59,SD= .966,N=200), and use of technology with data (M= 2.61, SD= .922, N=200) were all rated lower than teacher candidates' Exit Survey evaluation of preparation in the areas of students with disabilities (M= 1.47, SD= .509, N=200), English language learners (M= 1.47, SD= .535, N=200), technology integration (M= 1.31,SD= .441,N=200), and use of technology with data (M= 1.44,SD= .561,N=200).
A one-way MANOVA revealed a significant multivariate main effect for survey respondents, Wilks' λ= .197,F(6,393)= 266.79,p < .05. The multivariate ƞ2 based on Wilks' λ was strong, .803. This indicates that 80% of the difference in variance of the areas of preparation can be accounted for by the difference in position which appears to be a very large effect. Given the significance of the overall test, Analysis of variance (ANOVA) tests on classroom environment, instruction, students with disabilities, limited English proficient students, technology integration and use of technology with data were conducted as follow-up tests to the MANOVA.
The results of these post hoc analyses illustrated in Table 3 revealed that all six areas of preparation were significant. These findings indicate there is a significant difference between teacher candidates' Exit Survey evaluation of preparation and principals' Principal Survey evaluation of first-year teachers' preparation in the areas of classroom environment, instruction, students with disabilities, limited English proficient students, technology integration, and use of technology with data during the years 2011-2013 in the state of Texas. Principal survey evaluation of firstyear teachers in areas of preparation were rated lower than teacher candidates' evaluation of preparation in all areas of preparation. Therefore, the null hypotheses are rejected.

Conclusion
These results suggest that Exit and Principal Survey questions used to determine program effectiveness and as an accountability tool are not in line with the standards guiding educator preparation programs. The conclusion here however is not that the Exit and Principal Surveys are asking the wrong questions. Rather, it may be inferred that there is a disconnect between theory and practice. ATE, TAC, and PPR standards detail the curriculum and professional development requirements for educator preparation and training, but survey questions intended to evaluate teacher effectiveness ask questions about practice. As these results suggest, the fact that educator preparation program standards and survey questions are not aligned only serve to illustrate how misguided teacher training may be compared to the demands of the teaching profession. Further, speculation might suggest that choices about the sources from which current standards originate may too be misguided.
Altogether, the results of this study revealed a significant multivariate main effect for survey respondents where 80% of the difference in variance of the areas of preparation can be accounted for by the difference in position which appears to be a very large effect. Follow up tests for individual areas of preparation were conducted. Results of these tests further emphasized the significant disparities between teacher candidate and principal evaluation of preparation in all six areas, but most significantly so in the area of instruction.
Principal Survey evaluation of first-year teachers' preparation in the area of classroom environment was rated significantly lower than teacher candidates' Exit Survey evaluation of preparation in the same area. In fact, teacher candidates rated themselves almost completely well-prepared for these scenarios at a mean of 1.27, where principals rated teachers only somewhere between sufficiently prepared to not sufficiently prepared at a mean of 2.54. For instruction, the disparity was even greater, where teacher candidates believed again that they were almost completely prepared (M= 1.27) but principals believed them to be quite poorly prepared (M= 3.24). In this first analysis of this study results indicated that the most well reflected standards in Exit and Principal Survey questions were those pertaining to classroom environment and instruction. When evaluated in real classroom settings however, principals reported that teachers are not well prepared.
Based on the findings of this study, the following are proposed as contributions for future discussion on teacher preparation reform: expand practical experiences to afford teacher candidates immersive, diverse, and engaging practice in real classroom settings wherein candidates can build a diverse repertoire of instructional strategies; develop and maintain strong and intentional partnerships with schools to enable more realistic and more comprehensive classroom learning experiences for the professional development of the teacher candidate and educator preparation programs alike; data-driven strategies for continuous improvement in educator preparation program curriculum and delivery; align educator preparation program standards with evaluation standards found on the Texas Teacher Evaluation and Support System (T-TESS).
Results of this analysis found significant differences between the two groups in all six evaluated areas of preparation. The conjecture is that teacher candidates are leaving their educator preparation programs with the naïve assumption that they are thoroughly prepared for classroom experiences, validating again concerns of a disconnect between theory and practice. Educator preparation program standards need to be developed with a more conscious understanding of the realities of classroom experiences and student demographics, and vitally inclusive of teacher competencies most critical to success in the classroom.