CRITERIA FOR EFFECTIVENESS
A. ACHIEVEMENT ANALYSIS
There is a relationship among school system effectiveness, the socioeconomic status of the community, and the educational levels of parents. Many studies have suggested that early childhood experiences affect learning and development, with children from impoverished environments generally achieving at lower levels than those from more enriching situations. An article in the April 21, 1997 issue of US News and World Report, for example, supports the notion that improving social and family conditions, particularly in the early years, can enhance development. In the late 1970s, neuroscientists discovered that children have a rapid increase in brain connectivity early in development. The research of pediatric neurologists suggests that formation of synaptic contacts in the human cerebral cortex occurs between birth and age ten. In other words, most of the brain gets built after birth. (See Peter R. Huttenlocher and Arun S. Dabholkar, “Regional Differences in Synaptogenesis in Human Cerebral Cortex,” The Journal of Comparative Neurology, 387:167-178 [October 20, 1997]). Higher intelligence is witnessed in individuals with more stimulation and protein in the early years of life. More recent documentation indicates that “age three seems more likely to mark only the beginning of a long developmental and maturational period during which environmental stimulation and experience do matter” (See John T. Bruer, “Neural Connections – Some You Use, Some You Lose,” Phi Delta Kappan, Vol. 81, No. 4, pp. 264-277, [December 1999]). Recent research has shown that the family’s social class has an effect on vocabulary growth and the development of language skills in children. Those from lower socio-economic classes tend be less fluent with language and have less capacity to teach their children language skills than those from higher socio-economic situations. These differentiations are quite pronounced by age three or four, and have significant implications for later cognitive and school performance. Effective schooling can reduce the consequences of lack of early exposure to strong language skills, and can, to some degree, equalize performance on language-based tasks. (See George Farkas and Kurt Beron, “Family Linguistic Culture and Social Reproduction: Verbal Skill from Parent to Child in the Preschool and School Years,” presented at the Session of Child Poverty and Deprivation, at the Annual Meeting of the Population Association of America, Washington D.C., 31 March 2001.) Achievement has also been associated with child rearing practices, assertive mothers generally rearing children who achieve at higher levels than those reared by less assertive mothers. (See Guy Odom, Mothers, Leadership, and Success, Houston: Polybius Press, 1989.) These concepts open up many opportunities for parents and schools to work together in improving learning potential. The effective school research has abundantly demonstrated that achievement levels between poor and affluent students can be narrowed considerably while increasing levels of achievement for all students. Nevertheless, the criteria of effectiveness are generally met at a higher level when student populations contain a small percentage of free and reduced lunch students (an indicator of poverty levels) and the educational level of parents is high school completion or above. We can generally assume that school systems which have better entry level characteristics than the state average will meet the criteria of effectiveness at a higher level than those whose entry level characteristics match or are below the state average. Entry level characteristics used for predicting student achievement levels are items such as:
In this audit, the items used to predict the levels at which the effectiveness criteria should be met by the School District of Osceola County include:
Educational researchers and auditors realize that predictions and judgments cannot be made from a single indicator. Therefore, the audit uses a multiple variable approach to making recommendations and arriving at conclusions. It is the same method used by various state departments of education to establish accountability and validate school effectiveness. It was initially developed by the South Carolina Department of Education (Division of Public Accountability) and has been used in several other states. If a number of variables indicate some trend, and all the variables are moving in the same direction, then the confidence level of the prediction, or the conclusion, is well established. This is a commonly accepted practice in educational auditing. To establish effectiveness levels for the School District of Osceola County, we used the following conditions, as reported by the school district for the 1999-2000 school year (sources noted if other than the school district):
With the above socio-demographic profile, and considering the large (and rapidly growing) size of the student population, the School District of Osceola County should establish the following effectiveness levels in order to place the district’s student and staff populations among the top 15-20% of performance for similar populations:
Section VIII, Effectiveness Levels, provides three significant categories for the entire school district and for each school:
Assuming the district’s student population has similar characteristics (e.g., poverty level, parent education level) at all grade levels, students should perform at equivalent or higher levels on the achievement tests given in different grades. This kind of consistent achievement at different grade levels may be thought of as “value added” achievement. When this does not occur, a more rigorous academic program should be provided at upper grade levels. The most appropriate way to gauge value added achievement in a school district is to follow each group of students as they progress from grade to grade throughout their entire twelve years of schooling. The SchoolMatch team encourages the leadership in the Osceola County School District to establish procedures to collect data and monitor progress in this way. In the absence of twelve years’ worth of data on one academic class, the SchoolMatch Audit team considered achievement data at each grade level for one year as a kind of “prism view” of performance across the grade levels in the Osceola County School District. The following discussion focuses on an analysis of student performance in the major subject areas of reading (Language Arts and Communications) and mathematics. In 1999-2000 and 2000-2001, students were tested in grade1 using the Stanford Early Achievement Test, Fourth Edition, and in grades 2-9 using the Stanford 9 Achievement Test, Ninth Edition. Tables 1A and 1B show performance across the grade levels in reading and mathematics, respectively for 1999-2000 and 2000-2001, compared to the cohort mean and effectiveness level.
Reading: In general, district-wide scores in reading increased across grades 1-9, after an initial 5+ national percentile rank drop in achievement between grades 1 and 2 (1999-2000) and a 14 national percentile rank drop in achievement between grades 1 and 2 (2000-2001). Further investigation of the alignment of the testing program to the scope and sequence of the primary grades reading and language arts curriculum may reveal information useful in more fully developing reading achievement potential demonstrated in grade 1 (see Graphs 1A and 1B). Observers will note that the grade 1 reading achievement level is not again attained until grade 7 (1999-2000) and not again until grade 9 (2000-2001). In the 1999-2000 school year, by grade 5, Osceola students are consistently achieving at a level comparable to the average performance in reading of their cohort group (48th percentile). By grade 9 in the 1999-2000 school year, reading achievement almost attains the effectiveness level (65th percentile), performance in the top 15-20% of student populations in the Osceola cohort group. In the 2000-2001 school year, by grade 8, Osceola students are consistently achieving at a level comparable to the average performance in reading of their cohort group (48th percentile). Value added achievement in reading occurs in the Osceola County Schools.
Mathematics: In mathematics, district-wide scores generally increase across the grade levels. Achievement in mathematics occurs more slowly, but also more evenly throughout the grade 1-9 continuum, with no significant differences in achievement from year to year, either positive or negative, for both years observed. By grade 4 in both the 1999-2000 school year and the 2000-2001 school year, Osceola students are consistently achieving at a level comparable to the average performance in mathematics of their cohort group. By grade 9 in the 1999-2000 school year, mathematics achievement is approaching the cohort effectiveness level demonstrated by the top 15-20% of comparison student populations, due to a significant increase in student achievement measured between grades 8 and 9 (61.9th). It is noted that the positive incremental increases in student achievement in mathematics are interrupted in grades 5 and 8 in both years observed. Value added achievement in mathematics also occurs in Osceola County Schools.
The Florida Comprehensive Assessment Tests (FCAT) were administered in Reading, Writing and Mathematics in 1999-2000 and in 2000-2001 throughout Florida. Scores have been reported as mean scale scores, raw scores, and percentage of students achieving in each of Levels 1-5. Because state-mandated tests vary greatly from state to state, it is not appropriate to compare results on the FCAT to results from other states on their statewide exams. Osceola students, as was required of all students in Florida, took the following criterion-referenced (Sunshine State Standards) tests in 1999-2000:
Osceola students, as was required of all students in Florida, took the following criterion-referenced (Sunshine State Standards) tests in 2000-2001:
Data to allow a conversion to percentage meeting or exceeding the standard in each subject at each grade level were not available as of this writing. Mean scale scores, however, offer a relative view of performance in Osceola County Schools when compared with Florida statewide results. Reading: Benchmark mean scale score numbers are different for each grade level. Observations, then, can be made between performance of Osceola students and performance of all students in the State of Florida on a grade-by-grade basis only. Crosssubject analyses are not appropriate because benchmark mean scale score numbers are, likewise, different for each subject. Although the FCAT was administered to students in grades 3-10 in year 2000-2001, for parallel comparison purposes, only performance in grades 5, 8, and 10 is shown. Osceola students, on average, perform below levels achieved by their Florida peers in reading as measured by the FCAT.
Mathematics: Benchmark mean scale score numbers are different for each grade level. Observations, then, can be made between performance of Osceola students and performance of all students in the State of Florida on a grade-by-grade basis only. Crosssubject analyses are not appropriate because benchmark mean scale score numbers are, likewise, different for each subject areas. Although the FCAT was administered to students in grades 3-10 in year 2000-2001, for parallel comparison purposes, only performance in grades 5, 8, and 10 is shown. Osceola students, on average, perform below levels achieved by their Florida peers in mathematics as measured by the FCAT.
Florida requires passage of all portions of the High School Competency Test (HSCT) as one of its graduation requirements. Results are published as percentage meeting or exceeding the standard for first-time test takers. Osceola County's cohort group of student populations achieves an average passage rate of 90% on criterionreferenced and standards-based tests. The top 15-20% of the comparison group achieves a passage rate of 94%. In 1999-2000, first-time test takers of the HSCT in Osceola County School District achieve passage rates in Communications of 45% to 74%, and in mathematics of 37% to 61%. 2000-2001 HSCT results were not available at the time of this writing. Upper Grade, Lower Grade Norm-Referenced and Standards-Based Test Results In general in the 1999-2000 school year, a parallel relationship appears in the development of achievement in both reading and mathematics from the primary years through grade 7. Achievement in reading on norm-reference tests is then more fully developed than achievement in mathematics in grades 8 and 9. Results of norm-referenced testing in 2000-2001 show positive incremental increases in student in reading grades 2-9. Achievement in mathematics vacillates positive and negative between grades 2-9, while generally moving in an upward direction. While criterion-referenced test results are limited to the High School Competency Test, the higher achievement levels in reading, language arts, and communication evidenced in norm-referenced outcomes also appear in results of the HSCT administrations, with a top-end 74% passage rate in Communications versus a top-end passage rate of 61% in mathematics. Overall, as observed in results of both kinds of test administrations across two school years, student achievement in reading/language arts is more fully developed by Osceola County Schools than student achievement in mathematics. Cognition is the process by which knowledge is acquired through the use and interpretation of environmental symbols. When groups of students achieve results that indicate aggregate performance at a particular level in one cognitive area, the group should be expected to achieve aggregate performance at that same level in other cognitive areas. In other words, groups of students can do just as well in one cognitive area as in any other cognitive area (e.g., reading and mathematics). Equalization of expectations of cognition among groups should not be applied to individuals within those groups.
The following discussion focuses on the major subject areas of reading and mathematics. Results from 1999-00 show that scores in mathematics on the SAT 9 exceed the cohort mean (48th percentile) beginning in grade 4 and approach the effectiveness level (65th percentile) in grade 9. Scores in reading achieve and even surpass the cohort mean in grade 1, and then do not again reach that level until grade 7. Scores in reading surpass mathematics scores in grades 7, 8, and 9, and more closely approach the effectiveness level in grade 9 than mathematics achievement in that same grade. Results from 2000-01 show that scores in mathematics on the SAT9 meet and then exceed the cohort mean (48th percentile) beginning, again, in grade 4. Student achievement in mathematics does show positive incremental growth after grade 4. While higher in grades 1-3, scores in reading are surpassed by scores in mathematics at grade 4. Reading achievement does, however, surpass mathematics achievement in grade 9. Overall, during the 1999-2000 school year, district-wide performance as measured by the SAT 9 is higher in reading than in mathematics, except in grades 4 and 6, as shown in Graph 3A. Likewise (see Graph 3B), for the 2000-2001 school year, district-wide performance as measured by the SAT9 is higher in reading than in mathematics, except in grades 4-8. Student achievement in both reading and mathematics is more fully developed during the 1999-2000 school year than are reading and mathematics achievement during the 2000-2001 school year. On the Florida High School Competency Test, student performance in communications is higher than in mathematics. Cross-Subject Norm-Referenced and Standards-Based Test Results While the data from these two testing instruments appear generally to agree in terms of cross-subject performance, it should be taken into consideration that the SchoolMatch team's observations are interrupted in the absence of criterion-referenced testing outcome data from earlier grades.
Benchmark mean scale score numbers on the standards-based Florida Comprehensive Achievement Test are different for each grade level. Observations, then, can be made between performance of Osceola students and performance of all students in the State of Florida on a grade-by-grade basis only. Cross-subject analyses are not appropriate because benchmark mean scale score numbers are, likewise, different for each subject areas. Performance of Osceola County students is generally lower in all grades and in all subjects than that of their Florida peers. The SAT 9, because it is a well-established test, and because it measures the performance of students from a much larger nationwide sample, may be a more reliable indicator of Osceola's ability to develop student achievement than criterion-referenced assessments. Osceola County School students may benefit from norm-referenced assessments throughout their high school career. District-wide performance on the ACT is between the cohort mean (18.4) and the cohort effectiveness level (20.4). On the SAT in 1999-2000, performance is below the mean (950) and effectiveness levels (986). (See graphs below).
Having an Advanced Placement program in the high school creates a stronger academic press for the students and higher expectations for student achievement. The ability of the district to be successful in this effort is related to its willingness to train its teaching staff and to establish a more rigorous curriculum. The means and effectiveness levels for enrollment, percent taking exams, and percent earning eligibility college credit are established for populations of high school Juniors and Seniors nationwide enrolled in AP classes. The enrollment of Juniors and Seniors in Advanced Placement courses achieves effectiveness.
The percent of juniors and seniors who take one or more AP exam achieved the effectiveness level in 1999-2000. Juniors and seniors taking the AP exams represent 5.2% of the total student population in grades 9-12.
The percent of junior and senior AP test-takers who were eligible for college credit by earning a 3, 4 or 5 on one or more AP exam was 36.10% in 2000-01. Graphs demonstrate that effectiveness level is not currently being met.
It is commendable that enrollment in AP courses and the number of students taking exams in the Osceola County Schools. The data indicate that interest in AP courses is high. District personnel should consider examining the reasons why only about one third of the students who take the exams score high enough to earn full or partial college credit. Grade Inflation Analysis An analysis of grade point averages indicates that, generally, the highest grade inflation occurs in the lowest achieving schools. This is suggestive of low expectations for those students who most need to have higher expectations. Schools with grade inflation tend to reward low quality work and grades may not reflect actual learning levels. The School District of Osceola County grade point average for the 1999-2000 senior class was calculated at 3.03. The mean (average) GPA for the cohort of similar student populations is 2.4, and the effectiveness level is 2.2.
An analysis of student performance data indicates that grade inflation does appear to exist in the School District of Osceola County. ACT and SAT scores for college bound students do not achieve effectiveness, and scores on statewide tests are consistently just at or only slightly above state averages. Performance on the FCAT-NRT in reading and mathematics, while near or above the mean in most grades, does not achieve effectiveness levels. In addition, only about one third (36.1%) of students who take AP exams achieve a score of 3, 4 or 5, needed to earn college credit for the AP courses. Given these observations, it appears to the Audit team that a GPA of 3.03 does not reflect the actual performance of Osceola County students. Achievement data suggest that a GPA closer to the cohort mean would be warranted. To establish effectiveness for overall Grade Point Average, the district should continue or establish the following practices:
The benefits of rewarding high quality work and holding high expectations for all students is generally evidenced in lower overall grade point average, higher performance on college entrance examinations, and a capacity to achieve at higher levels in Advanced Placement and other rigorous academic programs. B. ATTENDANCE ANALYSIS
The average student attendance rate during the 1999-2000 school year was 94%. The effectiveness level is 95% and the mean is 92%. District-wide student attendance levels for this school year approach, but do not quite achieve, effectiveness.
With some effort, the district should be able to attain effectiveness in this area. Disclosure of attendance goals to students and parents is often helpful in attaining higher levels of student attendance. Examination of practices in schools and school systems which have achieved effective student attendance percentages may reveal methods to be applied to increasing student attendance in the district. The average number of days of teacher absence in the School District of Osceola County Schools for the 1999-2000 school year was 8.88 days per teacher 4. The effectiveness level is 5 days per teacher per year, and the mean is 7 days. Generally speaking, the higher the teacher absence rate, the less effective is a school system (e.g., continuity of instruction; monitoring student achievement; positive attendance role modeling for students; cost of substitute teachers depleting funds which could be used for instruction). Typically, only discretionary absences are included in the calculation of the teacher absence rate (see Appendix F). Data on consecutive days of absence were not provided to SchoolMatch, Currently, the teacher absence rate in the School District of Osceola County is above the mean, and has not yet achieved effectiveness.
Teacher attendance may be improved by publicizing absence rates, designing policy to recognize and reward teacher attendance, and implementing the district adopted from buildings where teacher attendance is effective. C. DROPOUT INFORMATION
The dropout rate in this audit is established for grades 9-12 on a one-year basis. In the School District of Osceola County, the one-year dropout rate for the 1999-2000 school year decreased from 6.1% in 1998-99 to 5.5%5. In the 2000-2001 school year, an even larger decrease in the dropout percentage occurred.
The effectiveness level for this demographic group is 2.6% or lower. The mean dropout rate for the cohort is 2.9%. The one-year drop out rate in Osceola County does not attain either the Mean or Effectiveness Level. It is noteworthy, however, that over the past five school years, Osceola County schools has been able to control dropout percentages and, in fact, implement strategies to cause a 1.3% decrease in the dropout percentage between the 1999-2000 and 2000- 2001 school years.
The most effective ways, overall, to reduce the number of students who drop out of school are to provide ways for students to succeed based on concrete, attainable goals and to promote participation in school activities. The principal bases for student success are the ability to read at or very near grade level and to receive effective instruction. Efforts to improve student achievement in reading will have short and long-term benefits in overall student success and, all other things equal, will have a positive effect on the school’s holding power over high school students. In addition to those initiatives currently practiced and contributing to decreasing dropout percentages over the past three academic years in the Osceola County Schools, these specific strategies may be effective in increasing the numbers of students who complete a high school education in the School District of Osceola County:
|
