CRITERIA FOR EFFECTIVENESS

There is a relationship among school system effectiveness, the socioeconomic status of the community, and the educational levels of parents. Many studies have suggested that early childhood experiences affect learning and development, with children from impoverished environments generally achieving at lower levels than those from more enriching situations. An article in the April 21, 1997 issue of US News and World Report, for example, supports the notion that improving social and family conditions, particularly in the early years, can enhance development.

In the late 1970s, neuroscientists discovered that children have a rapid increase in brain connectivity early in development. The research of pediatric neurologists suggests that formation of synaptic contacts in the human cerebral cortex occurs between birth and age ten. In other words, most of the brain gets built after birth. (See Peter R. Huttenlocher and Arun S. Dabholkar, "Regional Differences in Synaptogenesis in Human Cerebral Cortex," The Journal of Comparative Neurology, 387:167-178 [October 20, 1997]). Higher intelligence is witnessed in individuals with more stimulation and protein in the early years of life. More recent documentation indicates that "age three seems more likely to mark only the beginning of a long developmental and maturational period during which environmental stimulation and experience do matter" (See John T. Bruer, "Neural Connections – Some You Use, Some You Lose," Phi Delta Kappan, Vol. 81, No. 4, pp. 264-277, [December 1999]).

Recent research has shown that the family’s social class has an effect on vocabulary growth and the development of language skills in children. Those from lower socio-economic classes tend be less fluent with language and have less capacity to teach their children language skills than those from higher socio-economic situations. These differentiations are quite pronounced by age three or four, and have significant implications for later cognitive and school performance. Effective schooling can reduce the consequences of lack of early exposure to strong language skills, and can, to some degree, equalize performance on language-based tasks. (See George Farkas and Kurt Beron, "Family Linguistic Culture and Social Reproduction: Verbal Skill from Parent to Child in the Preschool and School Years," presented at the Session of Child Poverty and Deprivation, at the Annual Meeting of the Population Association of America, Washington D.C., 31 March 2001.)

Achievement has also been associated with child rearing practices, assertive mothers generally rearing children who achieve at higher levels than those reared by less assertive mothers. (See Guy Odom, Mothers, Leadership, and Success, Houston: Polybius Press, 1989.)

These concepts open up many opportunities for parents and schools to work together in improving learning potential.

The effective school research has abundantly demonstrated that achievement levels between poor and affluent students can be narrowed considerably while increasing levels of achievement for all students.

Nevertheless, the criteria of effectiveness are generally met at a higher level when student populations contain a small percentage of free and reduced lunch students (an indicator of poverty levels) and the educational level of parents is high school completion or above. We can generally assume that school systems which have better entry level characteristics than the state average will meet the criteria of effectiveness at a higher level than those whose entry level characteristics match or are below the state average.

Entry level characteristics used for predicting student achievement levels are items such as:

In this audit, the items used to predict the levels at which the effectiveness criteria should be met by the Portsmouth School Department include:

Educational researchers and auditors realize that predictions and judgments cannot be made from a single indicator. Therefore, the audit uses a multiple variable approach to making recommendations and arriving at conclusions. It is the same method used by various state departments of education to establish accountability and validate school effectiveness. It was initially developed by the South Carolina Department of Education (Division of Public Accountability) and has been used in several other states.

If a number of variables indicate some trend, and all the variables are moving in the same direction, then the confidence level of the prediction, or the conclusion, is well established. This is a commonly accepted practice in educational auditing.

To establish effectiveness levels for the Portsmouth School Department, we used the following conditions, as reported by the school department for the 1998-99 school year (sources noted if other than the school department):

• percent of student population on free and reduced lunch is 3.5%;
• percent of students whose principal language is not English is .5%;
• percent of school system residents living in poverty is 3.6%;
  (Source: SchoolMatch databases; US Census data)
• percent of school system families receiving public aid is 1.07%;
  (Source: SchoolMatch databases; US Census data)
• the average per capita income in the community is $18,531;
  (Source: SchoolMatch databases; US Census data)
• the education level of school system residents is 13.2 years of schooling;
  (Source: SchoolMatch databases; US Census Data)
• the general fund expenditure per pupil, excluding capital funds is $7173;
• percent of the student population deemed ready for kindergarten is 93%;
• percent of student population performing on grade level in grades 1-3 is 76%;
• percent of teaching staff having a Master's degree or higher is 60%;
• percent of teaching staff having temporary certificates or letters of authorization is 2%

With the above socio-demographic profile, the Exeter Region Cooperative School District -- SAU 16 should establish the following effectiveness levels in order to place the department’s student and staff populations among the top 15-20% of performance for similar populations:

• The grade point average of the senior class should be 2.6 (non-weighted).
• The District attendance rate (ADA/ADM) should be 96.5%.
• Achievement on norm-referenced tests for Total Reading should be at—or above—the 84th percentile.
• Achievement on norm-referenced tests for Total Mathematics should be at—or above—the 84th percentile.
• On criterion-referenced tests, 98% of the student population should achieve mastery in both Reading and Mathematics.
• 20% of the juniors and seniors should be enrolled in the Advanced Placement program. Of all juniors and seniors, 16.1% should be taking at least one AP examination. Of the students taking AP examinations, 80% should be eligible for college credit.
• The one-year drop-out rate should be .6% or lower;
• The drop-out rate, cumulative grades 9-12, should be 2.4% or lower.
• The Teacher Absence Index (TAI) should be at or below 4 days (See Appendix C).
• On the ACT, the school system mean should be 24.6 or higher.
• On the SAT, the school system mean should be 1031 or higher.
• The average of the responses for each correlate of effectiveness for parents, teachers, and administrators should be at, or below, 2.00. (See Section V, Correlates of Effectiveness).

Section IV, Effectiveness Levels, provides three significant categories for the entire school department and for each school:

1. The mean (AVERAGE) performance for the comparison population for each variable;
2. The performance standard when effectiveness is achieved in the school department at one standard deviation above the mean (the EFFECTIVENESS LEVEL); and
3. The current status of variables in the Portsmouth School Department and individual schools within the School Department.

2. New Hampshire Educational Improvement and Assessment Program (NHEIAP)Test Analysis

Because the Exeter Region Cooperative District--SAU 16 does not utilize nationally norm-referenced standardized tests, the SchoolMatch audit team examined the results of the 1999 New Hampshire Educational Improvement and Assessment Program (NHEIAP) test scores from the entire state of Hew Hampshire based on demographic indicators that have been shown to have impact on student success on standardized tests. We present in this section an analysis of the performance of Exeter’s students on the 1999 NHEIAP as compared to populations of New Hampshire students with similar demographic characteristics.

The SchoolMatch audit team recognizes the importance of parent involvement in motivating their children to learn. We further recognize the success which many parents of limited education and income have demonstrated in motivating their children to do well in school. Unfortunately, national databases do not reflect this motivational factor for individual homes. Consequently, the team relies on data which most closely parallel research on demographic impact.

Significant research has indicated that the education level of adults with whom youngsters have contact, particularly mothers, and the percentage of children living in poverty within a school community have a great impact on a school system’s ability to produce high achievement on standardized tests. Therefore, SchoolMatch auditors and researchers rely on these and other related demographic characteristics to conduct their analyses (see Section I for a discussion of demographic features evaluated).

SchoolMatch researchers analyzed the demographic characteristics of the student populations in each of the school systems in New Hampshire, and identified statewide groups of school system and school populations with similar characteristics (the cohort group). We then examined NHEIAP test data from every school system and school in the state of New Hampshire, and calculated the average performance level (mean) within each of the demographically similar cohort groups.

Once the mean scaled scores for each NHEIAP cohort in each subject and at each grade level were established, the effectiveness level (one standard deviation above the mean), which is the average performance level achieved by the top 15% to 20% of schools and school systems within each NHEIAP cohort group, was determined.

NHEIAP performance levels are based on the scaled score achieved by a student, school or school system. The New Hampshire State Department of Education defines a scaled score as the sum of the number of "points" earned for responses to questions in each of the subject areas tested. Each student receives an individual scaled score in each subject area; the mean scaled score of a school or school system is the average of all scaled scores attained by each student who took the tests.

Scaled scores on the NHEIAP range from 200 to 300. This relatively narrow band of possible scores and the relatively small number of school systems in the state resulted in what appears to be a small range of scores between the means and effectiveness levels on the charts and graphs below. When reviewing these scores, it is worth keeping in mind, however, that a change in one or a few points in a school’s or school system’s mean scaled score represents a change of a few to several points in nearly all individual student scores.

These results apply only within the State of New Hampshire. In addition, because the cohort groups for the NHEIAP are based on statewide, not national data, the NHEIAP cohorts developed for this performance indicator may differ from the national cohorts developed for other performance indicators, presented in the district and individual school data charts in Section VIII (beginning on page 58) of this report.

Analysis of the primary demographic characteristics of community poverty and education levels of adults revealed the Exeter Region Cooperative District--SAU 16 to be in the group with the second highest education levels and lowest poverty levels. Detailed analysis of these factors, in conjunction with secondary demographic determiners such as the size of the student population and the percent of students qualifying for free and reduced lunch, indicates that Exeter Region Cooperative District--SAU 16 is in the second highest overall NHEIAP cohort group.

The effectiveness levels presented in the following tables indicate the average scaled score achieved by schools and school districts in the top 15% to 20% of Exeter’s NHEIAP cohort group in the state, and the NHEIAP cohort group for each individual school. The SchoolMatch audit team suggests that administrators consider the achievement of performance in the top 15% to 20% of these cohort groups to be initial goals to be achieved in each of the subject areas of the NHEIAP exams. In other words, to be considered effective, a school or school district should attain or exceed the average scaled score of students in districts performing in the top 15% to 20% of similar student populations. This score is designated the Effectiveness Level. While in some cases, the NHEIAP cohort Effectiveness Levels may fall within the range of average scaled scores considered the "Novice" or "Basic" levels of performance on the NHEIAP tests, the SchoolMatch team presents these goals as attainable first steps toward systemwide improvement on these statewide tests. It is certainly reasonable and desirable to expect to move to higher levels of performance.

b. NEW HAMPSHIRE EDUCATIONAL IMPROVEMENT AND ASSESSMENT PROGRAM TEST RESULTS

As noted in the previous section, the absence of nationally norm-referenced achievement test data for the Exeter Schools does not permit the audit team to compare the performance of Exeter’s students to similar student populations nationwide (see Section I and discussion in part a above). However, knowing that the New Hampshire Educational Improvement and Assessment Program provides standards-based achievement test data for all New Hampshire schools, the audit team analyzed these data to develop demographically-based NHEIAP cohorts for New Hampshire districts and schools. The mean (average) scores on the NHEIAP exams for each of these cohorts can be compared (because of similar socio-demographic characteristics) with the Exeter district and school mean scores.

Comparison of the district’s 1999-2000 NHEIAP performance levels to the average (mean) and effectiveness levels of Exeter’s NHEIAP cohort group (see Table 1) indicates the following:

• 10th Grade: The scaled scores for grade 10 students at Exeter High School are above state averages in all subjects tested. Compared with the NHEIAP cohort, scores in all subjects exceed the effectiveness level.
• 6th Grade: Scores for 6th graders at Exeter Cooperative Middle School are above state averages in all subjects tested. Compared with achievement at school systems in New Hampshire with similar student populations, however, scores are below the NHEIAP cohort effectiveness levels.

EXETER REGION SCHOOL DISTRICT -- SAU 16

1999 NHEIAP SCORES

GRAPH 1: EXETER REGION COOPERATIVE SCHOOL DISTRICT -- SAU 16 NHEIAP RESULTS, GRADE 6

GRAPH 2: EXETER HIGH SCHOOL NHEIAP RESULTS, GRADE 10

Not applicable, since only one middle school and one high school exist in the Exeter Region Cooperative.

3. Lower Grade/Upper Grade Analysis

Assuming the district’s student population has similar characteristics (e.g., poverty level, parent education level) at all grade levels, students should perform at equivalent or higher levels on the achievement tests given in different grades. This kind of consistent achievement at different grade levels may be thought of as "value added" achievement. The most appropriate way to gauge value added achievement in a school district is to follow each group of students as they progress from grade to grade throughout their entire twelve years of schooling. The SchoolMatch team encourages the leadership in the Exeter Region Cooperative School District to establish procedures to collect data and monitor progress in this way. In the absence of twelve years’ worth of data on one academic class, the SchoolMatch Audit team considered achievement data at each grade level for one year as a kind of "prism view" of performance across the grade levels in the Exeter Region Cooperative District--SAU 16.

In all subject areas tested on the New Hampshire Educational Improvement and Assessment Program (NHEIAP) tests Exeter students at all grade levels are performing above the average of their peers statewide.

Inspection of Table 1shows that relative to NHEIAP cohort means, performance in grade 10 exceeds the effectiveness levels in all subjects. In grade 6, however, only performance in mathematics achieves the mean. However, the data show that relative performance does remain constant or improve from the grade 6 tests to the grade 10 tests.

4. Cross-Subject Analysis

Learning is the process by which knowledge and skills are acquired through the use and interpretation of environmental symbols. Achievement test results reflect student learning of various school subject areas. When a group of students achieves results at a particular level in one subject area, the group should be expected to achieve at a similar level in other subjects. In other words, student groups should do just as well in one subject area as in any other subject area (e.g., reading and mathematics). Because of the sometimes significant differences in individual students’ learning styles, this expectation should not be applied to individual students within those groups. When test results represent sufficiently large numbers of students (i.e., at the school or district level), however, the effects of individual student differences in learning styles tend to cancel each other out.

On the NHEIAP, Exeter Region Cooperative students perform above the average of their peers statewide in all subjects tested.

Inspection of Table 1 indicates that relative to NHEIAP cohort means in each subject area tested by the New Hampshire Educational Improvement and Assessment Program (NHEIAP), Exeter Region Cooperative School District students are demonstrating generally the same kind of achievement across different subject matter areas. In other words, there is little, if any, relative difference in student achievement among the subject areas tested at each grade level (even though the district mean scaled scores on the various NHEIAP tests actually show large differences among themselves).

5. Achievement on College Entrance Exams: SAT and ACT

Performance on the SAT college entrance examination is above the cohort mean but did not reach the effectiveness level in 1999-2000. (See graphs below). A little less than half (40.03%) of the juniors and seniors took the SAT in 2000. No results were reported for the ACT. For comparison, in New Hampshire, 72% of graduating seniors took the SAT statewide in 2000, and 6% took the ACT in 2000.

GRAPH 3: EXETER HIGH SCHOOL 2000 SAT SCORES

Advanced Placement (AP) courses provide college-level coursework and the potential to earn college credits while in high school for high-achieving students. Having an Advanced Placement program in the high school creates a stronger academic press for the students and higher expectations for student achievement. The ability of the district to be successful in this effort is related to its willingness to retrain its teaching staff and to establish a more rigorous curriculum.

Data for 1999-2000 indicate that 4.47% of the total Junior/Senior population at Exeter High School enroll in AP classes. The mean for the cohort group of similar student populations is 19% and the effectiveness level is 20%. Approximately 2.75% of all Juniors and Seniors take one or more AP exam; and of those tested, 81.25% achieve a score of 3, 4 or 5 on one or more exam, which can qualify them for college-level credit. The mean percent of test-takers for this cohort group is 9.1% and the effectiveness level 16.1%. The mean percent earning a score of 3, 4 or 5 is 70% and the effectiveness level is 80%. (See Graphs on the following page.)

The percentage of test-takers qualifying for college credit (that is, earning a score of 3, 4 or 5 on one or more AP exams) exceeds the effectiveness level in this category based on the Exeter High School. The percentage of juniors and seniors who take the AP exams and the percentage of students enrolled in the courses do not meet the mean of the cohort group. The District does not yet reach effectiveness in these two categories. Furthermore, only about 61.5% of those enrolled in AP courses took the exams.

TABLE 2: Advanced Placement Data Summary: Exeter High School

7. Achievement / Grade Point Average:

a. Overview

Schools with grade inflation tend to reward low quality work with grades that may not reflect actual learning levels. In other words, for example, student work that is average (C) or below average (D), may be assigned grades that would suggest higher achievement (grades of C, B or A). An analysis of grade point averages nationally generally indicates the highest grade inflation occurs in the lowest achieving schools. This is suggestive of low expectations for those students who most need to have higher expectations.

Grades are typically assigned a numerical value on a descending scale from 4 to 0, with an "A" being worth 4 points and an "F" worth 0 points. Students’ grades in courses are averaged each year to determine relative achievement levels among students and to establish such things as honor rolls, National Honor Society membership and class rank. This average of points constitutes a student’s Grade Point Average (GPA).

Often, especially at lower-performing schools where grade inflation exists, Grade Point Averages are high, while achievement on standardized tests such as college entrance examinations (the SAT and the ACT) is lower than expected.

Both student performance on standardized tests and grades awarded for classroom work reflect an assessment of student academic performance. When analyzing both kinds of student performance indicators, a consistent relationship between the two should be apparent. In other words, if the student body performs in the midrange of all students taking a standardized test, and the GPA is between 2.0 and 2.4 (in aggregate), there is consistency in assessment mechanisms. However, when the GPA diverges significantly from standardized test scores, there may be a problem, either with calibrating student learning to the test measuring it, or in grading policies. Often, grade inflation is identified when aggregate grades awarded are significantly higher, on average, than performance on standardized tests would warrant.

b. Analysis of GPA and Grade Inflation in the Portsmouth School Department

The Grade Point Average for the Exeter Region Cooperative School District in 2000 was 2.5 The mean GPA for the cohort of similar student populations is 2.8 and the effectiveness level is 2.6 (see Graph below). The GPA reaches the cohort effectiveness level.

Based on an analysis of the data, little to no grade inflation appears to exist at Exeter High School.

B. ATTENDANCE ANALYSIS

1. Student Attendance

The average student attendance rate during the 1999-2000 school year was 95%. The effectiveness level is 96.5% and the mean is 94%. Districtwide student attendance levels for this school year approach, but do not quite achieve, effectiveness.

With some effort, the District should be able to attain effectiveness in this area. Disclosure of attendance goals to students and parents is often helpful in attaining higher levels of student attendance. Examination of practices in schools and school systems which have achieved effective student attendance percentages may reveal methods to be applied to increasing student attendance in the District.

The average number of days of teacher absence in the Exeter Region Cooperative Schools for the 1999-2000 school year was 5.6 days per teacher. The effectiveness level is 4 days per teacher per year, and the mean is 6. Generally speaking, the higher the teacher absence rate, the less effective is a school system (e.g., continuity of instruction; monitoring student achievement; positive attendance role modeling for students; cost of substitute teachers depleting funds which could be used for instruction).

Only discretionary absences are included in the calculation of the teacher absence rate (see Appendix C).

Currently, the teacher absence rate in Exeter Region Cooperative School District is between the mean and effectiveness level, and has not yet achieved effectiveness.

The drop-out rate in this audit is established for grades 9-12 on a four-year cumulative basis. At Exeter High School, the calculated cumulative drop-out rate is 3.28% (in other words, approximately 3 of every 100 freshmen who began high school in the 1996-97 school year have separated from public education by their senior year in 1999-2000). The effectiveness level for this demographic group is 2.4% or lower. The mean cumulative drop-out rate for the cohort is 3.6%.

The drop out rate in Exeter High School is between the mean and effectiveness level, but does not approach effectiveness. The most effective ways, overall, to reduce the number of students who drop out of school are to provide ways for students to succeed based on concrete, attainable goals and to promote participation in school activities.

The principle bases for student success are the ability to read at or very near grade level and to receive effective instruction. Efforts to improve student achievement in reading will have short and long-term benefits in overall student success and, all other things equal, will have a positive effect on the school’s holding power over high school students.

In addition to those initiatives currently practiced in the Exeter High School, these specific strategies may be effective in increasing the numbers of students who complete a high school education in the Exeter Region Cooperative School District:

• Alternative educational opportunities
• Economic incentives for high-risk students
• Mentoring programs with high-profile community leaders
• Mentoring programs linking high school students with younger
• students in the school system
• Work/study options
• Block scheduling
• Class acceleration scheduling
• Community partnerships aimed at drop-out reduction
• Intensive counseling sessions for high-risk students
• Reclamation programs for students who have dropped out of school
• Intensive awareness programs aimed at families of high school students

Previous Section: School System Policies and Governance

Back to Table of Contents

Next Section: Correlates of Effectiveness