CRITERIA FOR EFFECTIVENESS

There is a relationship among school system effectiveness, the socioeconomic status of the community, and the educational levels of parents. Many studies have suggested that early childhood experiences affect learning and development, with children from impoverished environments generally achieving at lower levels than those from more enriching situations. An article in the April 21, 1997 issue of US News and World Report, for example, supports the notion that improving social and family conditions, particularly in the early years, can enhance development.

In the late 1970s, neuroscientists discovered that children have a rapid increase in brain connectivity early in development. The research of pediatric neurologists suggests that formation of synaptic contacts in the human cerebral cortex occurs between birth and age ten. In other words, most of the brain gets built after birth. (See Peter R. Huttenlocher and Arun S. Dabholkar, "Regional Differences in Synaptogenesis in Human Cerebral Cortex," The Journal of Comparative Neurology, 387:167-178 [October 20, 1997]). Higher intelligence is witnessed in individuals with more stimulation and protein in the early years of life. More recent documentation indicates that "age three seems more likely to mark only the beginning of a long developmental and maturational period during which environmental stimulation and experience do matter" (See John T. Bruer, "Neural Connections – Some You Use, Some You Lose," Phi Delta Kappan, Vol. 81, No. 4, pp. 264-277, [December 1999]).

Recent research has shown that the family’s social class has an effect on vocabulary growth and the development of language skills in children. Those from lower socio-economic classes tend be less fluent with language and have less capacity to teach their children language skills than those from higher socio-economic situations. These differentiations are quite pronounced by age three or four, and have significant implications for later cognitive and school performance. Effective schooling can reduce the consequences of lack of early exposure to strong language skills, and can, to some degree, equalize performance on language-based tasks. (See George Farkas and Kurt Beron, "Family Linguistic Culture and Social Reproduction: Verbal Skill from Parent to Child in the Preschool and School Years," presented at the Session of Child Poverty and Deprivation, at the Annual Meeting of the Population Association of America, Washington D.C., 31 March 2001.)

Achievement has also been associated with child rearing practices, assertive mothers generally rearing children who achieve at higher levels than those reared by less assertive mothers. (See Guy Odom, Mothers, Leadership, and Success, Houston: Polybius Press, 1989.)

These concepts open up many opportunities for parents and schools to work together in improving learning potential.

The effective school research has abundantly demonstrated that achievement levels between poor and affluent students can be narrowed considerably while increasing levels of achievement for all students.

Nevertheless, the criteria of effectiveness are generally met at a higher level when student populations contain a small percentage of free and reduced lunch students (an indicator of poverty levels) and the educational level of parents is high school completion or above. We can generally assume that school systems which have better entry level characteristics than the state average will meet the criteria of effectiveness at a higher level than those whose entry level characteristics match or are below the state average.

Entry level characteristics used for predicting student achievement levels are items such as:

In this audit, the items used to predict the levels at which the effectiveness criteria should be met by the Portsmouth School Department include:

Educational researchers and auditors realize that predictions and judgments cannot be made from a single indicator. Therefore, the audit uses a multiple variable approach to making recommendations and arriving at conclusions. It is the same method used by various state departments of education to establish accountability and validate school effectiveness. It was initially developed by the South Carolina Department of Education (Division of Public Accountability) and has been used in several other states.

If a number of variables indicate some trend, and all the variables are moving in the same direction, then the confidence level of the prediction, or the conclusion, is well established. This is a commonly accepted practice in educational auditing.

To establish effectiveness levels for the Portsmouth School Department, we used the following conditions, as reported by the school department for the 1998-99 school year (sources noted if other than the school department):

• percent of student population on free and reduced lunch is 24.8%;
• percent of students whose principal language is not English is 1.55%;
• percent of school system residents living in poverty is 6.9%;
  (Source: SchoolMatch databases; US Census data)
• percent of school system families receiving public aid is 4.7%;
  (Source: SchoolMatch databases; US Census data)
• the average per capita income in the community is $15,557;
  (Source: SchoolMatch databases; US Census data)
• the education level of school system residents is 13.1 years of schooling;
  (Source: SchoolMatch databases; US Census Data)
• percent of teaching staff having a Master's degree or higher in the regular high school is 59%;
• percent of teaching staff having temporary certificates or letters of authorization in the regular high school is 6%.

With the above socio-demographic profile, the Portsmouth School Department should establish the following effectiveness levels in order to place the department’s student and staff populations among the top 15-20% of performance for similar populations:

• The grade point average of the senior class should be 2.6 (non-weighted).
• The School Department attendance rate (ADA/ADM) should be 96%.
• Achievement on norm-referenced tests for Total Reading should be at—or above—the 76th percentile.
• Achievement on norm-referenced tests for Total Mathematics should be at—or above—the 76th percentile.
• On criterion-referenced tests, 96% of the student population should achieve mastery in both Reading and Mathematics.
• 15% of the juniors and seniors should be enrolled in the Advanced Placement program. Of all juniors and seniors, 5.2% should be taking at least one AP examination. Of the students taking AP examinations, 75% should be eligible for college credit.
• The one-year drop-out rate should be 1.6% or lower;
• The drop-out rate, cumulative grades 9-12, should be 6.4% or lower.
• The Teacher Absence Index (TAI) should be at or below 4.5 days (See Appendix C).
• On the ACT, the school system mean should be 22.8 or higher.
• On the SAT, the school system mean should be 1026 or higher.
• The average of the responses for each correlate of effectiveness for parents, teachers, and administrators should be at, or below, 2.00. (See Section V, Correlates of Effectiveness).

Section IV, Effectiveness Levels, provides three significant categories for the entire school department and for each school:

1. The mean (AVERAGE) performance for the comparison population for each variable;
2. The performance standard when effectiveness is achieved in the school department at one standard deviation above the mean (the EFFECTIVENESS LEVEL); and
3. The current status of variables in the Portsmouth School Department and individual schools within the School Department.

2. New Hampshire Educational Improvement and Assessment Program (NHEIAP)Test Analysis

Because the Portsmouth School Department does not utilize nationally norm-referenced standardized tests, the SchoolMatch audit team examined the results of the 1999 New Hampshire Educational Improvement and Assessment Program (NHEIAP) test scores from the entire state of Hew Hampshire based on demographic indicators that have been shown to have impact on student success on standardized tests. We present in this section an analysis of the performance of Portsmouth’s students on the 1999 NHEIAP as compared to populations of New Hampshire students with similar demographic characteristics.

The SchoolMatch audit team recognizes the importance of parent involvement in motivating their children to learn. We further recognize the success which many parents of limited education and income have demonstrated in motivating their children to do well in school. Unfortunately, national databases do not reflect this motivational factor for individual homes. Consequently, the team relies on data which most closely parallel research on demographic impact.

Significant research has indicated that the education level of adults with whom youngsters have contact, particularly mothers, and the percentage of children living in poverty within a school community have a great impact on a school system’s ability to produce high achievement on standardized tests. Therefore, SchoolMatch auditors and researchers rely on these and other related demographic characteristics to conduct their analyses (see Section I for a discussion of demographic features evaluated).

SchoolMatch researchers analyzed the demographic characteristics of the student populations in each of the school systems in New Hampshire, and identified statewide groups of school system and school populations with similar characteristics (the cohort group). We then examined NHEIAP test data from every school system and school in the state of New Hampshire, and calculated the average performance level (mean) within each of the demographically similar NHEIAP cohort groups.

Once the mean scaled scores for each NHEIAP cohort in each subject and at each grade level were established, the effectiveness level (one standard deviation above the mean), which is the average performance level achieved by the top 15% to 20% of schools and school systems within each NHEIAP cohort group, was determined.

NHEIAP performance levels are based on the scaled score achieved by a student, school or school system. The New Hampshire State Department of Education defines a scaled score as the sum of the number of "points" earned for responses to questions in each of the subject areas tested. Each student receives an individual scaled score in each subject area; the mean scaled score of a school or school system is the average of all scaled scores attained by each student who took the tests.

Scaled scores on the NHEIAP range from 200 to 300. This relatively narrow band of possible scores and the relatively small number of school systems in the state resulted in what appears to be a small range of scores between the means and effectiveness levels on the charts and graphs below. When reviewing these scores, it is worth keeping in mind, however, that a change in one or a few points in a school’s or school system’s mean scaled score represents a change of a few to several points in nearly all individual student scores.

These results apply only within the State of New Hampshire. In addition, because the cohort groups for the NHEIAP are based on statewide, not national data, the NHEIAP cohorts developed for this performance indicator may differ from the national cohorts developed for other performance indicators, presented in the school department and individual school data charts in Section VIII (beginning on page 56) of this report.

Analysis of the primary demographic characteristics of community poverty and education levels of adults revealed the Portsmouth School Department to be in the group with the second highest education levels and second lowest poverty levels. Detailed analysis of these factors, in conjunction with secondary demographic determiners such as the size of the student population and the percent of students qualifying for free and reduced lunch, indicates that Portsmouth School Department is in the second highest overall NHEIAP cohort group, statewide.

The effectiveness levels presented in the following tables indicate the average scaled score achieved by schools and school departments in the top 15% to 20% of Portsmouth’s NHEIAP cohort group, and the NHEIAP cohort group for each individual school. The SchoolMatch audit team suggests that administrators consider the achievement of performance in the top 15% to 20% of these cohort groups to be initial goals to be achieved in each of the subject areas of the NHEIAP exams. In other words, to be considered effective, a school or school department should attain or exceed the average scaled score of students in systems performing in the top 15% to 20% of similar student populations. This score is designated the Effectiveness Level. While in some cases, the NHEIAP cohort Effectiveness Levels fall within the range of average scaled scores considered the "Novice" or "Basic" levels of performance on the NHEIAP tests, the SchoolMatch team presents these goals as attainable first steps toward systemwide improvement on these statewide tests. It is certainly reasonable and desirable to expect to move to higher levels of performance.

As noted in the previous section, the absence of nationally norm-referenced achievement test data for the Portsmouth Schools does not permit the audit team to compare the performance of Portsmouth’s students to similar student populations nationwide (see Section I and discussion in part a above). However, knowing that the New Hampshire Educational Improvement and Assessment Program provides standards-based achievement test data for all New Hampshire schools, the audit team analyzed these data to develop NHEIAP cohorts for New Hampshire systems and schools. The mean (average) scores on the NHEIAP exams for each of these NHEIAP cohorts can be compared (because of similar socio-demographic characteristics) with the Portsmouth School Department and school mean scores.

Comparison of the 1999 NHEIAP performance levels to the average (mean) and effectiveness levels of Portsmouth’s NHEIAP cohort group (see Table 1) indicates the following:

• 10^th Grade. Students at Portsmouth High School performed above state-determined anticipated score ranges and above statewide averages in all subjects tested. Portsmouth High School ranked among the top ranked schools statewide in terms of performance on the NHEIAP. Scores in all subjects exceed the effectiveness levels for the NHEIAP cohort.

• 6^th Grade. Students in 6th grade in Portsmouth School Department perform within the state-determined anticipated score range. However, scores in Social Studies are below the statewide averages. Scores in Mathematics, English Language Arts and Science meet or exceed the statewide average. Scores in English Language Arts, Mathematics and Science meet the NHEIAP cohort means; Mathematics scores reach the NHEIAP cohort effectiveness level.

• 3^rd Grade. Third graders in the Portsmouth School Department performed within state-determined anticipated score ranges at the statewide averages in Mathematics, but slightly below the statewide mean in English Language Arts (students are not tested in Science and Social Studies in grade 3). Performance in Mathematics is below the mean score of the NHEIAP cohort. Performance in English Language Arts meets the NHEIAP cohort mean.

PORTSMOUTH SCHOOL DEPARTMENT

1999 NHEIAP AVERAGE SCALED SCORES

Cohorts were developed for each school, based on demographic factors, as compared to other schools across the state. In some cases, an individual school NHEIAP cohort group is different from the School Department’s overall NHEIAP cohort group because the school department NHEIAP cohort reflects an aggregated description of the entire student body and school department community. Variations in individual school communities account for the differences among school cohorts.

Comparison of individual school performance levels to the average (mean) and effectiveness levels of their school NHEIAP cohorts indicates the following:

1999 NHEIAP ELEMENTARY SCHOOLS: Grade 3

Assuming the school department’s student population has similar characteristics (e.g., poverty level, parent education level) at all grade levels, students should perform at equivalent or higher levels on the achievement tests given in different grades. This kind of consistent achievement at different grade levels may be thought of as "value added" achievement. The most appropriate way to gauge value added achievement in a school department is to follow each group of students as they progress from grade to grade throughout their entire twelve years of schooling. The SchoolMatch team encourages the leadership in the Portsmouth School Department to establish procedures to collect data and monitor progress in this way. In the absence of twelve years’ worth of data on one academic class, the SchoolMatch Audit team considered achievement data at each grade level for one year as a kind of "prism view" of performance across the grade levels in the Portsmouth School Department.

Based on the analysis of the statewide test data, although a few "dips" in performance occur, performance can be said to be holding steady or improving across the grade levels.

Learning is the process by which knowledge and skills are acquired through the use and interpretation of environmental symbols. Achievement test results reflect student learning of various school subject areas. When a group of students achieves results at a particular level in one subject area, the group should be expected to achieve at a similar level in other subjects. In other words, student groups should do just as well in one subject area as in any other subject area (e.g., reading and mathematics). Because of the sometimes significant differences in individual students’ learning styles, this expectation should not be applied to individual students within those groups. When test results represent sufficiently large numbers of students (i.e., at the school or system level), however, the effects of individual student differences in learning styles tend to cancel each other out.

Inspection of Table 1 shows that, although differences in scaled score values are evident, relative to cohort means, performance in all subject areas is consistent. Students appear to be achieving about as well in any given subject as in any other, based on the cohort comparisons within the state.

Performance on the SAT is higher than the cohort mean but does not yet achieve the effectiveness level for the school department overall (See graphs below). Approximately 70% of the seniors took the SAT in 1999-2000. No results were reported for performance on the ACT. For comparison, in New Hampshire, 72% of graduating seniors took the SAT statewide in 2000, and 6% took the ACT in 2000.

Advanced Placement Program

Advanced Placement (AP) courses provide college-level coursework and the potential to earn college credits while in high school for high-achieving students. Having an Advanced Placement program in the high school creates a stronger academic press for the students and higher expectations for student achievement. The ability of the system to be successful in this effort is related to its willingness to retrain its teaching staff and to establish a more rigorous curriculum.

Data for 1999-2000 indicates that 10.51% of the total Junior/Senior population enroll in AP classes. The mean for the cohort group of similar student populations is 13% and the effectiveness level is 15%. Approximately 10.51% of Juniors and Seniors take one or more AP exam (100% of those enrolled in AP courses); and of those tested, 54.9% achieve a score of 3, 4 or 5 on one or more exam, which can qualify them for college-level credit. The mean percent of test-takers for this cohort group is 4.3% and the effectiveness level 5.2%. The mean percent earning a score of 3, 4 or 5 is 70% and the effectiveness level is 75%. (See Graphs on the following page.)

The percentage of juniors and seniors who take AP exams exceeds the effectiveness level in this category based on the population of Portsmouth High School. The percentage of students enrolled in AP courses, however, does not meet the mean of the cohort group. In addition, the percentage of students who achieve a 3, 4 or 5 on the exams is below the cohort mean and effectiveness level.

TABLE 3: ADVANCED PLACEMENT, PORTSMOUTH HIGH SCHOOL

7. Achievement / Grade Point Average:

a. Overview

Schools with grade inflation tend to reward low quality work with grades that may not reflect actual learning levels. In other words, for example, student work that is average (C) or below average (D), may be assigned grades that would suggest higher achievement (grades of C, B or A). An analysis of grade point averages nationally generally indicates the highest grade inflation occurs in the lowest achieving schools. This is suggestive of low expectations for those students who most need to have higher expectations.

Grades are typically assigned a numerical value on a descending scale from 4 to 0, with an "A" being worth 4 points and an "F" worth 0 points. Students’ grades in courses are averaged each year to determine relative achievement levels among students and to establish such things as honor rolls, National Honor Society membership and class rank. This average of points constitutes a student’s Grade Point Average (GPA).

Often, especially at lower-performing schools where grade inflation exists, Grade Point Averages are high, while achievement on standardized tests such as college entrance examinations (the SAT and the ACT) is lower than expected.

Both student performance on standardized tests and grades awarded for classroom work reflect an assessment of student academic performance. When analyzing both kinds of student performance indicators, a consistent relationship between the two should be apparent. In other words, if the student body performs in the midrange of all students taking a standardized test, and the GPA is between 2.0 and 2.4 (in aggregate), there is consistency in assessment mechanisms. However, when the GPA diverges significantly from standardized test scores, there may be a problem, either with calibrating student learning to the test measuring it, or in grading policies. Often, grade inflation is identified when aggregate grades awarded are significantly higher, on average, than performance on standardized tests would warrant.

The Portsmouth School Department reports that the Grade Point Average (GPA) for the senior class is 2.9. The mean GPA for the cohort of similar student populations is 2.6 and the effectiveness level is 2.4 (see Graph below).

An analysis of student performance data indicates that grade inflation may exist to some degree in the Portsmouth School Department.

Although the students who take the college entrance exams perform well compared to their peers from demographically similar student populations, less than half of the juniors and seniors in Portsmouth take the SAT. Furthermore, participation rate is below the statewide averages of 74% for the SAT.

In the absence of norm-referenced test data other than the college entrance exams (SAT), the audit team bases its conclusions on reported trend data from the New Hampshire Educational Improvement and Assessment Program (NHEIAP). Student performance on the NHEIAP exams, when compared to statewide averages, is above statewide averages in the 10th grade in all subjects, and above the state average in all subjects except mathematics in the 6th grade. In the 3rd grade, performance in English Language Arts is below the state average, but in mathematics performance is at the state average level.

1. Student Attendance

The average student attendance rate during the 1998-99 school year was 94.7%. The effectiveness level is 96% and the mean is 93%. Systemwide student attendance levels for this school year approach, but do not yet achieve, effectiveness.

Graph 9

With some effort, the Portsmouth School Department should be able to attain effectiveness in this area. Disclosure of attendance goals to students and parents is often helpful in attaining higher levels of student attendance. Examination of practices in schools and school systems which have achieved effective student attendance percentages may reveal methods to be applied to increasing student attendance.

The average number of days of teacher absence in the Portsmouth Schools for the 1999-2000 school year was 5.1 days.

Currently, the teacher absence rate in the Portsmouth School Department is between the mean and effectiveness level, and has not yet achieved effectiveness.

Teacher attendance may be improved by publicizing absence rates, designing school department policy to recognize and reward teacher attendance, and implementing practices across the system adopted from buildings where teacher attendance is effective.

The drop-out rate in this audit is established for grades 9-12 on a four-year cumulative basis. In the Portsmouth School Department, the calculated cumulative drop-out rate is 3.42% (in other words, approximately 3 of every 100 freshmen who began high school in the 1996-97 school year in Portsmouth School Department have separated from public education by their senior year in 1998-99). The effectiveness level for this demographic group is 6.4% or lower. The mean cumulative drop-out rate for the cohort is 7.6%.

The drop out rate in Portsmouth achieves effectiveness, a commendable achievement.