CORRELATES OF EFFECTIVENESS

The Educational Testing Service conducted surveys in 55 cities across the nation to measure dissatisfaction rates among school department constituents (See 10/27/99 Education Week, "ETS Analysis Tracks Parent Dissatisfaction."). Of the many important observations, one stands out:

The SchoolMatch Survey of Perceptions gauges the perceptions of three major school system constituents: administrators, teachers and parents. Survey questions are based on the Correlates of Effectiveness and specific related elements in the following areas:

1. Strong instructional leadership
2. A comprehensive curriculum with an emphasis on learning
3. A positive school climate
4. Regular monitoring of performance and attendance
5. High expectations supported by staff and students

More information about the Correlates of Effectiveness and effective school practices may be found in
the Introduction, Part A of this report

Data Collection:
Information related to the Correlates of Effectiveness is collected in two ways:

• through interviews with school officials and leaders, and
• through questionnaires administered to randomly selected groups of parents, teachers and administrators
  (all administrators in small school systems).

The data obtained are used to establish the degree to which the correlates are perceived by each group to be operating in the Portsmouth School Department.

The questionnaire uses a scale of 1-4 to evaluate the various elements of an effectiveness correlate:

• 1 indicates an element is perceived to be very strong;
• 2 indicates an element is perceived to strong;
• 3 indicates an element is perceived to be weak; and
• 4 indicates an element is perceived to be very weak.

An average score of 2.0 or lower for an effectiveness correlate indicates that the correlate is perceived to be established in the school department. Correlates whose average is greater than 2.0 need attention. Likewise, individual elements within a particular correlate can be improved when behaviors associated with effectiveness correlates and their individual elements are established; that is, they are consistently displayed by school personnel. Values less than 2.0 show increasingly positive support, while values over 2.0 indicate declining support for an element or correlate, and are said to "approach effectiveness."

It is important to note the SchoolMatch Survey of Parents, Teachers and Administrators occurred in September of 2000. The data are reflective of perceptions regarding the correlates of effectiveness at that time. Shortly prior to the survey, a new central administration was employed by the School Department, including a new Superintendent and Assistant Superintendent. The Administration is encouraged to revisit perceptions subsequent to leadership changes.

The discussion of perception survey results can take many forms. In some cases, auditors may consider variances among responses to individual elements, may comment on the most and least effective elements identified by each group, may identify patterns of agreement and disagreement, or may combine these, and other, approaches. In all cases, the form of the discussion is tailored to the unique variables and patterns within a particular school system.

In analyzing the perception survey data, it is important to look for significant differences and similarities in the aggregate group responses. Identification of such patterns offers school system leadership both pathways to improvement and opportunities to communicate the positive things happening in the school department. The site visit team has identified some of the most significant findings related to each correlate of effectiveness. Please review the summary data presented on the pages following the discussion of results for each correlate in order to locate other patterns which may be helpful in enhancing that correlate within the department.

For this report, SchoolMatch consultants searched for patterns of agreement that revealed the areas of greatest effectiveness and greatest concern for each of the five correlates. Areas of greatest effectiveness were indicated by the lowest mean scores given by each group to individual elements. Conversely, areas of greatest concern were indicated by the highest mean scores given by each group to individual elements. The discussions focus on those elements to which more than one group gave one of their lowest or highest mean scores, even when those scores meet or fall below the 2.0 level.

The following perception survey results were obtained from an analysis of all complete parent, teacher and administrator perception surveys received in the SchoolMatch offices.

This factor supports clear efforts to improve school effectiveness and student learning. The system is well organized in support of student priorities when the perceptions meet the value of 2.0. Values less than 2.0 show increasingly positive support, while values over 2.0 indicate declining support for an element or correlate.

An analysis of the perception data for the LEADERSHIP correlate indicates that the 2.0 level of effectiveness is met by parents and administrators; teacher perceptions approach 2.0. The results:

• Parents: 1.989
• Teachers: 2.060
• Administrators: 1.945

The data suggest the main challenge in the area of Leadership at the time of the survey was lack of communication. Since the survey took place, changes occurred in leadership that may have had a significant positive impact in this area. In the site visit, the Audit team saw evidence of measures to greatly improve communication initiated by the new superintendent. To reinforce these measures, School Department leaders should consider a periodic review, perhaps through surveys of perceptions as conducted for this Audit, of how information is communicated and how personnel in all areas of operation work together to achieve goals. This ongoing review could assist with assessing the effectiveness of current initiatives, lead to development of additional effective strategies for creating common plans for improvement, and contribute to a greater sense of community and shared objectives.

The results of this survey indicate that those surveyed agree that building administrators establish school objectives and communicate them to students and staff (element C), all groups perceive problems with overall communication practices and channels (element B). In addition, although teachers and administrators hold the positive perception that the system has assembled a high quality staff and utilizes them effectively (element I), these same two groups do not perceive the school department to be effective in building teamwork (element J).

Overall Patterns:

• All groups agree one element of the Leadership correlate is not established (Element B):
  Clear and concise communication is practiced; open, functioning channels exist throughout the school system
• Parents and teachers perceive element F as not being established:
  The building administrators visit classrooms frequently
• Teachers and administrators agree element J is not established:
  The district is effective in building cohesive teamwork among staff

• All three groups agree elements C and M are among the most effective elements of Leadership:
  C: The building administrators establish school objectives and communicate them to staff and students.
  M: The principal is visible, accessible.
• Teachers and administrators identify element I as one of the most effective elements:
  I: The district has assembled a high quality staff and utilizes personnel effectively.

Areas of Greatest Concern:

• All three groups identify elements B and J as being among the least effective:
  B: Clear and concise communication is practiced; open, functioning channels exist throughout the school system.
  J: The district is effective in building cohesive teamwork among staff.
• Parents and teachers agree element F is among the least effective:
  F: The building administrators visit classrooms frequently.

The Site Visit Team believes it is important to note, again, that while the above stated concerns may have been present when the Perception Survey was completed, careful on-site observations suggest that strong efforts have been made to improve instructional leadership, promote cohesive teamwork and enhance communication relative to the development of the academic program.

This factor indicates that the schools' main priority is the learning of students. Distractions and interruptions of instruction and learning are minimized when this correlate is met at the 2.0 level. Values less than 2.0 show increasingly positive support, while values over 2.0 indicate declining support for an element or correlate.

An analysis of the perception data for the EMPHASIS ON LEARNING correlate indicates that the 2.0 level is met by all three groups. The values given to the correlate are:

• Parents: 1.931
• Teachers: 1.877
• Administrators: 1.849

While overall perceptions of the Emphasis on Learning correlate are positive, two areas stand out as in need of attention. All groups hold the perception that instructional materials are not given the highest priority in the allocation of resources (element J) and that distractions and interruptions of classroom learning need to be reduced (element F).

Overall Patterns:

• All three groups perceive element J as not being established:
  J: Instructional materials are given the highest priorities in the allocation of resources.
• Teachers and Administrators agree element F is not established, and parents’ perceptions
  approach effectiveness on this element:
  F: Distractions and interruptions of classroom learning are minimized.
• Teachers identify an additional element as not established:
  N: The district provides students with computer access and makes special arrangements for children who do not have computers at home.

• All groups agree element M is among the most effective elements of the Emphasis on Learning:
  M: The building administrators are visible, accessible.

• All groups identify elements F and J among the least effective elements of this correlate:
  F: Distractions and interruptions of classroom learning are minimized.
  J: Instructional materials are given the highest priorities in the allocation of resources.

This factor is manifest in student respect for the physical plant, strong parent involvement and positive staff and student morale. Values less than 2.0 show increasingly positive support, while values over 2.0 indicate declining support for an element or correlate.

An analysis of the perception data for the SCHOOL CLIMATE correlate indicates that the level of 2.0 is met by parents and administrators; teachers’ score approaches 2.0. The values given to the correlate are:

• Parents: 1.843
• Teachers: 1.992
• Administrators: 1.951

Overall, parents, teachers and administrators surveyed hold positive perceptions of the School Climate correlate. Teachers and administrators, however, express some concerns in the same areas, related to participation of parents (elements A and H) and student groups (element G) in school activities, committees and functions.

Some disagreement exists regarding the contributions of parent volunteers (element A). While teachers and administrators perceive this element to be one of the least effective areas of this correlate, parents perceive it as one of the most effective. It could be that many parents who completed this survey were themselves active in the schools, and so perceive greater participation of parents than do teachers or administrators. It could also be that although parents are volunteering in many areas of school functioning, more volunteers are needed. Whatever the reasons, Portsmouth School Department leaders should consider working with representatives from all constituent groups to identify ways to increase and support parent involvement.

Overall Patterns:

• Teachers and administrators agree that three elements of the School Climate correlate are not established:
  A: Parent volunteers contribute widely in the functioning of the school.
  G: All student groups participate broadly in school activities.
  H: There is wide parent participation in school committees and functions.
  (Parent perceptions approach effectiveness on this element.)

Areas of Greatest Effectiveness:

• All groups agree element F is among the most effective elements of this correlate:
  F: There is acceptance of diverse student backgrounds.
• Teachers and administrators agree that element I is among the most effective:
  I: The school has a positive image in the local media.

Areas of Greatest Concern:

• All groups agree element G and H are among the least effective element of the School Climate correlate:
  G: All student groups participate broadly in school activities.
  H: There is wide parent participation in school committees and functions.
• Teachers and administrators agree element A is among the least effective:
  A: Parent volunteers contribute widely in the functioning of the school.

This factor indicates that systematic procedures exist for measuring student achievement. Such procedures document achievement in specific areas, establish need for instructional improvement and develop priorities for the allocation of resources. Values less than 2.0 show increasingly positive support, while values over 2.0 indicate declining support for an element or correlate.

Analysis of the perception data for the MONITORING correlate indicates that the 2.0 level is met by all teachers and administrators; parent score approaches 2.0. The values given to the correlate are:

• Parents: 2.018
• Teachers: 1.900
• Administrators: 1.894

All groups express concern about the regular collection and review of data on student learning (element H), and about the clarity and focus of criteria for assessment of student learning (element A). On the other hand, those surveyed have positive perceptions that clear learning objectives exist (element B), attendance data are effectively reviewed and followed up (element D), and that teachers provide individual assistance to students (element G).

One issue these data reveal is that while these groups believe effective learning objectives may be in place, and procedures to measure student progress may have been developed, the criteria used to assess progress toward learning objectives are not clear. The strong perception that data on student learning are not regularly collected and reviewed (element H) supports this observation, and further suggests that assessment criteria, as well as data on outcomes, are not being communicated clearly to the entire school community.

Overall Patterns:

• All groups agree element H is not established:
  H: Data on student learning are regularly collected and reviewed with all members of the school community.

• All groups identify element D as being among the most effective elements of the Monitoring Student Progress correlate:
  D: Attendance data are regularly reviewed, follow-up contacts made with families, and progress recorded.
• Parents and teachers agree element G is one of the most effective:
  G: Teachers provide students with individual assistance throughout the day.
• Teachers and administrators agree element B is one of the most effective areas of this correlate:
  B: Clear learning objectives exist throughout the curriculum

Areas of Greatest Concern:

• All three groups agree elements A and H are among the least effective elements of this correlate:
  A: Criteria for the assessment of learning gains are explicit; student knowledge and skills development are stressed.
  H: Data on student learning are regularly collected and reviewed with all members of the school community.

This factor indicates the school staff is dedicated to having each student learn at the highest possible level. Challenging experiences are conducted to have students contribute their best work. Values less than 2.0 show increasingly positive support, while values over 2.0 indicate declining support for an element or correlate.

Analysis of the perception data for the HIGH EXPECTATIONS correlate indicates the 2.0 level is met by all three groups. The values given to the correlate by each group are:

• Parents: 1.812
• Teachers: 1.796
• Administrators: 1.871

Perceptions of the High Expectations correlate are strongly positive overall. Concern does exist, however, that test scores on standardized tests are not as high as expected (element I) and that learning activities and materials may not be at a sufficiently demanding cognitive level (element G). Some attention may need to be given to these areas, particularly since increasing the rigor of classroom learning activities may contribute to higher achievement levels on tests.

• Parents perceive element I as not being established (teachers’ perceptions approach effectiveness on this element):
  I: Test scores on standardized tests exceed expected scores for comparable schools.

Areas of Greatest Effectiveness:

• All groups agree element B is one of the most effective elements of the High Expectations correlate:
  B: Rules regarding attendance and promptness are made explicit in class.
• Parents and teachers agree element D is one of the most effective:
  D: Students are expected to be fully prepared for their classes.
• Parents and administrators identify element H as highly effective:
  H: Students are held responsible for missed work.
• Teachers and administrators find element A to be a highly effective element of this correlate:
  A: Teachers regularly require learning beyond the classroom; e.g. homewrok assignments.

• All three groups perceive elements G and I as being among the areas of greatest concern:
  G: Classroom learning activities and materials are at a demanding cognitive level; e.g., materials require analysis, generalizations, and evaluation over memorization and recall.
  I: Test scores on standardized tests exceed expected scores for comparable schools.

Previous Section: Criteria for Effectiveness

Back to Table of contents

Next Section: General Observations