Maybe not. A new working paper (abstract; PDF) by Maria Fitzpatrick and Michael Lovenheim finds that offering early retirement to experienced schoolteachers doesn't have a negative effect on students' test scores, and in some cases leads to an improvement. The abstract:
Early retirement incentives (ERIs) are increasingly prevalent in education as districts seek to close budget gaps by replacing expensive experienced teachers with lower-cost newer teachers. Combined with the aging of the teacher workforce, these ERIs are likely to change the composition of teachers dramatically in the coming years. We use exogenous variation from an ERI program in Illinois in the mid-1990s to provide the first evidence in the literature of the effects of large-scale teacher retirements on student achievement. We find the program did not reduce test scores; likely, it increased them, with positive effects most pronounced in lower-SES schools.