Alleged Cultural Bias in the Mass. Licensure Exam to Become a Teacher

More than 50 percent of African American and Hispanic teacher candidates in Massachusetts are failing the state's licensure exam to become teachers, commonly know as the MTEL (Mass. Tests for Educator Licensure).

Education reporter Tracy Jan wrote about the high minority-failure rate in a page-one article in the Boston Globe last month.

The rate has been "demonstrably higher [than the rate for White test takers] since the test's inception nearly a decade ago [in 1998]," she reported.

Moreover, the test results raise questions about the MTEL exam being "culturally biased" and whether minority teacher applicants receive a good quality of education.

Jan also noted that a class-action lawsuit was slated to be filed against both the Mass. Department of Education (Mass. DOE) and the test maker, on behalf of minority teachers who failed the MTEL exam multiple times and lost their jobs.

In response to the high failure rate, the Mass. DOE announced on May 31 of this year that it would create a committee - called the "MTEL Pass Rate Study Group" - to "try and better understand the barriers to passing these tests."

New England Ethnic News spoke to three education and testing professionals for their opinions concerning the MTEL exam:

• BOB BICKERTON, senior associate commissioner at the Mass. DOE and a member of the MTEL Pass Rate Study Group

• JOSEPH PEDULLA, senior research associate at the Center for the Study of Testing, Evaluation, and Education Policy at Boston College (BC); associate professor at BC, School of Education; and a member of the MTEL Pass Rate Study Group

• CLARKE FOWLER, professor at Salem State College, Department of Education.

New England Ethnic News invited Dana Mohler-Faria and Carol R. Johnson to speak for this interview roundup, but they were unavailable due to first-day-of-school and other conflicts. Dr. Mohler-Faria is the president of Bridgewater State College as well as Gov. Deval Patrick's special advisor for education. Dr. Johnson is the new superintendent of Boston Public Schools.

AS A VETERAN EDUCATION PROFESSIONAL, WHAT ARE SONE UNDERLYING ISSUES WORTH LOOKING AT, CONCERNING THE HIGH MINORITY-FAILURE RATE FOR THE MTEL TEACHERS EXAM?

B. BICKERTON: First of all, it's key that we have diversity in the teaching workforce...to reflect the students who we're teaching. People need to see reflections of themselves as models in our classrooms. I believe that...we've done a lot to make sure the tests are fair and unbiased, but that's an area we always have to be open to improving. We need to look at whether we're preparing and supporting everybody who wants to become a teacher adequately. Are they getting all the skills they need so that they can meet the competencies to teach?

J. PEDULLA: There are a number of issues. I've posed questions at [the MTEL Pass Rate Study Group's] first meeting [about] procedures used to eliminate any biased items [and] the procedures used to establish the passing scores on the test. Those are two major issues that I need information about....There seems to be an implicit assumption that the test is right....They [i.e., the MTEL] can be in error, and we have procedures in place in the profession to try to minimize those errors, but I haven't seen what procedures we use with the MTEL yet.

C. FOWLER: One [issue] is that, given the adverse impact of the test, there should be a careful examination of the technical aspects of the test. Has it been validated properly? To my knowledge, that has not been done. [Validating the test means] simply: Have [the test questions] been properly prepared [and] properly tested? Has the test maker...followed nationally-recognized standards for making a test? There's reason to believe they [i.e., the test maker] haven't.

THE HIGH MINORITY-FAILURE RATE HAS BEEN AN OLD PATTERN - SINCE THE MTEL EXAM WAS FIRST GIVEN IN 1998. WHY NOW, 10 YEARS LATER, DO YOU SUPPOSE IT'S CAUSING AN INVESTIGATIVE REVIEW AT THE MASS. DOE LEVEL?

B. BICKERTON: Let me just [correct some] errors. In 1998, we started testing. We issued a report in '99 that was the only report over this period of time that highlighted pass rates by racial, ethnic and linguistic groups. That was discontinued. About 2-1/2 years ago, when I was asked to take this [MTEL portion of my job], I was concerned that we weren't generating any reports or data that helped us understand the extent to which this was an issue. Anecdotally we were hearing it's a problem. We accepted, in fact, that it was likely a problem, but there was no data.

The first question that I asked was: What percent of people taking the test are, in fact, voluntarily providing their racial, ethnic, linguistic group affiliation. It turned out to be higher than 95%, so we learned [that] there's more-than-enough people answering...to then understand how it's working. We generated our first report a year ago, and that's what set in motion the steps [leading to] convening the Study Group.

J. PEDULLA: I'm not in a position to answer that definitively. I suspect - and it's just speculation - that the pending class-action lawsuit...against the test has spurred some of this activity. That's typically what's happened in other states. But I don't know that for a fact [for this case in Massachusetts].

C. FOWLER: My sense is that there was a group of deans that were very concerned about this issue. They had a meeting with people from the [Mass.] DOE. I think it was clear to the DOE that these people were not going to go away. So partly, I think, they [i.e., the Mass. DOE] were shamed into it by the deans. I also think that having a new governor who has some [test] concerns [may have played a part]. I think [the Mass. DOE] thought [it] would feel pressure from the governor's office to do something and to be responsive.

IS A TEACHING STAFF - THAT LACKS RACIAL DIVERSITY - THE PRICE OF HAVING QUALIFIED TEACHERS FOR CHILDREN AND TEENS?

B. BICKERTON: That should absolutely not be the case. Our students now have to meet high standards to graduate because they need to have the skills to go on to higher education and work and succeed in life. That means we need teachers who can meet just-as-comprehensive standards for knowledge, skills and abilities.

The fact is, there should be no gap in the racial, ethnic or linguistic makeup of the people who can meet those standards. Yet that's what we're confronted with. That's a problem we have to solve, and we should not be satisfied until we solve it.

J. PEDULLA: I would certainly hope not....That would be a very sterile teaching environment....We really need a diversity in the teaching force that students can identify with. To have a lack of diversity in teachers - especially in new teachers and young teachers - I think would be a shame. I think that there are potentially ways around this issue; there's something that I will propose in this Pass Rate committee to try to mitigate [this] adverse impact.

We certainly don't want [teachers] in the classroom who shouldn't be there. But by the same token, we don't want to preclude people from being in the classroom who should be there....There are two kinds of decisions, two kinds of mistakes that can be made. We can certify people who shouldn't be certified....but we can also not be certifying people who we should be certifying, and I'm concerned about those people as well.

C. FOWLER: No, I don't think so. I don't agree with that. That would imply that people of color can't qualify for...different skills. I do know that our schools are not doing a good-enough job educating all children to the levels we'd like them to be educated. My sense is that our teaching test is eliminating higher percentages of minority [prospective teachers] than it should be eliminating.

So maybe we're not doing as good a job preparing as many people to be teachers who are from minority backgrounds, but my sense is that this test is eliminating even higher percentages of minority prospective teachers.

WHAT DO YOU THINK ABOUT HAVING AN INDEPENDENT THIRD PARTY LOOK AT THE TEACHERS EXAM TO CHECK ITS VALIDITY?

B. BICKERTON: The people who check it for validity now are large teams of teachers from Massachusetts. What would be helpful at some point is to take a look at the process that leads to development of the test...agreeing to what the objectives would be for the test. And then, what are the right kinds of items? And then, where should we put the passing score?

So this is vetted by large groups of people to achieve that validity now....They are teachers from schools, administrators from schools, people from higher education who are in preparation programs or in arts and sciences departments. They're people with no affiliations to the Department of Education or to the contractor who does the testing. They actually are involved in the beginning to [sic] the development of the test. The test doesn't get released until it goes through that process - or each new test that's either developed or re-developed [goes through that process] because every year we're developing some new tests. And often we're re-developing tests.

J. PEDULLA: I think that's an excellent idea, and I think it should be done. It's something that a number of us - community professionals such as myself - for years have talked about on all kinds of high-stakes exams. And I think the argument might be made that these tests are subject to review by teachers in the development process, and by professors in education in the development process, so that some of that has been done.

But I think the key word that you used is "independent." And typically in the test-development process, it's the test publisher who controls that process and sets the parameters for the review. Having an independent group look at these tests...hopefully we'll have an opportunity to do that on this [MTEL Pass Rate] Study Group.

C. FOWLER: I think that we've been asking for an independent review of these tests for over nine years. We have yet to receive one. I was part of a [group] that was trying to study the test on our own, and we actually published a paper on it in "Education Policy Analysis Archives." We found that there WERE some technical problems with the test. Therefore, we asked for a fully-independent investigation into the exams, but that has never occurred.

Question for Bob Bickerton: WHAT WILL THE MTEL PASS RATE STUDY GROUP DO TO KEEP THE PUBLIC INFORMED OF ALL ITS FINDINGS AND ACTIONS?

B. BICKERTON: At the end of this, we'll be doing a summary of recommendations. The findings are recommendations, and those findings and recommendations will go to the [Mass. DOE's] EPAC [Educational Personnel Action Committee, to which the MTEL Pass Rate Study Group reports], which will then look at them with a broader policy view, and those will be public.

Question for Joseph Pedulla and Clarke Fowler: AS AN EXPERT IN EDUCATIONAL POLICY AND TESTING, HOW SURPRISING IS IT THAT MINORITY APPLICANTS ARE FAILING A STATE TEACHERS-LICENSURE EXAM AT SUCH HIGH RATES?

C. FOWLER: Unfortunately it's not that surprising. This has also happened with other [states'] testing programs. What is surprising is that...other states publish their scores so that people can both study the exam and also talk about ways to deal with this particular problem....What's happened in Massachusetts is that...we're only starting now what we should have started nine years ago.

The essence of education reform is transparency and accountability for results. Everybody...[is] subject to that. The...entity that is not subject to accountability and transparency in the entire educational-reform movement [is] those who make the teacher tests. That's happened here in Massachusetts....

The [Massachusetts] Department of Education has been willfully blind to these issues for nine straight years.

J. PEDULLA: Unfortunately it is not terribly surprising. It's a pattern that we see repeated in almost every state - certainly in every state that I've looked at or worked in....But this adverse impact typically happens in a similar vein to the way it's happening in Massachusetts right now. You see this adverse impact [regarding the] pass rates with typically Black students. They're the biggest group of minority test takers performing, or passing, at a much lower rate.

It's not just the [Mass.] DOE [that could improve by being transparent]. It is, in fact, the test publisher as well. I learned, for example, at this first meeting [of the MTEL Pass Rate Study Group] that there's no state budget to support the Mass. DOE in the teacher testing program.

All of the money that runs the program comes from the fees that the test taker pays the test publisher to take the test.

So I think the Mass. DOE could be more forceful in demanding that the test publisher provide good information - and information that's demanded by the standards of the profession that they have been in the past.

BRIEFLY, IS THERE ANYTHING ELSE YOU'D LIKE TO SAY?

B. BICKERTON: I think that there are many places for us to look and find solutions. We need to look at the test. We do need to look at preparation.

Are teachers who might ultimately have difficulty [with] the writing part of the MTEL test, that's the one that people have the most trouble with - we have to ask ourselves: Why that would be the case? What is it we can do to better prepare people so that writing is something that's not challenging...?

We need to look at every stage of this process, and then we have to [act].

J. PEDULLA: I'd like to say that whenever we're making...critical decisions...about people, we should incorporate information other than just a test in making that determination.

I think someone who fails a test by one item or one point should have an opportunity to compensate for that..., whether that be through their performance in academic courses or their performance in the classroom, through some kind of review, but some other data ought to be brought to bear to help in making the ultimate decision.

These [test] instruments are not precise enough to be able to be confident that someone who falls just below the cut score should, in fact, not be certified to teach.

C. FOWLER: I hope that the current ongoing effort to look at this test with the new [MTEL Pass Rate Study Group] will be productive.

Copyright 2007 New England Ethnic News. All rights reserved. This material may not be published, broadcast, rewritten or redistributed without the express permission of the source. Contact NEWz for more information.