vs BEST; APPLES, ORANGES & PEANUTS
Theodore A. Klein, Jr.
I think, therefore I must be. During my past three years of involvement
in adult education and ESL in the Austin area, after decades of working
elsewhere in this same field, I have spent a great deal of time looking
at ESL measurement, as it is now done. The key to ANY training or education
program, is some sort of objective measure of where we are and where we
will go, and to find out if we have achieved our goals. The test should
never be the PURPOSE of any training. The results should be simply be a
way to evaluate how we are doing.
The official instrument of the moment in AE in Texas, has for some time been the BEST Test, developed under contract to the Center for Applied Linguistics (CAL) in Washington, D.C.
CAL has for decades been a reputable outlet for useful language materials and information. I first became acquainted with this organization in 1963, on my way with a Fulbright Grant, to teach English in an engineering college in Saudi Arabia. I was impressed, at that time, by the facilities and staff.
When I first saw the BEST administered, 36 years later, I had a sense that it would be a suitable instrument for putting new students in with people, at perhaps a similar level of English language skills. When I found out that this test was also used to measure "progress," I became cynical.
I contacted a friend of mine in Virginia, whose wife was with CAL. I found out that the test had originally been designed for one time use and placement only.
This was reinforced by several factors:
A. There are only two forms of the test. Many other cultures have better memories than Americans, after coming from countries where the educational systems are based on memorization. After a student takes the test once and talks to friends about the other form, the test is no longer even slightly valid. The ALCPT has 60 forms and each one has been administered literally to thousands of adult military and civilian personnel from all over the world, who represent speakers of many languages and many levels of education. There will be a new computerized BEST soon, which will increase the number of "forms" available. However it would depend on all of the people fully trained in how to administer it, to remain in the area to make the results valid. It is unlikely that three years from now, these persons will still be THE testers. The ALCPT can be administered by a competent administrative person and remain valid. The BEST still won't measure PROFICIENCY, which is useful and necessary information.
B. There are very few people around now who have had COMPLETE and proper training on the administration of the BEST, even for placement purposes. I have observed at least ten people administering the BEST and find very little consistency in interviewer speech patterns or scoring procedures.
C. Some people seem concerned that the BEST tells about speaking ability and the ALCPT doesn't. The BEST tells about speaking ability in a limited and compromised context. The ALCPT measures language proficiency, in terms of how much English a person knows from all sources. IF that person has been in a balanced ESL training environment stressing first listening and speaking, followed by reading and writing, spoken proficiency will be predictable, IF that information is needed. However, for placement purposes, it is preferable to have students together who are ASSIMILATING the same amount of input at the same level. There are cultural and phonological differences as well as pedagogically different ESL backgrounds, that can hold students back orally for a while. However, if they are accurately placed with others at their level of comprehension, based on listening and reading skills, they will catch up with their classmates.
D. To my knowledge, there is only ONE spoken-language test in existence that is even close to objective. It is the Oral Proficiency Interview (OPI) developed by the Central Intelligence Agency and now used by the Foreign Service Institute and Defense Language Institute, English and Foreign Language Centers. Training persons to administer this test is VERY difficult. Even among experienced language specialists there is something like a 40% failure rate during training. Frequent "tuneups" of interviewers are required. This test is used for people who must have high oral proficiency in their jobs, such as air traffic controllers, communications specialists and spies. It is not needed, or probably even useful for classroom placement in most specialties.
E. From a practical viewpoint, the only way the AE programs are going to get anything close to a valid language measure is to use a standard objective proficiency test of listening and reading skills that is valid. I recommended the ALCPT because it is much less expensive than its commercial counterparts, and has an exceptional track record. The failure rate, in follow-on training due to "language problems," of allied military personnel, who have reached their required ECL (=ALCPT) scores, is around 2%.
F. I keep hearing that the ALCPT doesn't measure people at very elementary levels. This should not be a surprise, as I emphasized this fact during pre-ALCPT training, both in my presentations around the State for TEA and in writing. Persons with scores of 30 or under should be placed in elementary classes together and given skill-based ESL training. "ALCPT Tentative Civilian Interpretations," which is attached, was given to all participants in the ALCPT training. It explains the significance and usability of different score levels. ALCPT scores in the 40-70 ranges are normally consistent and quite useful for placement and estimates of future needed ESL training. Scores between 70 and 80 and above can be objectively used for exit strategies. We have recently used the ALCPT successfully, several times at Austin Community College to help persons make decisions regarding how much more ESL they needed, or didn't need, to move on into GED or higher education.
G. If the ALCPT has to be used to show "progress," students MUST receive between 100-120 hours in class to demonstrate some comprehension growth. This is true of ALL proficiency tests. Most students in the field trial apparently didn't have this many hours in class. If progress must be measured more often, due to administrative requirements, ACHIEVEMENT tests have to be used to measure specifically what was taught in the class. The alternative to either would be to measure ESL classroom performance skills. I am currently designing an alternative skill measurement system, the ESL Criteria Performance Measure (ESLCPM), that could prove useful in this context, although it is not a global proficiency measure. In my opinion, the ALCPT remains viable if it is properly used after an adequate amount of competent ESL training. The profiles of AE students that I have taught and observed indicate that many of them have family and work responsibilities outside of the classroom, that make it impossible, in many locations, to achieve stability in attendance. This makes the entire "progress" concept much more difficult to deal with. In some ways, the requirement itself has become part of the problem.
H. There was a comment from one participant in the TEA survey that the ALCPT was a "military English" test. It isn't. Out of 100 items, three or four questions may include rank such a "sergeant" or a term like "rations." This will not really affect the final score. This test was designed as a general English proficiency measure and has served thousands of students, military and civilian, in over 80 countries.