This assessment was created to evaluate my level-two Arabic students on their language achievements after completing unite 4 and 5 from Al-Kitaab textbook. It is part of their quarterly assessment for the current school year. During homework assignments and classroom activities, I can form an idea about my students’ progress; but tests give me and the stakeholders an assurance about their language achievement. Tests also allow us to get a ‘second opinion’ about our students’ progress – they can help confirm our own assessments and help us make decisions about students’ need with more confidence, Understanding Language Testing (Douglas 2010).
: In this assessment, I modified two questions from the online textbook review resources, one is “fill in the blanks” and the other is a writing question. I created the rest of the test elements which include another three questions. The “fill in the blanks” question was designed as four paragraphs, with a visual for each one to present the theme. There are 3 to 4 verbs between brackets in each paragraph. These verbs need to be put in the right form. Students need to use their comprehension and reading skills in order to form the verb to match the proper tens, gender and or singular / plural within the sentences. In this question, I changed both the visual and verb choices in each paragraph to better fit the theme of the test objectives for unite 4 and 5. For the writing questions, students have five different pictures to write a caption for. I changed the pictures to focus on what the students worked on during the past quarter. For the rest of the assessment, I incorporated the learned unites' objectives with new types of questions that I never introduced to my students before. Accordingly, I introduced these types of the questions as classroom activity and homework assignments ahead of the assessment. I also added detailed directions to make it easy to follow.
Target population: This assessment is for two 15-years-old students in 10th grade at a gifted high school program. Their Arabic proficiency level is advanced novice / low intermediate. One student is a heritage learner who speaks some Egyptian at home. Her Arabic speaking skill is higher than other language skills. The other student had no knowledge of Arabic prior to this class. This is the second year studying Arabic for both of students. Purpose / Results use: This is a summative assessment to evaluate the students’ learning at the end of the third quarter. Usually, I provide students written feedback on their performance, which will count towards their report card.
The consequences of the assessment start by sharing the results with the head of the world languages department and each student’s achievement and recommendation. This result is added to the student’s report card. Lastly, we share it with the parents and the students.
OVERALL DESIGN:
Language skill(s): This test assesses reading, writing, listening and speaking skills.
The construct of this assessment is to evaluate the students' understanding of vocabulary in areas like ; weather reports, reading a Middle Eastern restaurant's menu and ordering food; and recognizing Arabic numbers by tens numerically and in words. It also evaluate students’ understanding for grammar rules such as; Nominal and Verbal sentence, Subject & Predicate, Tanween تنوين . In addition, culture concepts (Middle Eastern Food an Weather around the Arab country).
Students should show good command of creating basic sentence structure and proper use of verb conjugation; and using a wide variety of vocabulary and expressions they learned in unites 4 & 5. They should also show a strong understanding of grammar rules; which includes present tense, Tanween . تنوين In the speaking section, students should show clear and accurate pronunciation of the long and short vowels, and proper distinction between deep and frontal letters. In addition, they should show the ability to give reasonable answers based on their listening skills.
Task/item types and number: This assessment has two parts. The first part covers grammar, writing and vocabulary. The second part covers reading, listening and speaking skills. This part has four questions. The first one evaluates the student’s understanding of the units’ vocabulary, with 10 fill-in-the-blank sentences. Students chose the correct answer from a word bank. The second question is writing captions for five pictures. Pictures include; a large family portrait, a weather report, a Middle Eastern restaurant menu, a city's down-town, and a picture of food items. Students are asked to write a caption that consists of at least five sentences, using the learned vocabulary, to describe / talk about each of the pictures. The third question evaluates grammar understanding, by putting the verbs that are between parentheses, in the proper form of conjugate verb, make an adjective that matches the noun in terms of gender, or use plural form. The fourth question is for the students to complete a table that has English number. Students need to translate them to the Arabic numeric and words form. The second part of the test is oral. I read a brief weather report, and then ask the students to answer three questions with short answers and one open ended question. The students were provided a paper to write their notes while listening. Response format: Students will answer part one of the assessment in writing in the same test paper, and will answer part two verbally. Scoring / rating: This assessment has subjective and objective scoring methods. There are three discrete point questions where scoring is objective and the process is reliable. These are “fill in the blank”, number writing and verb change form. I created an answer key sheet for an easy scoring process. There is only one right answer for each of these questions, which makes the scoring for this type of question precise, with no room for judgment. For the writing caption question and the integrative part, I used subjective scoring which is a lengthier process. I provided the students a rubric / checklist prior to the test for practice, which is the same one I used for scoring.
I provide clear and constructive feedback with the results. I highlight achievements and provide examples of better answers where students lost points. Scores and feedback are shared with all stakeholders.
ADMINISTRATION:
I administer the test during regular class time. This assessment took 40 minutes for both parts (20 minutes each). There is a five minutes break between the two parts. For part one of the test, students need pencils to write their answers on the test sheet. The listening and speaking part is verbal. I read the listening piece up to three times. For the speaking part, students do not need to record their utterance, I use my rubric to rate their answers as they speak.
ANALYSIS and EVALUATION:
The following table explains the students’ assessment results:
One student scored 95% and the other scored 93%. My Arabic students liked discrete point questions. As a teacher, I also like it since its scoring process is reliable and suits a summative evaluation. There is only one correct answer for each of the three types of the discrete points in this assessment, which makes it very reliable to score. Both students received full credit for these type of questions. They also understood the importance of the integrated questions as it presented a snap-shot from real life experience. Integrative tests enable students to use various language skills at the same time, however, one disadvantage is that that it is hard to score and the reliability remains uncertain.
measures whether the assessment tool can bring consistent results and accurate indicators of the true results students can get. Some of the elements that can indicate that this assessment is reliable are that 1) there are clear and explicit instruction for each question, 2) students were familiar with the test format and technique; 3) three out of the five questions are objective, 4) students received a rubric to prepare for both the writing and integrated question prior to the test. However, reliability for this assessment can certainly improve. There wasn't a second rater available for this assessment. From, the results; I can see that the students achieved results close to the level I expected them to reach based on their performance during class. I can say that the discrete point questions were certainly reliable, but there is room to improve with the subjective questions (mentioned at the end of the report)
During test preparation, students got familiar with new question types and techniques. They also had the appropriate tools to practice the test objectives using a variety of their Arabic language skills. Practice writing assignments and integrative drills encouraged the students to bring their best ability to the class, in order for them to assure good performance in the test. It started with great motivation to gain grades, however the amount of communication both in writing and speaking during the prep sessions, greatly enhanced the students ability to use their Arabic skills in more effective ways. It added fluency, accuracy and speed to their responses. One student said that “I feel that the answers come more naturally now”. There was undoubtedly a good amount of class time spent on test prep, which I considered as part of the learning process. I suspect that had the student not receive that amount of practice, that their grads would not have been that high, or that they would feel the same level of confidence in their communication and language usage.
The test Impact on the students was evident. It increased the students' confidence in using Arabic more often in communicating with native Arabic speakers outside the classroom. They feel the pride that they can effectively communicate in Arabic, not just recognize grammar and memorize vocabulary. In addition, both of my students are interested in studying political science and they feel very good about their dream career when they receive high grades in Arabic, since it strengthen their college application. One of the students will use her results to apply for an Arabic summer program, to earn additional credits toward her high school diploma.
The assessment has a strong content validity since it adequately evaluates what the students learned in these two unites.
It also has strong face validity because stakeholders feel that the assessment evaluates what it is supposed to assess, and the results count towards and impacts the student’s quarterly report card.
egarding the consequential validity, the results of this assessment is part of the student’s report card and directly impacts their GAP.
Assessment Fairness: Since the test evaluates the students in elements that they learned, and since they are familiar with the questions’ type. Students were also provided with a rubric for the subjective question, to understand what is expected from their answers and to motivate them to improve their language production.
I also think that the assessment is practical, since students take the test during their regular class time and do not need any special arrangements or tools.
I believe that assessment showed a considerable amount of authentic components such as ordering food from a Middle Eastern restaurant, understanding weather reports and introducing one’s family.
Improvement
I would like to improve a few elements in this assessment in the future. First, I would like to have more test items to improve its reliability. It has been demonstrated empirically that the addition of further items will make a test more reliable Testing for Language Teachers (Hughes, 2003). Second, I fear that the students may have been over prepared for this assessment. There is a set of relationships, intended and unintended, positive and negative between teaching, learning and testing, (Cheng and Curtis 2012) The Impact and Washback: Implications for Teaching and Learning In Christin Coombe, Peter Davidson, Barry O’Sullivan, Stephen Stoynof, The Cambridge Guide to Second Language AssessmentCambridge. Lastly, I want to peruse extra training in the area of subjective scoring. I would like to improve my writing question’s rubric. I will also record the student’s verbal answers and rate it twice, to ensure I reach consistent results. Lastly, I plan to reach out to other colleges, to have a second rater.