Pre-test
Experimented
group
* Self-Reflection Questionnaire * Learner attitude toward oral communication Mediation
Post-test
Learner awareness Questionnaire
Portfolio assessment ? Record-keeping ? Self/peer evaluation ? Teacher-student conference Learner awareness Questionnaire Traditional assessment ? Paper-and-pencil test ? Oral proficient test Controlled
group
* Self-Reflection Questionnaire * Learner attitude toward oral communication experiment
Figure 3. The process of the 3.3 Data Analysis
The analysis centered on emergent patterns induced from the observations, the questionnaires, and student portfolio. The classroom observation portrayed the ways students interacted with teacher and student?s attitude toward evaluation process. For data collected from interest survey and student learning attitude questionnaire, categories included: a.) The student? interests toward oral communication (Appendix I). b.) The general attitude toward practicing oral communication (Appendix II). For data collected from student perception and feedback questionnaire (Appendix IV), categories included: a.) Q1-4: students? perception of the connection between assessment, learning, and instruction. b.) Q 5-8: students?
16
attitude toward authenticity through engagement. c.) Q9-12: students? development of autonomous learning. Besides, data collected from student portfolios showed the evidence for self-reflection and the evaluation process. The language samples were tape-recorded, transcribed, and analyzed. 4. Findings
4.1 Classroom observation
In the experimented group classroom, most of EFL college students had no previous experience of the portfolio assessment. The advanced learners were more self-regulated, active in following teacher?s instructions, participating classroom activities, and responding to peer-evaluation sheets. However, the less skilled learners reluctantly engaged at first and easily got lost. Therefore, the teacher/researcher adjusted the instructional goals (refer to Appendix V): Task 4: self-selected topic good for independent searching and thinking. And, Task 5: teamwork by which lower learners could benefit from peer assistance. 4.2 Self-reflection
According to background and interest survey, it was found out
17
that student preferred communicative, authentic, and applicable topics: pop song (66%), food (58%), travel (47%), shopping (45%), and films (41%). The difficulties they encountered most often derived from lack of confidence on speaking out and listening in, whereas they thought their proficiency level were enough to read independently. The experimented students showed their stronger commitments to learning and willingness to collect their recording works.
The following tables suggested the questionnaire pre-tested general attitude toward practicing oral communication (Appendix II) before the implementation of assessments. It was shown that both groups demonstrated similar patterns. They all felt interested in oral communication courses but the current instruction hours are not sufficient and the conversational activities are not so motivating. Three out of fourths expected that their communicative abilities needed further improvements. Most importantly, toward questions 10 and 11, they had great concern with teacher?s error correction and scoring system. Still, they hardly spoke well in new context and had difficulties in employing appropriate expressions (questions 8 and
18
9).
Figure 4: A comparative study of learner’s attitude toward oral communication
Learner's Attitude Towad Oral Communication403020100123456789101112strongly disagreedisagreeagreestrongly agree Table (a) Experimented group
Learner's Attitude Toward Oral Communication35302520151050
strongly disagreedisagreeagreestrongly agree123456789101112
4.3 Students’ perceptions of portfolio assessment
The questionnaire revealed that the portfolio (experimented)
Table (b) Controlled group
group had a feeling of growing awareness, higher interests and controlling progress. More than 95 % of participants choose “agree” or “strongly agree” with the portfolio approach arising learning interests. About a half of portfolio group agreed the number of language task was acceptable, but still a half of this group thought there was too much to carry out. As a matter of fact, the
19
content of the portfolios required the integration of four skills. Furthermore, over three-fourth “agree” or “strongly agree” with the concepts of commitments to real-life situations and connection to daily basis. They had attempted to seek for external resources for assistance with serious attitude. As for the degree of responsibility for self-learning, the results indicated that more than three-fourths certified “portfolios recorded learning achievement” and “portfolios improved speaking and listening “. Although over 95% of learners responded to “agree” or “strongly degree” with “portfolio enriched the content of conversation”, more than 65% of them failed to fulfill “portfolios helped self-monitor my errors” partially because of the limited linguistic knowledge.
On the contrary, non-portfolio (controlled) group expressed that traditional assessment led both oral communication course to be less motivating and conversational) activities to be less diverse. The results reflected over 85% of this group thought the learning pressure existed in standardized tests. And, it was not so possible for traditional assessment to connect language use in real-life context. Based on traditional assessment, learners preferred
20