CESI Conference January 2002 St.Patrick's College

Gabrielle O’Callaghan,Centre for Research in IT in Education,Department of Computer Science,Trinity College, Dublin

Abstract
In this knowledge-based society of the information age, e-learning, particularly in the context of lifelong learning, is being touted as “the killer app.” In order to test the hyperbole, and more specifically, student perception of their learning in an online learning environment, e-learning was employed as an adjunct to the teaching of course modules to students at further education level, both on-site and off-site. A web-based questionnaire using specifically designed software was utilised to quantify the students’ assessment of learning gains and to quantify teachers’ expectations of e-learning outcomes.

Introduction
Much of the literature on the subject of e-learning extols the potential of the technologies for enhancing the learning experience. There is very little hard evidence, however, regarding learners’ attitude to online learning; their responses to online as opposed to traditional classroom learning; their future use of online learning in their lives and the particular elements in online learning which learners find useful.

Much of the literature also berates traditional educators for their slow take-up of online learning elements in their teaching. Most commentators attribute this to teachers’ lack of ability and enthusiasm for incorporating e-learning into the traditional classroom experience. Many of the corporate e-learning ventures stress the importance of incorporating a wide range of communicative elements within the e-learning environment without producing the empirical evidence for their usefulness for learners.

A group of teachers and students who had recently completed courses which incorporated a significant amount of e-learning agreed to respond anonymously to a questionnaire which aimed to evaluate their response to their own e-learning experience and the gains which they perceived they had made from the various online elements of the course. Where appropriate, the respondents’ perceptions of gains from offline and online components were requested for comparison. The questionnaires also aimed to evaluate respondents’ likely future use of e-learning.

Teacher Expectations
In this study, 9 of a group of 22 teachers responded to the Student Assessment of Learning Gains (SALG) online questionnaire. The SALG is a web-based instrument consisting of statements about the degree of “gain” (on a five-point scale) which students perceive they have made in specific aspects of the class. Instructors can add, delete, or edit questions. The author’s questionnaire was designed to elicit students’ perceptions of the learning gains they had made from both the traditional and online elements of the course.

Respondents are guaranteed anonymity in their responses. Respondent identification is assigned by the program and is used only for the purpose of checking that all members have completed the survey.

The scale chosen for the instrument is not a true Likert scale that has a neutral mid-point with two options above and below it. The SALG authors wished to give respondents the option to distinguish between four possible levels of “gain” from “very little” to “a great deal,” as well as a “no gains” and a “not applicable” option. The authors say that instructors may regard averages on particular questions that are above 3.0 as “positive,” and averages close to 4 or above as indicating a “good” or “very good” level of perceived respondent gain.

The questions were formulated to obtain respondents’ expectations of the usefulness of online elements in learning and teachers’ proficiencies in these areas. The teachers responding to the questions were also reflecting their own experiences of e-learning.

The responses revealed a very positive overall reaction to e-learning components. Out of 63 questions, 53 responses attracted ratings of 3 or above.

Online elements which received ratings of under 3 included:

The most highly-rated online elements included:

The following tables reflect the main findings from the teacher SALG questionnaire. In each case where there was direct comparison of offline and online features - question 1 below - the offline elements were preferred by the respondents. Questions 2 to 5 inclusive reflect the teachers’ ability and self-confidence in using online learning and indicate their future use of e-learning elements.

The responses in questions 2 to 5 inclusive reflect a high level of self-confidence in the teachers’ use of online learning tools and bode very well for teachers’ future use of e-learning. The use of online resources for research and communication feature particularly highly in their responses.

Student Responses
The author currently teaches several modules on a one-year, full-time, further education course entitled Electronic Communications and Multimedia (ECM). All of the students work part-time in order to support themselves financially and many have other family commitments. As a result, none of the students achieve full attendance, and for many, their attendance level is quite low.

This has resulted in frustration for both teacher and learners on the course and was the impetus for the author’s initial investigation into the feasibility of providing an online learning resource for the students. As the students study the modules The Internet, Web Authoring and Multimedia as part of the traditional course, and as all have access to a computer with connection to the Internet at home, providing an online learning resource for them seemed an obvious partial solution to the problem.

The author used Blackboard Online Learning Environment to create an online course for her students, which served as an adjunct to the traditional classroom course. At the end of the academic year, the author utilised the Course Statistics feature in Blackboard to track and quantify the extent of students’ usage of the various sections of the online course.

The OECD report Student Views on Learning with Information and Communication Technology (OECD 2000) reflects many of the frustrations that both students and teachers experience when using the Internet as a learning tool. Many of the problems stemmed from the inexperience - both technological and pedagogical - of teachers in a new, unfamiliar milieu. The students’ responses show both embarrassment (for the teachers’ lack of knowledge) and frustration at a less than useful experience.

Hara and Kling (2000) note that there are few systematic analytical studies of students who have experienced new technologies in higher education. Their article presents a qualitative case study of a small graduate-level web-based distance education course at a major US university. It examines students’ distressing experiences due to communication breakdowns and technical difficulties. They assert that this topic is glossed over in much of the distance education literature written for administrators, instructors and prospective students.

In order to avoid similar experiences for her students, the author waited until the students were totally Internet-literate and competent before she introduced them to the online environment. The students were competent in email, web navigation, online chat, online messaging, file transfer protocols, discussion board forums and computer file management before they began work within the online course. The author demonstrated the online learning environment to the students and ensured that all the students were able to navigate around the environment and utilise all its features.

The Blackboard online environment is made up of four main sections:

· Content (Main Page, Course Information, Staff Information, Course Documents, Assignments, External Links)

· Communication (Send Email, Student Homepages, Group Homepages, Posted Discussion Message, Virtual Chat Room, Student Roster)

· Students (Check Grade, Send file to instructor, Edit Homepage, Search)

· Groups (Access Group Area, Send Group Email, Send File to Group, Group Virtual Chat)

The author anticipated that as group project work forms a major part of the ECM course, students would profit most from the interactive communication and group sections. As the author had ensured that students had full hard copy information on the issues that feature in the content section, she anticipated that the students would use this section least.

At the end of the academic year, the author accessed and analysed the statistics for student access of each of the various sections and sub-sections of the learning environment. The following charts and tables illustrate the results.

Chart and table showing statistics of student usage of Blackboard Online Learning Environment:

Student usage of Blackboard Areas:

Chart and table showing statistics of student usage of Contents section of Blackboard Online Learning Environment:

Student usage of Content Sections:

Chart and table showing statistics of student usage of Communication section of Blackboard Online Learning Environment:

Student usage of Communication Sections:

Chart and table showing statistics of student usage of Student section of Blackboard Online Learning Environment

Student usage of Students Sections:

Chart and table showing statistics of student usage of Group section of Blackboard Online Learning Environment

Student usage of Groups Sections:

Contrary to the author’s expectations, Content proved to be the section of most interest to the students (87%). Despite all the hyperbole in the literature, Communication attracted only 9% of student usage. Student and Group sections were hardly utilised by students - 3% and 1% respectively.

From the above statistics, it can be inferred that the items of most use to students in the online course were:

Student Assessment of Learning Gains
In this study, 7 of the 15 students responded to the Student Assessment of Learning Gains (SALG) online questionnaire. The questions were formulated so that a comparison could be made of students’ perceived gains from traditional classroom elements and online elements of the course.

The following tables give the averages of students’ responses to the questions. Question 1 compared online with classroom elements of the course; questions 2 to 4 inclusive reflect the students’ perceptions of the gains they made in developing online skills and question 5 attempts to evaluate the students’ future use of e-learning.

In informal discussion with the students and from observation, the author discovered that the students made considerable use of email and online chat, both for general communication purposes and communication with each other and that file attachments to email was the students’ main means of distributing large amounts of information. When questioned about their very low use of email and chat within the Blackboard environment, the students responded that it was easier to use web-based email such as Hotmail, and online chat environments (for example Microsoft Messenger) rather than having to log into Blackboard and navigate to the relevant section.

Although understanding of e-learning rated extremely negatively at 1.75, how to learn online was rated 3.4. A rating of 4.33 was recorded for students who believed that they would use online learning in the future and students rated their ability to use online learning resources at 3.67. This apparent anomaly can be explained by some of the students who approached the author after replying to the questionnaire to say that they had never heard of the term e-learning and did not know what it meant. The students’ responses do indicate a very positive attitude to online learning and its place in their future lives. Online research methods, online communication with others, online communication with teachers were regarded very positively as elements that the students would remember and carry with them into other aspects of their lives. Students responded very positively to appreciating the learning possibilities provided by the Internet such as the ability to use online learning resources and feeling comfortable with online communication.

The rest of the responses can all be rated as positive, but in each case where there was direct comparison of offline and online features, the offline elements were preferred by the students.

Conclusion
It is important to emphasise that this was very small-scale local research. It does indicate, however, that overall teachers’ and students’ attitudes to e-learning were very positive - both in terms of their previous experience and its future role in their learning and research. Indeed, online resources for research and information gathering would appear to figure highly in their perceptions of their future use. Respondents indicated a high level of self-confidence in their abilities to use online learning tools.

The students’ main use of the Blackboard environment was accessing its material contents, rather than utilising its communicative tools. All responses seemed to indicate that participants regarded online resources as a major tool for research and for communication.

Some online elements - such as MOO’s MUD’s and Discussion Boards - were unpopular with respondents. The most popular elements were email, chat, network peer resources and, for the teachers, video conferencing. Students indicated that they preferred dedicated communicative programs - such as Hotmail - rather than the communicative tools bundled within the Blackboard environment. The high level of gains perceived by respondents in their interactions with the course teacher appear to bolster the importance of the teacher in the learning experience - whether online or offline.

One of the most interesting results, however, was the marked preference on both the part of the teachers and the students for offline rather than online resources. This was particularly evident in respondents’ preference and perceived gains from teacher and peer interactions and would appear to firmly re-establish the importance of real-world over virtual interaction in the learning environment.

References

Noriko Hara, and Rob Kling (2000). Student distress in a web-based education course. Information, Communication and Society 3.4 Available: http://www.infosoc.co.uk/00112/ab6.htm.

OECD (2000). “Student Views on Learning with Information and Communication Technology.” Aix-en-Provence, France: Organisation for Economic Co-operation and Development.

OECD (2001). “E-learning: The Partnership Challenge.” Paris: Organisation for Economic Co-operation and Development.