abstract

many university students have been mainly exposed to teacher-centered learning and do not generally have student-centered learning experience. an appropriate learning and teaching strategy needs to be applied, so that students can acquire the necessary skills for employment and become active learners. the author has undertaken a learning and teaching project to introduce the problem-based learning (pbl) approach, which uses problem cases that enable students to acquire problem-solving, self-directed learning, creative thinking, and collaborative learning skills. web technology plays an important role in supporting the pbl processes to foster learning and teaching.

this paper describes an approach to teaching information systems analysis through pbl with technological support to enhance learning and teaching. a statistical analysis of the feedback from 51 students comprised of ‘a’ level school leavers and technical institute graduates, is presented. this includes the perceived value of using the pbl approach, and the similarities and differences in the learning behavior of the two groups of students. the results obtained from this study indicate that the success of pbl with technological support is dependent on a number of factors. the problems that have been encountered by the teacher in the pbl process are also included.

keywords: internet-based educational system, problem-based learning, computer-assisted learning and instruction, computer-supported collaborative learning

1. introduction

the author undertook a learning and teaching project to introduce information systems analysis through problem-based learning (pbl) in september 1999. the subject is for undergraduates in the higher diploma in information systems and the students were comprised of ‘a’ level school leavers and technical institute (ti) graduates. these two groups of full-time students have different educational backgrounds and capabilities, in that the ‘a’ level school leavers do not in general have a good computing background, whereas the ti graduates already have two years of education in computing. however, students with an ‘a’ level background are in general more proficient in english. this group of students has been mainly exposed to teacher-centered learning and does not generally have student-centered learning experience.

the subject of information systems analysis requires students to develop logical, analytical, and critical thinking skills for problem-solving. it has been observed in the past that students do not prepare or read sufficiently. also, the majority of our undergraduate students do not have work experience, therefore they should be exposed to real-world problems. an appropriate learning and teaching strategy needs to be applied and envisaged that students can learn better with the pbl approach, which is facilitated by technological advances. this paper describes an approach to teaching information systems analysis through pbl to achieve quality learning and teaching. a statistical analysis of the feedback from 51 students comprised of ‘a’ level school leavers and technical institute graduates, is presented. this includes the perceived value of using the pbl approach, and the similarities and differences in the learning behavior of the two groups of students. the results obtained from this study indicate that the success of pbl with technological support is dependent on a number of factors. the problems that have been encountered by the teacher in the pbl process are also included.

2. adoption of a hybrid approach to pbl

problem-based learning is a curriculum design and teaching/learning strategy which recognizes the need to develop problem-solving skills as well as the necessity of helping students to acquire the necessary knowledge and skills (boud & feletti, 1997). the amount of direct instruction is reduced in problem-based learning as students assume greater responsibility for their own learning. a hybrid approach to pbl was chosen and considered to be more appropriate for undergraduates. a full pbl approach with less lecturing, more students working in small groups, and more self-directed learning, may not be workable as most students are accustomed to teacher-centered learning and may need some time to get used to the pbl environment. since their academic achievements may not be the best, it may be too ambitious to expect students to be able to read well and provide answers to ill-structured problems. a change to a full pbl approach needs to be gradual. the information systems analysis subject is one of the few to promote pbl in the higher diploma curriculum, thus students might perceive different learning and teaching treatments, which require more self-directed learning to which they were not accustomed. students received two hours of lectures and a one-hour tutorial per week for 14 weeks. it is an institutional constraint that teaching activities need to follow these patterns. however, there was a gradual reduction in formal lectures for knowledge dissemination, and these were replaced with more class discussion activities towards the end of the semester.

a web-based support system has been designed to facilitate the various stages of the problem-based learning /teaching strategy. students are able to access teaching materials on-line. they can use the web for the purposes of discussion, development work and learning resources, and good pieces of work are posted for information sharing and exchange. apart from the technological considerations, team formations coupled with an effective assessment mechanism are required to encourage students’ participation in group-learning situations. a pbl environment is a student-centered environment, in that it requires students to work individually or as a team.

3. the www facilitates pbl

development of good learning practice through knowledge transmission

the www is an effective mechanism for distributing educational materials to students. there is a growing trend that universities are using the www in direct support of teaching and learning. the use of the web as a teaching resource can benefit students (milheim & harvey, 1998; sloan, 1997). students can access learning resources on-line (lecture materials, self-assessment questions, references and urls for topic addresses). information of this nature is useful to students and can be updated easily. essential materials are presented in note form. review questions on the different topics are made available to ensure that students understand the subject matter. the www also solves the problem of photocopying overhead projector transparencies in the production of handouts.

having teaching materials on the www not only facilitates knowledge transmission but also supports problem-based learning. when students are in the process of problem-solving, they can make reference to the learning resources, which provide background information and can pave the way to initial information-seeking. students should be encouraged to use the www for information resources, in addition to information-gathering in the library, as they are able to learn at their convenience from a wealth of information beyond the classroom and beyond the campus. posting teaching materials on the web can aid the development of students’ good learning practice. they are encouraged to prepare ahead of class sessions and to raise questions for discussion. there is no doubt that students understand the subject better. the atmosphere of a lecture session is enhanced when there is increased participation and discussion in the class. this should promote more active rather than passive learning.

the www supports self-directed and small group learning

to be effective in pbl, students work in small groups. after receiving a problem case, each team has to identify the learning issues and apply them to the problem situation. a project plan, with all tasks in logical sequence and schedule, needs to be drawn up. the assignment of individuals to each of the tasks is determined by the team leader to ensure that every member has an even share of work. their work plan is put on the web, through which the teacher can provide feedback via the web or in tutorial sessions. the completion time of each task is marked. thus, students practise project management through planning and controlling by viewing all activities on the web. their skills are refined as they progress with more practice on different problem cases.

it has been observed that students used the www more than the library for information searching. the www is an excellent tool for searching but students often encounter an information overload problem, as there is a lot of information and they do not know how to filter it. students should be trained to select the relevant information rather than all the information. more importantly, they should have skills and knowledge in the use of the various www search engines, and in employing efficient search strategies in information-seeking.

the www is an automatic development environment through which individuals’ work and the results of their teamwork can be accessed, commented on and refined by team members electronically. it enables students to create knowledge. students learn from each other within a team and also observe good work from other teams. the progress of the tutorial sessions can be more effective and productive, since both students and teacher can review the work done by each individual in each team before the tutorials. they have increased time for reflection on the problem. shorter meetings and higher learning productivity are to be achieved. it is often the case that students cannot see the work done by other team members before a meeting and thus time is wasted unnecessarily during meetings. the use of the www for brainstorming enables creative ideas to be generated among the team members. small group discussions enable them to exchange experience through communication skills and to learn about the experience of other team members. this approach to learning facilitates the development of students' interpersonal communication and social skills (chimi 1997).

display of good work to enhance learning

one of the important teaching and learning issues is to motivate students to learn better and to work harder. the purpose of having a posting facility is to create a co-operative learning environment where students benefit from the activities of their peers. at the same time, the posting of good works by students will provide higher motivation for them to perform well.

4. other considerations for effective pbl

team formation for collaborative learning

the formation of student teams has an impact on collaborative learning. the class was divided into teams, with six students per group. the teams were formed with mixed academic abilities and genders, so that the stronger students tended to encourage and motivate the weaker students to achieve better performance levels. at the same time, the stronger students could practise their leadership skills and serve as consultants for their teammates (buffington, 1998). however, each student should have the opportunity to develop leadership skills, although students should be adaptable to a new team situation with people having different personalities and experiences. teamwork can be enhanced with technological advances through better communication and electronic means of sharing information.

assessment of students

the assessment of this subject is continuous, without examinations. students are assessed by a peer review system, with the intention that every student should contribute positively in a teamwork situation. peer-grading can be performed after each problem case has been completed, and the individual performance, such as presentation, tutorial contribution, individual work and team contribution can be measured. there are three components in the formal assessment of each student. they include individual contribution, group work as a whole and peer evaluation by students themselves. individual assessment is based on the work produced individually and also the contribution to team work discussions. also, each individual is requested to submit a short report reflecting what (s)he has learnt. it is common that individual and group marks are given equal weighting. good students should not be penalized if they have been assigned to work with other less hard-working students. students are allowed to assess other team members’ efforts, as this encourages each team member to contribute to the teamwork situation, and the team leaders and/or good students deserve higher weighting. the peer evaluation of students relies on the judgement and honesty of the students in making such evaluations. however, good students in general have been rated highly by their teammates.

5. problems encountered by the teacher

virus

although students had been warned to have their pcs disinfected with anti-virus software, it was found that viruses existed in files in which the students submitted their work. it would be somewhat inconvenient to scan each of the files before opening it or install a modern anti-virus software to periodically scan the system, as this would slow down the system for other functions.

heavy workload for teachers

the workload for the teachers includes: monitoring the students’ discussion on the web and providing them with timely feedback, printing reports for assessment, assessing and commenting on their reports and presentation files, check their peer evaluation, calculating the results of their peer evaluation for each case, and posting their best reports for review. the workload has been found to be very heavy compared with the traditional teaching method.

file name standard

when students submitted their work, both as individuals and teams, they used all sorts of file names which lead to the report identification problem of not knowing which teams had submitted their work. the content of the file had to be read for identification, and sometimes revealed no identity. it would have been better if some standard file name had been introduced, such that the individual or team and the problem case could be easily identified.

printer requirement for on-line work submission

the courseware allows students to submit their work on-line, and students enjoy this facility. however, it is not easy to view the large volume of students’ work on the screen. the amount of printing, including the report and presentation file, for one problem case is considerable. this created some problems for the teacher, as he does not have a laser printer and has to share a common laser printer with other members of staff. the generation of the reports might take some time when the printer was busy and in particular when other people did not attend to their print jobs, which held up the printing process.

6. statistical analysis

51 students participated in the pbl study. all of these students were requested to fill in a questionnaire consisting of 60 questions in eight sections, namely parts a to h (see table 1). there were 26 students who were ‘a’ level school leavers, and 25 who were ti graduates. the statistical analyses are based on the responses of these 51 students. various statistics analyses have been performed to see if there are any differences between the two groups of students as regards the learning effectiveness of the pbl approach, self-directed learning, and problems encountered in pbl. the first part of the analysis looks into each of the questions. the second part is to see whether there is any significant difference on each section, and the correlation between the sections. finally, a regression model is established to determine the significance of sections that can affect the learning effectiveness of the pbl approach.

table 1
description of questionnaire

part number of questions description
a 12 pbl approach to learning
b 5 your perception of learning effectiveness
c 4 viewing good work of other teams
d 4 video-taping your presentation
e 5 having peer evaluation of your team members
f 9 the facilities in the web site support
g 10 learning effectiveness
h 10 problems in the pbl processes

a more conservative non-parametric test (mann-whitney u test (sprent, 1993), a non-parametric two sample t-test) was employed to test the differences between the two groups of students on each question. students were asked to rank each question on a five-point likert scale (1-representing ‘strongly disagree’ and 5 representing ‘strongly agree’). as the distribution of the responses of the two groups of students was not normal, it has been found that:

1. the ‘a’ level students are more confident in oral communications than the ti graduates (with p-value = 0.047 < 0.05) in part a, pbl approach to learning.

2. the ti graduates have more problems than the ‘a’ level group in the following areas:

§ “expressing their satisfaction and dissatisfaction” (p-value = 0.018) in part e, having peer evaluation of your team members.

§ “information exchange and sharing” (p-value = 0.006) in part f, facilities in the web-based support.

§ “project controlling” (p-value = 0.004) in part h, problems in the pbl processes.

§ “report writing” (p-value = 0.019) in part h, problems in the pbl processes.

however, the ti group members found that they learned more from the teamwork approach than the ‘a’ level group (p-value = 0.046) in part g, learning effectiveness.

with each section (factor/variable), the differences with other sections have been compared, on average, between the two groups of students. the variances of the two groups have been validated by levene’s test (f-test of ratio of variances) of equality of variances. it has been found that there is no evidence of unequal variation existing between these two groups on each of the sections. hence, the independent t-test was utilized. it was found that the ‘a’ level graduates found the facilities in the website support more helpful to them than the ti graduates (p-value = 0.044).

in order to measure the association among the eight sections (factors/variables).