Computer-based assessment of mathematical competencies with WebCT

In this colloquium we describe theoretical and practical issues of computer-based assessment of mathematical competencies, which were applied to mathematics students in the Department of Mathematics, University of Joensuu, Finland.
The courseware platform WebCT provides an item database and allows to implement web-based tests.
In this contribution we give a short introduction to WebCT and a glimpse to the theoretical framework which we have used to assess procedural and conceptual knowledge about mathematical functions and binary operations.

Contents

1. The working environments we have used

The second step was to improve the worksheets into html forms
which could send the students' answers as email documents to the teacher, and
the correct answers or hints could be offered to the students immediately after they sent their answers.

Both of these ways of testing caused a great deal of manual correcting and organizing work.

Since most of the questions could be formulated as multiple choice or short answer formats,
we decided to embed our test materials in the
course management system WebCT.

2. WebCT

• it allows automatic checking of most test problems
• it supports a variety of data manipulations, and also because
• it was chosen around 2000 to be the only course management system that was technically supported by our University
• it has been widely in use at the Finnish Universities (later also Moodle has been adopted)

Thus, aside the WebCT tests, the WebCT course pages only contain general course information and links to the course material outside the system.
Not even the internal communication or discussion systems have been used, since the students seem to prefer ordinary email correspondence.

The test problems can be authored using plain text style or html code.
The latter allows the most important features we need:

• mathematical formulae
• static pictures and
• interactive Java applet figures

Link to WebCT (no longer valid!)

The problem body text and formulae have been first coded using LaTeX and converted to html in such a way that the formulae are either font supported text or table structures. The pictures and interactive figures are drawn in the document using links to www servers. The result seen on the screen is still far from a printable document, but usually quite comprehensible and quickly downloadable.

There are several advantages in using WebCT:

• it is easy to use for the students, and many students know the system in advance
• after attending the test the students can see the whole work sheet equipped with their own answers together with the correct answers and comments written by the teacher
• examples:  Functions, Internal binary operations, External binary operations
• the worksheet html source code can be sent automatically by email to the teacher
• at its best, a test can be corrected automatically; usually some revisions must be made
• data can be examined, manipulated and stored in many ways
• the system can be maintained and monitored from any internet connected workstation
• the system is supported by the institution

However, there is a modest aritmetical “calculated” problem type, which allows simple randomized problems to be used, and the system can check a numerical answer up to some tolerance level.
With this feature it is possible to create problems containing for example vectors.

Our test worksheets contain, in addition to multiple choice questions, also “short answer” questions, which can, at least in principle, corrected automatically according to a list of acceptable answers provided by the teacher. However, since the checking is done by comparing character strings, not all good answers are identified by the system.

Let us summarize the problems or deficiencies that we have found:

• the lack of support for (higher) mathematics
• the system is not easy to use for the authors, e.g. navigation is complicated and running slow
• the evaluation and teacher comments cannot be seen before submitting all the answers
• therefore the test system cannot be used efficiently for delivering learning material:
• NO “learning when doing”,  “exam as a learning tool” (except self-tests)
• it is not possible to correct manually all the answers to a certain problem consecutively

3. Moodle