Maple T.A. in the School of Mathematical Sciences
Semester 1, 2006
Overview
The School of Mathematical Sciences used the Maple T.A. Building Block in Blackboard for assessment in several large courses during semester 1 of 2006. This document summarises some of difficulties we encountered, outlines solutions to the problems that we resolved, and underlines some of the changes that still need to be made to it in order for the product to function more effectively within our environment.
Notes on Terminology: Blackboard is the software on which the MyUni online learning environment is currently implemented. Blackboard allows “Building Blocks” (which might also be described as plug-ins) to be registered in order to implement additional functionality. Maple T.A. is a Building Block that allows online testing and assessment of mathematically and scientifically oriented material.
Deployment
The Maple T.A. Building Block to Blackboard was made available in the production MyUni environment on January 25, 2006; post that date it has been available to any course registered in Blackboard, but to my knowledge the only courses in which it has been deployed are first year Mathematics courses. In particular, Dr David Parrott deserves sustained plaudits for having the courage to blaze a wide trail through this new territory. In semester 1 of 2006 he oversaw the use of Maple T.A. for fortnightly assessment in the following subjects:
- Maths 1A (595 students enrolled)
- Maths 1MA (200 students enrolled)
- Introduction to Financial Mathematics I (240 students enrolled)
By the end of semester we had collected around 10,000 marks from 30 different assignments set within these subjects.
Feedback and Load Considerations
Staff
As with any new system of assessment, there has been a larger than usual load on many of the staff who have been involved in subjects using Maple T.A. this semester. Unsurprisingly, different staff have adapted to the software to a greater or lesser degree, from those who have negotiated for their assignments to be set by an assistant to those who have taken the time to set a series of relatively challenging questions within the Maple T.A. environment. The School of Mathematics allowed staff who had not been trained in the use of Maple T.A. to have their assignments set by a skilled assistant for this trial, but will seek to minimise this herein. Given that all of the Mathematics staff are competent users of LaTeX (a mathematical typesetting language that can be used to construct assignments in Maple T.A.) there is a very low barrier to entry involved; we believe that one to two hours of general and LaTeX-specific training should prove adequate to construct assignments involving all question types (except perhaps of the rather open-ended Maple Graded type).
Students
Student feedback has been somewhat mixed, and this seems to be coloured to a significant extent by the restrictions on the feedback mentioned in the problems section below. Submission numbers for the large first year subjects have been similar to those achieved using the much-less-sophisticated testing mechanism built in to Blackboard; given the increased flexibility of the Maple T.A. based tests and the teething problems that we had this return rate is gratifying. Overall, however, we feel that student feedback will be decidedly more positive once the system has settled; in particular, all indications are that the feedback restriction will be lifted in the next major revision of Maple T.A., which is due late in 2006.
Informal conversations with students indicate that the actual process of answering a Maple T.A. assignment is quite straightforward: some of this can be attributed to the fact that many questions in the first semester assignments were simply mimicking the relatively straightforward questions asked in previous years when we were using less flexible software, but even answers to the newer, mathematically subtler questions showed little evidence of problems in navigating the Maple T.A. syntax and interface. There were occasional difficulties involving issues such operator precedence (eg, students entering “x^1/2” and thinking that would be interpreted as “x^(1/2)”) and capitalisation of variables (students failing to realise that x and X are not the same variable), but these really pointed to the need for clearer instructions about the submission of answers: that has now been implemented.
Technical Problems
Solved
Spaces in Surnames
One mysterious early difficulty was that certain students were not having their authentication credentials passed through correctly from Blackboard to Maple T.A.. Essentially, they became “unknown” students to Maple T.A., and were presented with a dialog that allowed them to sign up to the corresponding course in Maple T.A. using any username and password! This problem was consistently affecting about 5 per cent of Maths 1A, and we eventually tracked it down to a rather inflexible notion of what constituted a valid name within the Maple T.A. environment.
Enquiries to Maplesoft support eventually yielded an undocumented solution to this difficulty: the format of a name could be manually overridden by specifying a regular expression within a particular file in the Maple T.A. installation. With this modification in place the problem no longer exists.
Stability and Load
There was an isolated incident in which the Maple T.A. application crashed during a test period. It was quickly restored and we have seen no difficulties in terms of load on the server or crashing of the application since. The application has also been moved to a more powerful server (bought as part of the enabling Strategic Initiatives Grant) since that incident.
One important aspect of MapleTA that is well worth noting is that assignment creation, editing, and testing are all very much real time processes: there is no noticeable lag at all. The comparison with Blackboard’s (ie MyUni’s) built in testing environment is quite startling. Alternative tools for test construction are very much optional with Maple T.A., whereas they had previously been essential (and still far from ideal, particularly given that Respondus is only available for PC’s running Windows).
Open
Restrictions on Feedback in Maple T.A. Assignments
One of the main selling points of Maple T.A. is the ability to provide timely, helpful feedback to students about their work. This ability is somewhat stymied in the current version of the Blackboard Building Block, at least in the manner in which it is being used in the large first year Mathematics classes. In particular, we require all students to take the same assignment, and allow a three day period in which they can enter their results. Upon submission the students are given a mark out of ten, but no feedback about which answers were correct and which incorrect. This is, of course, to prevent the circulation of correct answers before the end of the assignment period. However, the way in which Maple T.A. parameters are mapped to the parameters of the corresponding object in Blackboard means that the closing date of the test is the date after which the object is no longer accessible within MyUni. Critically, this means that students cannot access the assignment after the closing date, and hence cannot compare their results to the correct versions.
Note: this capability is present in Maple T.A.; indeed, instructors can view a student’s answers via the administration palettes in the Maple T.A. control panel, but the student has no access to the same display.
Modification of Assignment Parameters
Whenever one or more students are taking an assignment (either actively, or if they have “suspended” the test using “Quit and Save”) that assignment is locked, and none of its properties can be modified. So, for example, if the solution to a problem is incorrect and an instructor wishes to change it the only hope is to “jump in” before any students have caused the locking of the assignment to occur or “between” such locking events. In a course with several hundred students this is almost impossible post the start date of the assignment, so errors are effectively locked into the system once the test is opened. Perhaps more importantly, simple properties such as the end date of an assignment are also locked in the same way: this removes the possibility of extending the time available for a test, or (for example) responding to difficulties with server access.
We have communicated the desire for easier and more practical ways to alter the assignment parameters within a live test and these will be present in the next version of Maple T.A.. While it is obvious that not everything can be changed while the assignment is “live”, we are hopeful that the more “external” properties such as end date will be readily extended without having to worry about whether a student is currently completing the assignment.
Migration of Courses
In the current version of Maple T.A. course registration is keyed on the Course Name in Blackboard. Unfortunately this is far from unique, as the same course in a different semester will have the same course name. This means that a course such as “Mathematics 1A”, which runs in both semesters, needs to be deregistered at the end of semester 1 before the semester 2 course can be registered, and the deregistration effectively locks staff out from the extant Maple T.A. content. This can be worked around by exporting the course content as a “Module” from within the Maple T.A. Control Panel, but it would certainly be preferable for multiple instances of a course to be able to access the Maple T.A. tools simultaneously.
With this is mind the saner choice of mapping to the Course ID in Blackboard has been requested, and the next version of Maple T.A. will provide a way of registering courses that will allows multiple instances of courses with the same course name to co-exist.
Summary
On the whole the move to Maple T.A. has been quite successful. After finding solutions to the problems mentioned above the load on coordinating staff (and affected students!) eased considerably, and the times when assignments were open passed extremely smoothly. It is easy to forget the pain of the old Blackboard test environment, but that pain (both in terms of the severe restrictions on question types and the time required to construct the questions) was considerable, and has now been largely eliminated. As detailed above, there are still aspects of the Building Block that require rewriting (and we are actively following up with the vendor about those aspects, with various positive indications as listed above), but it is already clear that the product has the potential to do exactly what we are seeking.
For semester 2 of 2006 we will
- continue the use of the product at Level I
- extend its use into subjects at other year levels
- take greater advantage of the flexibility available within Maple T.A. through the use of more mathematically sophisticated assignments and question types
In addition, we aim to encourage the adoption of Maple T.A. within other Faculties by
- presenting introductory seminars and
- producing documentation regarding the introductory and advanced use of the software
Paul McCann ( paul.mccann@adelaide.edu.au )
School of Mathematical Sciences
University of Adelaide