Developing criteria for assessing CIS / CMC software - Jim K.

**** Original document dated 26 Nov 1997 ****

**** [xxxx - 2006] = small amendments made in 2006 to anonymise this document. ****

Having made a preliminary review of a limited number of available CMC/CIS software tools, I think there are a few issues that I would like to raise and clarify. In particular, if we can draw up some specific criteria at this stage, then we will be able to reduce the number of packages under consideration quite substantially.

Hopefully the points that I am raising here should complement the results of [xxxxx - 2006]'s user survey - I am trying to approach the systems from a more technical/functional perspective. The points can thus be divided into:

• functional criteria
• technical criteria

For reference only, I have also made some preliminary notes on the systems that I have looked at.

Views, feedback and additions to the following would be appreciated:

Functional criteria

There are 3 points here, upon which I would welcome some discussion:

1. The overall approach of a CIS system.

Having looked now at a few "courseware" software packages, I think that we may need make a distinction between two different "approaches" to the problem:

(i) at one end of the spectrum, we could go for a "Web-based information system", which simply helps to collate and organise lecturers' existing materials, bringing all of these resources together into one central accessible structure.

(ii) at the other extreme, many Courseware systems on the market (probably most of them) seem to be geared towards the creation of "Web-based learning applications", whereby the software tries to define your entire environment for creating and accessing the material. Many of these focus on the construction of entirely Web-based courses, and incorporate on-line assessment facilities.

Examples of the two approaches:

Fundamentally, a "Web-based information system" (type i) need consist of little more than a file upload tool, plus a set of scripts to manage the creation of HTML pages containing links to the uploaded files. CourseInfo is the only satisfactory example of type (i) that I have seen so far. You just upload Excel, Word, HTML or Powerpoint files, or links to external Web pages, and the software stores them and makes them accessible in an organised structure. The principle is really analogous to the simplistic script that is included on the links page for this project.

Examples of "Web-based learning applications" (type ii) include TopClass and Asymetrix ToolbookII - authors have to prepare their information in a way that is defined by the software (e.g. in TopClass you need to pass your documents through a filter which converts them from RTF, via HTML, to "TopClass Plug-n-Play format" - whatever that is).

Comments:

The more elaborate courseware systems such as TopClass undoubtedly have more "bells & whistles", and slicker interfaces. But what they gain in consistency, they seem to lose in flexibility. If we want to provide a system that is flexible, and accessible to a wide range of users with differing needs, without a huge initial learning curve, then should we be looking for a type (i) tool, rather than type (ii) ? A simple system that does a few things easily may be better than a complex, structured "virtual course environment".

The advantage of type (i) is that lecturers who are already creating timetables in Excel, and slides in Powerpoint, can just upload these files, without having to pass them through HTML conversion filters. We just assume that the user has got properly-configured helper applications, which will be true at least in the classrooms.
(One disadvantage is that someone will inevitably upload a virus-infected Word document at some point !)

2. CMC tools within CIS products

It is very obvious that there are *lots* of CMC (conferencing) products to choose from, but there seem to be fewer available CIS (course information) products. Many CIS tools also contain built-in CMC components, but I doubt whether the best CIS tool for us will also, fortuitously, contain the best CMC system. I suggest that we should evaluate CIS and CMC tools separately (i.e. not throw out a good CIS tool because it contains a bad CMC component). This may result in a recommendation of two different systems, for the two purposes.

3. Threading or non-threading in CMC tools

It would help to cut down the "field" of available CMC products if we were to decide whether we want a "threaded" or "non-threaded" type of CMC system.

By "threaded" I mean:

• the discussion is usually represented graphically, in a "tree"-type structure
• sub-discussions can "branch off" at any point from the main initial subject of conversation
• example: Ceilidh

By "non-threaded" I mean:

• once a new topic for discussion is created, any subsequent posts on that subject can only be added to the end of a growing, linear list of replies.
• usually, the linear series of replies is displayed on a single Web page - the user may then have to scroll down in order to read them all (long topics may be split into several pages).
• example: Caucus

I am quite prepared to be out-voted on this, but personally I think non-threaded is better (for reasons of simplicity again). Here are some of the factors that have occurred to me:

• threaded structures have the advantage that they give a graphical overview of the "conversation", which many people find helpful and appealing.
• however, unless the software is very clear, it is extremely easy to get "lost" in a threaded discussion (I do that frequently).
• a threaded discussion can easily get fragmented into a lot of simultaneous sub-arguments. In a linear structure, there is much more of a sense that the conversation is being guided in a single, productive direction, and everyone is "on board".
• it can be a pain that, in a linear discussion, you have to scroll down through all of the responses in order to get to the new ones (which are usually added at the bottom of the page). Some people, however, see this as an advantage, because it means that the user is forced at least to glance at the previous discussion before making their own contribution.
• another big advantage of having a linear discussion on a single Web page - users can print out the entire discussion in one go.
• with a threaded system, it takes 1 or 2 mouse clicks (and associated page-loading delays) to read each individual message. This rapidly gets wearisome, especially if you are waiting 30 seconds to read a message which turns out to be empty or un-interesting.

Here are some more arguments on the subject (from the Caucus forum).

Technical criteria