Multiple choice guessing
Knowledge measurement audit
Item bank services
Excel and Office RAQ
Free Excel Add-in
Free PowerPoint Add-in
If you conduct multiple
choice testing, consider the following questions:
Are all of our questions
fair and well-written?
Does a test result reliably
indicate competence or otherwise?
Can we prove we have tried
to ensure our tests are effective?
Does our testing
useful feedback on the effectiveness of our training?
The iOTA (intelligent
objective test analysis) programme provides
an easy way to analyse your multiple choice testing and answer all of these
questions. Provided you currently use an automated testing system (e.g.
AdTest, TestMaster) or you transfer your manual marking to a database or
spreadsheet, iOTA can be geared to work automatically on data you already
The software analyses the
available sitting data for each multiple choice question and test, producing
a straightforward report, commenting on:
Facility – how easy is the
question and the test overall?
Discrimination – how well
does the question distinguish between a more able and a less able
Soundness – how far does it
appear that the correct answer has been identified and is being marked as
Distractor effectiveness –
which, if any, of the distractors (incorrect options) are not credible and
therefore reducing test effectiveness?
Correlation – how does the
ability of the candidates compare with previous sittings? Has the training
been effective for this group?
In financial services, iOTA
represents an excellent resource with which to face the
Regulator. You can
provide evidence that proper steps have been taken to ensure
testing produces a
reliable indication of competence.