I  have long been interested in computer-assisted learning. It has been fun to watch the progress of this field during the past half century. Over the years, Computer-assisted learning materials have gotten better. I have been especially interested in the roles that Artificial Intelligence and computer simulations have played in this progress.  Recently my friend Dexter Fletcher sent me copies of two reports  from the Institute of Defense Analysis. These were publicly available reports on research being done on an intelligent tutor system that can help train navel personnel (DARPA, 2011).  Certain aspects of military training lend themselves to use of Intelligent Computer-assisted Learning materials. Trainees are learning to diagnose and repair faults in complex instruments and other devices. There are steady streams of new recruits who have to be quickly trained and educated to a high professional level of expertise.

The following is quoted from DARPA (2011). It suggests that the Digital Tutor system being studied is much more effective than two of the traditional human-taught courses being used.

Quote of Report Summary

 This report presents findings from two assessments of the Digital Tutor (DT) being developed by the Education Dominance Program, which is sponsored by the Defense Advanced Research Projects Agency. This tutor is providing initial specialized skill training (“A” school and some additional “C” school training) for the Navy’s Information Systems Technology (IT) rating.

The assessments determined how well the DT was meeting IT training requirements and preparing students for Fleet IT duties. These assessments were performed in April 2010, when students had 4 weeks of DT training available for their use and again in November 2010, when students had 8 weeks of DT training available for their use. Both assessments were performed at the Navy’s Center for Information Dominance (CID), Corry Station, Pensacola, Florida.

The April assessment used a Written Knowledge test to compare IT knowledge acquired by the 4-week DT students with that of students who had finished the “A” school Integrated Learning Environment (ILE) training, which takes on average 10 weeks to complete. The DT students scored significantly higher than ILE students, with an effect size (“sigma”) of 2.81 on the knowledge test. The DT students also scored significantly higher, with an effect size of 1.25, than CID IT instructors on the test.

The November assessment compared IT capabilities of four groups: the 8-week DT students, students who had completed ILE training, IT of the Future (IToF) students who had completed its 19 weeks of training, and CID IT instructors. The assessment again used a Written Knowledge test, which was taken by all four groups. However, it was supplemented for the DT and IToF students by Practical Troubleshooting exercises, Packet Tracer exercises, and interviews by a three-member Oral Examination Board whose members did not know which of the two training programs the interviewees had taken.

The DT students outscored:

• ILE students on the Written Knowledge test, with an effect size of 4.68.
• IToF students, with an effect size of 1.95.
• Instructors, with an effect size of 1.35.
•  IToF students in the Practical Troubleshooting exercises, with an effect size of 1.90.
• IToF students in the Packet Tracer exercises, with an effect size of 0.74 for scores not weighted for difficulty and with an effect size of 1.00 for scores that were weighted for difficulty.
• IToF students on the Oral Reviews, with an overall effect size of 1.34.

Final Remarks

Wow! Learn much better and in much less time! Of course, you must keep in mind that this is a type of technical training that lends itself to use of intelligent computer-assisted learning, and that a lot of money is being spent in developing these materials. The project provides good evidence of progress that is occurring in certain aspects of computer-as-tutor systems.

Reference

DARPA (2011).  Education Dominance Program: April 2010 and November 2010 Digital Tutor Assessments. Institute for Defense Analyses, 4850 Mark Center Drive, Alexandria, VA 22311-1882