Written Tests in Industrial Training - Five Tips

Did the title of this post give you the jitters? Assessment and evaluation of trainees has been a busy topic of recent in heavy industrial maintenance. Some have been asking for more written tests while others have expressed disappointment in trainee test results.
Why all of the interest in tests? Perhaps it is way of asking trainees to appreciate the expenditure and investment being made and to take the training seriously. This motivation is sound enough given the economic conditions under which many companies are operating this year. The written test can be a useful tool for reviewing key content covered in the training and for assessing the transfer of knowledge to a certain level. It also has some obvious limitations. Below are some of my thoughts on the written test.

Written Test Limitations

As a key tool in assessing worker performance the written test has limitations. Here's a few;

1. For many, our early school experiences cause our whole body physiology to change to an extremely nervous state when we're told that we'll have to take a test. I know that I am often in this camp. This factor by itself can influence how effective the training session will be when many of the participants are obsessing with the test to come and worried that they might perform poorly. Are they really in a position to get the most from the training session?

2. A written test that is not well designed for the for the learning styles of the trainee group, turns out to be a test in how well one can take a test. Basic trainee literacy issues can surface at this point as well. Computer hosted tests are notorious for these issues.

3. Tests questions that primarily require factual recall, only test short term memory. Most of us who haven't taken the written component of a driver's test in a few years know that we have forgotten some of the details in the laws that govern traffic and highway use. Many companies use a written test to document their diligence in safety training. But if the training is not refreshed until many months later, are the trainees still knowledgeable on matters of safety?

4. A written test does not clearly indicate what a person can actually do. In the field of applied skills (all of industrial maintenance) it is very important to both 'know' and be able to 'do'.

Five Tips to Get Value from Written Tests

1. Align the test and the training sessions

Make sure that the test is well matched to the content and objectives of the training session. Using questions from a standard test bank or library may make for easy test construction, but the validity of the test will be low. If the training sessions have allowed for a flexible delivery where the specific needs of the trainee have been addressed during the sessions, you may need to modify and edit the test at the last moment to account for a change in content emphasis.

2. Help the class prepare for the test

Let them know well in advance how much of the test requires factual recall and memorization vs. understanding of general principles and concepts. Let them know what resources they'll have access to during the test. Emphasize key test material during the actual training sessions.

3. Use a variety of test item types

An entire test made up of multiple choice and true or false items is easy to mark but a poor indicator of true knowledge retained by the learner. It is important to include test items that allow the trainee to write a short answer in his/her own words (fill in the blank and short answer questions). A test item that asks trainees to match a series of terms to their symbols or definitions is another example of good test item variety.

You might also choose to design an open-book portion of the of the test. This is a test or a portion of a test that asks the trainee to use other resources such as manuals, drawings, schematics and formula tables. In this case you are not asking the trainee to memorize but rather to demonstrate resourcefulness and various technical communication competencies.

You might choose to have some actual physical machine parts and components labeled by letter or number and require the trainees to identify them in writing on the test. This type of item brings the test to near crossover between written and practical evaluation.

4. Review the test together as a group

Once the test is complete, you may choose to grade and review the test in the classroom as a group. If you are concerned about participants changing their answers during this stage you might choose to supply one color of pen during the writing of the test and then collect them and pass out another color for the grading/reviewing stage. The importance of this group review is that as a trainer you get one more chance to reinforce critical concepts and gauge the group's understanding. You also give the trainees a chance to defend their answers, especially the written ones where sole interpretation on the trainer's part is much more difficult.

5. Combine written testing with practical tests

If the results of a written test are to be recorded as a part of an employee's formal performance record, combine the test with other practical assessments. Again, the field of industrial maintenance is about 'know' and 'do'. If a trainee has done well on a written test on the topic of shaft alignment for example, then a planned matching practical evaluation might include observing the trainee make the correct preparations for shaft alignment in the field.

A written test is but one component in an overall evaluation strategy that must include assessment of visible job performance. For those of you who are familiar with Kirkpatrick's training assessment model, this is called a level three, behavior evaluation. Ultimately what you do, and can do is more important than what you supposedly know.

Your thoughts on written tests?

Hang On To Your Passion and Principles!

Are economic conditions improving for your operation? If they are not already improving, they will! Hang on to your passion and your principles for doing things right!

The downturn has been wreaking havoc on common sense management within heavy industry. In my visits with clients over the last two months, I noticed that many of the hard fought battles to establish disciplined maintenance were being surrendered. Maintenance departments are surrendering to corporate management's request to spend almost nothing.

Absolutely No Training Allowed

One client at a very complex plant told me that he would probably be found face down floating in a nearby river if he asked management for a budget to build some troubleshooting job aids that he needs. In response to my queries he said "not only are we not conducting maintenance training, we're not even doing basic maintenance". He went on to tell me that they will gamble on maintenance for the next six months and pay the price later. He said that he knew that the price would be very high.

Did this client really want me to believe that there was a threat to his life for proposing an investment in skills and performance? Of course not. But he was making it very clear that a serious edict had come down from above. He was also making clear that maintenance management progress was now in reverse.

Stay in the Game!

I have to wonder, does this mean that many of us who earlier fought hard to make progress in maintenance, have now let go of that passion? I can understand that if the company is nearly out of cash, that gambling on the maintenance of the plant might mean the difference between continuing to operate versus shutting down and laying everyone off. I suppose that like anyone else, a maintenance manager is thinking of his own family needs and his grocery bills and mortgage payment. If the company then instructs you not to spend on anything, you have to go along. Who wants to get laid off?

Another client who directs maintenance training and development for a large organization told me that his new game is tennis. He told me that serious liabilities come with the ball when it lands in his court. He told me that as soon as the ball arrives, he now hits it hard to get the ball as far from himself as possible. While describing the troublesome politics that plague his work I asked him to just imagine what the ideal solution might be. He told me that retirement was only a few years away now. Clearly he has removed himself from constructive and optimistic participation.



Basic Skills

This final story is a brain warper. I visited a client where we are producing job aids for a complex machine. I was there to gather additional data. The superintendent sent me out on to the plant floor with a supervisor. The supervisor first had me stop off in the machine shop. He had me help a group of tradesmen with a problem completely unrelated to the purpose of my visit.

They were struggling over a problem with a hydraulic cylinder that was creeping when it should not be. It was causing a lot of problems in the plant. After a few minutes of discussion in the machine shop it became apparent that the tradesmen were struggling with basic directional valve and hydraulic cylinder knowledge. I taught them what I could in the time I had during my visit. I took them online to show them where our free training resources are kept on our website. They were very grateful for the help.

When I got back to the superintendent's office I mentioned the discussions in the machine shop and that a two-day seminar that we provide would be very helpful. I was told that even though the plant was operating at near capacity, upper management had insisted that no training take place at this time as a cost saving measure.

Training Budget Trimmed?

Should expenditure on training be looked upon as a burdensome cost of production? Is it realistic to treat training and human development budgets in the same way as office supply and freight budgets? In the end a plant full of machinery only runs, or only runs well if a sharp and skilled crew is looking after it. In fact, if the plant cannot hire more help for the maintenance department, then you have to do better with the people you have. If the market has put pricing pressure on the plant's products, then the machinery has to run better than ever.

Fight - Put it on The Line!

Is training and development still at the forefront of your plant's operating philosophy? In the end it's all about people isn't it? We may once again have to fight for the good stuff,...the training as needed in our respective industrial departments.