Can diagnostic problem-solving competences of car mechatronics be validly assessed using a paper-pencil test?
In this study, an authentic paper-based key-feature test for electrotechnical diagnostic problem-solving competence was developed, administered to car mechatronic apprentices (N = 206) and validated using diagnostic problem-solving scores. It was hypothesized that the paper-based test scores represent the same construct as the problem-solving scores. The written test turned out to have a relatively high reliability (EAP/PV = .75). Additionally, it was found that the paper-based scores and problem-solving scores were highly correlated (r = .76) but represented empirically distinct dimensions. Presumably, the written test especially covered diagnostic knowledge and failed to cover other relevant subcomponents of diagnostic problem-solving competence. It is argued that this unexpected finding might be caused by construct underrepresentation and construct-irrelevant variance of the paper-based key-feature test.
Keywords: Professional competence, domain-specific problem-solving, key-feature items, construct representation, construct-irrelevant variance