|
|
Advantages of DIFdetect
DIFdetect represents several advances in the field of detecting DIF. These are briefly delineated here.
DIFdetect uses ordinal logistic regression techniques to detect DIF. This technique has several advantages over both item response theory (IRT) and non-IRT-based methods of detecting DIF.
DIFdetect does not require group assignment to be dichotomized. For many groups that have been considered when looking for item bias, dichotomization is present from the beginning (for example, gender bias). However, other demographic characteristics are not dichotomous. For example, we are interested in tests of cognitive functioning, and item bias based on age and years of education. Searching for DIF using age or years of education with IRT based methods or Mantel-Haenszel techniques would require dichotomization of these variables into older/younger and more/less educated groups. In this way, information is lost, and power to detect DIF is lost.
The techniques we offer do not require this loss of information for non-binary group memberships.
DIFdetect does not require special software or advanced programming skills. Unlike item-response theory techniques, standard statistical packages (like SPSS and STATA) are able to do everything necessary for powerful, full information DIF detection.
DIFdetect is able to look for DIF with either dichotomous or polytomous test response categories, even if some response categories are never chosen by members of one of the groups. We analyzed a 41-item test of cognitive functioning for DIF. We attempted to use PARSCALE, an IRT package from Scientific Software International, but were unable to do so because despite having very large groups of test takers (roughly 2000 in each group), there were some categories of some items that did not have any respondents in one of the two groups. At that point, PARSCALE gave up. DIFdetect will not give up, even when some response categories are empty. The ordinal logistic regression technique employed by DIFdetect is equivalent to logistic regression if it finds dichotomous rather than polytomous item response categories.
Finally, DIFdetect enables the evaluation of multiple measures of ability. We have found less DIF when IRT-based scoring techniques are used rather than standard scoring. This flexibility allows the advantages of IRT to be used without the usual IRT limitation of requiring dichotomization of continuous group characteristics (like age and educational level).
DIFdetect thus represents a flexible, powerful, full information technique for evaluating DIF. We are convinced that this package represent the best available techniques for this purpose.