Twoblock wrote on Sep 6
th, 2008 at 3:53am:
The empirical data on which you base your In conclusions is biased by the limited scope of subjects whos responses form your baseline. Example: conclusions as simplistic as the following statement;
"Every murderer that I have questioned drinks milk."
"You drink milk."
"Therefore you are a murderer".
For MMPI results to be truely meaningful, everyone would have to be tested and the data tabulated taking into consideration almost countless variables.
I am not sure I follow? Do you mean you think the
normative control group used as the baseline is biased or somehow not representative of the normal population? You realize the data was derived via statistical comparison between criterion groups (those known to have the disorder of a particular subscale)
AND a "normative" control group (normal individuals to used as a baseline for statistical comparison), right? The data is not restricted to just a clinical sample. If it was, how would we know what the deviant (scored) response should be? We have to know that a normal population would consistently endorse the item differently in order to establish what the normal response would be. If the norm group didn't endorse the item different than the criterion group at a high rate, then the item is worthless. The developers threw these items out. The ones that make up the test are the ones that show
statistically significant differences between norm group and criterion group. It allows us to see the
true differences between the groups (normal vs psychiatric). It's NOT an absolute conclusion (as you stated), it is a "degree of probability" (the degree of probability depends on how many standard deviations the T score is elevated above the norm), because the test is using items that have extemely high statistical differentiation properties. Your "milk" example would be an example of an item that would have been tossed out during the test's development because its ability to differentiate is highly flawed. If we take a sample of milk drinkers and run this same statistical procedure that MMPI-2 used for its development, we would see the true statistical trend. (i.e., there is no true correlation between the 2 when we do an empirical test of this hypothesis). We find out that most milk drinkers are not murders, and thus we realize your item example is a
statistically weak discriminator. Items with weak statistical properties like this did not make it into the test. This prevents the occurrence of this logic leap error you are referring to. We don't use items like that. Only ones with high statistical reliability for
true group differentiation. However, more importantly, you are not taking into account the real issue of the statistical logic at work here. Its the
aggregate sum of
many endorsements consistent with the criterion group that raises the red flags. Not just one endorsement, as in your example. We do not make snap judgments based on one item, or even several items. That's why the test has so many items, 567 to be exact. You've heard the old saying "One swallow doesn't make a summer", right? Well, we have too.
Your second paragraph demonstrates a poor understanding of multivariate statistics, and
again, misconstrues the purpose of psychometric assessment. The reason we have statistics is because what you proposed is obviously impossible. Statistics are powerful estimators of group trends. Multiple Regression, linear regression, and structural equation modeling procedures are well validated, scientifically accepted procedures for quantification of the frequency of a event or trait in a large data sample. If you have a problem with these procedures and methods, then you should have a problem with ALL of science and modern day medicine! Guess how risk factors for disease are calculated? They certianly don't survey everyone, right? Keep in mind the MMPI-2 is based on statistical base rates and likelihood of item endorsement consistent with a criterion group
over a certain threshold. The threshold is set very very high (i.e., 4, or 5 endorsements consistent with the criterion group is statistically meaningless. It takes 15, sometimes 20 endorsements consistent with a criterion group to raise the T score above "normal"). When you have an subscale elevated above the designated T score, the statistics of the test tell us that this elevation being the result of chance alone (i.e., error) are very very small. This tells us that you are indeed endorsing items consistent with those with the disorder at a very very high rate with a very very small likelihood that this could have happened by chance alone. In other words, you end up "looking like" (on paper) this clinical population. But again, this does NOT equate to diagnosis, nor does this automatically mean you would necessarily meet the clincial threshold for the disorder. You simply have alot of similarities with this particular population, since you have endorsed so many of the same opinions they did. You have alot of traits of that disorder. The
clinician delineates whether there is
clinically significant meaning behind these elevations after taking into account all the data from the clincial interview and any other tests that have been given. As you can see, the MMPI-2 is a powerful statistical
tool, nothing more. It is just one piece of clinical information, but, one that is completely empirical, statistically objective, and highly reliable. It does NOT diagnose, but that in no way means the data derived is "meaningless." Statistically, they are very meaningful. Official diagnoses should indeed be made cautiously, taking into account the many many factors and the numerous environmental and situational variables (as you pointed out) that the clinician obtains during the clincial assessment process.