Re: odd RELIABILITY .../SUMMARY output

[address@hidden]

If it's any consolation, I also didn't immediately realize what was happening.

The problem here is missing data, which is always a pain in the butt.

Computing a scale score as a mean is an effective way to compute a score in a sample with missing responses (assuming you don't mind mean imputation, which isn't a bad default).

But Reliability has very different (and inflexible) approach to missing data.

-Alan

On 9/17/2023 4:00 AM, Quandt, Markus wrote:

Alan, thanks a lot, that of course clarifies it. Strange how I couldn’t remember that reliability would use sum scores for the scale, not means.

 

Best, M.

 

From: pspp-users-bounces+markus.quandt=gesis.org@gnu.org <pspp-users-bounces+markus.quandt=gesis.org@gnu.org> On Behalf Of Alan Mead
Sent: Sunday, September 17, 2023 3:15 AM
To: pspp-users@gnu.org
Subject: Re: odd RELIABILITY .../SUMMARY output

 

You're comparing apples and oranges. The "Scale Mean If Item Deleted" looks plausible to me. 

You calculated insttrust as the average of the 18 items, but Reliability will calculate it as the sum. You got a mean of -0.13 so the mean for the sum score is 18 * (-0.13) = -2.34.

That is, the mean of the scale being analyzed by Reliability, is -2.34 (when you use all 18 items). (And, actually, that's the mean in a sample, but not THE sample that Reliability is using... because of how mean() and reliability() handle missing data, your mean of -0.13 is based on N=59147 but the listwise deletion inherent in Reliability means that sample is N=42666).

And then those "Scale Mean If Item Deleted" values are calculated dropping that target item. If you drop an item with a mean between -2 and +2 (and mostly closer to zero), it's not implausible that you would get values like -2.65, -3.15, etc.

But it's easy enough for you to check with:

compute insttrust_minus115 = sum(v116 to v132).

desc insttrust_minus115.

BUT that's only a direct test if you run that on the N=42666 sample that is used in the Reliability commend (which automatically does listwise deletion). I'm not sure, off hand, how you easily remove anyone missing any one or more of the 18 responses.

If you did that analysis, in the correct sample, and the mean was not -2.65, then something's wrong.

If it's too much to bother getting that exact N=42666 sample, I'd just run that syntax on all cases, and see if the result is close to -2.65. If it is, then probably -2.65 is correct...

You could also, for the purposes of checking, recode missing values (on variables v115 to v132) into the mean value of -0.13 and then redo all the analyses on complete data and sample sizes shouldn't differ.

The other numbers look plausible...

-Alan

 

On 9/16/2023 5:14 AM, Quandt, Markus wrote:

Okay, I am running pspp 2.0.0-pre1g968e22 on Windows 10. I believe that is one of the moderately recent nightly builds.

 

I do wonder about the item-total statistics that RELIABILITY is outputting.

 

See below, go to the very bottom for the question:

 

 

Syntax:

 

compute insttrust=mean(v115 to v132).

desc insttrust.

reliability /var=v115 to v132 /summary=all /missing=exclude.

 

 

Output:

 

 

 

 

 

 

 

Question: How is the “Scale Mean If Item Deleted” computed, so that it is lower than the minimum of the values of any item in the scale?

 

(Also note that the ‘C’ of ‘Cronbach Alpha’ is cut off in the Reliability Statistics table)

 

Thanks,

Markus

-- 

Alan D. Mead, Ph.D.

President, Talent Algorithms Inc.

science + technology = better workers

https://talalg.com

God created war so that Americans would learn geography.

-- Mark Twain

-- 

Alan D. Mead, Ph.D.
President, Talent Algorithms Inc.

science + technology = better workers

https://talalg.com


God created war so that Americans would learn geography.

-- Mark Twain