?

Log in

No account? Create an account
That white firefighters case..... - Input Junkie
June 29th, 2009
01:03 pm

[Link]

Previous Entry Share Next Entry
That white firefighters case.....
Does anyone have details about the test which the white firefighters scored so much better on?

I can't imagine what would be on a test which legitimately produced such results, so I'm wondering if there was dishonest scoring.

(17 comments | Leave a comment)

Comments
 
[User Picture]
From:cakmpls
Date:June 29th, 2009 05:19 pm (UTC)
(Link)
Given that the entity that gave the test, the city, was so unhappy with the result that they threw the whole thing out, I kind of doubt that there was dishonest scoring.

Do you know how many people took the test? If it was a fairly small number, there's no particular reason (that I can see) that the distribution couldn't have just happened to turn out this way. The larger the number who took the test, the more likely there was some kind of bias.

[User Picture]
From:agrumer
Date:June 29th, 2009 05:28 pm (UTC)
(Link)
Also, if you give a test to hundreds of mixed groups, you'll get a few cases where all members of one group do better than the other, even if there isn't any overall tendency for members of one group to do better in the society as a whole.
[User Picture]
From:nancylebov
Date:June 29th, 2009 06:03 pm (UTC)
(Link)
That's another plausible explanation, probably as plausible as bad scoring.
[User Picture]
From:subnumine
Date:June 29th, 2009 06:25 pm (UTC)
(Link)
New Haven asked that the original Federal court decision be sealed; there's probably something embarassing.
[User Picture]
From:cakmpls
Date:June 29th, 2009 08:39 pm (UTC)
(Link)
It's possible; it's also possible that it had the individuals' names in it.
[User Picture]
From:madfilkentist
Date:June 29th, 2009 05:39 pm (UTC)
(Link)
There are often significant educational disparities between different ethnic groups within a local area. When this happens, it's quite easy for members of one group to get better scores than another.

After beating racial discrimination in court, why do these people now have to be subjected to groundless accusations of cheating?
[User Picture]
From:redaxe
Date:June 29th, 2009 05:41 pm (UTC)
(Link)
The more typical reason for test results to have been biased in terms of race/ethnicity in the past is that the test itself contained social/ethnic biases. I don't know that this was the case in Connecticut, but I'd find it easier to believe than I would believe dishonest scoring.

I must go read the opinion at SCOTUSBlog, but that will wait until later.
[User Picture]
From:agrumer
Date:June 29th, 2009 05:55 pm (UTC)
(Link)
According to Wikipedia, 118 firefighters took the exams. New Haven's city charter has a rule called the "Rule of Three", which states that an open civil service position be filled with one of the three people who scored highest on the qualifying exam. There were seven vacancies for the Captain position, so those would be filled from the top nine scorers, all of whom were white or Hispanic. For the Lieutenant exam, there were eight vacancies, and the ten top scorers were all white.
[User Picture]
From:cakmpls
Date:June 29th, 2009 07:27 pm (UTC)
(Link)
That entry says further that the Captain exam was taken by 25 whites, 8 blacks, and 8 Hispanics. The top 9 scorers were 7 whites and 2 Hispanics. That means that 28% of whites were in the top scorers, 25% of Hispanics, and 0% of blacks. Given the small numbers who took the test, that doesn't look particularly suspicious to me. 18 whites were NOT in the top ten--more than the total of blacks and Hispanics (14) who were not.

The Lieutenant exam was taken by 43 whites, 19 blacks, and 15 Hispanics. 23% of the whites were in the top ten scorers, and 0% of the blacks and Hispanics. That's somewhat more skewed, but the numbers are still pretty small for drawing any conclusions. 33 whites were NOT in the top ten--and a total of 34 blacks and Hispanics were not.
[User Picture]
From:nancylebov
Date:June 29th, 2009 08:21 pm (UTC)
(Link)
Is there a statistician in the house? Those results don't look terribly likely to me, but I don't know what the odds are of results like that happening by chance.
[User Picture]
From:cakmpls
Date:June 29th, 2009 08:36 pm (UTC)
(Link)
I don't think there is any way of determining "odds" for something like this. It isn't a case of drawing lots out of a hat.

On the Captain's test, we're looking at 9 top scorers out of 41, so we're looking for the top 22% (rounded). More than 22% of the white testees scored in that group, but so did more than 22% of the Hispanic testees.

If the spread had been perfectly proportional by race/ethnicity, 4.5 whites, 1.8 blacks, and 1.8 Hispanics would have been in the top 9, compared to 7, 0, and 2 actual. That doesn't seem particularly unlikely to me.
[User Picture]
From:nancylebov
Date:June 29th, 2009 09:16 pm (UTC)
(Link)
Thanks. The results probably weren't as unlikely as I thought.
[User Picture]
From:bradhicks
Date:June 29th, 2009 10:32 pm (UTC)
(Link)
For what it's worth, it's also been a consistent result of multiple-choice tests in general, even when comparing two groups of equivalent educational and economic background. For reasons not even vaguely well known, African Americans do much worse on multiple-choice tests than Americans of European or Asian descent do. There are some hypotheses, having to do with dialect, especially vocabulary and grammar rules differences between the people writing the test (who are, unsurprisingly, nearly all white) and those taking the test.

From 1971 until today, to defend a test that had that disparate an impact, you would have had to prove that answering multiple-choice questions was a bona fide occupational qualification (BFOQ) for the job, that the person who got the job was going to have to spend all or part of his day filling out multiple-choice tests. An overtly silly and indefensible argument, when it comes to fire-fighters.

But today's particular big of legislation from the bench (and you can't call it anything else, I'm sorry) is entirely consistent with Antonin Scalia's long-held political philosophy on discrimination cases, which is that no matter what Congress says, and no matter what the results are, the only actionable discrimination is overt, explicit, and intentional discrimination, and no court should accept anything less than the most explicit and compelling evidence with regard to intent or state of mind. For example, the only reason that the Santerians won in Hialeah versus Santeria was that the Hialeah city council's own minutes showed that the intent of the ordinance was to close down a Santeria church. If they had managed to hide this intent from the evidentiary record, they would have won.
[User Picture]
From:nancylebov
Date:June 30th, 2009 12:34 pm (UTC)
(Link)
http://ta-nehisicoates.theatlantic.com/archives/2009/06/thoughts_on_the_ricci_case.php#comment-217345

The study materials cost $500, but people from fire-fighter families from the area would already have had them. And in a couple of cases, it took a month and a half for the study materials to arrive. So there was some de facto bias there.

I wonder what would have happened if the city had compensated everyone who'd taken the test for their time (an estimate of study time, not just the time spent taking the test) and money.
[User Picture]
From:cakmpls
Date:June 29th, 2009 08:43 pm (UTC)
(Link)
On the Lieutenant's test, we're looking at picking the ten top scorers out of 77 candidates. That's the top 13% (rounded). Perfectly proportional, it would include 5.59 whites, 2.5 blacks, and 1.9 Hispanics; actual was 10, 0, and 0.
[User Picture]
From:subnumine
Date:July 2nd, 2009 04:39 pm (UTC)
(Link)
Not particularly difficult; the real difficulty is deciding on the exact question being tested (does it matter that some Hispanics got into the range eligible for promotion). The odds are on the order of 1000 to one against, assuming everybody had an equal chance; but if one assumes that there are some 200 similar exams out there, a thousand to one shot will show up about a sixth of the time. (And no statistics of the exam can tell you whether the inequality was the test scoring, the study kits, or the New Haven educational system.)
From:(Anonymous)
Date:July 4th, 2009 05:44 am (UTC)
(Link)
It's easy to find sample questions for the SATs, which produce similar results.
nancybuttons.com Powered by LiveJournal.com