Friday, July 22, 2011

The Flaw in Mr. Franken's Argument, or: How to Manipulate Data for Bigotry, Comedy and Civil Rights.

Before I begin, let me say now that this is a paper about how to use data and think critically about statistical reports. I am not defending Focus on the Family. I love Al Franken and would be tempted to have his babies if I were not CJ's "unmarried partner." Now that you are clear on my personal point of view towards this issue, let's talk about the interpretation of statistic and critical thinking.

The recent hearings on the DOMA act have brought to light a rather unassuming little report. The report in question is called, "Family Structure and Children's Health in the United States: Findings From the National Health Interview Survey, 2001-2007," conducted by the Department of Health and Human Service. This study has been painfully abused. This is a lesson in how not to interpret a study. (Read it for yourself here: http://www.cdc.gov/nchs/data/series/sr_10/sr10_246.pdf)


This type of statistical report is designed to determine whether one thing or condition makes a different thing or set of conditions more likely. In this instance the report was designed to determine whether the existence within a nuclear family versus other types of families, increases the likelihood of certain positive outcomes for children. Nuclear family is defined as: "one or more children living with two parents, who are married to one another and are each biological or adoptive parents of all the children in the family." The results were, simply put, yes.

Tom Minnery of Focus on the Family used this positive correlation in his report to assert that these positive outcomes can only be expected in a parentally heterosexual family, and this is the point Franken takes him to task for.

The problem here is this: though the wording of the study does not explicitly exclude families with same sex parents, their data source absolutely did, by requiring them to be "married." Here are the goods:

Where does the Department of Health and Human Services get its data from? In other words, they are pulling a sample of American families from what list of American families? The data for the HHS study came from a program called the National Health Interview Survey. In the information on their website about their sampling methods, they state that they use census data for their sampling(1), this is also stated in the report. The data used for the HHS study came from 2001-2007 NHIS, which were derived partly from projections of the 1990 census and partly from projections of the 2000 census. Here things get a bit sticky. Because the census relies on self reporting, same sex households could report, in 1990 and 2000, their sex and that they were "husband/wife," which, though kind of repugnant, would have put gay families on the data retrieval map. However, in 1990 (3), the census excluded same sex families by just changing one of the individual's sex designation to make it look like a heterosexual couple. In 2000, thanks to DOMA, the census bureau could not legally recognize these couples as "married" and therefore systematically went through and changed all such cases to the designation of "unmarried partner." (2) Just so we are clear on this, it is illegal for the Census Bureau to have any type of record that implies that a same sex couple with children could be "married."

So, because the data for the first half of this study does not recognize such a thing as gay families, we can safely say none were involved. As for the second half, because the federal government does not recognize same sex unions, same sex folks with kids would have been filtered out of any search for "nuclear families" to include in the study, as they do not carry the same legal designation as married heterosexual couples. It is possible that gay families could have been involved in this study (though extremely unlikely), but it is impossible for them to have been categorized as a "nuclear family," (i.e. the preferred condition in this study.)

If read objectively, the report says nothing, absolutely nothing, about same sex families, except that their children would greatly benefit from their parents being afforded the right of marriage. (This is brought up later in the hearing and Franken actually gets Minnery to agree, which is truly the masterstroke of the whole ordeal.)

So, when Franken asks if, "a same sex married couple with biological or adoptive children would also meet [the nuclear family] definition," he is confusing the wording of the study's creators with what can be statistically derived from a particular set of data. That is not to say that there isn't tons of data to support that children in a family with two same sex parents do not share the positive outcomes mentioned in this particular study, but it is not this data.

It has been widely reported (4, 5, actually, click any news story about this) that the study says it does not matter the gender of the parents, because that was the driving moment of the proceeding, but it isn't technically true. It is what the words imply, not what the particular information can support. It's amazing to me that no has caught this.

Franken is really over stepping when he says that the report, "does not say what you say it says." It actually does because the bias in this data specifically exempts parentally same sex families from the preferred condition. Minnery is not making a false assumption when he says, "I would think that when it cites nuclear families, the study would mean a family headed by a husband and wife." Though the study may have laid out their definitions in a gender neutral way, and may have even done so to be intentionally inclusive, the specific information they are using is not sufficiently broad to support their assertions.

If Minnery had based his assumptions on fact as opposed to his entirely mythical belief system he would have been able to point this out, effectively taking the wind right out of Franken's sails. But, as extremists often do, he saw in the data what he wanted to see, not what is factual. His assertion is a belief based on faith; to him there is no need for any additional supporting arguments. This exactly why extremism is so bad.

As for Al, maybe he knew he was being a bit equivocal or maybe he didn't really look beyond the words of the report either. Even if he had, this was a still a good play. It requires the barest minimum of intuition to figure that a guy like Minnery is so limited by his bigotry that his attempt at evidence without a Bible to lean on would be halfhearted at most.


On a lighter note, watch this because it is the best kind of gay!