Bias, we don't need no stinkin' bias!

On Monday the Miami Herald published an interesting article by Oscar Corral about the appeal of the "Miami 5" and how the appeals court tribunal may have been influenced in its decision to overturn the convictions of the five based on a survey conducted by a former FIU professor named Gary Moran.

In the article we learn that the research, when scrutinized by experts on polling, doesn't cut the mustard. I was particularly interested in this story because my day job (I do have one!) involves a lot of market research. Specifically, I work with surveys a lot. Ironically the survey which was conducted to measure the amount of bias in the potential jury pool in south Florida, is now being looked at closely because of the bias of the survey designer, Moran, and how that bias made its way into the work product.

So my curiosity was aroused and I wanted to read questionnaire in its entirety. Many times two people can reach far different conclusions from survey results, particularly if you haven't seen the actual questionnaire. We've all heard about "push polls" which are conducted during electoral campaigns to raise the level of fervor about political issues. Those polls are comprised of a series of leading questions. Such a question might sound something like this:

Q. Are you in favor of the early release of violent criminals including rapists and pedophiles?

Of course that's a loaded question because nobody in their right mind would be in favor of such a thing, but each criminal has his own story and the word "early" could mean two months early or 20 years early. The answers to that question might be different if it was rephrased to something like:

Q. Are you in favor of the releasing certain convicted felons, even violent offenders, before they complete their entire sentence, if the felons in question meet certain criteria that indicates that they have been rehabilitated?

Mr. Moran is now accused of using the same type of leading questions (as the first example above) in his survey. So I have decided to analyze the survey in its entirety. I obtained the survey, which is contained in an affidavit submitted to the court, from the Miami Herald. I have transcribed it from the PDF and it's below. My commentary continues below the survey.
_____________________________________

Hello my name is _____ from focus on Miami. We are conducting a survey of South Florida voters to see how they feel about the upcoming trial of some people charged in federal court with Spying for Castro’s Cuba. Your house has been randomly selected to provide a participant for this survey. We are not selling anything. Your information will be kept strictly confidential and you need not ever give us your name and address.

First, how many people in your house are registered voters in Miami-Dade? ____(If none, thank you and terminate).

I am supposed to speak with the registered voter in your house with the most recent birthday.

Before I begin asking you questions I want you to know that there are no right or wrong answers, just honest ones, and that you are free to answer “I don’t know” to any question… Okay?

(Brief Pause – Continue)

Are you aware of the case involving the alleged Cuban spies who were arrested in Miami?

Yes
No

Section 1

I am going to read a series of statements. For each one I would like to if you strongly agree, agree, disagree, or strongly disagree with the statement. Here is the first statement.

1. Cuban born persons carrying false identification and engaging in intelligence gathering activities in south Florida are Castro Spies.

Agree Strongly
Agree
Disagree
Disagree strongly
Don’t Know – No Answer

2. These defendants are charged with setting up the ambush of the Brothers to the Rescue planes in which for people were killed. This type of activity is characteristic of the Castro regime.
3. The aim of Castro is to undermine legitimate Cuban exile organizations
4. An aim of Castro is to infiltrate U.S. military bases in South Florida.
5. Castro’s agents have attempt to disrupt peaceful demonstrations such as the Movimiento Democracia’s flotillas which honor fallen comrades
6. Castro’s Cuba is an enemy of the United States.
7. Castro poses a real threat to the lives of Cuban exiles.
8. Castro’s spies should not be given a public trial if this threatened national security.
9. Because of my feelings and opinions about Castro’s government I would find it difficult to be a fair and impartial juror in a trial of alleged Cuban spies.
10. You have told me that you would find it (difficult/not difficult) to be a fair and impartial juror. Are there any circumstances that would change your opinion?

No
Yes

If so, what? _______

11. Suppose your jury found these spy defendants not guilty. How worried would you be that you might be criticized in your community.

Very Worried
Worried
Somewhat Worried
Not Worried
Don’t Know – No Answer
________________________________________

The questionnaire continues with basic demographic questions that are usually used to make sure the correct proportion of respondents is being used to represent the community at large that survey sample is supposed to represent. For example males vs. females, white vs. black.

Below are my opinions as to what is wrong with this survey and the conclusions that Mr. Moran draws from them.

The very first question in the survey asks whether or not the respondent was aware of the case against the alleged spies. 54% of the respondents and 48.5% of the Hispanic respondents were not aware of the case. This does not suggest the kind of large-scale media coverage that often creates a need a for a change of venue for a trial. Knowing that, Moran argues in his affidavit that it's not the media surrounding the case but instead the poisoned atmosphere in the jury pool based on decades of negative publicity about the Castro regime. But it seems incongruous that in such a poisoned atmosphere that half of the respondents didn't even know about such an important case.

I agree with Rutgers University Professor Cliff Zukin, president of the American Association for Public Opinion Research, who the Herald interviewed. Zukin told the Herald "The first eight questions that come before the ninth question should come after it... It's really pretty biasing. All those items are very one-sided. They are all anti-Castro and framed in the same way so an agreed statement would be anti-Castro.''

I'll add that not only are the questions leading, meaning that they are seeking a certain response but that the proposition of all 8 questions could be true in FACT (as opposed to only in the minds of the respondents) and a potential juror could still be unbiased in a courtroom situation. One can only assume that Mr. Moran believes that the only way the defendants could get a fair trial would be with a jury of Castro sympathizers. But the intent of these questions is clearly to poison the well so that when the all-important question of impartiality comes up, that the respondents would doubt their ability to be fair. Question 8 doesn't even disclaim the fact these were alleged spies. It convicts them before the trial even takes place. This is an inexcusable offense for a researcher.

In addition to the leading nature and order of the questions, the next thing that merits analysis is Mr. Moran's calculations. In his affidavit Mr. Moran adds the "agree strongly" and "agree" to get a "net agree" percentage. This is a common practice to gauge general direction of opinion. But showing the actual values for EACH response helps to better understand the intensity of the feeling behind the response. Mr. Moran includes the raw numbers in his notes that he provides in the affidavit but not in the topline summary where he makes his recommendations. Invariably for the first 8 questions, the less intense "agree" answer had a much greater response than the more intense "Agree Strongly".

Even given all of the above, more than 60% of the respondents that answered question 9 disagreed with the idea that they would not be able to be impartial as a juror in the trial of the alleged spies. Mr. Moran points out that normally that number is closer to 80% (in similar surveys for other trials) and thus claims this as proof of an unsuitable venue for the trial. But I wonder if the other surveys that Mr. Moran refers to were as leading as his.

If Mr. Moran wanted definitive proof that another venue would be more suitable he could have very easily conducted the survey in a control market, perhaps Atlanta, Georgia or Cleveland, Ohio or some other place. He could have used the same survey instrument and detected any difference in the response rates between the control market and the test market. But I suspect the survey is so biasing that there would not have been a remarkable difference.

The answers to question 11 are also revealing. 59.3% of the total sample said that they wouldn't be worried about being criticized by the community if they were part of a jury that returned a not guilty verdict and among the Hispanics in the survey the number was only slightly lower at 54.7%. Contrast that with the 8.3% overall and 8.5% of Hispanics that said they'd be "very worried". Among those that had any degree of worry the most popular response was the least intense "somewhat worried".

Mr. Moran omits a very important fact from his affidavit. And that's his personal views on Cuba and Castro. He demanded that the jury pool either be completely neutral towards Castro or sympathetic to him in the name of fairness because he thinks that people who have negative feeling towards Castro couldn't be impartial towards the defendants but he completely dismisses the idea that his personal feelings of admiration towards Castro might influence his impartiality in designing, fielding, and interpreting the results of the study. Moran told the Herald ''I'm not ashamed of my opinion that Fidel Castro is a serious Cuban patriot, doing his best for the Cuban people." So Mr. Moran you're capable of doing what mere mortals aren't, separating your personal views from the task at hand?

In my opinion, the trial judge was correct in disregarding Moran's survey and the three-judge appeals court panel was wrong to give it credence. What the full appeals court feels about this survey will quite likely determine whether the convictions will be upheld or whether a new trial will be needed. I'm quite confident that the end result will be a conviction in any event. There's a lot of evidence in this case that will stand up regardless of the venue. I have confidence in our federal courts.