Difficulties With Methodology in Social Science Research With Controversial Issues Regarding Human Sexuality.

Social science is commonly used in debates about controversial issues, especially for those concerning human sexuality. However, caution must be exercised in interpreting such social science literature, because of a variety of methodological and theoretical weaknesses that are not uncommon. Families are complex structurally and over time; such data are not easily analyzed. Merely determining the number of, for example, sexual minority families has been a difficult task. While some new theories are popular with social scientists, for example, sexual minority theory, they are often used to the exclusion of other, equally valid theories and often are not well tested empirically. Some types of families remain relatively unexamined. Social scientists can be biased by their own values, which are reflected in weak use of theory and in a variety of methodological problems. Eight studies are presented as examples of probable confirmation bias, in which methods and theory were modified in unusual ways that may have affected the outcomes and conclusions. Suggestions for improving social science include greater attention to effect sizes rather than statistical significance per se, deliberately minimizing the politicization of science, developing a culture of humility with respect to social science, deliberately reducing common biases, and maintaining a deeper curiosity about social science than is often seen. Scientists must be open to seeing their best "sacred cow" ideas or theories disproven or modified with increases in research on such issues. Summary: In controversial areas of social science, there can be numerous threats to the validity of science. Here, some of the more common risks for social science research and theory are examined, with several specific illustrations of how bias appears to have crept into social science, often as confirmation bias. Recommendations are made for reducing bias in future research.

Is Research on Transgender Children What It Seems? Comments on Recent Research on Transgender Children with High Levels of Parental Support.

Recent research on transgender children who have had support from their parents for their transitioning has concluded that their mental health is virtually no different than that of nontransgender children. Such research has been extensively cited, over 370 times in the past three years. Most of the hundreds of reviews received the stated results of the studies with little caution. However, the research featured numerous statistical errors and omissions, the implications of which would likely lead neutral observers to conclude that the mental health of transgender children, even when supported by their parents, was poorer than that of the groups of control children. In particular, levels of anxiety as reported by both parents and their transgender children appear to be significantly higher, and the transgender children's reports of self-worth appear to be significantly lower. Although reports regarding depression are not as significantly different, the effect sizes were generally in a similar direction as the other outcomes, being less favorable for the transgender children. Such issues highlight the need for careful examination of statistical research, even when published in highly regarded medical journals. As with other research, findings from the early stages of controversial research may often be premature. Further research is needed to explore factors underlying these results. SUMMARY: Some scholars have believed that if transgender children were supported by their parents before the children reached puberty, the generally higher rates of mental illness experienced by many transgender persons might be prevented or alleviated. Dr. Kristina Olson of the Department of Psychology at the University of Seattle was the first scholar to have studied groups of transgender children who were being supported by their parents and to have compared them to a control group of children and to siblings of the same transgender children. Her conclusion was that there were minimal, if any, differences in anxiety, depression, and self-worth among the groups of children; her research has since been cited extensively as having found just that. We reanalyzed her raw data and found that, to the contrary, the transgender children, even when supported by their parents, had significantly lower average scores on anxiety and self-worth. Often, a significantly higher percentage of transgender children, compared to controls, featured preclinical or clinical levels of anxiety. Parental support of transgender children may temporarily reduce levels of poor mental health for some transgender children, but it does not appear to eliminate those problems for all transgender children. Our findings should serve as a warning against accepting research at a surface level, which can lead to acceptance of invalid information and pursuit of ineffective interventions.

Research anomalies in criminology: How serious? How extensive over time? And who was responsible?

A variety of ways to detect questionable research practices in small sample social science surveys have been discussed by a variety of authors. However, some of those approaches (e.g., GRIM test, SPRITE test) do not work well for results obtained from larger samples. Here several approaches for detecting anomalies in larger samples are presented and illustrated by an analysis of 78 journal articles in the area of criminology, 59 by Dr. Eric Stewart, published since 1998 with similar methods and/or authors. Of all 59 articles, 28 (47.5%, p < .001, d = 0.94) had two or more major anomalies compared to none of the 19 control group articles. It was also found that the larger the role of Dr. Stewart in article authorship, the greater the number of anomalies detected (p < .001, d = 1.01) while for his coauthors, there were few significant relationships between their roles and total anomalies. Our results demonstrate that extensive problematic results can remain undetected for decades despite several levels of peer review and other scientific controls; however, use of two types of control groups and the use of statistical methods for measuring and evaluating anomalies can improve detection.

Methodological decisions and the evaluation of possible effects of different family structures on children: The new family structures survey (NFSS).

Every social science researcher must make a number of methodological decisions when planning and implementing research projects. Each such decision carries with it both advantages and limitations. The decisions faced and made by Regnerus (2012) are discussed here in the wider context of social science literature regarding same-sex parenting. Even though the apparent outcomes of Regnerus's study were unpopular, the methodological decisions he made in the design and implementation of the New Family Structures Survey were not uncommon among social scientists, including many progressive, gay and lesbian scholars. These decisions and the research they produced deserve considerable and continued discussion, but criticisms of the underlying ethics and professionalism are misplaced because nearly every methodological decision that was made has ample precedents in research published by many other credible and distinguished scholars.

Children of homosexuals more apt to be homosexuals? A reply to Morrison and to Cameron based on an examination of multiple sources of data.

Ten narrative studies involving family histories of 262 children of gay fathers and lesbian mothers were evaluated statistically in response to Morrison's (2007) concerns about Cameron's (2006) research that had involved three narrative studies. Despite numerous attempts to bias the results in favour of the null hypothesis and allowing for up to 20 (of 63, 32%) coding errors, Cameron's (2006) hypothesis that gay and lesbian parents would be more likely to have gay, lesbian, bisexual or unsure (of sexual orientation) sons and daughters was confirmed. Percentages of children of gay and lesbian parents who adopted non-heterosexual identities ranged between 16% and 57%, with odds ratios of 1.7 to 12.1, depending on the mix of child and parent genders. Daughters of lesbian mothers were most likely (33% to 57%; odds ratios from 4.5 to 12.1) to report non-heterosexual identities. Data from ethnographic sources and from previous studies on gay and lesbian parenting were re-examined and found to support the hypothesis that social and parental influences may influence the expression of non-heterosexual identities and/or behaviour. Thus, evidence is presented from three different sources, contrary to most previous scientific opinion, even most previous scientific consensus, that suggests intergenerational transfer of sexual orientation can occur at statistically significant and substantial rates, especially for female parents or female children. In some analyses for sons, intergenerational transfer was not significant. Further research is needed with respect to pathways by which intergenerational transfer of sexual orientation may occur. The results confirm an evolving tendency among scholars to cite the possibility of some degree of intergenerational crossover of sexual orientation.

Evidence of pro-homosexual bias in social science: citation rates and research on lesbian parenting.

Three refereed journal articles concerning lesbian family life were identified and compared for content and methodological quality. The articles shared the same authors, same academic institution of origin, the same samples, similar dates of publication, and even the same journals. However, one article reported evidence less supportive of lesbian parenting while two articles reported evidence more supportive of lesbian parenting. Citation rates for the articles were compared. The supportive reports have been cited 28 to 37 times since their publication compared to only two citations for the less supportive report, in spite of its possibly better methodological qualities. The citation rate differences were statistically significant. Implications for this possible evidence of pro-homosexual bias in the social sciences are discussed.

A comparison of citations across multidisciplinary psychology journals: a case study of two independent journals.

Citation rates and impact factors are often used in an attempt to evaluate the apparent prestige of scholarly journals and the quality of research published by individual scholars. However, the apparent prestige of "top tier" journals may reflect aggressive marketing and advertising efforts as much as scholarship. Some journals have retained their independence from professional organizations and the funding, marketing, and advocacy policies that may be associated with such organizations. While lacking as much visibility as organizational journals and sometimes considered "lower tier," independent journals may be able to provide comparable scientific quality as measured by citation rates. To test this, the citation rates of 169 articles published by a frequently cited scholar were compared across first- and second-tier journals, including many sponsored and marketed by large professional organizations, and to rates for two independent journals combined, Psychological Reports and Perceptual and Motor Skills. Citation rates were higher for first-tier journals but for most comparisons, especially those that controlled for heterogeneity of variance, results did not differ in statistically significant ways among the three tiers of journals, though some nonsignificant trends (p < .15) were found. If citation rates of articles are any indication of scientific quality, tiered classifications of journals appear to be a relatively weak indicator of scientific merit; journals at any tier contain articles that are useful and of good quality.

Statistical requirements for properly investigating a null hypothesis.

Issues involved in the evaluation of null hypotheses are discussed. The use of equivalence testing is recommended as a possible alternative to the use of simple t or F tests for evaluating a null hypothesis. When statistical power is low and larger sample sizes are not available or practical, consideration should be given to using one-tailed tests or less conservative levels for determining criterion levels of statistical significance. Effect sizes should always be reported along with significance levels, as both are needed to understand results of research. Probabilities alone are not enough and are especially problematic for very large or very small samples. Pre-existing group differences should be tested and properly accounted for when comparing independent groups on dependent variables. If confirmation of a null hypothesis is expected, potential suppressor variables should be considered. If different methods are used to select the samples to be compared, controls for social desirability bias should be implemented. When researchers deviate from these standards or appear to assume that such standards are unimportant or irrelevant, their results should be deemed less credible than when such standards are maintained and followed. Several examples of recent violations of such standards in family social science, comparing gay, lesbian, bisexual, and transgender families with heterosexual families, are provided. Regardless of their political values or expectations, researchers should strive to test null hypotheses rigorously, in accordance with the best professional standards.

Dimensionality and reliability of a modified version of Norton's 1983 Quality Marriage Index among expectant and new Canadian mothers'.

A slightly modified version of Norton's 1983 Quality Marriage Index was administered to 61 expectant mothers prior to giving birth and within 3 mo. after giving birth. Mothers' ages ranged from 19 to 43 years (M = 30, SD = 5.01) and their partners' ages ranged from 21 to 48 years (M = 32, SD = 6.02). Mothers were presented an opportunity to participate in this study during prenatal classes held at hospital and community health centers. The only requirement for participation was that the mother be residing with her child's father for the duration of the study. The six items of the modified index showed high internal consistency (alpha > .90) and substantial test-retest reliability with a Pearson zero-order correlation of .65 across the two administrations. Maximum likelihood factor analysis indicated moderate support for unidimensional factor structure for the modified index, but removing one item from the pre- and postnatal administration improved the factor structure. In the first administration, the last item (overall current satisfaction with partner) fit poorly with the factor structure, while at the second administration, the second item (our relationship is very stable) fit poorly. Possible implications of the results are discussed.

Re-evaluation of the "no differences" hypothesis concerning gay and lesbian parenting as assessed in eight early (1979-1986) and four later (1997-1998) dissertations.

Academic and policy effects of eight early dissertations on gay and lesbian parenting are discussed with a focus on their having been cited at least 234 times in over 50 literature reviews, beginning with Gottman in 1989 and 1990. Most literature reviews, referencing these eight early dissertations and agreeing with Gottman's early conclusions, have reiterated the theme that parenting by gay men or lesbians has outcomes no different than parenting by heterosexual parents. Here it is proposed that certain potential adverse findings may have been obscured by suppressor effects which could have been evaluated had multivariate analyses been implemented. Further, several adverse findings were detected by reanalyzing data where sufficient information was yet available. Some of the dissertations' results (absent controls for social desirability and other differences between homosexual and heterosexual parents) supported the 2001 "no differences" hypothesis discussed by Stacey and Biblarz. Yet, differences were also observed, including some evidence in more recent dissertations, suggesting that parental sexual orientation might be associated with children's later sexual orientation and adult attachment style, among other outcomes. Odds ratios associated with some of the apparent effects were substantial in magnitude as well as statistically significant. Also, more recent research on gay and lesbian parenting continues to be flawed by many of the same limitations as previous research in this area of study, including overlooked suppressor effects.

Dimensionality of subscales for emotional support in the Bosch Support Measure for survivors of domestic abuse.

Items from the Emotional Support Subscale of the Bosch Support Measure were administered in a small survey of 56 formerly abused rural women in the midwestern USA. Maximum likelihood factor analyses were performed on a total of 21 items. A 5-factor solution appeared to produce the best results, but only 3 factors suggested viable subscales with Cronbach alphas greater than .80. Informal support networks in the rural areas studied here appeared to provide survivors of domestic violence with general emotional support to a greater extent than guidance on accessing professional resources and help.

Relationship satisfaction for heterosexual women compared to lesbians and men in a sample of faith communities from Topeka, Kansas.

A modified version of the Kansas Marital Satisfaction Scale was administered to 239 women who belonged to 8 different faith communities in Topeka, Kansas in 2001. 12 members of a Metropolitan (gay and lesbian) church identified themselves as lesbians. Lesbian respondents reported significantly higher relationship satisfaction scores than heterosexual women (Cohen's d = 0.69). Controlling for number of children and social desirability reduced the regression coefficient for sexual orientation to a statistically nonsignificant level. Using a matched sample of 12 heterosexual women compared with the lesbians yielded a nonsignificant result (ES = 0.31). Nevertheless, although not statistically significant, comparisons between lesbians and heterosexual women continued to feature effect sizes that represented higher satisfaction for lesbians with no children or with only one child when compared to heterosexual women with no children or only one child. It is apparent that methodology made a difference in the results obtained in this comparative study of lesbian and heterosexual relationships. It was also observed, among members of churches other than the Metropolitan church, that relationship satisfaction was significantly lower (d = 0.22) among females than males, including among wives compared to husbands; significant linear and cubic relationships between a single-item measure of relationship social desirability and relationship satisfaction were also observed.

Marital satisfaction before and after deployments associated with the Global War on Terror.

Previously, McLeland and Sutton (2005) assessed marital satisfaction among 46 male civilians and Reserve Component personnel, some of the latter having been alerted for a future deployment. Subsequently, marital satisfaction was also assessed for 74 male Active Component Army personnel who had recently returned from duty in a combat zone. Marital satisfaction scores were compared across personnel status. Mixed results were obtained from an analysis of variance comparing the five subgroups of personnel; however, when the three types of deployment groups were contrasted with the civilian and never-deployed military groups, substantial (ES = 0.55) and significant (p < .005) differences in marital satisfaction were observed. Although exploratory, these data indicate that predeployment and postdeployment situations are stressful for military members and may be associated with lower marital satisfaction for months before and months after a deployment, as well as during the deployment. Furthermore, changes in marital or relationship satisfaction may be more sensitive and timely as indicators of any stressful effects of deployments and separation on military families than changes in marital or relationship stability.

Dimensionality of subscales for General Support, Informational Support, and Physical Support in the Bosch Support Measure for survivors of domestic abuse.

Items from the General Support, Informational Support, and Physical Support subscales of the Bosch Support Measure were developed and administered in a small survey of 56 formerly abused rural women in midwestern USA. Maximum likelihood factor analyses were performed on the total of 16 items. A five-factor solution appeared to produce the best results, but only three of the associated subscales yielded Cronbach alphas greater than .70. Limitations of the study are discussed along with implications for professionals working with abused rural women.

Neurologic adverse events associated with smallpox vaccination in the United States--response and comment on reporting of headaches as adverse events after smallpox vaccination among military and civilian personnel.

BACKGROUND: Accurate reporting of adverse events occurring after vaccination is an important component of determining risk-benefit ratios for vaccinations. Controversy has developed over alleged underreporting of adverse events within U.S. military samples. This report examines the accuracy of adverse event rates recently published for headaches, and examines the issue of underreporting of headaches as a function of civilian or military sources and as a function of passive versus active surveillance. METHODS: A report by Sejvar et al was examined closely for accuracy with respect to the reporting of neurologic adverse events associated with smallpox vaccination in the United States. Rates for headaches were reported by several scholarly sources, in addition to Sejvar et al, permitting a comparison of reporting rates as a function of source and type of surveillance. RESULTS: Several major errors or omissions were identified in Sejvar et al. The count of civilian subjects vaccinated and the totals of both civilians and military personnel vaccinated were reported incorrectly by Sejvar et al. Counts of headaches reported in VAERS were lower (n = 95) for Sejvar et al than for Casey et al (n = 111) even though the former allegedly used 665,000 subjects while the latter used fewer than 40,000 subjects, with both using approximately the same civilian sources. Consequently, rates of nearly 20 neurologic adverse events reported by Sejvar et al were also incorrectly calculated. Underreporting of headaches after smallpox vaccination appears to increase for military samples and for passive adverse event reporting systems. CONCLUSION: Until revised or corrected, the rates of neurologic adverse events after smallpox vaccinated reported by Sejvar et al must be deemed invalid. The concept of determining overall rates of adverse events by combining small civilian samples with large military samples appears to be invalid. Reports of headaches as adverse events after smallpox vaccination appear to be have occurred much less frequently using passive surveillance systems and by members of the U.S. military compared to civilians, especially those employed in healthcare occupations. Such concerns impact risk-benefit ratios associated with vaccines and weigh against making vaccinations mandatory, without informed consent, even among military members. Because of the issues raised here, adverse event rates derived solely or primarily from U.S. Department of Defense reporting systems, especially passive surveillance systems, should not be used, given better alternatives, for making public health policy decisions.

A discriminant analysis of Whissell's New Testament data: on the statistical trail of the author of Hebrews.

Discriminant analyses of data on 6 dimensions from Whissell's analyses of 27 books of an English translation of the New Testament seem to support Clement of Alexandria's contention that Luke was involved in the authorship, but it is difficult after centuries to rule out definitively a few other contenders.

Re-analysis of Sageman's (2004) and Pape's (2005) data predicting Al'Qaeda membership and suicide terrorism.

Sageman's (2004) and Pape's (2005) arguments for the development of Al'Qaeda members and suicide terrorism are examined statistically. Presence of foreign occupying troops and foreign support for disliked local regimes appeared to be related to either the number of Al'Qaeda suicide terrorists (Pape) or the number of known Al'Qaeda members (Sageman) reported within 34 predominately Sunni nations. Results support the theory that nationalism fuels much of Middle Eastern resentment against allied nations.

A Review and Critique of Research on Same-Sex Parenting and Adoption.

Are the outcomes for children of gay, lesbian, or bisexual parents in general the same as those for heterosexual parents? That controversial question is discussed here in a detailed review of the social science literature in three parts: (1) stability of same-sex parental relationships, (2) child outcomes, and (3) child outcomes in same-sex adoption. Relationship instability appears to be higher among gay and lesbian parent couples and may be a key mediating factor influencing outcomes for children. With respect to part 2, while parental self-reports usually present few significant differences, social desirability or self-presentation bias may be a confounding factor. While some researchers have tended to conclude that there are no differences whatsoever in terms of child outcomes as a function of parental sexual orientation, such conclusions appear premature in the light of more recent data in which some different outcomes have been observed in a few studies. Studies conducted within the past 10 years that compared child outcomes for children of same-sex and heterosexual adoptive parents were reviewed. Numerous methodological limitations were identified that make it very difficult to make an accurate assessment of the effect of parental sexual orientation across adoptive families. Because of sampling limitations, we still know very little about family functioning among same-sex adoptive families with low or moderate incomes, those with several children, or those with older children, including adolescents or how family functioning may change over time. There remains a need for high-quality research on same-sex families, especially families with gay fathers and with lower income.