clock menu more-arrow no yes

Filed under:

IRB Basics and How Willingham Got It All Wrong

New, comments

The release of Mary Willingham's research application has generated many questions about the IRB process, and her comments following this revelation demonstrate the disconnect between how research of this type should be conducted and what she actually did.

Jeremy Brevard-USA TODAY Sports

A vital but otherwise normally obscure governing body at UNC was thrust into the spotlight this week as Mary Willingham's research application was released in response to a Freedom of Information Act request.  The Institutional Review Board (IRB) operates as a part of the university's Office of Human Research Ethics, which is just what it says: the office responsible for the monitoring of research involving human subjects. Without going into an alphabet soup of research minutiae, here is a quick primer of how the IRB works.

Despite some confusion on the interwebs about what an IRB is and how it works, it's actually a very simple process. When someone at a university wants to do a research study, they submit a research proposal to the IRB. Pretty much every college or university has an IRB, and at a large research institution like UNC, the IRB literally receives hundreds of research proposals every year. The IRB is a committee that determines whether the research involves human subjects. According to the UNC IRB site,

What is a human subject?

A human subject is defined by Federal Regulations as "a living individual about whom an investigator conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable private information." (45 CFR 46.102(f))

If the research is deemed to involve human subjects, then specific permission to conduct the study is required and a rigorous set of protocols is invoked to ensure legal and ethical treatment of the study subjects and their data. If the study is determined to not deal directly with human subjects or data that can directly identify specific people, then IRB approval is not required.

Mary Willingham's initial application to the UNC IRB in 2008 indicated that the data she intended to use had identifiers such as names but that those identifiers would be removed and coded. This is standard procedure in this type of research. For example, instead of listing, say, "Marvin Austin" followed by his associated data, it would say "Student-Athlete 001" so that specific scores could not be tied to an individual. Given that she indicated she would be using de-identified data, also known as secondary data (meaning only numbers, as opposed to primary data gathered from or about an individual), the IRB ruled her research did not involve human subjects and did not therefore require nor receive IRB approval.

Based on her 2008 application, on the surface it would seem to be a pretty standard research process. Willingham indicated she was researching the occurrence of ADHD and learning disabilities in entering freshman student-athletes. The university had a long-standing contract with psychologist Lyn Johnson, who was administering screening tests for learning disabilities. The types of tests mentioned by Willingham in her research abstract are consistent with the types of tests Johnson would be administering, and Willingham included consent forms on the letterhead of NC Neuropsychiatry, which was Johnson's practice at the time. Standard research protocol would be for Johnson to collect the screening data, remove anything that could tie a specific individual to its results, and provide that data to Willingham for analysis.

But that is apparently not what was happening. In an article in the News and Observer following the release of her applications, Willingham indicated that Johnson largely filled out the application and did the analysis, and then coded the data so Johnson did not know the identifying data. This is 180 degrees turned around from how it should have worked. Johnson should have been the one de-identifying the data and Willingham performing the analysis. It would seem that Willingham's research was twisted around from the start.

In early 2013, Willingham amended her IRB application to increase her sample size to 184 entering student-athletes, up from the original 46. This time the IRB flagged a concern about using identifiable data but she specifically indicated the data would be de-identified before it was entered into SPSS, a statistical analysis software. Based on that statement, the IRB again determined this involved secondary data and did not require IRB approval.

But again, Willingham's statements in the Friday N&O article indicate she was already off target as it related to both the stated purpose and use of data. Willingham said that her research stopped being about learning disabilities and veered into reading levels because "she saw little difference in identifying learning disabilities versus reading levels." It is difficult to believe that a learning "expert" would make such a preposterous statement. She also attempts to justify the switch in research purpose by citing an email to Provost Jim Dean citing her conclusions. But Dean would have had no way to know what her original research statement was, nor would he be in a position to approve it.

The complex part about Willingham's research purpose is that her positions at UNC, both in athletics and later in the Center for Student Success, allowed her access to the data she was using in her study. The problem is that, given her position, the data was tied directly to individual student-athletes. She indicated publicly that she had access to grades, test scores, and other data and used them in her study. This is in clear contravention to the second part of the definition of human subjects, which specifically addresses data. Once the IRB became aware of this, they advised her she needed to seek specific IRB approval. They did not "suspend her research" as the narrative went because they had never sanctioned her research in the first place given her statement on the use of secondary data. Moreover, she says in the same article that she needed the test scores to assist in her position of helping athletes. The only problem with that is she stopped working with athletes in early 2010.

Based on her statements, it seems she thought that as long as there were no identifiers on any of the data she shared or results she shared, then she was meeting the secondary data requirement. She told the N&O that others could not link the data back to specific athletes and none have been identified (well, except for the 2005 basketball team and the 2013 football team). But in order to meet federal and institutional guidelines, she should not have known who the data belonged to; otherwise it becomes primary data and is subject to IRB guidelines.

There has also been a lot of message board buzz about the revisions to the IRB application that added Richard Southall and Jay Smith as researchers. The most likely scenario there is a move to allow Southall and Smith to legally and ethically access her raw data. Southall is at least a noted researcher on college athletics, so most of the message board ire has been directed at Smith. Since Smith and Willingham are writing a book, however, it seems pretty clear where that comes from.

In short, from the very beginning, Willingham operated in contravention to IRB guidelines regarding research purpose and use of data. Whether this was by deceit or incompetence is up for debate. Given her comments in the N&O, it would seem incompetence is the most likely scenario, given her apparent misapplication of primary versus secondary data and her contention that there is little difference in learning disabilities and reading levels. There is also the fact that ostensibly her research was to support her masters thesis, but the original study of 46 student-athletes merited only part of a single paragraph near the end of a 30-page paper.

Again, her conclusions had already been destroyed, and now her motives and methods are in tatters. This calls into question anything she has had to offer, whether on the AFAM mess or the O'Bannon case.