Let me apologize right now for the negative tone of many recent blog posts. This past field season was my most difficult ever. In 2017 I found it difficult to motivate one of my field assistants to complete data collection at all of her assigned lakes each day, and her complaints and weak performance hurt team morale. But last year was doubly trying, because a second field assistant falsified her data. In truth, this was the third instance of falsification that I had observed during the study.
One can dismiss data falsification by one or two field workers as flukes. When we found that Laura was not going to some of the lakes on her circuit in 1996 and was faking her arrival and departure times at some others, we were horrified. Having never encountered this problem before, though, we eventually decided that something was wrong with Laura for her to have betrayed our trust. Margaret, my head field assistant that year, announced, “I always thought she was a spoiled brat”, and I quickly agreed. When Frank, another team leader, caught Chelsea faking her data in 2005, I allowed myself to reach a similar conclusion. The problem was Chelsea, not us. I assured Frank that I would be more careful in screening potential field assistants in the future.
The third instance of faking data, which occurred last year, forced me to confront an unpleasant possibility: something about the way we select student assistants or go about data collection makes our project prone to data falsification. One aspect of this problem is obvious. While we are able to cover more ground and collect more data per observer than most other studies by having each observer work independently, solitary data collection opens the door to cheating. A single weak person in a weak moment who decides to fake data has no check on their behavior, whereas neither of two observers working in a pair would likely risk discovery by proposing data falsification to their teammate.
Fortunately, the design of our data collection also makes it easy to detect falsification. You see, we systematically rotate observer visits to lakes to limit the impact of observer bias. Observer bias — the innocent and natural tendency of each individual observer to detect and record certain events in their environment and not others — occurs in all observational studies. We limit such bias by making sure that all observers visit all study lakes and that no two consecutive visits to a given lake are by the same observer. This protocol gives us a means to detect faking of data, because any occasion when one visitor’s observations at a lake fail to match those of the previous visitor — like one observer reporting a failed nest and a different observer a week later reporting chicks — is immediately flagged for further scrutiny. Rotation of lake visits should also discourage cheating, because field assistants know that the lakes that they are assigned to visit on a certain day will be visited by a teammate in a week’s time. In effect, we are all constantly checking on each other’s observations. We are like a community of paranoiacs!
Our unintentional safeguard against data falsification is lucky, but it only makes the repeated occurrence of cheating more puzzling. Our field assistants are young people, eager to gain experience and to show their field skills to scientists who might write them strong reference letters. Their application to work with me must include names of three references and considerable information about their academic background and training. Why would such people risk fallout from a severe violation of academic integrity by falsifying data? We can fully never solve this puzzle, I suppose. Needless to say, exit interviews with data cheats are not feasible. In the three instances of falsification: 1) Laura simply denied faking data, despite having been caught red-handed doing so, 2) Chelsea admitted falsifying data and commented, “I cannot believe that I took that chance”, and 3) our faker from last year also admitted her wrongdoing, but insisted that she faked data only on the single occasion when she was caught.
I suspect that I will expose myself to ridicule with the following conclusions about cheaters’ motivations, but I will proceed. First, I think the adage that “cheaters never expect to be caught” applies here. Perhaps young people get caught up in the present and underestimate the power of rotating observer visits to detect inconsistencies in data. That is, assistants might decide that if they are not actually observed cheating in the field, they will not be found out. Second, I think young people are perhaps a bit more selfish now than they were 30 years ago, as revealed by Chelsea’s comment. If your main priority is to promote your own future rather than to safeguard the integrity of scientific data, then the bar is much lower for cheating.
What now? Can we screen research assistant applications more effectively and discard those that show signs of a defective moral compass? Is there a simple and inexpensive personality test that we can run to detect a proclivity for cheating?