Questionable Research Practices: Attitudes & Predictors

Introduction to Questionable Research Practices

Questionable Research Practices (QRPs) represent a critical and increasingly scrutinized area within the philosophy and practice of science. While the pursuit of knowledge relies fundamentally on integrity and unbiased reporting, QRPs encompass a range of actions that violate established scientific norms and best practices, yet often fall short of outright research misconduct, such as fabrication or falsification. These practices pose a significant threat to the validity, reliability, and ultimately, the reproducibility of scientific findings across all disciplines, particularly psychology and medicine. Understanding the mechanisms that drive researchers to engage in QRPs requires a deep examination of both individual attitudes and the systemic pressures inherent in the modern academic environment. The discussion of QRPs is not merely an ethical exercise; it is central to ensuring that the scientific enterprise maintains the public trust necessary for its continued function and funding.

The distinction between honest error, QRPs, and outright fraud is often subtle, residing primarily in the intent and the severity of the deviation from accepted methodology. Honest errors are typically unintentional mistakes in data recording or analysis, whereas QRPs involve deliberate, though often rationalized, choices designed to improve the chances of achieving a statistically significant or publishable result. These behaviors create a gray area where researchers may feel justified in bending rules to achieve what they perceive as a worthwhile scientific outcome, thereby undermining the foundational principles of objective inquiry. The difficulty in identifying and measuring the true prevalence of QRPs stems from the fact that they are often subtle, easily disguised, and rely heavily on researchers’ self-reporting, which is subject to strong social desirability bias.

The rising awareness of the “replication crisis” across various scientific fields has amplified the focus on QRPs as primary contributors to unreliable literature. When research is conducted using practices that inflate statistical significance or selectively report findings, the resulting body of evidence becomes unstable, leading to wasted resources and potentially harmful applications, especially in clinical settings. Therefore, a comprehensive understanding of the attitudes that normalize these behaviors, alongside the predictive factors that make certain researchers or environments more susceptible to them, is essential for developing effective preventative and remedial strategies aimed at restoring rigor and transparency to the scientific process. This analysis will explore these underlying psychological and environmental drivers in detail.

Defining Questionable Research Practices

Questionable Research Practices are defined as actions taken during the research process that, while not meeting the strict definition of fraud (Fabrication, Falsification, and Plagiarism, or FFP), severely compromise the integrity of the research findings. These practices typically involve manipulating data collection, analysis, or reporting procedures to achieve a desired outcome, most frequently a statistically significant result (a low p-value) suitable for publication in high-impact journals. The insidious nature of QRPs lies in the fact that they are often performed under the guise of flexibility in data analysis or methodological optimization, allowing researchers to mentally distance themselves from the ethical implications of their actions.

One of the most widely studied and frequently cited QRPs is P-hacking, also known as data dredging or significance chasing. P-hacking involves conducting multiple statistical tests or analyses on a dataset until a significant result (p < .05) is obtained, without adequately correcting for the increased probability of Type I errors (false positives). This can take several forms, including stopping data collection early once significance is reached, excluding outliers post-hoc based on outcome rather than predefined criteria, transforming data in multiple ways until the desired result emerges, or running several different statistical models and only reporting the one that yields significance. The consequence of P-hacking is the inflation of the scientific literature with findings that are merely artifacts of statistical selection bias rather than true effects, thus severely limiting their generalizability and reproducibility.

Another critical QRP is HARKing, which stands for Hypothesizing After the Results are Known. HARKing occurs when a researcher analyzes the data, discovers an unexpected but significant finding, and then presents the finding in the final research report as if it were the result of a pre-planned, deductive hypothesis. This practice fundamentally misrepresents the scientific process. Science relies on a clear distinction between exploratory (hypothesis-generating) research and confirmatory (hypothesis-testing) research. By framing an unexpected exploratory finding as a planned test of a theory, HARKing misleads readers about the true evidential support for the claim, creating a narrative that is deductively strong when the process was, in fact, inductively driven by the data. This practice severely biases the literature toward neat, clean narratives that often mask the true complexity and messiness of real-world data collection.

The Spectrum of Research Misconduct

Research misconduct exists on a continuum, ranging from minor procedural deviations to outright fraud, with QRPs occupying the expansive middle ground. At one end of the spectrum are honest errors—unintentional mistakes in methodology, coding, or statistical application that are typically corrected upon discovery. At the opposite end lies FFP (Fabrication, Falsification, and Plagiarism), which constitutes deliberate, intentional deceit and is universally recognized as grounds for severe professional sanction. QRPs bridge this gap, characterized by intentional actions designed to bias results but often rationalized by the researcher as acceptable practices within a competitive environment.

The primary factor distinguishing QRPs from FFP is the level of intent to deceive the scientific community. Fabrication involves making up data entirely; falsification involves manipulating existing data or results (e.g., changing data points or omitting inconvenient findings) such that the research is not accurately represented. While P-hacking and HARKing manipulate the *interpretation* or *presentation* of data, they typically do not involve outright creation or wholesale alteration of raw data, which is why they are considered “questionable” rather than fraudulent. However, repeated and severe QRPs can accumulate to create an outcome that is functionally equivalent to falsification, blurring the ethical line considerably and making the researcher’s self-justification increasingly tenuous.

The normalization of QRPs often stems from a psychological coping mechanism where researchers minimize the ethical harm. They may argue that the underlying phenomenon they are seeking is “real,” and that the QRP is merely a necessary shortcut to bypass the stringent and often arbitrary requirements of the publication system. This self-deception allows researchers to maintain a positive professional identity while engaging in behaviors that violate the core tenets of scientific objectivity. Understanding this psychological gray area is key, as addressing QRPs requires not just punitive measures, but cultural and systemic shifts that remove the incentives for such rationalizations.

Attitudes Shaping Research Behavior

The willingness of a researcher to engage in QRPs is profoundly influenced by their personal and professional attitudes, particularly concerning statistical flexibility, peer review rigor, and career demands. Explicit attitudes relate to what researchers openly state about scientific integrity, which is almost universally positive. However, implicit attitudes—the subconscious beliefs and perceived norms regarding acceptable behavior—often drive engagement in QRPs. If a researcher perceives that their peers or supervisors view minor P-hacking as a necessary evil for career success, that perception becomes a powerful predictive factor for their own behavior, regardless of their stated commitment to ethical ideals. This attitude of normative acceptance is highly corrosive to research integrity.

A significant contributing attitude is a belief in the flexibility of data analysis. Researchers may view statistical modeling not as a rigorous, predefined test of a hypothesis, but rather as an art form or a tool to coax meaning from ambiguous data. This attitude encourages the exploration of countless analytical pathways (the “garden of forking paths”), where the researcher feels entitled to choose the path that yields the most interesting or significant result. This attitude is often coupled with a belief in scientific exceptionalism, where the researcher feels their primary duty is to advance knowledge quickly, and minor ethical compromises are justified if they lead to a major discovery. This prioritized goal orientation, focused on output quantity rather than methodological quality, fundamentally skews ethical decision-making.

Several key attitudinal factors are empirically linked to higher QRP engagement:

• Cynicism about the Peer Review Process: A belief that peer review is arbitrary, biased, or primarily focused on novelty rather than methodological rigor can lead researchers to conclude that the system itself demands QRPs for survival.
• Low Perceived Risk of Detection: The attitude that QRPs are difficult to detect or prove acts as a powerful disinhibitor. Since raw data sharing has historically been uncommon, researchers often feel shielded from scrutiny, encouraging risky behavior.
• High Competition and Career Anxiety: An intense focus on personal career advancement and the perceived scarcity of high-impact publication slots creates an attitude that publication success is a zero-sum game, justifying aggressive tactics like QRPs.
• Belief in the “Realness” of the Effect: The conviction that the phenomenon under study genuinely exists, even if the current data presentation requires minor manipulation, allows the researcher to rationalize QRPs as merely helping the truth emerge.

Individual Predictors of QRP Engagement

While systemic pressures provide the context for QRPs, certain individual characteristics and professional circumstances increase the likelihood of a researcher resorting to these practices. One primary predictor is the level of statistical and ethical literacy. Researchers who lack rigorous training in advanced statistical modeling or who have been taught outdated or permissive analytical techniques may engage in QRPs unintentionally, simply because they do not recognize the ethical implications of procedures like sequential data analysis or inappropriate outlier exclusion. Ignorance, while not an excuse, often plays a role in the initial adoption of QRPs, which then become normalized habits.

Beyond training, certain personality traits have been linked to QRP engagement, though these findings require careful interpretation. Researchers exhibiting high levels of narcissism or low conscientiousness may be more prone to QRPs. Narcissistic individuals, driven by a need for recognition and superiority, may prioritize the achievement of groundbreaking results over methodological integrity. Conversely, low conscientiousness—characterized by disorganization and a lack of diligence—can lead to sloppy methodological choices that veer into QRP territory, such as inconsistent data handling or failure to pre-register design choices. These traits interact complexly with the high-pressure environment of academia, amplifying the risk of integrity breaches.

Perhaps the strongest individual predictor is the degree of career pressure and competitive drive. Early-career researchers (ECRs) who face imminent tenure reviews, funding deadlines, or dissertation defense requirements are under immense pressure to produce statistically significant, novel results. This “publish or perish” environment forces individuals to make difficult trade-offs between speed, novelty, and rigor. When a study yields ambiguous or null results, the temptation to engage in P-hacking or HARKing to secure a publication—which is directly tied to their economic and professional survival—becomes overwhelming. This pressure is less about inherent malice and more about rational self-preservation within a flawed incentive structure.

Systemic and Environmental Predictors

The engagement in QRPs is not merely a matter of individual failure; it is powerfully shaped by the organizational and systemic structures of the academic ecosystem. Institutional pressures are paramount. Universities and research institutions often use metrics like grant funding acquisition and publication count in high-impact journals (measured by Impact Factor) as primary criteria for hiring, promotion, and tenure decisions. This creates an environment where the perceived quality of research is conflated with its statistical significance and novelty, rather than its methodological soundness or contribution to cumulative knowledge.

Journal policies and the culture of scientific publishing serve as major systemic predictors. Journals generally exhibit a strong preference for positive, novel findings, leading to a phenomenon known as the file drawer problem, where studies yielding null results are rarely submitted or accepted for publication. This bias incentivizes researchers to manipulate their data until they achieve a publishable positive result, thereby skewing the entire scientific literature towards inflated effect sizes and false positives. Furthermore, the speed required for publication often discourages the time-intensive processes necessary for robust research, such as large sample sizes and comprehensive replication attempts.

The structure of research teams and supervisory dynamics also plays a crucial predictive role. In hierarchical laboratory settings, power imbalances can lead to QRPs. Trainees or junior researchers may feel unable to question or resist a senior researcher or principal investigator who encourages or demands practices that compromise integrity. Lack of robust, transparent oversight within large research teams can allow QRPs to flourish unnoticed. Systemic factors that predict QRPs include:

1. Funding Structures: Short grant cycles that demand rapid, high-impact output.
2. Tenure Requirements: Over-reliance on quantitative metrics (counts of papers, Impact Factor scores) rather than qualitative evaluation of methodological rigor.
3. Lack of Replication Incentives: Scientific journals and institutions rarely reward or prioritize direct replication studies, thus failing to punish the publication of unreliable initial findings.
4. Inadequate Institutional Review Board (IRB) Scrutiny: While IRBs focus heavily on ethical treatment of human subjects, they often lack the expertise or mandate to scrutinize statistical analysis plans and methodology to prevent QRPs proactively.

Mitigating QRPs: Ethical Training and Reform

Addressing the widespread issue of QRPs requires a multi-pronged approach focused on both reforming systemic incentives and enhancing individual ethical competence. The most critical reform involves a widespread adoption of Open Science practices, which directly target the mechanisms underlying P-hacking and HARKing. The cornerstone of this movement is preregistration, where researchers publicly register their study hypotheses, design, sample size, and analytical plan before data collection begins. This practice renders HARKing impossible and severely limits the opportunity for post-hoc P-hacking, forcing researchers to adhere to confirmatory methods or clearly label deviations as exploratory.

Institutional and educational reforms must shift the focus from punishment to prevention and cultural change. Ethical training must move beyond abstract discussions of morality to practical, case-based instruction on statistical best practices. Researchers need to be explicitly taught the dangers of analytical flexibility and the importance of statistical rigor. Furthermore, institutions must overhaul their evaluation metrics for tenure and promotion, placing greater weight on methodological quality, transparency, data sharing, and the publication of robust null findings or replication studies, thereby reducing the pressure to manufacture positive results.

Finally, the scientific publishing landscape must evolve to support these reforms. Journals should embrace registered reports—a format where the study protocol is peer-reviewed *before* data collection—guaranteeing publication regardless of the outcome, provided the methodology is sound. Furthermore, mandatory data and code sharing policies, coupled with robust infrastructure for data archiving, increase the transparency and detectability of QRPs, thereby increasing the perceived risk of engaging in them. Ultimately, mitigating QRPs requires cultivating a culture where rigor and transparency are not merely ethical obligations but are the primary drivers of professional reward and recognition.