There are no shortcuts to theory.

Almaatouq et al. claim that the integrative experiment design can help "develop a reliable, cohesive, and cumulative theoretical understanding." I will contest this claim by challenging three underlying assumptions about the nature of scientific theories. I propose that the integrative experiment design should be viewed as an exploratory framework rather than a means to build or evaluate theories.

The logical structure of experiments lays the foundation for a theory of reproducibility.

The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship among openness, replication experiments and results reproducibility has been obscure. We analyse the logical structure of experiments, define the mathematical notion of idealized experiment and use this notion to advance a theory of reproducibility. Idealized experiments clearly delineate the concepts of replication and results reproducibility, and capture key differences with precision, allowing us to study the relationship among them. We show how results reproducibility varies as a function of the elements of an idealized experiment, the true data-generating mechanism, and the closeness of the replication experiment to an original experiment. We clarify how openness of experiments is related to designing informative replication experiments and to obtaining reproducible results. With formal backing and evidence, we argue that the current 'crisis' reflects inadequate attention to a theoretical understanding of results reproducibility.

Rigorous exploration in a model-centric science via epistemic iteration.

Urgent attention is needed to address generalizability problems in psychology. However, the current dominant paradigm centered on dichotomous results and rapid discoveries cannot provide the solution because of its theoretical inadequacies. We propose a paradigm shift towards a model-centric science, which provides the sophistication to understanding the sources of generalizability and promote systematic exploration. In a model-centric paradigm, scientific activity involves iteratively building and refining theoretical, empirical, and statistical models that communicate with each other. This approach is transparent, and efficient in addressing generalizability issues. We illustrate the nature of scientific activity in a model-centric system and its potential for advancing the field of psychology.

The case for formal methodology in scientific reform.

Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (i) that reproducibility is the cornerstone of science; (ii) that data must not be used twice in any analysis; and (iii) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.

Scientific discovery in a model-centric framework: Reproducibility, innovation, and epistemic diversity.

Consistent confirmations obtained independently of each other lend credibility to a scientific result. We refer to results satisfying this consistency as reproducible and assume that reproducibility is a desirable property of scientific discovery. Yet seemingly science also progresses despite irreproducible results, indicating that the relationship between reproducibility and other desirable properties of scientific discovery is not well understood. These properties include early discovery of truth, persistence on truth once it is discovered, and time spent on truth in a long-term scientific inquiry. We build a mathematical model of scientific discovery that presents a viable framework to study its desirable properties including reproducibility. In this framework, we assume that scientists adopt a model-centric approach to discover the true model generating data in a stochastic process of scientific discovery. We analyze the properties of this process using Markov chain theory, Monte Carlo methods, and agent-based modeling. We show that the scientific process may not converge to truth even if scientific results are reproducible and that irreproducible results do not necessarily imply untrue results. The proportion of different research strategies represented in the scientific population, scientists' choice of methodology, the complexity of truth, and the strength of signal contribute to this counter-intuitive finding. Important insights include that innovative research speeds up the discovery of scientific truth by facilitating the exploration of model space and epistemic diversity optimizes across desirable properties of scientific discovery.

Self-prediction and patient health: influencing health-related behaviors through self-prophecy.

People asked to make a self-prediction about a socially normative behavior are significantly more likely (than a comparable control group) to perform the behavior in a manner consistent with social norms. Making a behavioral self-prediction has been demonstrated to increase attendance to a health club, consumption of healthy snacks, and commitment to a health and fitness assessment. Empirical evidence indicates that this self-prophecy effect is due to dissonance-based motivation generated by the prediction request. In this article, we present self-prediction as a practical and effective tool that health care professionals can use to favorably influence a variety of health-related, patient behaviors. Previous studies on health behaviors are aggregated using meta-analytical techniques to determine the magnitude of self-prediction effects on health-related behaviors. To account for potential errors of exclusion in our analysis, a file drawer analysis is also conducted. Our analysis suggests that self-prophecy manifests as a small- to medium- effect size when used in the context of modifying health-related behaviors. Providing support for the robustness of this effect, our file drawer analysis indicated that 270 further studies with null results would be needed to negate our conclusions regarding the effect. Based on previous research and findings of the current meta-analysis, we are confident that health care professionals can effectively employ self-prediction as a method for promoting healthier behaviors and lifestyles among their patients. Implications for medical practice and allied health fields, as well as areas of future research, are identified.