4 ICRC ‘impact information’ treatments: Questions and tests

Rather than poetically characterizing our results, we try to keep the narrative short to let the data and statistical measures speak for themselves. The tests below follow from the questions asked and procedures proposed in our pregistration.

In the preregistration we say …

We supplement this with exploratory work and robustness considerations.

This is followed by Bayesian inference probing the ‘tight bounds’ of the null effect.

This naturally leads into the following section, conducting meta-analysis across related experiments.*

* For now, I am conducting that meta analysis in the present “ICRC” repository because we may not have full permission to share the data in the other (public) repository.

I can mirror (hard link) the data and code putting together “those other experiments” here to make this more cohesive.

4.1 Preregistered tests

4.2 Exploratory analysis

4.3 Bayesian intervals, equivalence tests, probing the ‘tight null effect’

Incorporate and adapt to material from ‘Dual process’ repo, file, dv_input_anal.Rmd, results hosted HERE

Bayesian Test of Difference in Proportions

(see other repo, some notes and headers included below)

Comparison of Posterior Probabilities

Distributional Plots

Comparison of MAPs, relative to the baserate

Maximum a Posteriori (MAP) estimates are the Bayesian analogy for the frequentist maximum likelihood estimates. They provide a point estimate for the most likely value of the parameter of the random variable (the coefficient of interest). (I.e., this is the mode of the posterior density.)

Forest Plot Comparison

4.4 Q: Does including impact information affect the amount raised?

Hypothesis tests and inference: Rank-sum and t-tests, confidence/credible intervals

Rank sum tests

#Ranksum test - include zeroes

wilcox.test(rev_rank ~ treatment, data = ., exact = FALSE, conf.int=TRUE)