13  Using G*Power

I hope by now you are convinced by the argument that a priori1 power analysis is an important part of designing experiments, and in the design of experiments where there are ethical considerations governing the numbers of experimental subjects, such as those using animals, in particular.

Unfortunately, power analysis is frequently overlooked by many scientists in their own experimental designs. The oversight is sometimes deliberate, and justified by arguments along the lines of “power analyses are too complex to perform”2 (Mayr et al. (2007)) Happily, there are widely-available software packages that make power calculations more accessible for all researchers.

In this chapter, we will walk through a calculation of power analysis using the widely-available and easy to use package G*Power(Faul et al. (2007)).

Note

G*Power is a powerful software tool that can compute power calculations for a very wide range of statistical analyses, and several types of power calculation, including:

  • a priori power calculation: conducted before the experiment is performed
  • post-hoc power calculation: conducted after the experiment is performed - I would strongly discourage the use of post hoc power calculations.
  • sensitivity: if the sample size is known, calculates for a given desired staistical power what effect size would be required to achieve that statistical power

It is beyond the scope of this workshop to consider all the possible circumstances that G*Power can accommodate, and our goal here is to introduce you to the basic functionality of the program.

13.2 G*Power Walkthrough: one-sample t-test

In this walkthrough we will consider a simple statistical analysis to determine whether the average weight of ten mice is statistically different from 25g.

13.2.1 Preliminary questions and assumptions

This is about as simple a case as we might want to consider, but we still need to answer some questions before we start:

  • We know that mean values of samples from populations tend to follow a Normal distribution, so this suggests a t-test is appropriate.
  • We are testing the mean of a sample against a single hypothesised central value, so this is a one-sample t-test.
  • As we want to know if the sample mean is different from that single stated value, we require a two-tailed test.
  • We want to determine an appropriate sample size for our experiment
  • As we want to know a good sample size for our experiment, before the experiment is conducted, this is an a priori power calculation.
  • We will use a standard funder’s requirements of 80% power at a \(P < 0.05\) threshold for statistical significance.
    • i.e. \(\alpha = 0.05, \beta = 0.2, 1 - \beta = 0.8\)
  • We will assume that a 10% difference in weight from the target 25g is meaningful, for an effect size of 2.5g.
  • We will assume that the weight of individuals in the sample has a standard deviation of about 1.25g
    • i.e. \(\sigma = 1.25\)

13.2.2 Walkthrough

  1. Open the G*Power application, to see the default settings (Figure 13.1).
Figure 13.1: The opening view of G*Power. This presents three panels: at the top is a space where output will be shown; in the middle, we can select the kind of statistical analysis we want to perform (Test family, Statistical test), and the kind of power analysis we’d like to carry out (Type of power analysis); below this is a set of fields where we can enter parameters for the power calculation, and where results will be displayed.
  1. Set the appropriate values for the statistical analysis we want to perform: one-sample, two-tailed t-test (Figure 13.2).
Figure 13.2: The appropriate G*Power settings for our analysis. The Test family was already set to t tests, but we had to change (i) the Statistical test option to Means: Difference from constant (one sample case), (ii) Type of power analysis to A priori: Compute required sample size - given $\alpha$, power, and effect size, and Tail(s) to be Two.
  1. Set input parameters for statistical power and significance. We have chosen a statistical significance threshold of \(\alpha = 0.05\) and power as \(1 - \beta = 0.8\), and can enter these values directly (Figure 13.3).
Figure 13.3: The Input Parameters pane of G*Power with appropriate values set for statistical power and significance.
  1. We still have to enter the effect size in the Input Parameters. G*Power expects something called Cohen’s d, not the absolute effect size of 2.5g - so we must calculate this. Click on the Determine button to open the effect size drawer and enter appropriate values for Mean H03, Mean H14, and $\sigma err prob5, then click Calculate and transfer to main window (Figure 13.4).

Cohen’s d is an effect size index, which is a way of turning the expected means and variations from comparisons at very different quantitative scales into a single comparable value. To calculate d, we use the equation:

\(d = \frac{\mu - \mu_0}{\sigma}\)

where

  • \(\mu\) is the mean of the null hypothesis
  • \(\mu_0\) is the mean of the alternate hypothesis
  • \(\sigma\) is the estimate fo the (unknown) standard deviation in the population
Figure 13.4: The G*Power effect size window with our hypothesised mean values, and standard deviation, reporting a Cohen’s d for this comparison of 2. The Calculate and transfer to main window button has been pressed, which populates the Effect size d field of the Input parameters pane.
  1. Click the Calculate button to obtain the results of the power calculation (Figure 13.5).
Figure 13.5: The results of the power calculation. The predicted sample size is provided in the Output parameters pane, and a graphical representation of the likely statistical outcomes given the input parameters is shown in the topmost pane. Clicking on Protocol of power analyses provides a summary of the power calculation settings.
  1. Click on the X-Y plot for a range of values button. This will bring up a new Power Plot window, automatically populated with the settings from our analysis (Figure 13.6).
Figure 13.6: A blank Power Plot window, autopopulated with values from our power analysis.
  1. Click on the Draw plot button to obtain a graph showing how required sample size varies with desired power, with our analysis settings (Figure 13.7).
Figure 13.7: The Power Plot window, showing how sample size varies with desired power, for our chosen settings.
  1. You can save the graph as a .pdf file as shown below, by choosing File -> Save x-y plot from the top menu (Figure 13.8).
Figure 13.8: The File -> Save x-y plot option to save the Power Plot graph.

Download PDF file.

13.2.3 Interpreting the output

The results indicate that we need a sample size of at least \(N = 5\) experimental units to ensure a power \(1 - \beta \geq 0.8\), with \(\alpha = 0.05\).

Important

The actual power achieved with this sample size \(N = 5\) is \(1 - \beta = 0.9088849\), which is higher than that requested. We can use the Power Plot graph output to investigate how the predicted power of the experiment would change if we modified the sample size \(N\).

13.3 Manual and Tutorial

The current manual is found at this link.

Download PDF file.

This tutorial is published as Mayr et al. (2007).

Download PDF file.

  1. Before the experiment takes place.↩︎

  2. Although it could be noted that, if the power analysis is too complex for the researcher, we should be open to the possibility that the experiment is too complex for the researcher.↩︎

  3. This is the null hypothesis - for us, the null hypothesis is that there is no difference between the mean of our sample and the constant value of 25g, so set this to be 25.↩︎

  4. This is the expected mean of the alternative hypothesis. Our assumed effect size of 2.5g implies that this expected mean is 27.5g.↩︎

  5. This is the estimate of the standard deviation of the population. Our estimate is that this is 1.25g.↩︎