Sample size calculation: Basic principles (2024)

Abstract

Addressing a sample size is a practical issue that has to be solved during planning and designing stage of the study. The aim of any clinical research is to detect the actual difference between two groups (power) and to provide an estimate of the difference with a reasonable accuracy (precision). Hence, researchers should do a priori estimate of sample size well ahead, before conducting the study. Post hoc sample size computation is not encouraged conventionally. Adequate sample size minimizes the random error or in other words, lessens something happening by chance. Too small a sample may fail to answer the research question and can be of questionable validity or provide an imprecise answer while too large a sample may answer the question but is resource-intensive and also may be unethical. More transparency in the calculation of sample size is required so that it can be justified and replicated while reporting.

Key words: Effect size, medical research, power, randomized control trial, research hypothesis, sample size, study design

INTRODUCTION

Besides scientific justification and validity, the calculation of sample size (‘just large enough’) helps a medical researcher to assess cost, time and feasibility of his project.[1] Although frequently reported in anaesthesia journals, the details or the elements of calculation of sample size are not consistently provided by the authors. Sample size calculations reported do not match with replication of sample size in many studies.[2] Most trials with negative results do not have a large enough sample size. Hence, reporting of statistical power and sample size need to be improved.[3,4] There is a belief that studies with small sample size are unethical if they do not ensure adequate power. However, the truth is that for a study to be ethical in its design, its predicted value must outweigh the projected risks to its participants. In studies, where the risks and inconvenience borne by the participants outweigh the benefits received as a result of participation, it is the projected burden. A study may still be valid if the projected benefit to the society outweighs the burden to society. If there is no burden, then any sample size may be ideal.[5] Many different approaches of sample size design exist depending on the study design and research question. Moreover, each study design can have multiple sub-designs resulting in different sample size calculation.[6] Addressing a sample size is a practical issue that has to be solved during planning and designing stage of the study. It may be an important issue in approval or rejection of clinical trial results irrespective of the efficacy.[7]

By the end of this article, the reader will be able to enumerate the prerequisite for sample size estimation, to describe the common lapses of sample size calculation and the importance of a priori sample size estimation. The readers will be able to define the common terminologies related to sample size calculation.

IMPORTANCE OF PILOT STUDY IN SAMPLE SIZE ESTIMATION

In published literature, relevant data for calculating the sample size can be gleaned from prevalence estimates or event rates, standard deviation (SD) of the continuous outcome, sample size of similar studies with similar outcomes. The idea of approximate ‘effect’ estimates can be obtained by reviewing meta-analysis and clinically meaningful effect. Small pilot study, personal experience, expert opinion, educated guess, hospital registers, unpublished reports support researcher when we have insufficient information in the existing/available literature. A pilot study not only helps in the estimation of sample size but also its primary purpose is to check the feasibility of the study.

The pilot study is a small scale trial run as a pre-test, and it tries out for the proposed major trial. It allows preliminary testing of the hypotheses and may suggest some change, dropping some part or developing new hypotheses so that it can be tested more precisely.[8] It may address many logistic issues such as checking that instructions are comprehensive, and the investigators are adequately skilled for the trial. The pilot study almost always provides enough data for the researcher to decide whether to go ahead with the main study or to abandon. Many research ideas that seem to show great promise are unproductive when actually carried out. From the findings of pilot study, the researcher may abandon the main study involving large logistic resources, and thus can save a lot of time and money.[8]

METHODS FOR SAMPLE SIZE CALCULATION

Sample size can be calculated either using confidence interval method or hypothesis testing method. In the former, the main objective is to obtain narrow intervals with high reliability. In the latter, the hypothesis is concerned with testing whether the sample estimate is equal to some specific value.

TYPES OF TRIALS

Parallel arm RCTs are most commonly used, that means all participants are randomized in two or more arms of different interventions treated concurrently. Various types of parallel RCTs are used in accordance with the need: Superiority trials which verify whether a new approach is more effective than the conventional from statistically or clinical point of view. Here, the concurrent null hypothesis is that the new approach is not more effective than the conventional approach. Equivalence trials which are designed to ascertain that the new approach and the standard approach are equally effective. Corresponding null hypothesis states that the difference between both approaches is clinically relevant. Non-inferiority trials which are designed to ascertain that the new approach is equal if not superior to the conventional approach. Corresponding null hypothesis is that the new approach is inferior to the conventional one.

PREREQUISITES FOR SAMPLE SIZE ESTIMATION

At the outset, primary objectives (descriptive/analytical) and primary outcome measure (mean/proportion/rates) should be defined. Often there is a primary research question that the researcher wants to investigate. It is important to choose a primary outcome and lock that for the study. The minimum difference that investigator wants to detect between the groups makes the effect size for the sample size calculation.[7] Hence, if the researcher changes the planned outcome after the start of the study, the reported P value and inference becomes invalid.[11] The level of acceptable Type I error (α) and Type II error (β) should also be determined. The error rate of Type I error (alpha) is customarily set lower than Type II error (beta). The philosophy behind this is the impact of a false positive error (Type I) is more detrimental than that of false negative (Type II) error. So they are protected against more rigidly.

Besides, the researcher needs to know the control arm mean/event rates/proportion, and the smallest clinically important effect that one is trying to detect.

THE RELATION BETWEEN PRIMARY OBJECTIVE AND THE SAMPLE SIZE

The type of primary outcome measure with its clear definition help computing correct sample size as there are definite ways to reach sample size for each outcome measure. It needs special attention as it principally influences how impressively the research question is answered. The type of primary outcome measure also is the basis for the mode of estimation regarding population variance. For continuous variable (e.g., mean arterial pressure [MAP]), population SD is incorporated in the formula whereas the SD needs to be worked out from the proportion of outcomes for binomial variables (e.g., hypotension - yes/no). In literature, there can be several outcomes for each study design. It is the responsibility of the researcher to find out the primary outcome of the study. Mostly sample size is estimated based on the primary outcome. It is possible to estimate sample size taking into consideration all outcome measures, both primary and secondary at the cost of much larger sample size.

ESSENTIAL COMPONENTS OF SAMPLE SIZE ESTIMATION

The sample size for any study depends on certain factors such as the acceptable level of significance (P value), power (1 − β) of the study, expected ‘clinically relevant’ effect size, underlying event rate in the population, etc.[7] Primarily, three factors P value (depends on α), power (related with β) and the effect size (clinically relevant assumption) govern an appropriate sample size.[12,13,14] The ‘effect size’ means the magnitude of clinically relevant effect under the alternative hypothesis. It quantifies the difference in the outcomes between the study and control groups. It refers to the smallest difference that would be of clinical importance. Ideally, the basis of effect size selection should be on clinical judgement. It varies with different clinical trials. The researcher has to determine this effect size with scientific knowledge and wisdom. Available previous publications on related topic might be helpful in this regard. ‘Minimal clinically important difference’ is the smallest difference that would be worth testing. Sample size varies inversely with effect size.

The ideal study to make a researcher happy is one where power of the study is high, or in other words, the study has high chance of making a conclusion with reasonable confidence, be it accepting or rejecting null hypothesis.[9] Sample size matrix, includes different values of sample sizes using varying dimensions of alpha, power (1-β), and effect size. It is often more useful for the research team to choose the sample size number that fits conveniently to the need of the researcher [Table 1].

FORMULAE AND SOFTWARE

Once these three factors are fixed, there are many ways (formulae, nomogram, tables and software) for estimating the optimum sample size. At present, there are a good number of softwares, available in the internet. It is prudent to be familiar with the instructions of any software to get sample size of one arm of the study. Perhaps the most important step is to check with the most appropriate formula to get a correct sample size. Websites of some of the commonly used softwares are provided in Table 2.[2,6]

The number of formulae for calculating the sample size and power, to answer precisely different study designs and research questions are no less than 100. It is wise to check appropriate formula even while using software. Although there are more than 100 formulae, for RCTs numbers of formulae are limited. It essentially depends on the primary outcome measure such as mean ± SD, rate and proportion.[6] Interested readers may access all relevant sample size estimation formulae using the relevant links.

SUMMARY

Sample size calculation can be guided by previous literature, pilot studies and past clinical experiences. The collaborative effort of the researcher and the statistician is required in this stage. Estimated sample size is not an absolute truth, but our best guess. Issues such as anticipated loss to follow-up, large subgroup analysis and complicated study designs, demands a larger sample size to ensure adequate power throughout the trial. A change in sample size is proportional to variance (square of SD) and inversely proportional to the detected difference.

REFERENCES