PREMIER Randomization and blinding of biomedical experiments
Background
One of the largest sources of experimental bias that influences outcome measures derives from studies in which experimental conditions are not fully or only partly known to the observer. The second largest source of bias arises from experiments in which the samples/ subjects are not randomly allocated to the different experimental conditions. While it is not always possible to control all forms of known or unknown bias, simple adjustments in the experimental design can greatly reduce its impact.
Randomization refers to the practice of using chance methods (random number tables, flipping a coin, etc.) to assign subjects to treatments.
Determine the number of blocks needed
Every experiment tests and compares at least 2 different conditions, for instance drug vs. control. In addition various drug concentrations, drug antagonists, positive and negative controls of extend the number of testable conditions. Power calculations can project how many samples each conditions needs to contain to answer the research question with statistically solid confidence. This forms a matrix of x number of samples times y numbers of conditions. Then x*y need to be arranged in experimental groups (blocks) that are similar to one another. An example for a block are number of animals undergoing MCAO surgery on a given day or the number of cell culture plates undergoing OGD. Sometime other nuisance variables have to be taken into account such as sex or genetic variants.
Two Different Designs
- Simple randomization: Randomized allocation of study subjects or sample to the different treatment groups based on one single sequence of random assignments. While this leads to a completely randomized sequence, the breakup into experimental units may either lead to imbalanced groups and group sizes.
- Stratified randomization or Block randomization: Allocation of study subjects or samples to blocks that share similar baseline characteristics followed by randomized allocation of the subjects/samples of each block to the different treatment groups.
This method adjusts for potential covariates (nuisance variables) across treatment groups and is therefore preferred.
Blinding is the continuous masking of the treatment allocation for the person(s) who perform the experiment, collect data and assess outcome. "Any process using a human as perceptor, rater, or interpreter should be as blind as possible for as long as possible"(Ref. 1)
How to achieve effective blinding
Effective blinding requires at least 2 people, one blinded and one unblinded person. If different persons are involved in various steps of the research process, make sure that they are sufficiently blinded. Exchange about the experiment its goals and expectations should ideally be limited.
The unblinded person is the keeper the blinding scheme. Blinding must occur prior to the start of the experiment and unblinding shall not occur until final analysis. It is of utmost importance that the inclusion and exclusion criteria are applied in a blinded fashion.
What is a blinding scheme?
A blinding schema contains a chronological, stepwise workflow of the experiment. Every step contains information about the conductor and participants and their subjective blinding status, which can be unblended, partially blinded or fully blinded. It also mentions the step(s) at which inclusion and exclusion criteria are applied. There are multiple web-based tools that you can use, e.g. the Experimental Design Assistant - EDA provided by NR3R or the Trial Randomization Tool available at the NCI.
Allocation concealment
The investigator should not be aware and/or have the choice to which treatment group a sample is allocated. Therefore the assignment to a specific group needs to be concealed and every sample should have the same chance to be assign to each of the groups. This can be achieved separating the assignment of identifiers and randomization of interventions into two independent processes and then merge the two.
Blinding Procedures
Blinding individual samples can be cumbersome because they usually part of a larger order system (cages, batches, or plates). But within these container systems allocation of treatments to all samples need to be randomized. It is utmost important that during measurement and analyses the executing person are unable to draw conclusions with regards to the conditions. However, this is sometimes not possible especially when certain cues cannot be blinded, such as skin color of a transgenic mice or color of a solution in a well.
Blinding requires at least 2 people, the blinded (BP) and the unblinded person (UP). The UP creates and holds on randomization and blinding key and masks the conditions appropriately. However, depending on the setup /duties several scenarios can occur, that require different workflows:
Scenario 1: In the absence of the BP, the UP provides the usually drug solutions or other intervention and applies them to experimental samples. For the BP, which is the rater and analyzer, no clues are given. The BP also applies the inclusion and exclusion criteria and marks excluded samples clearly. The UP then provides information, which samples belong to the same group and the BP applies the statistical analysis from the study plan. In the final step, the BP becomes unblinded.
Scenario 2: Similar to scenario 1, except that the BP, which is rater and analyzer, applies the intervention to the samples. Therefore, when providing the intervention, the UP must mask them. At this step, many experimenters often practice pseudoblinding by simply masking the common substance container. This provides grouping information to the BP, which now tends to harmonize rating of samples that belong to the same group. This can be avoided by the UP providing the substances or intervention in single units designated for one sample only. This requires special caution when labeling those samples and when they are applied.
Scenario 3: There is no UP to help. This should be avoided at any cost. There are occasions, for instance, on weekends or late hours, no personnel is available to help with blinding. Planning in advance can avoid such situations. Drugs could be prepared and blinded earlier or on a Friday.
Hidden cues in sample names
Often measurements occur long after treatment is applied. This can be the case when tissue samples are taken and further analyzed and videos of behavioral experiments were taken, but are scored at a much alter time point. Often immediate measures are already analyzed and unblinded. It is important to maintain the blinding status for the rater as long as needed.
In addition many hidden clues exist that can partially unblind the rater. This could be an additional sample information on a glass slide or vial, a folder or file name on data storage system or even time /date of files that could give away temporal or sequential information or access to analyzed and unblinded parts of data, e.g. shared electronic laboratory notebook project. During experimental planning phase, such potential cues must be identified and prevented.
Most often publications contain the standard sentence like: "All experiments were randomized and outcomes measurement were blinded." to satisfy publication guidelines. Even ARRIVE guidelines contain stricter requirements on reporting of blinding and randomization. In times of combating the replication crisis, reporting should contain:
- How was randomization achieved?
- What tool was used?
- Were randomized blocks used? - if yes, size of the blocks
- Was allocation concealment practiced? If no, explain why.
- How was ensured that the entire study was conducted in a blinded manner? If any, which parts were not blinded?
Were inclusion and exclusion criteria applied in a blinded manner? If not, explain why.
- Rosenthal, R., 1978. How often are our numbers wrong? American Psychologist 33, 1005–1008
- Clinical trials : study design, endpoints and biomarkers, drug safety, FDA and ICH guidelines. Brody, Tom. 1st ed. Amsterdam : Elsevier/AP, 2012. NLM ID: 101575674.