{"text_format":"3","workbook":"xlsx","n_sheets":11,"sheets":["Citation","Criterion1","Criterion2","Criterion3","Criterion4","Criterion5","Criterion6","Criterion7","Criterion8","Overall","All_decisions"],"not_rendered":{"images":8,"defined_names":1},"read":"whole","date_system":"1900"}

{"sheet":"Citation","index":0,"n_rows":3,"n_cols":1,"rows_kept":3}
["Last updated: 08 May 2021"]
["Please cite as: Konno K, Livoreil B, Pullin AS. 2021. Collaboration for Environmental Evidence Critical Appraisal Tool version 0.2 (prototype)."]
["© 2021 the authors."]

{"sheet":"Criterion1","index":1,"n_rows":45,"n_cols":4,"rows_kept":45,"merged":[[0,0,0,3],[2,0,5,0],[2,1,5,1],[2,3,5,3],[6,0,11,0],[6,1,11,1],[6,3,11,3],[12,0,16,0],[12,1,16,1],[12,3,16,3],[17,0,22,0],[17,1,22,1],[17,3,22,3],[23,0,28,0],[23,1,28,1],[23,3,28,3],[29,0,38,0],[29,1,38,1],[29,3,38,3],[39,0,39,3],[40,0,42,1],[40,3,42,3],[43,0,43,3],[44,0,45,3]]}
["Criterion 1: Risk of confounding"]
["Category","Checklist questions","Response option","Answer"]
["General (please answer)","1.1. Is it possible for the impact of the exposure or the effectiveness of the intervention to be confounded (or influenced by third variable) in this study?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
["Conditional (answer if Y/SY to 1.1, otherwise select 'Not applicable')","1.2. Did the author(s) control for all the important potential confounding factor(s) that are likely to be predictive of the outcome?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
[null,null,"Not applicable"]
["Conditional (answer if N/SN/Unclear to 1.2, otherwise select 'Not applicable')","1.3. Is there any justifiable reason for not controlling for all the important potential confounding factor(s) (so that omission of some of the important potential confounding factors is unlikely to influence the assessment of the effectiveness or impact)? (e.g. select ‘yes’ or ‘seemingly yes’ when there is evidence that omission of some of the important potential confounding factors does not affect the assessment of effectiveness or impact)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Not applicable"]
["Conditional (answer if Y/SY to 1.2 or 1.3, otherwise select 'Not applicable')","1.4. Were the important potential confounding factor(s), that were controlled for, likely to be measured accurately and precisely enough in this study? (measurements of confounding factor(s) may be nominal (categorical), ordinal (ranks) or scale)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
[null,null,"Not applicable"]
["Conditional (answer if you have answered 1.4, otherwise select 'Not applicable')","1.5. Did the author(s) analyse the effect appropriately by taking into account the important potential confounding factors, as well as the issue of accuracy and precision of the measurements of the potential confounding factor(s)?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
[null,null,"Not applicable"]
["Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)","1.6. What are the predicted magnitude and the direction of bias due to confounding? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result) (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)","Intervention or exposure intolerably favoured"]
[null,null,"Intervention or exposure tolerably favoured "]
[null,null,"Comparator intolerably favoured"]
[null,null,"Comparator tolerably favoured"]
[null,null,"Intolerably towards no effect"]
[null,null,"Tolerably towards no effect"]
[null,null,"Intolerably away from no effect"]
[null,null,"Tolerably away from no effect"]
[null,null,"Unpredictable"]
[null,null,"Skip"]
["Judgement about risk of bias for the criterion"]
["Please finalise your judgement about risk of bias for this criterion usign the algorithm.",null,"Low risk of bias"]
[null,null,"Medium risk of bias"]
[null,null,"High risk of bias"]
["Reason for deviationo from the suggested judgement about risk of bias"]
["Reason for deviation: NA"]

{"sheet":"Criterion2","index":2,"n_rows":67,"n_cols":4,"rows_kept":67,"merged":[[0,0,0,3],[2,0,11,0],[2,1,11,1],[2,3,11,3],[12,0,21,0],[12,1,21,1],[12,3,21,3],[22,0,26,0],[22,1,26,1],[22,3,26,3],[27,0,32,0],[27,1,32,1],[27,3,32,3],[33,0,38,0],[33,1,38,1],[33,3,38,3],[39,0,44,0],[39,1,44,1],[39,3,44,3],[45,0,50,0],[45,1,50,1],[45,3,50,3],[51,0,60,0],[51,1,60,1],[51,3,60,3],[61,0,61,3],[62,0,64,1],[62,3,64,3],[65,0,65,3],[66,0,67,3]]}
["Criterion 2: Risk of selection bias"]
["Category","Checklist questions","Response option","Answer"]
["General (please answer whichever suitable)","2.1.a. Was the selection of subjects or areas into the study (or analysis) random or systematic (i.e. random or systematic sampling) and the selection was not altered afterwards? (This applies when an attempt was not made to collect data of the entire, or nearly entire, inference population)\nOR\n2.1.b. Was the entire, or nearly entire, population of inference included in the study (or analysis), and the selection was not altered afterwards? (This applies when an attempt was made to collect data of the entire, or nearly entire, inference population)\n\n(Assessors may accept slight alteration (e.g. exclusion of subjects may be considered as ‘no alteration’ if it is acceptable))","Yes (2.1.a)"]
[null,null,"Seemingly yes (2.1.a)"]
[null,null,"Seemingly no (2.1.a)"]
[null,null,"No (2.1.a)"]
[null,null,"Unclear (No) (2.1.a)"]
[null,null,"Yes (2.1.b)"]
[null,null,"Seemingly yes (2.1.b)"]
[null,null,"Seemingly no (2.1.b)"]
[null,null,"No (2.1.b)"]
[null,null,"Unclear (No) (2.1.b)"]
["General (please answer whichever suitable)","2.2.a. Was/were the researcher(s) unaware (or blinded) of the selection until subjects or areas were assigned to or grouped as intervention/exposure and comparator, and the selection was not altered afterwards? (when the study conducted the selection or grouping)\nOR\n2.2.b. Had the selection or grouping taken place before the conduct of the study (so that the researcher(s) were not involved in the selection or grouping at all) and the selection was not altered afterwards?\n\n(Assessors may accept slight alteration (e.g. exclusion of subjects may be considered as ‘no alteration’ if it is acceptable))","Yes (2.2.a)"]
[null,null,"Seemingly yes (2.2.a)"]
[null,null,"Seemingly no (2.2.a)"]
[null,null,"No (2.2.a)"]
[null,null,"Unclear (No) (2.2.a)"]
[null,null,"Yes (2.2.b)"]
[null,null,"Seemingly yes (2.2.b)"]
[null,null,"Seemingly no (2.2.b)"]
[null,null,"No (2.2.b)"]
[null,null,"Unclear (No) (2.2.b)"]
["General (please answer)","2.3. Did the start of follow-up (when specific individuals or particular areas, such as plots, who had received intervention/exposure or control treatments were followed up) or the start of post-intervention or post-exposure selection of subjects or areas (when subjects or areas were selected after intervention or exposure; this might be the case if individuals or areas were not specifiable so that follow-up to the same individuals or exact areas was not applicable due to the nature of study) coincide for most subjects or areas? (If only one subject or area was studied, select 'yes' or 'seemingly yes')","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
["Conditional (answer if N/SN/Unclear to 2.1, 2.2 or 2.3, otherwise select 'Not applicable')","2.4. Was the follow-up or the selection of subjects or areas after intervention or exposure likely to be associated with any characteristics of the subjects or areas (that have different qualitative or quantitative values for different entities; e.g. pH, temperature, subspecies, age, sex, etc.)?","Yes"]
[null,null,"Probably yes"]
[null,null,"Probably no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
[null,null,"Not applicable"]
["Conditional (answer if Y/SY/Unclear to 2.4, otherwise select 'Not applicable')","2.5. Were the characteristics of the subjects or areas, that are likely to be associated with the follow-up or the selection of subjects or areas after intervention or exposure, also likely to be related to status of the intervention or exposure (or correlated with intervention or exposure variable(s); e.g. intervention intensity)?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
[null,null,"Not applicable"]
["Conditional (answer if Y/SY/Unclear to 2.5, otherwise select 'Not applicable')","2.6. Were the characteristics of the subjects or areas, that are likely to be associated with the follow-up or the selection of subjects or areas after intervention or exposure, also likely to be predictive of the outcome?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
[null,null,"Not applicable"]
["Conditional (answer if N/SN/Unclear to 2.1, 2.2 or 2.3, otherwise select 'Not applicable')","2.7. Did the author(s) estimate the potential bias(es) (systematic error(s)) due to haphazard or altered selection, or unequal timing of selection of subjects or areas into the study, and found that the bias(es) was/were minimal? Alternatively, is there evidence that the effect is measured accurately enough in relation to bias arising from selection of subjects or areas into the study or analysis after intervention or exposure?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
[null,null,"Not applicable"]
["Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)","2.8. What are the predicted magnitude and the direction of bias arising from systematic differences in the selection of subjects or areas into the study or analysis after intervention or exposure? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)","Intervention or exposure intolerably favoured"]
[null,null,"Intervention or exposure tolerably favoured "]
[null,null,"Comparator intolerably favoured"]
[null,null,"Comparator tolerably favoured"]
[null,null,"Intolerably towards no effect"]
[null,null,"Tolerably towards no effect"]
[null,null,"Intolerably away from no effect"]
[null,null,"Tolerably away from no effect"]
[null,null,"Unpredictable"]
[null,null,"Skip"]
["Judgement about risk of bias for the criterion"]
["Please finalise your judgement about risk of bias for this criterion usign the algorithm.",null,"Low risk of bias"]
[null,null,"Medium risk of bias"]
[null,null,"High risk of bias"]
["Reason for deviationo from the suggested judgement about risk of bias"]
["Reason for deviation: NA"]

{"sheet":"Criterion3","index":3,"n_rows":33,"n_cols":4,"rows_kept":33,"merged":[[0,0,0,3],[2,0,6,0],[2,1,6,1],[2,3,6,3],[7,0,11,0],[7,1,11,1],[7,3,11,3],[12,0,16,0],[12,1,16,1],[12,3,16,3],[17,0,26,0],[17,1,26,1],[17,3,26,3],[27,0,27,3],[28,0,30,1],[28,3,30,3],[31,0,31,3],[32,0,33,3]]}
["Criterion 3: Risk of misclassification bias"]
["Category","Checklist questions","Response option","Answer"]
["General (please answer)","3.1. Were the intervention or exposure (group) and the comparator (group) of interest clearly defined and the definitions are appropriate for measuring the effect of intervention or exposure (so that readers can understand, and the definitions allow the effect to be measured accurately and precisely enough)? (select ‘no’ or ‘seemingly no’ when classification is poorly defined and/or the choice of measure of exposure or intervention is unlikely to be accurate or precise enough, for example, when the use of an imprecise or inaccurate biomarker is defined as exposure)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
["General (please answer)","3.2. Were the intervention or exposure (group) and comparator (group) pre-defined (i.e. before initiation of treatment or analysis) and there were no alterations of the definitions afterwards? (e.g. select ‘yes’ or ‘seemingly yes’ when there is a study or analysis protocol defining the intervention or exposure as well as comparator, and there are no alterations of the definitions afterwards)\n\n(Assessors may accept slight alteration (e.g. minor alteration of definition may be considered as ‘no alteration’ if it is acceptable))","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
["General (please answer)","3.3. Might classification of exposure, intervention or comparator (group) have been incorrect due to influence of some knowledge, experience or desire? (e.g. intentional misclassification of exposure to yield a desired outcome; unintentional misclassification due to prior knowledge or cognitive bias)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)","3.4. What are the predicted magnitude and the direction of bias due to misclassification of intervention, exposure and/or comparator of interest? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)","Intervention or exposure intolerably favoured"]
[null,null,"Intervention or exposure tolerably favoured "]
[null,null,"Comparator intolerably favoured"]
[null,null,"Comparator tolerably favoured"]
[null,null,"Intolerably towards no effect"]
[null,null,"Tolerably towards no effect"]
[null,null,"Intolerably away from no effect"]
[null,null,"Tolerably away from no effect"]
[null,null,"Unpredictable"]
[null,null,"Skip"]
["Judgement about risk of bias for the criterion"]
["Please finalise your judgement about risk of bias for this criterion usign the algorithm.",null,"Low risk of bias"]
[null,null,"Medium risk of bias"]
[null,null,"High risk of bias"]
["Reason for deviationo from the suggested judgement about risk of bias"]
["Reason for deviation: NA"]

{"sheet":"Criterion4","index":4,"n_rows":49,"n_cols":4,"rows_kept":49,"merged":[[0,0,0,3],[2,0,7,0],[2,1,7,1],[2,3,7,3],[8,0,13,0],[8,1,13,1],[8,3,13,3],[14,0,19,0],[14,1,19,1],[14,3,19,3],[20,0,25,0],[20,1,25,1],[20,3,25,3],[26,0,31,0],[26,1,31,1],[26,3,31,3],[32,0,42,0],[32,1,42,1],[32,3,42,3],[43,0,43,3],[44,0,46,1],[44,3,46,3],[47,0,47,3],[48,0,49,3]]}
["Criterion 4: Risk of performance bias"]
["Category","Checklist questions","Response option","Answer"]
["Conditional (answer if experimental treatments are applied in the study, otherwise select ‘not applicable’)","4.1. Were any of the persons, who applied treatments (intervention, exposure, alternative intervention, alternative exposure, or control), aware of the hypothesis that was being tested or the comparison that was being made to measure impact or effectiveness?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
[null,null,"Not applicable"]
["Conditional (answer if experimental treatments are applied in the study, otherwise select ‘not applicable’)","4.2. Were there any alterations of intervention/exposure or control treatments of interest that might have an impact on the effectiveness of the intervention or the impact of the exposure? (e.g. deviated initiation, implementation and/or discontinuation)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
[null,null,"Not applicable"]
["Conditional (answer if Y/SY/Unclear to 4.2, otherwise select 'Not applicable')","4.3. Were these deviated treatments unbalanced between intervention or exposure groups (when comparing two interventions or exposures; i.e. intervention of interest or exposure vs. alternative one), or inaccurately taken into account (when comparing intervention or exposure vs. control (no intervention or exposure); e.g. nitrogen fertilizer was mistakenly applied more than initially planned for one group but this deviation is not reflected on the data collection sheet, i.e., not occurred as recorded), and thus it might have influenced the measure of impact or effectiveness?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
[null,null,"Not applicable"]
["Conditional (answer if experimental treatments are applied in the study, otherwise select ‘not applicable’)","4.4. Were both exposure/intervention and comparator treatments initiated and implemented as intended (or occurred as recorded) for all or nearly all subjects or areas?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
[null,null,"Not applicable"]
["Conditional (answer if Y/SY/Unclear to 4.2, or N/SN/Unclear to 4.4, otherwise select 'Not applicable')","4.5. Are the used analysis methods of the impact of the exposure or the effectiveness of the intervention appropriate in relation to bias due to altered treatment procedure of interest?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
[null,null,"Not applicable"]
["Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible. Select ‘not applicable’ if no experimental treatments are applied)","4.6. What are the predicted magnitude and the direction of bias due to altered treatment procedure of interest? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)","Intervention or exposure intolerably favoured"]
[null,null,"Intervention or exposure tolerably favoured "]
[null,null,"Comparator intolerably favoured"]
[null,null,"Comparator tolerably favoured"]
[null,null,"Intolerably towards no effect"]
[null,null,"Tolerably towards no effect"]
[null,null,"Intolerably away from no effect"]
[null,null,"Tolerably away from no effect"]
[null,null,"Unpredictable"]
[null,null,"Not applicable"]
[null,null,"Skip"]
["Judgement about risk of bias for the criterion"]
["Please finalise your judgement about risk of bias for this criterion usign the algorithm.",null,"Low risk of bias"]
[null,null,"Medium risk of bias"]
[null,null,"High risk of bias"]
["Reason for deviationo from the suggested judgement about risk of bias"]
["Reason for deviation: NA"]

{"sheet":"Criterion5","index":5,"n_rows":38,"n_cols":4,"rows_kept":38,"merged":[[0,0,0,3],[2,0,6,0],[2,1,6,1],[2,3,6,3],[7,0,11,0],[7,1,11,1],[7,3,11,3],[12,0,16,0],[12,1,16,1],[12,3,16,3],[17,0,21,0],[17,1,21,1],[17,3,21,3],[22,0,31,0],[22,1,31,1],[22,3,31,3],[32,0,32,3],[33,0,35,1],[33,3,35,3],[36,0,36,3],[37,0,38,3]]}
["Criterion 5: Risk of detection bias"]
["Category","Checklist questions","Response option","Answer"]
["General (please answer)","5.1. Was there any way for the outcome measure to be affected by knowledge of the exposure, intervention, subjects or areas, or desire for certain outcome (e.g. data collectors, who measured the outcome, were aware of the details of the study)?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["General (please answer)","5.2. Was/were the person(s), who assessed the effectiveness of the intervention or the impact of the exposure, aware of the exposure or intervention received by subjects or areas?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["General (please answer)","5.3. Were the methods for measuring and analysing the outcome data the same across the groups?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
["General (please answer)","5.4. By looking at the available materials for this study, was/were any bias(es) in outcome measurement that was/were related to (or correlated with) the assessed exposure or intervention?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)","5.5. What are the predicted magnitude and the direction of bias arising from systematic differences in measurement of outcomes of interest? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)","Intervention or exposure intolerably favoured"]
[null,null,"Intervention or exposure tolerably favoured "]
[null,null,"Comparator intolerably favoured"]
[null,null,"Comparator tolerably favoured"]
[null,null,"Intolerably towards no effect"]
[null,null,"Tolerably towards no effect"]
[null,null,"Intolerably away from no effect"]
[null,null,"Tolerably away from no effect"]
[null,null,"Unpredictable"]
[null,null,"Skip"]
["Judgement about risk of bias for the criterion"]
["Please finalise your judgement about risk of bias for this criterion usign the algorithm.",null,"Low risk of bias"]
[null,null,"Medium risk of bias"]
[null,null,"High risk of bias"]
["Reason for deviationo from the suggested judgement about risk of bias"]
["Reason for deviation: NA"]

{"sheet":"Criterion6","index":6,"n_rows":45,"n_cols":4,"rows_kept":45,"merged":[[0,0,0,3],[2,0,6,0],[2,1,6,1],[2,3,6,3],[7,0,11,0],[7,1,11,1],[7,3,11,3],[12,0,22,0],[12,1,22,1],[12,3,22,3],[23,0,28,0],[23,1,28,1],[23,3,28,3],[29,0,38,0],[29,1,38,1],[29,3,38,3],[39,0,39,3],[40,0,42,1],[40,3,42,3],[43,0,43,3],[44,0,45,3]]}
["Criterion 6: Risk of attrition bias"]
["Category","Checklist questions","Response option","Answer"]
["General (please answer)","6.1. For the analysis/analyses of impact or effectiveness, were relevant outcome data available for all the included subjects or areas (or for nearly entire samples that could be equivalent of all included subjects or areas)?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
["General (please answer)","6.2. Were any subjects or areas excluded after the start of the study because of missing some relevant data that was/were required for the analysis/analyses of the impact or effectiveness?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["Conditional (answer whichever suitable if N/SN/Unclear to 6.1, or Y/SY/Unclear to 6.2, otherwise select 'Not applicable')","6.3.a. Are the percentages of missing data the same, or almost the same, across the groups? (This applies when an attempt was not made to collect data of the entire, or nearly entire, inference population)\nOR\n6.3.b. Are subjects or areas still representative of the population of inference? (This applies when an attempt was made to collect data of the entire, or nearly entire, inference population)","Yes (6.3.a)"]
[null,null,"Seemingly yes (6.3.a)"]
[null,null,"Seemingly no (6.3.a)"]
[null,null,"No (6.3.a)"]
[null,null,"Unclear (No) (6.3.a)"]
[null,null,"Yes (6.3.b)"]
[null,null,"Seemingly yes (6.3.b)"]
[null,null,"Seemingly no (6.3.b)"]
[null,null,"No (6.3.b)"]
[null,null,"Unclear (No) (6.3.b)"]
[null,null,"Not applicable"]
["Conditional (answer if N/SN/Unclear to 6.1, or Y/SY/Unclear to 6.2, otherwise select 'Not applicable')","6.4. Is there any evidence that missing data has no or minimal (e.g. statistically non-significant) impact on the study findings (effectiveness of intervention or impact of exposure)?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (No)"]
[null,null,"Not applicable"]
["Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)","6.5. What are the predicted magnitude and the direction of bias due to systematic differences in missing data between intervention/exposure and comparator groups? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)","Intervention or exposure intolerably favoured"]
[null,null,"Intervention or exposure tolerably favoured "]
[null,null,"Comparator intolerably favoured"]
[null,null,"Comparator tolerably favoured"]
[null,null,"Intolerably towards no effect"]
[null,null,"Tolerably towards no effect"]
[null,null,"Intolerably away from no effect"]
[null,null,"Tolerably away from no effect"]
[null,null,"Unpredictable"]
[null,null,"Skip"]
["Judgement about risk of bias for the criterion"]
["Please finalise your judgement about risk of bias for this criterion usign the algorithm.",null,"Low risk of bias"]
[null,null,"Medium risk of bias"]
[null,null,"High risk of bias"]
["Reason for deviationo from the suggested judgement about risk of bias"]
["Reason for deviation: NA"]

{"sheet":"Criterion7","index":7,"n_rows":33,"n_cols":4,"rows_kept":33,"merged":[[0,0,0,3],[2,0,6,0],[2,1,6,1],[2,3,6,3],[7,0,11,0],[7,1,11,1],[7,3,11,3],[12,0,16,0],[12,1,16,1],[12,3,16,3],[17,0,26,0],[17,1,26,1],[17,3,26,3],[27,0,27,3],[28,0,30,1],[28,3,30,3],[31,0,31,3],[32,0,33,3]]}
["Criterion 7: Risk of reporting bias"]
["Category","Checklist questions","Response option","Answer"]
["General (please answer)","7.1. Are the reported relevant outcome data (or effect estimate) likely to be of (or based on) selected measurements of the outcome?","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["General (please answer)","7.2. Are relevant outcome data likely to be unreported for some subgroup(s)? (i.e. only outcome data on certain subjects or areas with certain characteristic(s) (e.g. taxonomic group) or in certain conditions (e.g. intervention intensity) are available)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["General (please answer)","7.3. Is/are the analysis/analyses of the causal relationship of interest (intervention-outcome or exposure-outcome) likely to be partially reported? (i.e. there is/are other relevant analysis/analyses of the causal relationship that is/are not reported)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)","7.4. What are the predicted magnitude and the direction of bias in reporting of study findings? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)","Intervention or exposure intolerably favoured"]
[null,null,"Intervention or exposure tolerably favoured "]
[null,null,"Comparator intolerably favoured"]
[null,null,"Comparator tolerably favoured"]
[null,null,"Intolerably towards no effect"]
[null,null,"Tolerably towards no effect"]
[null,null,"Intolerably away from no effect"]
[null,null,"Tolerably away from no effect"]
[null,null,"Unpredictable"]
[null,null,"Skip"]
["Judgement about risk of bias for the criterion"]
["Please finalise your judgement about risk of bias for this criterion usign the algorithm.",null,"Low risk of bias"]
[null,null,"Medium risk of bias"]
[null,null,"High risk of bias"]
["Reason for deviationo from the suggested judgement about risk of bias"]
["Reason for deviation: NA"]

{"sheet":"Criterion8","index":8,"n_rows":35,"n_cols":4,"rows_kept":35,"merged":[[0,0,0,3],[2,0,6,0],[2,1,6,1],[2,3,6,3],[7,0,12,0],[7,1,12,1],[7,3,12,3],[13,0,18,0],[13,1,18,1],[13,3,18,3],[19,0,28,0],[19,1,28,1],[19,3,28,3],[29,0,29,3],[30,0,32,1],[30,3,32,3],[33,0,33,3],[34,0,35,3]]}
["Criterion 8: Risk of analysis bias"]
["Category","Checklist questions","Response option","Answer"]
["General (please answer)","8.1. Is it likely that there is/are error(s) in the applied descriptive statistics? (e.g. miscalculations of sample sizes, means, medians, variances, ranges for intervention/exposure and comparator groups)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
["Conditional (answer if inferential statistics are applied, otherwise select ‘not applicable’)","8.2. Is it likely that there is/are error(s) in the applied inferential statistics (including null hypothesis testing, estimation, coding)? (e.g. null hypothesis and alternative hypothesis mislabelled, miscalculations of differences between intervention/exposure and comparator, errors in coding, etc.)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
[null,null,"Not applicable"]
["Conditional (answer if inferential statistics are applied, otherwise select ‘not applicable’)","8.3. Were assumptions for the applied inferential statistics violated or the applied inferential statistics methods inappropriate? (e.g. normality not assumed when conducting a parametric test, equal or unequal variances not tested when testing for a difference, no justification for the choice of dependent and independent variables, a Pearson’s correlation test was used when analysing a causal relationship, inappropriate comparison of multiple models to support the provided statement when some of the models do not relate to impact or effectiveness, inappropriate modelling which may affect an estimate of effectiveness or impact)","Yes"]
[null,null,"Seemingly yes"]
[null,null,"Seemingly no"]
[null,null,"No"]
[null,null,"Unclear (Yes)"]
[null,null,"Not applicable"]
["Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)","8.4. What are the predicted magnitude and the direction of bias due to error in applied statistical methods? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)","Intervention or exposure intolerably favoured"]
[null,null,"Intervention or exposure tolerably favoured "]
[null,null,"Comparator intolerably favoured"]
[null,null,"Comparator tolerably favoured"]
[null,null,"Intolerably towards no effect"]
[null,null,"Tolerably towards no effect"]
[null,null,"Intolerably away from no effect"]
[null,null,"Tolerably away from no effect"]
[null,null,"Unpredictable"]
[null,null,"Skip"]
["Judgement about risk of bias for the criterion"]
["Please finalise your judgement about risk of bias for this criterion usign the algorithm.",null,"Low risk of bias"]
[null,null,"Medium risk of bias"]
[null,null,"High risk of bias"]
["Reason for deviationo from the suggested judgement about risk of bias"]
["Reason for deviation: NA"]

{"sheet":"Overall","index":9,"n_rows":9,"n_cols":8,"rows_kept":9,"merged":[[0,0,0,7],[1,0,1,7],[4,0,4,7],[5,0,5,5],[6,1,6,5],[6,6,8,6],[6,7,8,7],[7,1,7,5],[8,1,8,5]]}
["Judgement about risk of bias for the seven individual criteria"]
["Once you have made judgements about risks of bias for all the criteria, the recorded judgements should appear below. If you did not record your judgements, the cells indicate '0', meaning you have not added any values in the corresponding sheets."]
["Criterion1","Criterion2","Criterion3","Criterion4","Criterion5","Criterion6","Criterion7","Criterion8"]
[{"$f":"Criterion1!D41","v":0},{"$f":"Criterion2!D63","v":0},{"$f":"Criterion3!D29","v":0},{"$f":"Criterion4!D45","v":0},{"$f":"Criterion5!D34","v":0},{"$f":"Criterion6!D41","v":0},{"$f":"Criterion7!D29","v":0},{"$f":"Criterion8!D31","v":0}]
["Overall judgement about risk of bias for this study result (estimate of effectiveness of intervention or impact of exposure)"]
["Overall risk of bias judgement",{"$nulls":5},"Reason for deviation","Overall judgement"]
["Overall low risk of bias","a study is considered to have low risk of bias for all risk-of-bias criteria for the findings ",{"$nulls":4},"Reason for deviation: NA"]
["Overall medium risk of bias","a study is considered to have medium risk of bias in at least one risk-of-bias criterion, but not to have high risk of bias for any risk-of-bias-criteria for the findings "]
["Overall high risk of bias","a study is considered to have high risk of bias in at least one risk-of-bias criterion for the findings"]

{"sheet":"All_decisions","index":10,"n_rows":4,"n_cols":61,"rows_kept":4,"merged":[[0,1,0,8],[0,9,0,18],[0,19,0,24],[0,25,0,32],[0,33,0,39],[0,40,0,46],[0,47,0,52],[0,53,0,58],[0,59,0,60]]}
["Study findings ID","Criterion 1",{"$nulls":7},"Criterion 2",{"$nulls":9},"Criterion 3",{"$nulls":5},"Criterion 4",{"$nulls":7},"Criterion 5",{"$nulls":6},"Criterion 6",{"$nulls":6},"Criterion 7",{"$nulls":5},"Criterion 8",{"$nulls":5},"Overall"]
[null,{"$f":"Criterion1!A3","v":"General (please answer)"},{"$f":"Criterion1!A7","v":"Conditional (answer if Y/SY to 1.1, otherwise select 'Not applicable')"},{"$f":"Criterion1!A13","v":"Conditional (answer if N/SN/Unclear to 1.2, otherwise select 'Not applicable')"},{"$f":"Criterion1!A18","v":"Conditional (answer if Y/SY to 1.2 or 1.3, otherwise select 'Not applicable')"},{"$f":"Criterion1!A24","v":"Conditional (answer if you have answered 1.4, otherwise select 'Not applicable')"},{"$f":"Criterion1!A30","v":"Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)"},"Please finalise your judgement about risk of bias for this criterion usign the algorithm.",{"$f":"Criterion1!A44","v":"Reason for deviationo from the suggested judgement about risk of bias"},{"$f":"Criterion2!A3","v":"General (please answer whichever suitable)"},{"$f":"Criterion2!A13","v":"General (please answer whichever suitable)"},{"$f":"Criterion2!A23","v":"General (please answer)"},{"$f":"Criterion2!A28","v":"Conditional (answer if N/SN/Unclear to 2.1, 2.2 or 2.3, otherwise select 'Not applicable')"},{"$f":"Criterion2!A34","v":"Conditional (answer if Y/SY/Unclear to 2.4, otherwise select 'Not applicable')"},{"$f":"Criterion2!A40","v":"Conditional (answer if Y/SY/Unclear to 2.5, otherwise select 'Not applicable')"},{"$f":"Criterion2!A46","v":"Conditional (answer if N/SN/Unclear to 2.1, 2.2 or 2.3, otherwise select 'Not applicable')"},{"$f":"Criterion2!A52","v":"Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)"},"Please finalise your judgement about risk of bias for this criterion usign the algorithm.",{"$f":"Criterion2!A66","v":"Reason for deviationo from the suggested judgement about risk of bias"},{"$f":"Criterion3!A3","v":"General (please answer)"},{"$f":"Criterion3!A8","v":"General (please answer)"},{"$f":"Criterion3!A13","v":"General (please answer)"},{"$f":"Criterion3!A18","v":"Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)"},"Please finalise your judgement about risk of bias for this criterion usign the algorithm.",{"$f":"Criterion3!A32","v":"Reason for deviationo from the suggested judgement about risk of bias"},{"$f":"Criterion4!A3","v":"Conditional (answer if experimental treatments are applied in the study, otherwise select ‘not applicable’)"},{"$f":"Criterion4!A9","v":"Conditional (answer if experimental treatments are applied in the study, otherwise select ‘not applicable’)"},{"$f":"Criterion4!A15","v":"Conditional (answer if Y/SY/Unclear to 4.2, otherwise select 'Not applicable')"},{"$f":"Criterion4!A21","v":"Conditional (answer if experimental treatments are applied in the study, otherwise select ‘not applicable’)"},{"$f":"Criterion4!A27","v":"Conditional (answer if Y/SY/Unclear to 4.2, or N/SN/Unclear to 4.4, otherwise select 'Not applicable')"},{"$f":"Criterion4!A33","v":"Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible. Select ‘not applicable’ if no experimental treatments are applied)"},"Please finalise your judgement about risk of bias for this criterion usign the algorithm.",{"$f":"Criterion4!A48","v":"Reason for deviationo from the suggested judgement about risk of bias"},{"$f":"Criterion5!A3","v":"General (please answer)"},{"$f":"Criterion5!A8","v":"General (please answer)"},{"$f":"Criterion5!A13","v":"General (please answer)"},{"$f":"Criterion5!A18","v":"General (please answer)"},{"$f":"Criterion5!A23","v":"Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)"},"Please finalise your judgement about risk of bias for this criterion usign the algorithm.",{"$f":"Criterion5!A37","v":"Reason for deviationo from the suggested judgement about risk of bias"},{"$f":"Criterion6!A3","v":"General (please answer)"},{"$f":"Criterion6!A8","v":"General (please answer)"},{"$f":"Criterion6!A13","v":"Conditional (answer whichever suitable if N/SN/Unclear to 6.1, or Y/SY/Unclear to 6.2, otherwise select 'Not applicable')"},{"$f":"Criterion6!A24","v":"Conditional (answer if N/SN/Unclear to 6.1, or Y/SY/Unclear to 6.2, otherwise select 'Not applicable')"},{"$f":"Criterion6!A30","v":"Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)"},"Please finalise your judgement about risk of bias for this criterion usign the algorithm.",{"$f":"Criterion6!A44","v":"Reason for deviationo from the suggested judgement about risk of bias"},{"$f":"Criterion7!A3","v":"General (please answer)"},{"$f":"Criterion7!A8","v":"General (please answer)"},{"$f":"Criterion7!A13","v":"General (please answer)"},{"$f":"Criterion7!A18","v":"Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)"},"Please finalise your judgement about risk of bias for this criterion usign the algorithm.",{"$f":"Criterion7!A32","v":"Reason for deviationo from the suggested judgement about risk of bias"},{"$f":"Criterion8!A3","v":"General (please answer)"},{"$f":"Criterion8!A8","v":"Conditional (answer if inferential statistics are applied, otherwise select ‘not applicable’)"},{"$f":"Criterion8!A14","v":"Conditional (answer if inferential statistics are applied, otherwise select ‘not applicable’)"},{"$f":"Criterion8!A20","v":"Optional (It is suggested that detailed rationale or empirical evidence be provided when predicting magnitude and direction of bias. Assessors may skip this optional checklist question if they feel unfeasible)"},"Please finalise your judgement about risk of bias for this criterion usign the algorithm.",{"$f":"Criterion8!A34","v":"Reason for deviationo from the suggested judgement about risk of bias"},"Please finalise your judgement about risk of bias for this study result usign the criteria.",{"$f":"Overall!G6","v":"Reason for deviation"}]
[null,{"$f":"Criterion1!B3","v":"1.1. Is it possible for the impact of the exposure or the effectiveness of the intervention to be confounded (or influenced by third variable) in this study?"},{"$f":"Criterion1!B7","v":"1.2. Did the author(s) control for all the important potential confounding factor(s) that are likely to be predictive of the outcome?"},{"$f":"Criterion1!B13","v":"1.3. Is there any justifiable reason for not controlling for all the important potential confounding factor(s) (so that omission of some of the important potential confounding factors is unlikely to influence the assessment of the effectiveness or impact)? (e.g. select ‘yes’ or ‘seemingly yes’ when there is evidence that omission of some of the important potential confounding factors does not affect the assessment of effectiveness or impact)"},{"$f":"Criterion1!B18","v":"1.4. Were the important potential confounding factor(s), that were controlled for, likely to be measured accurately and precisely enough in this study? (measurements of confounding factor(s) may be nominal (categorical), ordinal (ranks) or scale)"},{"$f":"Criterion1!B24","v":"1.5. Did the author(s) analyse the effect appropriately by taking into account the important potential confounding factors, as well as the issue of accuracy and precision of the measurements of the potential confounding factor(s)?"},{"$f":"Criterion1!B30","v":"1.6. What are the predicted magnitude and the direction of bias due to confounding? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result) (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)"},"Risk of bias for criterion 1",null,{"$f":"Criterion2!B3","v":"2.1.a. Was the selection of subjects or areas into the study (or analysis) random or systematic (i.e. random or systematic sampling) and the selection was not altered afterwards? (This applies when an attempt was not made to collect data of the entire, or nearly entire, inference population)\nOR\n2.1.b. Was the entire, or nearly entire, population of inference included in the study (or analysis), and the selection was not altered afterwards? (This applies when an attempt was made to collect data of the entire, or nearly entire, inference population)\n\n(Assessors may accept slight alteration (e.g. exclusion of subjects may be considered as ‘no alteration’ if it is acceptable))"},{"$f":"Criterion2!B13","v":"2.2.a. Was/were the researcher(s) unaware (or blinded) of the selection until subjects or areas were assigned to or grouped as intervention/exposure and comparator, and the selection was not altered afterwards? (when the study conducted the selection or grouping)\nOR\n2.2.b. Had the selection or grouping taken place before the conduct of the study (so that the researcher(s) were not involved in the selection or grouping at all) and the selection was not altered afterwards?\n\n(Assessors may accept slight alteration (e.g. exclusion of subjects may be considered as ‘no alteration’ if it is acceptable))"},{"$f":"Criterion2!B23","v":"2.3. Did the start of follow-up (when specific individuals or particular areas, such as plots, who had received intervention/exposure or control treatments were followed up) or the start of post-intervention or post-exposure selection of subjects or areas (when subjects or areas were selected after intervention or exposure; this might be the case if individuals or areas were not specifiable so that follow-up to the same individuals or exact areas was not applicable due to the nature of study) coincide for most subjects or areas? (If only one subject or area was studied, select 'yes' or 'seemingly yes')"},{"$f":"Criterion2!B28","v":"2.4. Was the follow-up or the selection of subjects or areas after intervention or exposure likely to be associated with any characteristics of the subjects or areas (that have different qualitative or quantitative values for different entities; e.g. pH, temperature, subspecies, age, sex, etc.)?"},{"$f":"Criterion2!B34","v":"2.5. Were the characteristics of the subjects or areas, that are likely to be associated with the follow-up or the selection of subjects or areas after intervention or exposure, also likely to be related to status of the intervention or exposure (or correlated with intervention or exposure variable(s); e.g. intervention intensity)?"},{"$f":"Criterion2!B40","v":"2.6. Were the characteristics of the subjects or areas, that are likely to be associated with the follow-up or the selection of subjects or areas after intervention or exposure, also likely to be predictive of the outcome?"},{"$f":"Criterion2!B46","v":"2.7. Did the author(s) estimate the potential bias(es) (systematic error(s)) due to haphazard or altered selection, or unequal timing of selection of subjects or areas into the study, and found that the bias(es) was/were minimal? Alternatively, is there evidence that the effect is measured accurately enough in relation to bias arising from selection of subjects or areas into the study or analysis after intervention or exposure?"},{"$f":"Criterion2!B52","v":"2.8. What are the predicted magnitude and the direction of bias arising from systematic differences in the selection of subjects or areas into the study or analysis after intervention or exposure? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)"},"Risk of bias for criterion 2",null,{"$f":"Criterion3!B3","v":"3.1. Were the intervention or exposure (group) and the comparator (group) of interest clearly defined and the definitions are appropriate for measuring the effect of intervention or exposure (so that readers can understand, and the definitions allow the effect to be measured accurately and precisely enough)? (select ‘no’ or ‘seemingly no’ when classification is poorly defined and/or the choice of measure of exposure or intervention is unlikely to be accurate or precise enough, for example, when the use of an imprecise or inaccurate biomarker is defined as exposure)"},{"$f":"Criterion3!B8","v":"3.2. Were the intervention or exposure (group) and comparator (group) pre-defined (i.e. before initiation of treatment or analysis) and there were no alterations of the definitions afterwards? (e.g. select ‘yes’ or ‘seemingly yes’ when there is a study or analysis protocol defining the intervention or exposure as well as comparator, and there are no alterations of the definitions afterwards)\n\n(Assessors may accept slight alteration (e.g. minor alteration of definition may be considered as ‘no alteration’ if it is acceptable))"},{"$f":"Criterion3!B13","v":"3.3. Might classification of exposure, intervention or comparator (group) have been incorrect due to influence of some knowledge, experience or desire? (e.g. intentional misclassification of exposure to yield a desired outcome; unintentional misclassification due to prior knowledge or cognitive bias)"},{"$f":"Criterion3!B18","v":"3.4. What are the predicted magnitude and the direction of bias due to misclassification of intervention, exposure and/or comparator of interest? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)"},"Risk of bias for criterion 3",null,{"$f":"Criterion4!B3","v":"4.1. Were any of the persons, who applied treatments (intervention, exposure, alternative intervention, alternative exposure, or control), aware of the hypothesis that was being tested or the comparison that was being made to measure impact or effectiveness?"},{"$f":"Criterion4!B9","v":"4.2. Were there any alterations of intervention/exposure or control treatments of interest that might have an impact on the effectiveness of the intervention or the impact of the exposure? (e.g. deviated initiation, implementation and/or discontinuation)"},{"$f":"Criterion4!B15","v":"4.3. Were these deviated treatments unbalanced between intervention or exposure groups (when comparing two interventions or exposures; i.e. intervention of interest or exposure vs. alternative one), or inaccurately taken into account (when comparing intervention or exposure vs. control (no intervention or exposure); e.g. nitrogen fertilizer was mistakenly applied more than initially planned for one group but this deviation is not reflected on the data collection sheet, i.e., not occurred as recorded), and thus it might have influenced the measure of impact or effectiveness?"},{"$f":"Criterion4!B21","v":"4.4. Were both exposure/intervention and comparator treatments initiated and implemented as intended (or occurred as recorded) for all or nearly all subjects or areas?"},{"$f":"Criterion4!B27","v":"4.5. Are the used analysis methods of the impact of the exposure or the effectiveness of the intervention appropriate in relation to bias due to altered treatment procedure of interest?"},{"$f":"Criterion4!B33","v":"4.6. What are the predicted magnitude and the direction of bias due to altered treatment procedure of interest? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)"},"Risk of bias for criterion 4",null,{"$f":"Criterion5!B3","v":"5.1. Was there any way for the outcome measure to be affected by knowledge of the exposure, intervention, subjects or areas, or desire for certain outcome (e.g. data collectors, who measured the outcome, were aware of the details of the study)?"},{"$f":"Criterion5!B8","v":"5.2. Was/were the person(s), who assessed the effectiveness of the intervention or the impact of the exposure, aware of the exposure or intervention received by subjects or areas?"},{"$f":"Criterion5!B13","v":"5.3. Were the methods for measuring and analysing the outcome data the same across the groups?"},{"$f":"Criterion5!B18","v":"5.4. By looking at the available materials for this study, was/were any bias(es) in outcome measurement that was/were related to (or correlated with) the assessed exposure or intervention?"},{"$f":"Criterion5!B23","v":"5.5. What are the predicted magnitude and the direction of bias arising from systematic differences in measurement of outcomes of interest? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)"},"Risk of bias for criterion 5",null,{"$f":"Criterion6!B3","v":"6.1. For the analysis/analyses of impact or effectiveness, were relevant outcome data available for all the included subjects or areas (or for nearly entire samples that could be equivalent of all included subjects or areas)?"},{"$f":"Criterion6!B8","v":"6.2. Were any subjects or areas excluded after the start of the study because of missing some relevant data that was/were required for the analysis/analyses of the impact or effectiveness?"},{"$f":"Criterion6!B13","v":"6.3.a. Are the percentages of missing data the same, or almost the same, across the groups? (This applies when an attempt was not made to collect data of the entire, or nearly entire, inference population)\nOR\n6.3.b. Are subjects or areas still representative of the population of inference? (This applies when an attempt was made to collect data of the entire, or nearly entire, inference population)"},{"$f":"Criterion6!B24","v":"6.4. Is there any evidence that missing data has no or minimal (e.g. statistically non-significant) impact on the study findings (effectiveness of intervention or impact of exposure)?"},{"$f":"Criterion6!B30","v":"6.5. What are the predicted magnitude and the direction of bias due to systematic differences in missing data between intervention/exposure and comparator groups? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)"},"Risk of bias for criterion 6",null,{"$f":"Criterion7!B3","v":"7.1. Are the reported relevant outcome data (or effect estimate) likely to be of (or based on) selected measurements of the outcome?"},{"$f":"Criterion7!B8","v":"7.2. Are relevant outcome data likely to be unreported for some subgroup(s)? (i.e. only outcome data on certain subjects or areas with certain characteristic(s) (e.g. taxonomic group) or in certain conditions (e.g. intervention intensity) are available)"},{"$f":"Criterion7!B13","v":"7.3. Is/are the analysis/analyses of the causal relationship of interest (intervention-outcome or exposure-outcome) likely to be partially reported? (i.e. there is/are other relevant analysis/analyses of the causal relationship that is/are not reported)"},{"$f":"Criterion7!B18","v":"7.4. What are the predicted magnitude and the direction of bias in reporting of study findings? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)"},"Risk of bias for criterion 7",null,{"$f":"Criterion8!B3","v":"8.1. Is it likely that there is/are error(s) in the applied descriptive statistics? (e.g. miscalculations of sample sizes, means, medians, variances, ranges for intervention/exposure and comparator groups)"},{"$f":"Criterion8!B8","v":"8.2. Is it likely that there is/are error(s) in the applied inferential statistics (including null hypothesis testing, estimation, coding)? (e.g. null hypothesis and alternative hypothesis mislabelled, miscalculations of differences between intervention/exposure and comparator, errors in coding, etc.)"},{"$f":"Criterion8!B14","v":"8.3. Were assumptions for the applied inferential statistics violated or the applied inferential statistics methods inappropriate? (e.g. normality not assumed when conducting a parametric test, equal or unequal variances not tested when testing for a difference, no justification for the choice of dependent and independent variables, a Pearson’s correlation test was used when analysing a causal relationship, inappropriate comparison of multiple models to support the provided statement when some of the models do not relate to impact or effectiveness, inappropriate modelling which may affect an estimate of effectiveness or impact)"},{"$f":"Criterion8!B20","v":"8.4. What are the predicted magnitude and the direction of bias due to error in applied statistical methods? (Note: quantitative assessment (e.g. through simulation) may be conducted to predict the magnitude and direction of bias for this study result)\n\n(Intolerable means that the study result should not be considered as valid enough in relation to the predicted magnitude of bias. Tolerable means that the study result could be considered as valid enough in relation to the predicted magnitude of bias.)"},"Risk of bias for criterion 8",null,"Overall judgement about risk of bias for this study result"]
["e.g. 1",{"$f":"Criterion1!D3","v":0},{"$f":"Criterion1!D7","v":0},{"$f":"Criterion1!D13","v":0},{"$f":"Criterion1!D18","v":0},{"$f":"Criterion1!D24","v":0},{"$f":"Criterion1!D30","v":0},{"$f":"Criterion1!D41","v":0},{"$f":"Criterion1!A45","v":"Reason for deviation: NA"},{"$f":"Criterion2!D3","v":0},{"$f":"Criterion2!D13","v":0},{"$f":"Criterion2!D23","v":0},{"$f":"Criterion2!D28","v":0},{"$f":"Criterion2!D34","v":0},{"$f":"Criterion2!D40","v":0},{"$f":"Criterion2!D46","v":0},{"$f":"Criterion2!D52","v":0},{"$f":"Criterion2!D63","v":0},{"$f":"Criterion2!A67","v":"Reason for deviation: NA"},{"$f":"Criterion3!D3","v":0},{"$f":"Criterion3!D8","v":0},{"$f":"Criterion3!D13","v":0},{"$f":"Criterion3!D18","v":0},{"$f":"Criterion3!D29","v":0},{"$f":"Criterion3!A33","v":"Reason for deviation: NA"},{"$f":"Criterion4!D3","v":0},{"$f":"Criterion4!D9","v":0},{"$f":"Criterion4!D15","v":0},{"$f":"Criterion4!D21","v":0},{"$f":"Criterion4!D27","v":0},{"$f":"Criterion4!D33","v":0},{"$f":"Criterion4!D45","v":0},{"$f":"Criterion4!A49","v":"Reason for deviation: NA"},{"$f":"Criterion5!D3","v":0},{"$f":"Criterion5!D8","v":0},{"$f":"Criterion5!D13","v":0},{"$f":"Criterion5!D18","v":0},{"$f":"Criterion5!D23","v":0},{"$f":"Criterion5!D34","v":0},{"$f":"Criterion5!A38","v":"Reason for deviation: NA"},{"$f":"Criterion6!D3","v":0},{"$f":"Criterion6!D8","v":0},{"$f":"Criterion6!D13","v":0},{"$f":"Criterion6!D24","v":0},{"$f":"Criterion6!D30","v":0},{"$f":"Criterion6!D41","v":0},{"$f":"Criterion6!A45","v":"Reason for deviation: NA"},{"$f":"Criterion7!D3","v":0},{"$f":"Criterion7!D8","v":0},{"$f":"Criterion7!D13","v":0},{"$f":"Criterion7!D18","v":0},{"$f":"Criterion7!D29","v":0},{"$f":"Criterion7!A33","v":"Reason for deviation: NA"},{"$f":"Criterion8!D3","v":0},{"$f":"Criterion8!D8","v":0},{"$f":"Criterion8!D14","v":0},{"$f":"Criterion8!D20","v":0},{"$f":"Criterion8!D31","v":0},{"$f":"Criterion8!A35","v":"Reason for deviation: NA"},{"$f":"Overall!H7","v":0},{"$f":"Overall!G7","v":"Reason for deviation: NA"}]