Ensuring quality of inputs to the ABDS
As part of the ABDS 2011, a quality framework was developed to report on estimates produced as part of the study. Several steps were taken to ensure the accuracy and relevance of the estimates in the ABDS:
- All standard inputs (such as the reference life table, disability weights and relative risks) were reviewed and assessed as appropriate by the Australian Burden of Disease Expert Advisory Group and Indigenous Reference Group for relevance and applicability in the Australian and Indigenous contexts.
- All data used in the ABDS were required to meet strict inclusion criteria via protocols endorsed by the Australian Burden of Disease Expert Advisory Group and Indigenous Reference Group.
- All models and inputs used in YLL, YLD and risk factor estimates were reviewed by clinical and other relevant experts to ensure their appropriateness for Australian and Indigenous populations.
- Where there were competing methods or data sources, sensitivity analyses were undertaken to compare the impact of the different choices. Final decisions were made in consultation with the Expert Advisory Group and Indigenous Reference Group.
The AIHW considered the two most commonly used measures of reliability—uncertainty analysis and scenario testing (see ABDS quality index for more details). However, it became clear from the case-study assessments that the amount of work required to develop a reasonable and defensible method of uncertainty estimation that could be used across all parts of the ABDS was not within the resources of the project.
In addition, the assessments confirmed that the amount of error that could be encapsulated within an uncertainty interval will generally be only a (possibly minor) part of the total error or uncertainty attached to an estimate. Ignoring or concealing the error that might arise from epidemiological or methodological choices could mislead users into placing unjustified reliance on patterns and differences that they see in estimates of burden.