Appendix A: Data quality information
This appendix provides information on the National Elective Surgery Waiting Times Data Collection (NESWTDC), including a Data Quality Statement summary relevant to interpreting the NESWTDC. It also contains further information on variation in hospital reporting that may affect the interpretation of the data presented in this report.
National Elective Surgery Waiting Times Data Collection
The AIHW has undertaken the collection and reporting of the data in this report under the auspices of the Australian Health Ministers’ Advisory Council, through the National Health Information Agreement.
The data supplied by state and territory health authorities were used by the AIHW to assemble the National Elective Surgery Waiting Times Data Collection (NESWTDC), covering waiting times and other characteristics of elective surgery in all public hospitals.
The NESWTDC covers most public hospitals that undertake elective surgery. Hospitals that were not included may not undertake elective surgery, may not have had waiting lists, or may have had different waiting list characteristics from those of reporting hospitals. Some smaller remote hospitals may have different patterns of service delivery to those of other hospitals because specialists providing elective surgery services visit these hospitals only periodically.
Prior to 2016–17, the elective surgery waiting list data were reported to two separate national minimum data sets (NMDSs), which are available on the AIHW’s Metadata Online Register (METEOR):
- Elective surgery waiting times (census data) NMDS (METEOR identifier 600059)
- Elective surgery waiting times (removals data) NMDS (METEOR identifier 600056)
From 1 July 2016, the Elective surgery waiting times NMDS comprises both removals and census data—that is, patients on, or removed from, elective surgery waiting lists (see METEOR identifier 623795).
Detailed information about the AIHW’s NESWTDC is in the Data Quality Statement. The Data Quality Statement is summarised below.
Data quality summary for National Elective Surgery Waiting Times Data Collection 2017–18
The NESWTDC provides episode-level data on patients added to or removed from elective surgery waiting lists managed by public hospitals. This includes private patients treated in public hospitals, and may include public patients treated in private hospitals. ‘Public hospitals’ may include hospitals that are set up to provide services for public patients (as public hospitals do), but are managed privately. Removals are counted for patients who have been removed for admission, or for another reason.
The data supplied for 1 July 2017 to 30 June 2018 are based on the ESWT NMDS for 2017–18.
The NESWTDC includes data for each year from 1999–00 to 2017–18.
Summary of coverage and key data quality issues
How has data coverage changed over time?
For the purposes of this report, the coverage of the NESWTDC is estimated by comparing admissions for elective surgery reported to the NESWTDC with elective surgical separations reported to the NHMD, expressed as a percentage. For more information on elective surgical separations and the estimate of coverage, see Appendix B.
As 2017–18 NHMD data are not yet available, the estimates of the coverage are preliminary, based on 2016–17 NHMD data. For 2017–18, the preliminary estimate of public hospital elective surgery covered by the NESWTDC was 95%. The estimated coverage was 100% in New South Wales, Queensland, Western Australia, Tasmania, the Australian Capital Territory and the Northern Territory. For Victoria and South Australia, the majority of public hospital elective surgery was covered by the NESWTDC (85% and 97%, respectively) (see Table A1, available to download in the data section). These estimates will be updated when the total number of elective surgery separations for public hospitals is available in the NHMD, early in 2019.
Between 2013–14 and 2017–18, the coverage of the NESWTDC fluctuated between 92% and 95% (excluding data for the Australian Capital Territory from the numerator for 2015–16). Coverage was highest for Principal referral and women’s and children’s hospitals and for Public acute group A and Public acute group B hospitals (see Table A2, available to download in the data section).
For 2017–18, the NESWTDC covered most hospitals that undertook elective surgery. Hospitals that were not included may not undertake elective surgery, may not have had waiting lists, or may have had different waiting list characteristics compared with other hospitals.
After adjusting for the changes in coverage for Victoria and Queensland, additions to elective surgery waiting lists were estimated to have increased by 2.2% on average each year and removals were estimated to have increased by 1.6% on average each year.
Changes in the number of hospitals reporting
Changes in the number of hospitals reporting
Between 2013–14 and 2017–18, the number of public hospitals that reported admissions from elective surgery waiting lists increased from 244 to 261 nationally, and there were changes in the number of hospitals that reported admissions for some jurisdictions.
A change in the number of hospitals reporting admissions over time does not necessarily represent a change in coverage of elective surgery data reported. For example, data provided by two separate hospitals for one period, may be combined and provided by a single hospital the following year. Changes that made a material difference to the coverage of elective surgery reported over time, are outlined in Table A4, available to download in the data section.
Between 2016–17 and 2017-18, the number of hospitals that reported admissions from elective surgery waiting lists changed for three jurisdictions, however this increase did not constitute a change in coverage between these time periods.
In addition, the number of hospitals reported here may underestimate the number of hospitals with elective surgery waiting lists, because the coverage of the data collection is incomplete. See ‘How has data coverage changed over time?’ above for more information.
New South Wales
- Bulli Hospital, Temora Hospital and Pambula District Hospital ceased providing elective surgery from 2016–17.
- Combined, these hospitals reported about 400 admissions from elective surgery waiting lists in 2015–16. This does not constitute a change in coverage.
- The Wodonga and Warnambool hospitals reported elective surgery data for the first time in 2016–17. This constituted a change in coverage
- The Women’s Hospital at Sandringham began reporting elective surgery data separately in 2015–16, whereas in previous years this data was reported with elective surgery data for the Royal Women’s Hospital. This change in organisational arrangements did not represent an increase in coverage over this period.
- The Sunshine Coast University Hospital opened in March 2017. This did not constitute a change in coverage, as elective surgery services were previously provided by a number of smaller hospitals in the region, which reported data for the NESWTDC
- For 2015–16, data for an additional 18 smaller hospitals were reported for the first time. This constituted a change in coverage as the activity was previously not reported for the NESWTDC
- The Lady Cilento Children’s Hospital opened in November 2014, replacing the Royal Children’s Hospital and the Mater Children’s Hospital. The Lady Cilento Children’s Hospital and the Royal Children’s Hospital are both included in the 2014–15 data. This did not constitute a change in coverage
- For 2014–15, data were not provided for 2 hospitals (which reported about 9,300 admissions (combined) from elective surgery waiting lists in 2015–16) and 5 months of data for a third hospital (which closed in late 2014 and had reported about 3,700 admissions in 2013–14). The 3 hospitals comprised 2 Principal referral and women’s and children’s hospitals and 1 Public acute group A hospital. These periods of missing data constituted changes in coverage.
- In June 2018, Princess Margaret Hospital closed and Perth Children’s Hospital opened, both hospitals were reported for 2017–18 , this did not constitute a change in coverage.
- Data was reported for Kalamunda hospital for the first time in 2017–18.
- The Fiona Stanley Hospital opened in October 2014, replacing the Royal Perth Hospital (Shenton Park campus) and Kaleeya Hospital. All 3 of these hospitals are included for 2014–15 in tables 2.1 and 2.2.
- In November 2015, the St John of God, Midland Public Hospital opened, replacing the Swan District Hospital. Both hospitals are included for 2015–16 in tables 2.1 and 2.2.
- In 2017–18, Southern Yorke Penninsula Health service elective surgery data was reported as part of Northern Yorke Penninsula health service. This did not constitute a change in coverage.
- 3 small hospitals ceased providing elective surgery between 2012–13 and 2015–16. This did not represent a change in coverage.
Australian Capital Territory
- For 2015–16, Australian Capital Territory data were not available at the time of publication.
Summary of key data quality issues
- Although there are national standards for data on elective surgery waiting times, methods to calculate waiting times have varied between states and territories and over time. For example, in Victoria, Queensland and Tasmania, for patients who were transferred from a waiting list managed by one hospital to that managed by another, the time waited on the first list is not included in the waiting time reported to the NESWTDC from the second hospital. Therefore, the number of days waited in those jurisdictions reflected the waiting time on the list managed by the reporting hospital only. For New South Wales, the waiting time reflects the complete waiting period from the original listing date to admission for surgery at the second hospital.
- In 2016–17 and 2017–18, the Northern Territory did not report the number of patients who were Transferred to another hospital’s waiting list.
- There is an apparent lack of comparability of the assignment of clinical urgency categories among jurisdictions, which may result in statistics that are not meaningful or comparable between jurisdictions.
- The quality of the data reported for Indigenous status for the NESWTDC has not been formally assessed; therefore, caution should be exercised when interpreting these data.
- A small number of intended procedures may be undertaken as non-admitted patient care, for example, cataract extraction in New South Wales. Therefore, any waiting times associated with this non-admitted activity will not be captured in the NESWTDC.
Additional information on data quality
Clinical urgency categorisation
As for earlier years, there is apparent variation in the assignment of urgency categories among states and territories for 2017–18.
This apparent lack of comparability of clinical urgency categories among jurisdictions means that measures based on these categories are also not comparable between jurisdictions. The concepts of the proportion ‘overdue’ and the ‘average overdue wait time’ may also not be meaningful or comparable because they depend on the urgency categorisation.
Despite the differences in how clinicians assign clinical urgency categories, interpreting state and territory waiting times statistics could be assisted by having context information about the proportion of patients in each urgency category.
For example, a state or territory could report relatively long 50th percentile waiting times in association with a relatively high proportion of patients assessed by clinicians in the state (or territory) as being in Category 3 (procedure clinically indicated within 365 days). Conversely, a state or territory in which a relatively high proportion of patients are assessed by clinicians as being in Category 1 or 2 (procedure clinically indicated within 30 days and 90 days, respectively) could have relatively short overall 50th percentile waiting times.
With the aim of promoting more nationally consistent and comparable elective surgery urgency categorisation, the AIHW worked with the Royal Australasian College of Surgeons (RACS) to develop national definitions for elective surgery urgency categories, including ‘not ready for care’.
The AIHW and the RACS reviewed the existing practices across Australia and reported the findings of their review and recommendations for action in the report National definitions for elective surgery urgency categories (AIHW 2013b) which was presented to the Standing Council on Health in late 2012.
The Australian Health Ministers’ Advisory Council was asked to progress the implementation of the report’s recommendations:
- Adopt a statement of an overarching principle for urgency category assignment.
- Adopt simplified, time-based definitions of urgency categories.
- A listing of usual urgency categories for higher volume procedures, to be developed by surgical specialty groups.
- Establish a national process to provide information on comparative urgency categorisation between states and territories.
- Adopt ‘treat in turn’ as a principle for elective surgery management.
- Clarified approaches for patients who are not ready for surgery because of clinical or personal reasons.
As a result of this work, revised definitions for clinical urgency categories and for the glossary items elective surgery, emergency surgery and other surgery were developed and were implemented in the ESWT NMDSs from 1 July 2015. In addition, the data element Intended procedure and the revised data element for Surgical specialty were implemented on 1 July 2016. Guidelines on the assignment of the ‘usual’ clinical urgency category for each intended procedure were released in 2015 (AHMAC 2015). With these changes, it is expected that the comparability of the urgency categorisation data will improve over coming years.
Before 2016–17, information about the specialty of the surgeon who was to perform the procedure was collected using the data element Elective surgery waiting list episode—surgical specialty (of scheduled doctor) (METEOR identifier 270146). It included 10 specific surgical specialties, and an ‘other’ category.
From 1 July 2016, the surgical specialty data element was revised to include Paediatric surgery, and some surgical specialties were relabelled (METEOR identifier 605195). The revised surgical speciality data element now contains 11 specific surgical specialties, and an ‘other’ category.
In previous years, records for which the surgical specialty may have been Paediatric surgery would have been allocated to another surgical specialty or as ‘Other’ (surgical specialty other than one of the 10 specified specialties). Therefore, the data for 2016–17 and 2017–18 are not comparable with data presented for earlier years.
Use of the Paediatric surgery category varied among jurisdictions.
- Paediatric surgery was reported by Western Australia, South Australia, Tasmania and the Australian Capital Territory.
- Paediatric surgery was reported by New South Wales, Western Australia, South Australia, Tasmania and the Australian Capital Territory.
The data by surgical specialty for jurisdictions that did report Paediatric surgery are not comparable with the data provided by jurisdictions that did not report Paediatric surgery.
Intended surgical procedures
Between 2015–16 and 2016–17, the data element Indicator procedure was replaced by Intended procedure in the ESWT NMDS. The Intended procedure (intended surgical procedure) data element (METEOR identifier 637500) contains 152 categories of surgical procedures, and includes the 15 procedures that were previously reported for the Indicator procedure data element (METEOR identifier 514033).
The following Intended procedures are considered equivalent to the corresponding Indicator procedures:
- Cataract extraction (with or without intra-ocular lens insertion)
- Cholecystectomy (open/laparoscopic)
- Coronary artery bypass grafting
- Hysterectomy (abdominal/vaginal/laparoscopic)
- Tonsillectomy (with/without adenoidectomy).
In addition, Myringotomy (without insertion of grommets) and Pressure equalising tubes—insertion of, combined, are considered to be equivalent to the indicator procedure Myringotomy.
There are some minor differences between the following Intended procedures and the corresponding Indicator procedures:
- Inguinal herniotomy/herniorrhaphy
- Total hip replacement
- Total knee replacement
- Varicose veins treatment.
The previous list of 15 Indicator procedures represented high-volume procedures that were potentially associated with longer waiting times. These are presented in this report, in Table 3.4 along with the 10 most commonly reported intended surgical procedures (that were not in the previous set of indicator procedures).
From 2016–17, 2 separate Intended procedures—Myringotomy and Pressure equalising tubes (grommets) - insertion of—are regarded as equivalent to the Myringotomy indicator procedure. The increase in admissions for Myringotomy between 2015–16 and 2016–17 reflects, in part, the inclusion of New South Wales admissions for Pressure equalising tubes (grommets) - insertion of, that had not previously been reported for NSW under the indicator procedure Myringotomy.
There was some variation in the reporting of intended procedures among jurisdictions, which may indicate that the data element was not completely implemented, or that there are differences among jurisdictions in the types of procedures that are managed through elective surgery waiting lists.
For 2017–18, the majority of states and territories provided some patient counts for most of the 152 intended procedures. For Tasmania, 97 of the 152 intended procedures were not reported and for New South Wales, 30 of the 152 intended procedures were not reported.
The Intended procedure data element includes an ‘Other’ category for procedures other than the 152 individual procedures. In 2017–18, nationally, 26.6% of intended procedures were categorised as ‘Other’. The proportion of admissions from public hospital elective surgery waiting lists where the intended procedure was reported as ‘Other’ ranged from 24.1% in Victoria to 48.1% in Tasmania.
Therefore, the data by intended procedure for jurisdictions that did not report against all intended procedure categories may not be comparable with the data provided by other jurisdictions.
For time series, the 15 indicator procedures are presented, including the 2016–17 and 2017–18 data based on the Intended procedure data element. There is also an ‘other’ category which contains data for procedures not included in the 15 indicator procedure categories.
For the 2017–18 data, a longer list of 25 intended surgical procedures is presented. This includes the 15 ‘indicator procedures’ and the 10 most common intended surgical procedures that were not ‘indicator procedures’.
Quality of Indigenous status data
The quality of Indigenous status information in the data provided for the NESWTDC has not been formally assessed. Therefore, the information presented for Indigenous status for elective surgery waiting times in Chapter 4 should be used with caution.
The AIHW report Indigenous identification in hospital separations data: quality report (AIHW 2013a) found that, nationally, about 88% of Indigenous Australians were identified correctly in hospital admissions data in the 2011–12 study period, and the ‘true’ number of separations for Indigenous Australians was about 9% higher than reported. This under-identification could similarly affect the NESWTDC data.
The following information has been supplied by the states and territories to provide some insight into the quality of Indigenous status data in the NESWTDC.
New South Wales
The New South Wales Ministry of Health advised that Indigenous status has been collected for elective surgery waiting times data from 2010–11.
The Victorian Department of Health reports that Indigenous status data is of acceptable quality, with valid information recorded for more than 98% of patients admitted and/or removed from elective surgery waiting lists. However, the number of identified Aboriginal and Torres Strait Islander patients is likely to be more accurate within the admitted patient care data, compared with the waiting list data.
Available evidence suggests that the number of Indigenous patients is understated in Queensland hospital data due to both non-reporting and misreporting of Indigenous status. Despite this, Queensland Health regards the Indigenous status data used in this report to be of an appropriate quality for publication.
The Western Australian Department of Health regards its Indigenous status data for elective surgery waiting times as being of good quality. Quality improvement activities, including cross-referencing across patient administration systems, continue to enhance the accuracy of this data element.
The South Australian Department for Health and Ageing reports that the quality of Indigenous status data in its elective surgery waiting times collection has improved over recent years and is of sufficient quality to be appropriate for publication.
The Tasmanian Department of Health and Human Services reports that the quality and level of Indigenous status identification, across public hospital information collections, are of a high standard. However, as with all data collections, there is continued work on maintaining and improving the collection of this data element, where needed.
Australian Capital Territory
The Australian Capital Territory Health Directorate advised that the quality of its Indigenous status data for elective surgery waiting times is of sufficient quality to be appropriate for publication.
The Northern Territory Department of Health considers the quality of its Indigenous status data for elective surgery waiting times patients to be good, with accuracy at over 90%. The department retains historical reporting of Indigenous status. All management and statistical reporting, however, is based on a person’s most recently reported Indigenous status.
What are the limitations of the data?
Overall, the quality of the data in the NESWTDC is sufficient for them to be published in this report. However, the limitations of the data should be taken into consideration when they are interpreted.
States and territories are primarily responsible for the quality of the data they provide. However, the AIHW undertakes extensive validations on receipt of data, checking for valid values, logical consistency and historical consistency. Where possible, data in individual data sets are checked against data from other data sets. Potential errors are queried with jurisdictions, and corrections and resubmissions may be made in response to these queries. Except as noted, the AIHW does not adjust data to account for possible data errors or missing or incorrect values.
Comparisons between states and territories and reporting years should be made with reference to the accompanying notes in the chapters and in Appendix A.
Caution should be used when interpreting the data presented in this report, as they have not been confirmed against the data on elective surgery in the National Hospital Morbidity Database (NHMD) because those data are not yet available. The NHMD includes information on patient characteristics and on the procedures performed, which can be used to check the data in the NESWTDC. These data will be reported in early 2019.
AHMAC (Australian Health Ministers’ Advisory Council) 2015. National Elective Surgery Urgency Categorisation Guideline April 2015.
AIHW 2013a. Indigenous identification in hospital separations data: quality report. Cat. no. IHW 90. Canberra: AIHW.
AIHW 2013b. National definitions for elective surgery urgency categories: proposal for the Standing Council on Health. Cat. no. HSE 138. Canberra: AIHW.
AIHW 2018. Australia’s hospitals 2016–17: at a glance. Health services series no. 85. Cat. no. HSE 204. Canberra: AIHW.