Welcome to Module 3 - One21seventy clinical audits, systems assessments and reports.

You’ll need to complete this module before logging in to do modules about interpreting and using audit report data.

I’ll be describing the One21seventy clinical CQI tools and resources available to services registered with one21seventy, and explaining the rationale for collecting and reporting on different types of data as part of the CQI process.

I’ll also explain how demographic, audit and systems assessment data are presented in One21seventy clinical audit reports.

We’ll look at the participatory process used by health teams to interpret report data and identify priorities and strategies for improvement in the delivery of care.

One21seventy data collection tools are used in step three of the One21seventy cycle.

The tools include the Health Centre and Community Survey, clinical audit tools and protocols, and the Systems Assessment Tool.

Non-clinical audit tools are also available, however this module focuses on clinical auditing.

The Health Centre and Community Survey is used to collect information about the health service, including governance arrangements, accreditation status, staffing, regular visiting staff, location, outlying clinics, Indigenous population and number of clients.

A new survey is done to update information at the start of each annual audit cycle, and survey data must be entered into the One21seventy database before audit reports can be generated.

This data entry is usually done by CQI facilitators or coordinators using a health centre’s login and password.

The Health Centre and Community Survey report is generated separately to a clinical audit report. Like other reports, it tracks changes over time when updated each year.
The report can be taken into account when interpreting audit and systems assessment data and when benchmarking results against other services.

It can help the team consider factors that contribute to results, and ways to overcome or manage barriers to providing high quality care.

The team can use survey information to consider how available resources might be used to achieve change.

A clinical audit is a systematic review of the information recorded in clinical client records about a particular type of health care.

For example, there are clinical audit tools for vascular and metabolic syndrome management, for maternal, child, preventive and mental health care, rheumatic heart disease and other areas of care.

The client records used can be paper or computer records (such as Communicare records) or a combination of both.

One21seventy clinical auditing collects demographic, clinical indicator and service delivery data about the health service population.

Auditing is based on a sample (or sub-set) of patient records specific to each audit tool.

For example, to be eligible for inclusion in a mental health audit a client must have a diagnosed mental illness or a mental health disorder that is likely to recur, have experienced symptoms for more than six months in the past, and meet other criteria listed in the mental health audit protocol.

Each audit tool has a step-by-step protocol to guide the auditing process.

Key items for service delivery, consistent with clinical practice guidelines, are audited.

Collected audit data is entered into the One21seventy database.

Why do CQI processes use service delivery and clinical indicator data?

Clinical indicators are measures of elements of clinical care.

When assessed over time, they provide a method of assessing the quality and safety of care at a system level2. For the purpose of quality improvement, clinical indicators provide data for comparison against standard care guidelines.

They can indicate the quality of care being delivered, and measure changes in clinical indicators in a population over time.

Service delivery data can show how a health centre is performing in relation to the scheduled services recommended in care guidelines.

Staff can benchmark their health centre’s performance by comparing data with data from other services at area, state and national levels.

It is possible to achieve improvements in service delivery and clinical indicators through CQI, because CQI focuses on improving local health centre systems for delivering client care.

Evidence tells us that even small changes in some clinical indicators can result in significant change in health outcomes.

A UK study of over 5000 hypertensive patients with type 2 diabetes found that a maintained 10 millimetres of mercury decrease in blood pressure from a mean of 154/87 to a mean of 144/82 millimetres of mercury recorded clinically important results including: 37% reduction in micro-vascular complications, 44% reduction in strokes and 32% reduction in deaths.

Resources:
1. The UK Prospective Diabetes Study (UKPDS) Group 1998, ‘Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes’ (UKPDS 38), British Medical Journal 1998, Se The UK Prospective Diabetes Study (UKPDS) Group 1998, ‘Tight blood pressure control and risk of macrovascular and microvascular complications in type 2 diabetes’ (UKPDS 38), British Medical Journal 1998, Sep 12; Volume 317, (7160), pp 703-13.p 12; Volume 317, (7160), pp 703-13.

2. Department of Health, Victoria, Understanding Clinical Practice Toolkit: Clinical Indicators, <http://www.health.vic.gov.au/clinicalengagement/downloads/pasp/clinical_indicators.pdf>

A system can be defined as a set of interacting parts, which form an integrated whole.

Think of a machine, with parts mechanically bound together. If one part of the system is removed or changed, the nature of the system is changed.

The same can be said of organisational systems.

The One21seventy Systems Assessment Tool has been developed to allow health centre teams to do a structured assessment of the strengths and weaknesses of the organisational systems that support client care.

Teams can use the information to decide how function might be improved.
Quality of care can be seriously limited by poor organisational systems.

Evidence shows that better functioning systems are more effective in producing the results they are designed to achieve.

Improving systems is therefore important in improving the quality of health care - improvement that can be measured in following audit cycles.

In addition, system change can sustain improvements, because a well functioning health centre system is not dependent on the ideas of individual practitioners, and it survives staff turnover.

The One21seventy systems assessment tool was developed by the Menzies School of Health Research as part of the Audit and Best Practice in Chronic Disease Project.

It is now well established as effective in Aboriginal and Torres Strait Islander primary health care settings.

The tool is based on internationally accepted models that reflect the way health systems work and interact at organisation, practice, patient and community levels: the Chronic Care Model and associated Assessment of Chronic Illness Care tool developed in the United States in the 1990s, and the World Health Organisation’s Innovative Care for Chronic Conditions Framework.

So - what does this systems interaction look like in practice?

Take, for example, the way clinical information systems work and interact at different levels.

At the organisation level, a health service's electronic clinical information system allows timely and integrated methods for reviewing pathology.

At the practice level, this information system provides a seamless flow of data, so practitioners can work as a team using the same pathology result to determine patient care and advice.

Patients hear a consistent story from everyone involved in their care; the community is happy because the organisation or practice is providing a coordinated approach.

The One21seventy systems assessment tool is designed for the workforce to assess how well these types of system interactions are functioning to achieve better client health outcomes.

The One21seventy Systems Assessment Tool assesses five key system components needed for improving chronic illness care across primary health care:

·delivery system design;
·information systems and decision support;
·self-management support;
·links with community, other health services and other services;
·and, organisational influence and integration.

Within each system component, individual items (or parts) are assessed.

For example, the ‘Information systems and decision support’ component refers to the clinical and other information structures and processes that support the planning, delivery and coordination of care.

Three items are assessed.

For each assessed item, such as ‘Maintenance and use of electronic client lists’, the tool lists elements for discussion, with prompt questions.

The systems assessment process is a whole of team discussion, ideally led by an outside facilitator to enable all staff to share their perspectives.

A Facilitators Guide is provided. Staff and managers discuss and come to consensus about how well their systems are working.

There are no right or wrong answers - each item is scored between zero and eleven and a score justification noted. Information is entered into the One21seventy database.

The item scores are averaged to reach one score for each of the five SAT components.

A separate systems assessment is done for each clinical audit (for example maternal health or preventive health).

This is because the quality of systems to support different aspects of care may differ within a service.
Listen to these real quotes from health service staff:

"The best thing about doing the Systems Assessment is getting the whole team together to talk about how well things are working and whether our clinic systems really support good patient care or get in the way. We don’t get to do that very often but it is really worthwhile"

"When programs come into the organisation, the information goes to management levels, which have always been responsible for data collection … Whereas, the SAT processes involve all staff perspectives - all the way from drivers, clinical staff, management and executive …"

Step 4 in the One21seventy cycle is data analysis and reporting.

Reports of analysed data are accessed through the One21seventy web site when audits are completed.

Each report explains the purpose of the report and the significance of the clinical audit information, and contains graphs and raw data and comparison data tables.

If a systems assessment has been done, a systems assessment data plot and table of scores and justifications are included.

The report also includes templates for identifying improvement priorities, setting goals, planning strategies and developing action plans.

Graph types used are bar graphs, stacked bar graphs, pie charts and a radar plot.

Click on the figures below to learn more about data presentation.

Report interpretation should be a participatory process. Local knowledge and different team member perspectives are recommended to achieve the best outcomes.

Teams discuss and link demographic, audit and systems assessment results to identify strengths and weaknesses of the health centre system and identify priorities for improvement.

Teams can then set improvement goals, decide practical strategies to achieve the goals and develop plans to put them into action.

One21seventy provides templates to help teams complete these processes. Interpreting reports and planning together is the first step towards working together to improve health centre systems.

Some health centres choose to interpret the clinical audit data before doing the systems assessment, so that what has been learnt from auditing informs the systems assessment process.

There is flexibility in the One21seventy processes to meet local needs and preferences.

You can now complete the assessment task.
You can either do the brief assessment quiz, or complete the optional task, which is designed for individual reflection or group discussion with colleagues who have completed the module.
The optional task won’t be assessed - so you are encouraged to discuss your work with your local CQI facilitator.
Once you begin the quiz, you can’t exit until all questions are answered.
If you would like to review module content before starting, use the navigation menu on the left to go back through the module.

Bar graphs display service delivery and clinical indicator data. The different heights of the vertical bars enable easy comparison of audit data values.

1.In One21seventy reports, the title of a graph explains who, and what was audited and the timeframe reviewed.
2.The x axis along the bottom shows audit dates and sample sizes - n - of each audit cycle.
3.The y axis along the left hand side shows the type of measure represented - in this case, percentage.
4.This graph shows us the percentage of clients who had their mental health goals reviewed in the three months leading up to the audit date. In the first cycle 30% had had their goals reviewed, and 2 years later, the proportion was 69%, a significant improvement.

One21seventy stacked bar graphs display service delivery and clinical indicator data. The bars are divided into subparts representing portions of the whole audit sample.

1.The title explains who, and what was audited and the timeframe reviewed
2.In this stacked bar graph, different coloured subparts represent percentages of smokers, non smokers and those whose smoking status was not recorded in their client files.
3.Audit dates and sample size are specified, equal to n. For example, n=80 means a sample size of 80. Three annual cycles can be compared.
4.In cycle 1, with 17% of mental health clients having no record of smoking status, we are not as confident with results on smoking and non smoking status. In cycles 2 and 3 the sample size is greater and the proportion of clients with no record of smoking status is less, so we can be more confident of the results that 26% of the group were non smokers in the 2013 audit.

Some bar graphs display a group of measures, as explained in the title and key.

1.The green bars show a slight improvement in measuring weight over 3 audit cycles.

2.Yellow bars show a greater improvement in measuring waist circumference over 3 cycles.

3.A small improvement in measuring pulse rate and rhythm has been recorded.

4.Note that cycle 3 included a much larger sample size. This makes the 2013 data more reliable.

Pie charts are used to display values as portions of a whole group.

In One21seventy reports, pie charts are used to present demographic data about the population sample - gender, Indigenous status and age group data. In this chart, different colours represent different age groups, explained in the key.

1.Segments are labelled as percentages of the audit sample.
2.Pie charts showing proportions of different age groups in the sample are useful in understanding the population breakdown of clients in a particular audit. In a preventive health audit, for example, the age breakdown of your sample should be similar to that of the overall adult population.

Systems assessment results are presented as radar plots. Axes radiating from the centre show overall scores for each component of the systems assessment.

1.The scores are measured outwards from the centre, from 0 to 11, the highest possible score.
2.Scores at the boundary (11) indicate a fully developed system; scores towards the middle indicate that parts of the system have potential for improvement.
3.Improvements over time (or cycles) are reflected in the widening of the points along the axes, as shown in this example.
4.A radar plot can display up to three annual cycles of results, as shown here and indicated in the key.

Clinical indicator and service delivery audit data are also presented as raw data tables.

1.Note the denominator - the number of client records in this sample.
2.Note the numerators or raw numbers. Within the sample of patient records, the numerator is the number of patients in this category, for example 18 out of 100. These numbers are converted to percentages.
3.The raw data tables include comparative percentage data from other de-identified One21seventy services that have consented to share data. Average percentage scores for the local cluster, state or territory and nationally are shown in the columns on the right hand side. These percentage data enable teams to benchmark their audit results against those of others, and can be useful when identifying areas for improvement - and areas of high performance - in your service.