Qualification achievement rates (QAR)
The Ofsted / Quality Dashboard provides indicative QAR calculations for Apprenticeship learners based on the ILR data within your Aptem account. The calculations within Aptem follow the published business rules and technical specifications provided by the DfE.
The official calculations are based on the ILR batch files your organisation has submitted to the ESFA. In comparison there are some constraints such that calculations based on data stored within Aptem may differ from the official version. This page sets out how we have followed the QAR rules and where potential differences may arise.
Note that currently the Quality dashboard only provides QAR calculations for apprenticeships. Future updates will include calculations for traineeships and other funded training programmes.
How is QAR calculated?
QAR calculations are based upon a Hybrid Year End which determines which learner data is included in calculations for a particular year. In summary the calculation is based on the number of apprentices completing or withdrawing in a particular Hybrid year and the number who achieved.
While the calculation is complex, the key points are summarised here:
- Leavers who are funded by the ESFA are included in the calculations – this excludes all Aptem users whose records do not contain valid ILR data.
- Learners are excluded from the calculations for a variety of reasons which include:
- Starting an apprenticeship but not completing the qualification period.
- Withdrawing from an apprenticeship but then starting a different apprenticeship within a certain time period (effectively a transfer).
- Transfers between one training provider and another.
- Learners who are taking a break in learning, with the exception of those where the break is overdue.
- Learners who are still in learning and have an ILR completion code of 1, except for those determined as an overdue continuing aim.
The QAR pages within the Ofsted/Quality dashboard provide QAR, Pass Rate and Retention calculations for the past two years and an estimation for the current academic year. For the current year we provide three figures, in all case the total count comprises all learners who have a hybrid year end matching the current year along with those who have a planned end year within the current year.
The calculations for Min / Max / Expected are as follows:
- Min: This is the worst-case scenario, that assumes that those in learning will not successfully complete this year, successes are only those who have already completed. Min = (Achieved) / (Learner count + In learning)
- Max: This is the best-case scenario, that assumes that in addition to those who have already achieved, all those in learning will achieve this this year. Max = (Achieved + In Learning) / (Learner count + In learning)
- Expected: This assumes that the achievement rate for those in learning will be the same as for those who have already completed this year. Expected = (Achieved ) / (Learner count)
Limitations to the QAR calculation in Aptem
Aptem stores your current ILR data which is then exported as a batch file. Currently we do not store copies of the ILR batch files submitted to the ESFA, so any QAR calculations are based on current ILR data in Aptem. The ESFA calculations are based upon the R14 submissions for the past five years.
There are potential differences between the data stored within Aptem and the data held by the ESFA which can result in discrepancies between our dashboard calculations and the official calculations. These differences result in our calculations for the total count of learners being lower or higher than the official calculations, and can occur for the following reasons:
- The hybrid year end is defined as the maximum of the apprenticeship Actual end year, Planned End year, Achievement year or reporting year. If ILR data for historical learners who withdrew in the past but have a planned end date in the current year have not been imported into Aptem then we may overestimate the achievement rate, whereas the official calculations will include these learners in the overall count.
- Non-identification of overdue continuing aims. Learner ILRs were previously submitted to the ESFA as in learning, for example using another system, but these aims are not included within your Aptem ILR data or have been tagged as exclude from batch (and thus excluded from our calculations). These learners will be tagged as overdue by the ESFA and assumed as subsequently withdrawn if there is no corresponding ILR aim record in the current year. This will result in a slight overestimation of the achievement rate.
- Some learners were shown as on a break in the most recent R14 submission, then their ILR record within Aptem was subsequently edited to a status of withdrawn. Learners on a break are excluded from the QAR calculation (their hybrid end year is pushed into the future). Our calculations will include these learners as withdrawn rather than on a break, thus adding to the count of overall learners, resulting in an underestimation of the achievement rate for the previous or current year depending on the date of the withdrawal.
Note that learners who were shown as on a break in the R14 ILR submission who subsequently returned to learning will have a restarted aim record and will appear in calculations for the current or future years. This will not affect QAR calculations for earlier years. - The batch files submitted to the ESFA have been edited outside of Aptem. For example, where training delivery is undertaken by subcontractors who provide a separate ILR batch file for these learners, and the file submitted to the ESFA is a merged file containing all learners for the prime provider. Aptem can only report on learner data within Aptem thus the QAR calculations will not match.
Note: Update Hybrid end year calculation within the Quality Report QAR Pages
To minimize some of these limitations, the reporting year calculation within the QAR calculation has been updated to include those learners who have had the “Force include in batch” completed on one of their learning aims.
- Using “Force include in batch” pushes the reporting year to the value that has been entered against that record.
- The hybrid end year then uses this reporting year to determine the maximum of the 4 years; Planned end year, Actual end year, Achievement year and Reporting year.
- Hybrid year examples.
- Planned end year = 2025/26, Actual end year 2023/24, Achievement year 2023/24, Reporting year (force include) 2024/25: Hybrid year = 2025/26
- Planned end year = 2022/23, Actual end year 2023/24, Achievement year 2023/24, Reporting year (force include) 2024/25: Hybrid year = 2024/25
Accountability metrics
In 2021 the Apprenticeship Provider Accountability Framework ( https://www.gov.uk/guidance/apprenticeship-training-provider-accountability-framework-20212022) was introduced. Alongside achievement rates, this framework monitors additional metrics including withdrawals, learners past their planned end date, breaks in learning and when apprentices are registered with an end-point assessment organisation. The metrics are based on data provided from your learner ILRs.
The Accountability page of the Quality dashboard provides headline indicator for the current year. The calculations are based on not only counts of those who have withdrawn/achieved but also learners who are either ‘in learning’ or on a break – and thus excluded from the QAR calculation.
The following assumptions have been made for the accountability calculations:
- Learner aims with invalid ILRs (excluded aims, no ULN) are not counted.
- Current year:
- Apprentices that are in learning with a start within or before the current year, this includes restarts.
- Apprentices who are on a break (regardless of actual end year) and do not have a more recent restart. (Those with a restart are not counted as being on a learning break.)
- Apprentices who completed or withdrew during the current year.
- Previous year:
- Apprentices that are in learning with a start before the end of the previous year, this includes restarts.
- Apprentices who are on a break (regardless of actual end year) and do not have a restart before the end of the year. (Those with a restart before the end of the year are not counted as being on a learning break.)
- Apprentices who completed or withdrew before the end of the previous year.
- Apprentices who have withdrawn, completed or gone on a break after the previous year end are counted as being in learning at the year end.
- All learners that have started after the year end have been excluded.
- Actual end dates that are after the year end have been ignored. The outcome and status have been assumed to be “in learning” prior to that.
- Learners that have returned from a break since the year end are counted as still on a break,
Note: For these calculations we have counted the number of unique apprentices, rather than the number of main aims.
Ability to filter Accountability Framework data by academic year
Users can use the 'Year' filter to filter on the current year and previous year and view the position as it existed at the end of the previous year.
The options are: “Current year” (which is today) and “Previous year” (31st July that is prior to today)