The functionality described in this article will be made available when the revised Ofsted Quality report is published. This is expected to be published by Monday, 27 June 2022.
Qualification achievement rates (QAR)
The Ofsted / Quality Dashboard provides indicative QAR calculations for Apprenticeship learners based on the ILR data within your Aptem account. The calculations within Aptem follow the published business rules and technical specifications provided by the DfE.
The official calculations are based on the ILR batch files your organisation has submitted to the ESFA. In comparison there are some constraints such that calculations based on data stored within Aptem may differ from the official version. This page sets out how we have followed the QAR rules and where potential differences may arise.
Note that currently the Quality dashboard only provides QAR calculations for apprenticeships. Future updates will include calculations for traineeships and other funded training programmes.
How is QAR calculated?
QAR calculations are based upon a Hybrid Year End which determines which learner data is included in calculations for a particular year. In summary the calculation is based on the number of apprentices completing or withdrawing in a particular Hybrid year and the number who achieved.
While the calculation is complex, the key points are summarised here:
- Leavers who are funded by the ESFA are included in the calculations – this excludes all Aptem users whose records do not contain valid ILR data.
- Learners are excluded from the calculations for a variety of reasons which include:
- Starting an apprenticeship but not completing the qualification period.
- Withdrawing from an apprenticeship but then starting a different apprenticeship within a certain time period (effectively a transfer).
- Transfers between one training provider and another.
- Learners who are taking a break in learning, with the exception of those where the break is overdue.
- Learners who are still in learning and have an ILR completion code of 1, except for those determined as an overdue continuing aim.
The QAR pages within the Ofsted/Quality dashboard provide QAR, Pass Rate and Retention calculations for the past two years and an estimation for the current academic year. For the current year we provide three figures, in all case the total count comprises all learners who have a hybrid year end matching the current year along with those who have a planned end year within the current year:
- Minimum: assumes that those currently in learning will not achieve, and for achievement purposes only those who have already achieved in the current hybrid year are counted. This provides a lower limit for the current hybrid year.
- Expected: assumes that those currently in learning will achieve at the same rate as those who have already completed/withdrawn in the current academic year. This fraction does not exactly match the pass rate for the current year as non-returning learners from breaks may be included in the count.
- Maximum: assumes that those currently in learning will achieve, along with a count of those who have already achieved within the current year.
Limitations to the QAR calculation in Aptem
Aptem stores your current ILR data which is then exported as a batch file. Currently we do not store copies of the ILR batch files submitted to the ESFA, so any QAR calculations are based on current ILR data in Aptem. The ESFA calculations are based upon the R14 submissions for the past five years.
There are potential differences between the data stored within Aptem and the data held by the ESFA which can result in discrepancies between our dashboard calculations and the official calculations. These differences result in our calculations for the total count of learners being lower or higher than the official calculations, and can occur for the following reasons:
- The hybrid year end is defined as the maximum of the apprenticeship Actual end year, Planned End year, Achievement year or reporting year. If ILR data for historical learners who withdrew in the past but have a planned end date in the current year have not been imported into Aptem then we may overestimate the achievement rate, whereas the official calculations will include these learners in the overall count.
- Non-identification of overdue continuing aims. Learner ILRs were previously submitted to the ESFA as in learning, for example using another system, but these aims are not included within your Aptem ILR data or have been tagged as exclude from batch (and thus excluded from our calculations). These learners will be tagged as overdue by the ESFA and assumed as subsequently withdrawn if there is no corresponding ILR aim record in the current year. This will result in a slight overestimation of the achievement rate.
- Some learners were shown as on a break in the most recent R14 submission, then their ILR record within Aptem was subsequently edited to a status of withdrawn. Learners on a break are excluded from the QAR calculation (their hybrid end year is pushed into the future). Our calculations will include these learners as withdrawn rather than on a break, thus adding to the count of overall learners, resulting in an underestimation of the achievement rate for the previous or current year depending on the date of the withdrawal.
Note that learners who were shown as on a break in the R14 ILR submission who subsequently returned to learning will have a restarted aim record and will appear in calculations for the current or future years. This will not affect QAR calculations for earlier years.
- The batch files submitted to the ESFA have been edited outside of Aptem. For example, where training delivery is undertaken by subcontractors who provide a separate ILR batch file for these learners, and the file submitted to the ESFA is a merged file containing all learners for the prime provider. Aptem can only report on learner data within Aptem thus the QAR calculations will not match.
In 2021 the Apprenticeship Provider Accountability Framework ( https://www.gov.uk/guidance/apprenticeship-training-provider-accountability-framework-20212022) was introduced. Alongside achievement rates, this framework monitors additional metrics including withdrawals, learners past their planned end date, breaks in learning and when apprentices are registered with an end-point assessment organisation. The metrics are based on data provided from your learner ILRs.
The Accountability page of the Quality dashboard provides headline indicator for the current year. The calculations are based on not only counts of those who have withdrawn/achieved but also learners who are either ‘in learning’ or on a break – and thus excluded from the QAR calculation.
The following assumptions have been made for the accountability calculations:
- Learner aims with invalid ILRs (excluded aims, no ULN) are not counted.
- For the current year we have included:
- all in learning apprentices with a start within or before the current year, this includes restarts.
- all apprentices who are on a break (regardless of actual end year) and do not have a more recent restart. Those with a restart are not counted as being on a learning break.
- all apprentices who completed or withdrew during the current year.
For these calculations we have counted the number of unique apprentices, rather than the number of main aims.