Top Mistakes in Assessment Sampling to Avoid

An assessor’s portfolio can look complete, orderly and fully mapped, yet still contain a decision that does not stand up to scrutiny. That is why the top mistakes in assessment sampling are rarely just administrative errors. They are usually failures to ask the right questions, select evidence purposefully and act clearly on what the sample reveals.

For Internal Quality Assurers, sampling is the practical means of establishing whether assessment decisions are valid, reliable, fair and consistent. It is not a process of simply checking a proportion of learner files. A well-designed approach gives a provider confidence in its assessment practice and gives assessors meaningful support to maintain and improve standards.

Why sampling needs professional judgement

Sampling must provide enough assurance to make a defensible quality decision, without becoming a duplicate assessment exercise. The balance will differ between settings. A small cohort with a new assessor, complex workplace evidence or a high proportion of referrals may require closer attention than a settled cohort assessed by an experienced practitioner with a strong record of consistent practice.

The sampling plan should therefore be a live quality tool, not a fixed percentage applied without thought. It needs to reflect risk, the qualification requirements, the delivery model and what previous sampling has found. If a plan cannot explain why the IQA selected particular learners, units, or decisions, the centre will struggle to show that quality assurance has been purposeful.

Top mistakes in assessment sampling

Treating sampling as a numbers exercise

A common error is setting a percentage of portfolios and assuming that this alone creates assurance. A figure can be useful for planning workload, but it does not describe the quality of a sample. Sampling 10 per cent of learners may miss all direct observations, all reasonable adjustments, all late claims or all work completed by a newly appointed assessor.

A stronger approach considers what needs to be represented. This may include different assessors, sites, employers, units, methods of evidence collection, learner groups and stages of the assessment journey. It should also include a mix of decisions, such as achieved, not yet achieved, referred and resubmitted work where relevant.

The aim is not to sample everything. It is to select evidence that can test whether assessment practice is working consistently across the risks that matter. If a programme has few learners or one particularly complex unit, a larger or more focused sample may be justified. If assessors have established strong practice and previous findings remain secure, the IQA may keep the sample proportionate while still making it sufficiently broad.

Using a generic risk rationale

Many sampling records contain phrases such as ‘random sample’ or ‘standard sample’ with little further explanation. Random selection has a place, particularly when checking whether a planned approach is missing unexpected issues. However, random does not mean risk-based, and it should not become the only rationale.

Real intelligence should inform risk. This can include new or inexperienced assessors, changes to assessment methods, delayed evidence, unusually high achievement rates, recurring action points, learner complaints, employer feedback or previous concerns about particular units. It may also include changes in staff, locations or digital systems.

A credible rationale makes the IQA’s professional judgement visible. Rather than record that the IQA sampled three portfolios, record why they selected those portfolios and what aspect of practice they tested. This creates a clearer audit trail and helps the team understand how quality activity connects to improvement.

Sampling evidence but not assessment decisions

The presence of evidence is not the same as a sound assessment decision. An IQA may find a detailed witness testimony, a completed professional discussion record or a strong observation report, but still needs to ask whether it meets the relevant criteria, is authentic, current and sufficient.

Sampling should test the assessor’s decision-making. Is the feedback accurate and constructive? Has the assessor identified gaps appropriately? Is there a clear link between the evidence and the assessment criteria? Where professional discussion has been used, does the record show what was asked, what the learner demonstrated and why this supports the decision?

This distinction is especially important where electronic systems make it easy to see completion ticks, uploaded files and progress indicators. Those tools can support efficient monitoring, but they cannot replace an informed review of the assessment judgement itself.

Overlooking the assessment journey

End-point sampling can confirm that a completed portfolio appears satisfactory, but it may not reveal whether the learner was assessed fairly throughout. Sampling only at claim stage can miss weak initial planning, delayed feedback, insufficient support or repeated missed opportunities to address gaps.

Build sampling across the learner journey. Early sampling can test whether assessment plans are realistic and whether learners understand expectations. Mid-point sampling can help IQAs identify stalled progress, inconsistent feedback, or evidence that assessors collect without a clear purpose. Pre-claim sampling can confirm that decisions are secure before certification activity takes place.

This does not mean every learner must be checked at every stage. It means the overall plan should provide visibility of practice over time. Where learners are on longer programmes, this approach can prevent small issues becoming embedded habits.

Failing to sample assessor feedback

Feedback is a direct expression of assessment quality. It shows whether an assessor has made a clear decision, supported learner progress and maintained the distinction between guidance and over-direction. Yet feedback is sometimes reviewed only for spelling, tone or whether a comment has been entered.

Quality sampling should examine whether feedback is timely, specific and linked to the evidence or criteria under review. For a successful decision, does it explain what the learner has demonstrated? For a referral, does it clearly identify what remains to be done without writing the learner’s response for them? Vague comments such as ‘good work’ or ‘add more detail’ provide limited value and offer little assurance.

Feedback also reveals consistency across the team. If one assessor routinely gives detailed developmental comments while another gives minimal statements, standardisation may be needed even where both have reached broadly correct decisions.

Recording actions that cannot be checked

An IQA record should do more than state that an issue was discussed. Actions such as ‘assessor to improve feedback’ or ‘monitor next month’ are difficult to follow up and do not demonstrate whether improvement has occurred.

Effective actions name the issue, the expected improvement, who is responsible and when it will be reviewed. For example, an action might require an assessor to make criterion references explicit in feedback for the next five sampled learners, followed by a targeted review at an agreed date. The purpose is development, not fault-finding.

Equally, avoid raising actions for every minor difference in style. The issue is whether practice affects validity, reliability, fairness, sufficiency or the learner experience. Proportionate feedback respects assessor professionalism while making standards non-negotiable.

Closing the sample without evaluating the pattern

A single sample may identify a local issue. Several samples may reveal a trend. The mistake is to close each record in isolation and fail to look across findings over a month, cohort or programme.

Quality managers and IQAs should review recurring themes: perhaps direct observation records vary between sites, feedback is consistently late in one area, or a particular assessment criterion is being interpreted differently. These patterns should inform standardisation, assessor development and the next sampling plan.

This is where sampling becomes a source of improvement rather than a compliance archive. It creates an evidence base for decisions about CPD, staffing support, learner protection and programme quality.

Building a defensible sampling approach

Start with a clear plan that states what will be sampled, when, by whom and why. Make room for planned sampling and responsive sampling. Planned activity gives structure, while responsive activity allows an IQA to investigate concerns that emerge through progress data, feedback or stakeholder information.

Keep records clear enough for another competent practitioner to understand the judgement. They should show the sample rationale, the evidence reviewed, the outcome, actions and follow-up. If no action is required, the record should still explain what provided assurance.

Most importantly, use sampling conversations constructively. Assessors need clarity about standards, but they also need space to discuss difficult evidence decisions, workplace constraints and learner needs. This is where professional community and focused CPD can strengthen confidence as well as consistency. BIAP recognises that high-quality assurance depends on informed practitioners who can apply standards thoughtfully, not simply complete a process.

A sampling strategy earns its value when it helps people make better assessment decisions tomorrow. Keep the focus there: purposeful selection, clear professional judgement and follow-through that improves practice for every learner.