How to do ... monitoring and evaluation of work-based learning?

Did you know that several countries in the West Balkan – such as Albania, Montenegro, Serbia and North Macedonia – recently launched programmes with a substantial part of work-based learning (WBL)? No wonder: WBL is considered a key element of providing labour market oriented vocational education and training (VET) and is high on the political agenda of many countries.

At the same time, there is still some uncertainty: what exactly is work-based learning?

In the narrow sense, WBL is understood as learning that takes place in a real working environment, e.g. by participating in the work process of producing goods or services at a company. In a wider sense, WBL can also take place in learning environments that simulate real work processes, e.g. at a vocational training centre. Here it is crucial that the entire business process is reflected in a holistic manner (as opposed to single, isolated activities or tasks). Examples for organising WBL are – as to name a few – formal or informal apprenticeships, traineeships, or internships.

For VET stakeholders, WBL can hold specific advantages. Employees developing and updating their skills in accordance with a company’s needs are a clear asset for enterprises, especially when leading to better services and products. Learners, again, may value the fact that their employability and career opportunities are enhanced – which corresponds, of course, just as well to the interest of policy makers.

Especially the latter have a further interest. Being a part of VET policy, implementing WBL is also a matter of governance. Good governance, again, depends not only on careful planning but also on reliable information on the effectiveness of a measure. This feedback is essential for adapting and updating the measure as necessary. The implementation of policy measures aimed at the promotion of WBL hence need to be monitored, and their results have to be evaluated. Yet as to design monitoring and evaluation (M&E) procedures, one needs to first clarify target groups and stakeholders to be involved and – along with that – the exact purpose of the procedure: who is supposed to use the results and to what end? Whose perspectives should be included?

It is quite likely that different VET stakeholders have different priorities when it comes to assessing the quality and performance of WBL. Government bodies with a focus on economic and labour market issues will mostly be interested in the learners’ integration into the labour market. Institutions in charge of education policy in a wider sense may assign greater priority to the pedagogical quality of programmes and the learners’ progression within the education system. VET schools may be interested in minimising drop-outs and secure high completion rates. Employers expect that skills delivered by WBL programmes match their needs. And learners might be interested in a broader set of knowledge and skills offering access to employment and/or further education. All of these perspectives are legitimate quality dimensions and should be taken into account when formulating a WBL evaluation framework. Likewise, data collection must ensure that information from all relevant stakeholder groups feeds into the process.

Now how is M&E for WBL set up? The IPOO model offers a general conceptual framework:

• INPUT: arrangements preceding the actual implementation, resources feeding into the process
• PROCESS: factors directly related to the intervention and the production of the targeted goods or services
• OUTPUT: immediate results in terms of goods or services produced, e.g. knowledge, skills and competences imparted on the learners
• OUTCOME: medium and long-term effects achieved through the utilisation of the output

In the case of WBL, these four dimensions can be represented by various characteristics such as governance and financing structures, the cooperation between enterprises and training institutions, or the learning process itself.

To take one example from the cases mentioned above, Montenegro has established a WBL model with the following features:

In the first year, learners spend one day per week at the workplace, two in the second and three in the third year. Planning of in-company training is done jointly by VET school coordinators and in-company tutors, with the latter being provided a pedagogical course. The Ministry of Education subsidies the pay of the students to the extend of at least 10% of the minimum net wage in the first grade and at least 15% in the second grade, while employers cover year three (20%). The programme is popular among employers and students alike: The total enrolments raised from 277 students in 2017/18 to 848 in 2019/20 – with 280 employers in 18 different occupations.

When countries introduce new programmes or systems with a substantial share of WBL, they face new challenges concerning M&E. Some indicators which have proven relevant for M&E of school based VET (such as enrolment rates, drop-out-rates, student-teacher ratios) are equally important for M&E of WBL. However, it is likely that further questions arise along the IPOO model.

When setting up an M&E system for WBL, it can be helpful to learn from other countries. Bearing that in mind, the Research Institute for Vocational Education and Training (f-bb), Germany, is supporting the ETF in collecting examples on monitoring and evaluation of WBL from different countries in order to widen the knowledge base on this topic. Against this backdrop, you are invited to join the debate.

Would you like to share an example from your country?

What are your experiences with monitoring and evaluating WBL?

Which challenges did/do you face?

References:

• European Training Foundation (ETF) (2018). Work-based learning: A handbook for policy makers and social partners in ETF partner countries. 2^nd edition. Torino: European Training Foundation.
• European Training Foundation (ETF) (2019). Assessing governance and financing of vocational education strategies: A methodology. Torino: European Training Foundation.
• European Training Foundation (ETF); International Labour Organization (ILO) (2020). Dual education in Montenegro: Practical training in three-year educational programmes. Torino: European Training Foundation.
• Sager, F.; Hinterleitner, M. (2014). Evaluation. In K. Schubert and N. C. Bandelow (eds.), Lehrbuch der Politikfeldanalyse (pp. 437-439). Munich: Oldenbourg.
• Brown, M. D.; Svenson, R. A. (1988). Measuring R&D productivity. Research Technology Management, 41:6, pp. 31-35.
• Heidegger, G. (2008). Evaluation Research. In F. Rauner and R. Maclean (eds.), Handbook of Technical and Vocational Education and Training Research (pp. 825-833). Dordrecht: Springer.