Expert external evaluation

  1. Home
  2. Projects and activities
  3. Expert external evaluation

General overview

Time period 2011–2017 
Funding Ministry of Education, Science and Sport
Contact person Contact person
Tanja Možina, PhD
Phone: +386 1 5842 577
E-mail: tanja.mozina@acs.si

Purpose and goals

Expert external evaluations are used to independently confirm the quality of adult education in selected fields within educational organisations. They encourage maintaining the already attained quality standards and implementing improvements and developments in quality.

The expert external evaluation provides the organisation with quality feedback about where it is already good and the possibilities shown for further improvement. Evaluations also help to upgrade the existing internal quality systems in the organisation.
 

Description

Expert external evaluation was used by organisations that wanted an external, expert opinion about the quality of certain areas within their organisation. The public invitation for participation was published in the SIAE e-Newsletter and on the Quality website, but the organisations had to meet six criteria before being allowed to participate. A committee nominated by the SIAE reviewed the applications, decided whether the criteria were met, and selected a certain number of educational organisations. The educational organisations were voluntary participants of the expert external evaluation.

We also included expert external evaluators who were experts in the fields of adult education, internal and external quality evaluation, and wanted to gain additional experience in the field of quality assessment. These experts responded to the invitation published in the SIAE e-Newsletter and on the Quality website, and had to meet general and specific criteria.

The expert external evaluation has four steps, as shown in the image below.  
 

Developmental milestones

The following milestones were important for the establishment and development of expert external evaluation:
 
2018
Self-evaluation study Evaluation of processes and effects of expert external evaluations in adult education (dodaj link)
 
2016
2 adult education organisations participate in expert external evaluations.
2015
Development of a web application where the organisations can prepare and submit self-evaluation and the expert external evaluators can review them.
2015
3 adult education organisations participate participate in expert external evaluations.
2015
Update of criteria for existing quality evaluation areas and the development of a new area: Developmental orientation of the adult education organisation.
2014
Development of a code of ethics for expert external evaluators
2014
3 adult education organisations participate in expert external evaluations.
2013
3 adult education organisations participate in expert external evaluations.
 
2012
3 adult education organisations participate in expert external evaluations.
2011
2 adult education organisations participate in expert external evaluations.
2011
Development of the expert external evaluation concept.

Key achievements

The concept of expert external evaluation was developed in 2011 as an improvement and upgrade of two SIAE initiatives: The Green Quality Logo and Siae awards for quality in adult education. 

To ensure successful implementation of expert external evaluations, we:
  • planned and organised an informational meeting and training for adult education organisations where we provided them with all necessary information for participation in expert external evaluation;
  • planned and organised a two-day training for expert external evaluators who were trained for implementation of expert external evaluations;
  • planned and prepared all the necessary documentation templates and instruments (template of the report on expert external evaluation, template of the organisation’s self-evaluation, etc.) for the planning and execution of the visit of the participating organisation;
  • planned and coordinated expert external evaluations;
  • organised and evaluation workshop every year to assess the progress we have made;
  • developed a code of ethics for expert external evaluators (2014);
  • updated the criteria for existing quality evaluation areas and developed a new area: Developmental orientation of the adult education organisation (2015);
  • developed a web application where the organisations can prepare and submit self-evaluation and the expert external evaluators can review them (2015);
  • implemented a self-evaluation study called Evaluation of processes and effects of expert external evaluations in adult education (implemented in 2017, published in 2018), where we presented the results of our self-evaluation and reflected on the role, the future development and placement of this type of external evaluation into the adult education system in Slovenia.

Areas of expert external evaluation

External expert evaluation of adult education organisations between 2012 and 2014 focused on two areas of quality. One area remained constant while the other changed.
  • Area 1: Internal system for quality assessment and development 
  • Area 2: Support for participants in adult education
In the years 2015 and 2016, three areas were assessed by expert external evaluation. One area was mandatory and two were elective. The evaluation thus included the mandatory area and one of the elective areas:
  • Mandatory area: Internal system for quality assessment and development 
  • Elective area 1: Support for participants in adult education
  • Elective area 2: Developmental orientation of adult education organisation
Every area was carefully defined by detailed quality criteria. The adult education organisations were informed of the detailed criteria and the entire process of expert external evaluation at the informational meeting and training held by the SIAE.
 

Education, training and evaluation

All participants in the project of Expert external evaluation were given additional education and training to ensure successful implementation of the evaluations. The expert external evaluations were also constantly monitored and evaluated.
 
The CEOs and coordinators of expert external evaluations – who were employees of the participating organisations – learned about the starting points, the purpose and the process of expert external evaluation. We discussed both areas of quality and the criteria that will be part of the external evaluation as well as the self-evaluation form. The organisation then had two months to implement the self-evaluation.
 
The participants of the Training for expert external evaluators first learned about the concept of external evaluations and the areas and criteria that were part of external evaluation in the given year. This was followed by a practical review and evaluation of a self-evaluation of an imaginary educational education. Next we prepared appropriate interview questions to ask during the visit of the educational organisation. We discussed how to prepare for and implement the visit of the educational organisation, and practiced the visit through role-playing. At the end of the workshop, the participants were given advice on the preparation of the final report on expert external evaluation. The SIAE prepared all necessary instruments, templates, and guidelines to help the workshop participants implemented expert external evaluations.
After every expert external evaluation we organised an evaluation workshop with all the project partners – expert external evaluators, participating educational organisations and the SIAE – to monitor and evaluate the result. The conclusions of these workshops were used to improve and develop the model of expert external evaluation in adult education.