Ministry of Education, Science and Sport; since 2004 also the European Social Fund
Contact person
Project leader at the SIAE:
Tanja Možina, PhD
Phone: +386 1 5842 577
E-mail: tanja.mozina@acs.si
Purpose and goals
The main purpose of the project Offering Quality Education to Adults - OQEA is to encourage educational organisations to establish and develop an internal quality system based on self-evaluation, and supplemented by external quality assessments.
The key objectives of the project Offering Quality Education to Adults - OQEA were:
Development of a self-evaluation approach, adapted to adult education organisations;
Development of quality indicators for adult education;
Development of basic and advanced training programmes for professionals who use the OQEA approach in practice;
Offering guidance to educational organisations regarding the implementation of self-evaluations in accordance with the OQEA approach;
Development of an informational and communicational support service and professional literature to help with self-evaluations.
Description
The Offering Quality Education to Adults - OQEA project was first introduced in 2001. By 2013, when the last group of organisations was trained to implement the OQEA model of self-evaluation, the project involves as many as 69 organisations for adult education, which equals to 53% of all educational organisation offering public and valid adult education in that study year.
Our work with the organisations took around two years. The role of the Slovenian Institute for Adult Education included mostly counselling and education, while each educational organisation formed a quality team to implement individual processes for the evaluation and development of quality.
The OQEA project has ended in 2013, but the ideas we developed as part of the project live on in our principles and processes. Based on experiences we gained with the OQEA Project, we developed measures for the evaluation and development of quality, which are now used in many adult education organisations. The motto Offering Quality Education to Adults is no longer just the name of the OQEA project, but now includes all activities for the evaluation and development of quality in Slovenian adult education. You can read more in the online collection of recommendations for the establishment and development of an internal quality system – Quality Mosaic.
The quality assessment and development model Offering Quality Education to Adults (OQEA) is based on self-evaluation; this means that the educational organisation, the adult educational manager and/or teacher(s) voluntarily decide:
to assess the quality of their work,
which parts of the educational process or effects they will monitor,
what quality standards they will set for themselves,
what they will do with the achievements,
what measures they will introduce based on the assessment received.
The model is adjusted in a way that it can be used in different adult education organisations (adult education centres, secondary and post-secondary education organisations, private education, and volunteer organisations, etc.). We use it to assess formal and informal education. The model can be used for self-evaluation of the entire educational organisation, or only one programme, department, or educational group; it can even be used by only one teacher.
Implementation of the model on the level of the educational organisation requires permanent cooperation of the management. During assessments, decisions, and measures the entire collective of the educational organisation should participate.
Image: Professional principles guiding the development of a system for quality in adult education
The development cycle for the quality assessment and development model implementation into an educational organisation lasted two years and had the following steps:
The management of the educational organisation nominated a quality group. The project itself did not specify who the members of the quality group should be. We advised that it included at least representatives from the management, the head/organiser of adult education, teachers' representatives, other staff (administrators, accountants, etc.).
If the educational organisation already had a quality group or committee, there was no need to nominate a new group to participate in the OQEA project - the work on the OQEA project was connected to the activities that the quality committee was carrying out already.
The quality group or committee participated in carrying out individual phases of quality assessment and development, organised different activities connected to the self-evaluation in the collective and motivated the staff for an active participation in the quality assessment and development.
Two members of the quality group took part in training and guidance workshops at the Slovenian Institute for Adult Education.
Together with the participants we reflected at length during the OQEA project on the basic strategic questions on quality in the educational organisation. Such reflection was a part of the determining quality processes.
The quality group in an educational organisation led the procedure in which the staff, together with the management, reflected their basic mission, determined or evaluated the existing vision, and discussed the basic values that led its work.
During this process, the organisation found the answers to the fundamental questions, for example: "What is our basic mission?", "What are the values that connect us?", or "How do we see ourselves in the future?" and recorded them as written statements.
Image: An example of forming a mission, vision and values at the Adult Education Centre »Ljudska univerza Jesenice«
The organisations began the quality assessment process according to the OQEA model by first completing the so called first quality assessment on all areas defined in the OQEA model.
This first assessment was done using the SWOT method. Such initial analysis of the situation helped the educational organisation decide in which area of quality it carried out a more thorough self-evaluation.
The Manual INDICATORS (the first collection of quality indicators, published in 2003, as the core of the Offering Quality Education to Adults (OQEA) self-evaluation model) helped the users chose appropriate indicators and led them through assessing their quality.
In 2013 SIAE published Reformed collection of quality indicators in adult education as a result of re-thinking of the first collection of quality indicators. We can use the reformed collection of quality indicators in adult education for internal quality assessment and development in the adult education organization (self-evaluation, self-assessment) and for external quality assessment and development in the adult education organization (external monitoring, external evaluation, accreditation).
The next thing to consider in the OQEA project was which interest groups were particularly important for the educational organisation. We thought about the groups that can importantly influence the quality of the work in the organisation, or which the organisation influenced with its work.
Every educational organisation:
determined the most important internal and external interest groups (employees, participants, employers, professional institutions ...).
It carried out an analysis of how much and how these groups have already participated in determining, assessing and developing quality,
and considered how they can cooperate in these processes in the future.
Image: An example of demonstration of important internal and external interest groups
After completing the first quality assessment in all areas of quality according to the OQEA model, the educational organisation chose one area and from two to four quality indicators: then carried out an in-depth self-evaluation.
After the educational organisation decided on the area and quality indicators that will be used to carry out an in-depth self-evaluation, it prepared a self-evaluation plan. The first step was formulating its own quality standards for the selected indicators. With this, the organisation defined what level of quality it wanted to achieve. The organisation later compared thus formulated standards of quality with the actual state of quality and evaluated where it had already reached the standards and where improvements or additions to the activities must be implemented to reach them.
An important step in the preparation of the self-evaluation plan was determining the sources from which the organisation gets its data, information, opinions, points of view and methods (e.g., questionnaires, interviews, benchmarking, focus groups). The organisation prepared instruments for quality assessment (e.g., questionnaires, interview reminders, framework questions for benchmarking, focus groups).
Educational organisations had the following at their disposal when prepared instruments for quality assessment:
A manual with a CD-rom with a list of questions for quality assessment;
A web-based collection of questions for quality assessment in adult education which enabled them to create and fill in questionnaires online.
Considering the selected methods, the quality assessment was carried out in different ways. In OQEA project, the educational organisations could try online or traditional questionnaires, as well as got experience conducting a guided debate, benchmarking, or focus groups.
Based on the conducted analysis of the collected data, the educational organisations prepared a self-evaluation report in which they interpreted the outcomes, evaluated the quality situation, and indicated how they intend to reach the set quality standards if they had not already.
It was an important moment when the quality group presented the self-evaluation findings in the collective. The quality group discussed the findings of quality assessment with the management and all the employees, as well as other important stakeholders (participants, employers, founders etc.).
The presentation of findings had been followed by a debate about the possible improvements and development. All parties agreed on the measures they would take in future to maintain or improve the quality of their work.
Image: A debate about self-evaluation findings at the Adult Education Centre »Ljudska univerza Velenje«
Educational organisations put the agreed measures to improve quality into their action plan to develop quality. When preparing the plan, each organisation began with disclosed deficiencies and detected problems, and planned activities to remedy them. Equally important, it planned activities that helped it maintain the level of quality already achieved.
The action plan is a practical work tool, therefore it includes definitions of precise tasks and agreed deadlines for completing the planned activities.
In the second year of participation in the project, the organisations allocated all the attention to quality development - the realisation of set measures for quality improvement. It is important that the realisation of measures doesn't remain limited to the quality group, but attracts a wider circle of employees to implementing improvements. Only then will the work that was put into in-depth self-evaluation yield results.
Every educational organisation monitored the realisation of the action plan and, if necessary, adopted additional measures for quality development.
When the organisation participated in the OQEA project, we monitored and evaluated the realisation of activities during our regular meetings with them. This also enabled the exchange of good practices among educational organisations participating in the OQEA project.
Image: Participants of group of organisations OQEA 3 wrote down the realisation of the action plan
Image: Monitoring the implementation of quality improvements at the Technical School Centre Kranj
Once the educational organisations successfully completed implementation of the set improvements they entered a new development cycle, when a new quality assessment began on the remaining areas of the OQEA model.
The implementation of the model on the level of the educational organisation required uninterrupted cooperation of the management. The entire collective of the organisation participated in all the assessments, decisions and taking measures.
Adult participants in education, employers, and the local community were important allies in implementation of the model and quality assessment.
Participating educational organisations
Based on the public tender from the Ministry of Education and Sports, a certain number of educational organisations are generally selected annually to participate in the project: once joining the project they remain with it for about two years. In 2001, when we started the project, we only included secondary schools that educate adults, because we wanted to test the model in those organisations that carry out fewer educational programmes. As early as 2002 we included adult education centres, and in 2004 also private educational institutions that carry out accredited programmes of adult education. In 2005 we introduced training according to the adjusted variant of the OQEA model within the framework of certain schools, members of the Consortium of Biotechnical schools.
Review of the participation of organisations in the project:
From 2001 to 2003 the project involved four secondary schools. They started with educational workshops and work in October 2001; the last workshop was held in December 2003. These four schools occupy a special role in the project, because they were the first who tested the model Offering Quality Education to Adults in practice. Their work was therefore not only to use the model, but also participation in formulating certain practical solutions.
We implemented around 100 hours of educational and counselling workshops during the first year of our participation in the project.
We covered the following topics:
Learning about the project and the OQEA model;
Principles and philosophy of quality;
Defining quality development in the organisation – purpose, vision and values;
Preparing an overview of important interest groups;
Self-evaluation as a method for the assessment and development of quality;
Importance of teamwork for the development of quality within the organisation;
Planning and implementation of quality evaluation;
Creation of instruments for self-evaluation;
Preparation of self-evaluation report.
Image: Teamwork at one of the OQEA 2 workshops
There were around 20 hours of educational and counselling workshops in our second year of participation, as the focus during this year was the work of the quality team or committee set up by the organisations and implementation of the action plans they prepared.
We covered the following topics:
Implementation and realisation of action plans for quality development;
Preparation of a Quality Decree;
Quality online;
Professional development and identity of adult educators as related to quality;
Guidelines for the development of quality in adult education – further development of quality in an educational organisation.
Based on the educational needs and interests of participants in individual groups and based on any problems they had, we added other topics to the programme of individual workshops.
Image: Peer review of web pages about quality in adult education
Developmental milestones
Key developmental milestones are divided into different periods:
2012–2013
OQEA 7 group – The handbook Quality indicators for adult education was published
More
Less
The participants of the workshops learned about the OQEA self-evaluation model, assessed their quality using a web questionnaire, learned about the benchmarking method in the processes of evaluation and development of quality, and prepared their own action plan for quality improvement, based on the results of self-evaluation. The Quality indicators in adult education handbook was also published in 2013, with an updated collection of quality indicators used by the OQEA Project. The OQEA 7 Group was the last OQEA group that was trained using this approach, developed and implemented in 2001 (two-year participation at workshops and guided work within the organisation).
2010–2011
OQEA 6 group – A web-based learning corner established; SWOT analysis added to the self-evaluation model
More
Less
In the 2007–2010 period, new educational organisations were not able to participate in the OQEA Project, which was a difficult challenge for the project. In addition to other obstacles we faced, the development team at the SIAE was aware that after such a long period of inactivity, the project might never be revived. But our fears were unfounded! The bureaucracy and other obstacles could not stop all the organisations that were already using the OQEA model in practice. In 2010, we first used the SWOT analysis as part of the self-evaluation concept. The OQEA 6 generation was the first to combine three methods – the SWOT analysis, the questionnaire, and the benchmarking method – for their own self-evaluation.
2006–2007
OQEA 5 group – A web-based collection of quality assessment questions developed; we tested qualitative methods for the first time: benchmarking and focus groups
More
Less
Much attention was given to the development of instruments for measuring the satisfaction of participants. We published a handbook Measuring the participant satisfaction in adult education. We tried the benchmarking method and the focus groups method for the first time. The experience was covered in two handbooks. At the same time, the educational organisations filled in questionnaires. They were the first group that was able to use the new online collection of questions for the evaluation of quality, which was developed in the same period. It already included more than 2000 questions.
2005–2006
OQEA 4 group – The collection of questions for quality assessment reinforced with new questions from the field of management
More
Less
In this group, the educational organisations focused on the topic of Leadership and management, which none of the previous groups chose as the focus for self-evaluation. We also improved the training by adding the topics of PR and an online presentation of quality. The idea of an online collection of questions for the evaluation of quality, with the aim of empowering users to create their own evaluation questionnaires online, and allowing the users to fill in and process the data in a web environment.
2004–2006
OQEA 3 group – The collection of quality assessment questions published, we consolidated the topic of including the employees into the quality work
More
Less
In collaboration with educational organisations, we developed a methodology for measuring the results and effects of adult education, and the collection of questions for evaluation was completed with new questions related to this field. During the training, we invited discussion about internal and external stakeholders important for educational organisations, and the way to encourage them to evaluate and develop quality. One problem we faced was how to encourage more of the employees to work on quality, which is why we put more focus on the topic of teamwork, working with others to improve quality, etc.
2003–2004
OQEA 2 group – The publishing of the manual Quality indicators, we consolidated the themes of vision, values, mission formation
More
Less
We allocated more time at our workshops to methodology, as it turned out the participants has less previous methodological knowledge than assumed. We wanted to challenge all personnel of organisations with broad strategic questions related to quality in adult education, which is why we included broader reflections on purpose, vision, and core values that should guide the work of adult educators.
2001–2003
OQEA 1 group – The piloting of the OQEA model in practice, tested structure and contents of the areas and OQEA quality indicators
More
Less
We supported educational organisations with methodological and content counselling, and helped them implement systematic self-evaluations. Together we created an evaluation instrument at educational and counselling workshops, which was later tested in schools.
Key achievements
Key achievements in the 2001–2013 period were:
Since 2001, we have been developing instruments for the evaluation of quality in adult education in collaboration with educational organisations.
In collaboration with educational organisations that participated in the OQEA project we also implemented two mobility projects:
Mobility Project – Self-evaluation and quality in vocational and professional adult education – Denmark (2002);
Mobility Project – Self-evaluation and quality in vocational and professional adult education – Finland (2005). This project received the national award Golden Apple of Quality 2007, awarded by the Centre of the Republic of Slovenia for Mobility and European Educational and Training Programmes (CMEPIUS).
Image: National Golden Apple of Quality Award
During the OQEA Project we also published two manuals to help organisations evaluate and develop quality in adult education:
We edited and published a Collection of questions for evaluation of quality in a printed version and on CD.
We developed an online collection of questions for evaluation of quality.
We tested qualitative methods for evaluation of quality in practice (focus groups method and benchmark method) and published two manuals:
Focus groups as a method for evaluation and development of quality in education (2007);
Benchmarkingas a method for evaluation and development of quality in education (2007).
We developed instruments for measuring the satisfaction of participants and published a handbook,Measuring the participant satisfaction in adult education (2007).
We developed an online learning corner where participating organisations could access study materials and publications and articles related to quality as well as a glossary.
If we include the year 2013, which was the year of the last training for self-evaluation using the OQEA model in this format, the OQEA model of self-evaluation was used by 69 organisations in total, which equated to 97% of adult education centres offering public adult education programmes in the 2012/2013 study year (31 of 32), 34% of secondary schools (26 of 77 secondary schools and school centres) and 57% of private educational organisations (12 of 21). This equals to 53% of all educational organisation offering public and valid adult education in that study year.
Image: Participants at the meeting of all OQEA generations celebrating the 10th anniversary of the project
Monitoring and evaluation of the project
The implementation and usability of the OQEA project were regularly monitored and evaluated: we could respond quickly to any potential deficiencies and the need for improvement. For this reason we have developed the following forms of monitoring and evaluation:
We regularly monitored the satisfaction of quality groups' members with the themes and the organisation of training for the project implementation support.
We have implemented the method of occasional consultation with the experts from the Slovenian universities who have worked with the quality of adult education.
We regularly monitored how the educational organisations enacted the individual aspects of quality which we talked about and shaped in the OQEA model: how much did the educational organisations, included in the OQEA project, spread and presented their vision, mission, and values in their practical work, and to what extent they did it via web pages and publications presenting their offer. We also paid attention to the way participating educational organisations dealt with the self-evaluation reports in their own collectives.
We also carried out more thorough evaluations of OQEA model implementation in educational organisations.
Among the numerous observations that our participants shared with us during our meetings, we selected some that meant a lot to us when we received them, and that kept motivating us for further collaborations. This is what the OQEA participants said about the project:
The OQEA became a meeting point for adult education organisations.
The training is very well-planned, offering us useful knowledge and new insights. We also get a different, more in-depth perspective of some topics.
The topics discussed at the workshop are not only important and useful for completion of the project, but also for the work we do and the fields we work in.
We gladly welcome external experts, as they can show some problems from a new point of view, not only the point of view of adult educators.
We are glad that you stand by our side and keep the OQEA project going.
