Ref: 022811
The assessment of higher education and research is quite a new thing in Lithuania. In order to avoid the subjective assessment, it is necessary to create the system of assessors selection. The computerisation of the system of assessors selection is highly appreciated. In this case it would help not only selection of assessors, but also management of peer reviews.
Keywords
The assessment of higher education and research, the system of assessors selection, management of peer reviews, the database of reserve of assessors.
In order to reach the best accuracy and reliability of the quality assessment in higher education and research, it is necessary: to direct aims; to distinguish conditions where aims should be reached; to make action plans; to take necessary measures for carrying out the action plans; to regulate actions according to variable conditions.
These requirements could be realised in the stages of management process. The process of management is divided into management functions, according to those stages. Results of higher education and research peer reviews depend on performance of management functions. In order to increase the efficiency of assessors management it is necessary to create a computerised system of assessors management.
Assessors are recommended into reserve by science and higher education councils, Lithuania Science Council, Lithuanian of Science, councils or administration boards of trade unions, chiefs of ministries or other institutions.
Data about a candidate are offered in a special form. This form should be signed by a candidate and an offering institution.
An assessor can be eliminated from the reserve if he does not correspond to requirements made.
However, several problems exist. It is important to choose right criterions of selection. The system of selection of assessors may decrease subjectivity but subjectivity depends not only on personal opinion of assessors. Lithuania is a small country and it is difficult to find assessors without any relations to the institution to be assessed. High requirements for receiving efficient decisions put great demands for the effective using of information resources. One of the reasons to make the right solution is to perfect the collection of information and creating of information security tools. It is important to create the right selection techniques as well. Serious problem is that results of peer reviews are verified only after a long time period.
Peer reviews are carried out by groups of assessors. The group of assessors contains from three to five persons. One of them is the head of the group. It is possible to invite assessors from abroad too. The reserve of assessors is divided into two parts. The first part includes active researchers; the second one includes some politicians, businessmen, artists etc.
Main requirements to assessors selection system are these:
· Assessor can not be an employer of the institution to be assessed;
· At least one assessor from the active group of assessors reserve should
be included into the branch of science to be assessed;
· The group of assessors should contain some assessors from
different branches of science;
· One person active in social work and a person from the second part of
assessors reserve should participate in the activities of institution or
scientific collective peer review;
· Assessor have to know a foreign language if scientific work to be
assessed is created in a foreign language;
· Assessor should have a scientific degree if he assesses scientific
activities;
· In case activity of scientific area of subject or a study program is
assessed, at least one assessor have to be from this area of subject and a half
or more than a half of assessors have to be active in the same scientific or
study field;
· One assessor of the group has to be active in social work if general
scientific or higher education institutions assessment is performed;
· At least one assessor have to know requirements of professional
qualification if he assesses a study program, providing professional
qualifications.
· Only one assessor could be selected not from science or higher education
institution.
Additional requirements to assessors are these:
· Number of assessors with higher qualification should be limited in
groups (max. or min);
· Number of assessors from different regions should be limited (max. or
min);
· Number of assessors from different scientific branches and reserve parts
should be limited;
· Number of assessors from higher education and research institutions
should be limited.
Desirable requirements to assessors are these:
· Frequency of using assessors should be even;
· Assessors whose assessment is better evaluated should be used more
frequently.
Solution (a group of assessors) made by system can be omitted or filled up according to customer's wish. The group of assessors can be filed up according to desire of group or the head of the group. The group of assessors can invite an expert to solve specific problems.
During selection it is important to conform the demands and the interests of
an assessor and a customer. It depends on flexibility of the mechanism of an
assessors selection. This mechanism should be a system which allows to
evaluate:
· business skills;
· personal skills;
Business skills can be defined by commission according to system of points. However, this method needs a lot of experienced and competent people, which is expensive and in this case it is impossible to avoid subjectivity.
Method evaluating business skills according to specially prepared tests is also a very tedious work. This way of definition needs a lot of time to prepare and process results of tests, therefore, this way is better for sociological inquest. However it does not solve personnel management problems.
The selection and evaluating of assessors could be improved using professional-demographic methods, which allows to apply computers. This model would allow to describe and assess the experts themselves. The professional-demographic model (PDM) is a complex of indices such as personal indices (communication skills, reliability etc.) and business indices (scientific activities, pedagogical activities etc.) ones.
Figure 1. PDM in relation with management functions
Relations between the professional-demographic model and management functions could be examined in Figure 1.
Different characteristics of assessors are divided into two blocks which are business (professional) skills and personal characteristics.
The reserve of assessors is divided into parts. Assessors from different parts
should have different business skills. Persons (scientists) from the first part
of reserve should have the following business skills:
· Scientific activity (theoretical, experimental, organisational) should
be active;
· Scientific activity (theoretical, experimental, organisational) should
be resultative;
· Scientific activity (theoretical, experimental, organisational) should
be irreproachable from the point of view of ethic of science.
Persons (public persons) from the second part of reserve should have the
following business skills:
· Persons should be active in politics, culture, business, industry;
· Persons have to take an active interest to science and higher
education;
· Persons should take an active interest in production of science and
higher education;
· Persons should be acquainted with the foreign experience (was worked
abroad, have relationships with foreign firms and organisations).
It is a pity that the model (including PDM) does not exist in practice which would allow perfectly evaluate personal characteristics. It is an especially big problem to evaluate personal characteristics of a new assessor. However, it is possible to define.
The most important personal characteristics of assessors are the following:
1. Intellect:
· ability to observe, summarise, select and analyse facts;
· logical thinking;
· ability to generalise information.
2. Association and partnership:
· respect for different opinions, ability to hear out, patience;
· ability to make contacts easily;
· ability to win confidence and respect;
· politeness and good manners;
· oral and written communication skills;
· ability to motivate people and to create atmosphere of benevolence.
3. Intellectual and emotional maturity:
· stability of actions and behaviour;
· ability to resist from pressing and influences of other people;
· ability to make independent conclusions.
4. Personal initiative:
· self-confidence;
· creativity;
· initiativity.
5. Ethics and conscientiousness:
· desire to help people;
· exceptional conscientiousness;
· knowing his own competence limits;
· recognition of own mistakes.
Person, having all these skills could become an absolutely ideal assessor, but as ideal people do not exist in reality. These skills are preferable for aspiration. Evaluation of these characteristics is a big problem. It is possible evaluate them testing people, but this method is not as good as it is described above in text. Besides that, a freshman assessor can be without some of these characteristics, because part of these characteristics can be cultivated only in practice when working.
Characteristics can be evaluated according the objective indices, therefore PDM is not completely perfect for entirely ideal employers, because it is difficult to estimate all the indices. The creation of selection systems lasts very long, therefore, indices may be accreted by the approaching method. Part of the characteristics are evaluated by scale system. However, different characteristics have different points of importance. It is necessary to multiply points by coefficients of importance.
On the other hand, using PDM we can realise additional requirements, because it
is possible to evaluate the level of assessors activity. Three levels of
assessors activity could be defined:
1) not active enough (null points),
2) active (one point),
3) very active (two points).
Besides, it is possible to evaluate quality of the assessors work. We can define the quality level of made peer review
in the following way:
1) unsatisfactory (null points),
2) satisfactory ( one point),
3) good (two points),
4) excellent (three points).
It is possible to definite qualification of assessors according to these points. The system stimulating to do work of better quality, therefore, it is necessary to introduce assessors with this system. The system also makes management of peer reviews more effective.
After the estimation of all earlier discussed things it is necessary to input
the following data to the database of the reserve of assessors:
1) data about identification and education of persons,
2) data about their work experience,
3) data about their scientific activities during the last five years for the
first part of the reserve of assessors and data about their interests in
science and higher education for the second part of the reserve of assessors,
4) data about their activity as assessor,
5) data about the quality of peer reviews.
Once in two years assessors have to make corrections in personal data.
The system of assessors selection allows to create assessors groups or to select single assessors, because the computerised program fulfils realise all requirements for composing the group of assessors. This system allows to use the reserve of assessors the most favourably. In this case subjectivity decreases, because selection is performed according to objective indices and mainly automatically. Data about the assessors is stored in the database, therefore, it is possible to receive full and concentrated information about assessors any time. This system allows us to manage peer reviews in more efficient way because PDM is used.
The system of assessors selection is a part of information system for assessment of higher education and research institutions. This system is under implementation in the Lithuanian Centre for Quality Assessment in Higher Education. The administration of the database of the reserve of assessors is performed by the group of the systematisation and information.
It is obvious that finding a perfect model and techniques for the selection of the assessors is a hard task. However, PDM could serve as good model for creation of the computerised system of assessors selection. The main part of the system is a database, containing data about the assessors. The Lithuanian Centre for Quality Assessment in Higher Education implements this system in its work when selecting assessors for assessment of higher education and research institutions.
Copyright EUNIS 1997 Y.E.