Projectbeschrijving D-PAC
Development, validation and effects of a digital platform for the assessment of competences (D-PAC)
Project Leader:
prof. dr. Sven De Maeyer, EduBROn, University of Antwerp
Co-Promotors:
prof. dr. Vincent Donche, EduBROn, University of Antwerp
prof. dr. Jan Vanhoof, EduBROn, University of Antwerp
prof. dr. Peter Van Petegem, EduBROn, University of Antwerp
dr. Tanguy Coenen, iMinds iLab.o, iMinds
prof. dr. Pieter Ballon, iMinds-SMIT, iMinds
prof. dr. Peter Vlerick, Dept. of Personnel Management, Work and Organisational Psychology, Ghent University
Project summary
Valorisation objectives
The main objective of the D-PAC project is to valorise an innovative approach (adaptive comparative judgement, ACJ) to assess competences in schools and other work settings in a more user-friendly, credible and efficient way. The project aims at (1) raising awareness regarding assessing competences in a credible fashion, (2) the development of a user-friendly, credible and efficient tool facilitating the assessment of competences by implementing the ACJ methodology and (3) amplify the impact of the feedback from assessment of competences on assessees and organisations.
The D-PAC tool will enable educational and social profit organisations (amongst others) to:
- be successful in a more user-friendly, credible and efficient assessment of competences;
- provide systematic, optimised feedback reports on crucial competences;
- take informed and credible decisions regarding important evaluation procedures;
- realise an evaluation culture in which assessment to enhance learning is important.
The objectives of the D-PAC project will be reached through a repeating process of developing, implementing and testing of the innovative approach for performance assessment in a variety of work contexts (e.g. educational and social profit organisations). This will result in the first place in the D-PAC tool: an open source, on-line available and modular (input, judgment and feedback modules) tool for performance assessment.
Scientific objectives
The main scientific aim of this project is to produce ground breaking research results on how a wide range of competences can be more credibly assessed by using a new adaptive comparative judgment technique which will be integrated into a digital platform for assessment of competences. In the project proposal 12 scientific objectives are distinguished which are part of three overarching scientific research objectives:
To provide evidence on the added value of adaptive comparative judgment in contrast to other traditional assessment procedures (e.g. filling in a rubric or completion of a multiple choice test which has often found to produce more unreliable and invalid results regarding complex competences);
To provide insight on the conditions, mechanisms and effects of the use of D-PAC in schools and organisations on both the individual and the organisational level;
To provide evidence on the added value of developing and evaluating performance assessment tools in the “real world” by using an innovative design methodology that integrates design research, action research and the Living Labs methodology.
Besides the use of literature studies and multiple case studies, quantitative and qualitative data gathering in this project will take place alongside four experiments.
The present vacancy focusses on the first scientific research objective, in the domain of psychometric research. More specifically, the research will focus on the decision-making of assessors. As such, qualitative data is needed to answer research questions on the validity of the scoring in the several conditions. Inductive analyses of the verbalisations will give insight in the key characteristics assessors take into account when they come to decisions about representations.