PISA - Models and the reality
Conference Proceedings - The Future of Education
The Programme for International Student Assessment – PISA – is the most ambitious endeavour of large-scale education systems evaluation ever implemented. The Organization for Economic Cooperation and Development – OECD – launched this exercise for the first time in 2000, and in the 2012 edition 65 education systems were assessed. According to OECD, the programme “[…] is a triennial international survey which aims to evaluate education systems worldwide by testing skills and knowledge of 15-year-old students.” And, “[…] tests are designed to assess to what extent students at the end of compulsory education, can apply their knowledge to real-life situations and be equipped for full participation in society.” Albeit being a prestigious programme, entrenched in sound theoretical grounds, and notwithstanding all the efforts made by PISA experts to mitigate shortcomings, the PISA is not exempt from criticisms of various kinds. When analysing the quotes mentioned above, and taking into consideration the applied methodologies, several questions can be raised and some concerns should be pointed out. The first question arising in the process of evaluation is that any measurement always affects, direct or indirectly, the system itself, disturbing its inner workings. This fact is particularly relevant when social systems are at stake. A second difficulty results when students from very different countries in what regards culture, tradition, and beliefs are subjected to the same test. Although all items are always carefully analysed by panels of experts in order to detect cultural bias or offending interpretations, there is no complete guarantee that the final set of items is adequate to evaluate all students. Another question regarding the fairness of PISA results is the fact that a paper-and-pencil (or computer) test, limited to three disciplinary domains, cannot encompass the possibly rich, diverse, and unsuspected knowledge and skills of 15-year-old students. There are also technical criticisms regarding the adopted approaches and methodologies, from the utilization of the Rasch model to negative remarks about the way data are collected and questions are coded. Some of what could be considered advantages of PISA – the literacy based instead of a curriculum based approach, the assessment of 15-year-old students instead of a particular school year pupils, and the definition of a large set of indicators, as is the case of ESCS – have been also severely criticised. Finally, some of the criticisms reside, not in the PISA methods and characteristics themselves but on an excessive focus on country rankings, primarily promoted by media, and consequently followed by political leaders. The main objective of this research is to reframe difficulties and artefacts together with virtuous results of PISA, putting in perspective praises and criticisms to foster a better understanding of this important programme.
Esta comunicação está associada ao registo seguinte: