Rethinking Assessment: From Measuring to Promoting Critical Competencies

Published: 18 September 2024| Version 1 | DOI: 10.17632/j9v9dwg5vj.1
Contributors:
Carlos Brito,

Description

The main theme-problem addressed in the technical report is related to university professors' social representations of assessment in higher education. The investigation seeks to understand how these professors interpret and apply the concept of assessment in their pedagogical practices. This issue is crucial because assessment is viewed not merely as a tool for measuring performance but as a dynamic process involving the construction of meanings and the development of competencies for both teachers and students. The general objective of the study is to investigate how university professors perceive and structure their assessment practices. To achieve this, the research aims to: (1) identify the central and peripheral elements in the social representations of professors regarding assessment; (2) analyze variations in perceptions among professors from different areas of knowledge; and (3) discuss the implications of these perceptions for pedagogical practice and teacher training. Regarding data replicability, the technique used was the Free Word Association Technique (TALP), which allows data collection by having participants list words associated with the central theme. This method is highly replicable and can be applied in various academic and institutional contexts, as long as the focus remains on the social representations of specific groups. The data analysis was facilitated by the IRaMuTeQ software, a robust and freely accessible tool, which enabled the prototypical analysis of social representations. Therefore, the study can be replicated in other higher education institutions, or even at other educational levels, provided that the central theme is adjusted to the audience and context in question. The analysis of the results was conducted through Prototypical Analysis, identifying the most frequently evoked words and terms related to the concept of assessment. The analysis was divided into quadrants, highlighting the Central Core, Peripheral Elements, and Contrast Zones of social representations. This approach offered an in-depth understanding of the more stable elements and those more susceptible to contextual variations, making it applicable in other settings where assessment practices are studied.

Files

Steps to reproduce

Here is the step-by-step guide for applying the technical and methodological procedures: Define the Central Theme and Target Population: Start by clearly defining the central theme of the study (e.g., "Assessment in Higher Education"). Identify and select the target population (e.g., university professors) that will participate in the study. Sample Selection: Choose a convenience sample from the target population. The size of the sample can vary depending on the context, but it should be large enough to capture a diverse range of perspectives. In the case of this study, 68 professors from different fields of knowledge were selected. Data Collection – Free Word Association Technique (TALP): Apply the Free Word Association Technique (TALP) to gather data. Ask participants to list five words that come to mind when they think of the central theme (e.g., "Assessment"). After listing the words, have them rank the words in order of importance (from 1 to 5) and provide a justification for each choice. Data Preparation: Once all data is collected, organize the words and justifications into a database. Each participant’s responses should be recorded separately. Ensure that the data is cleaned and formatted for analysis, removing duplicates and standardizing terms where necessary (e.g., singular vs. plural forms). Analysis using IRaMuTeQ Software: Use the IRaMuTeQ software to conduct a Prototypical Analysis. This analysis will categorize the words into four quadrants based on frequency and order of evocation: Central Core: High frequency and low order of evocation (immediately recalled). First Periphery: High frequency but evoked later. Contrast Zone: Low frequency and low order of evocation (specific to subgroups). Second Periphery: Low frequency and high order of evocation. Interpretation of the Results: Analyze the quadrants generated by the software, focusing on the Central Core for the most significant terms and the Peripheral Zones for context-specific variations. This will reveal the shared social representations of the target population regarding the central theme. Report Findings and Discussion: Present the findings, discussing the implications of the Central Core and Peripheral Elements. Relate these findings to the broader theoretical framework (e.g., Social Representations Theory) and explore their potential for informing future pedagogical practices or institutional policies. This methodological procedure can be replicated in other contexts by adjusting the central theme and target population.

Institutions

Universidade Municipal de Sao Caetano do Sul

Categories

Education, Higher Education, Higher Education Teacher, Feedback, Learning Sciences

Licence