Analysis of Final Year Project Syllabi in Physical Education Teacher Education: Is The Assessment Formative?

Análisis de las guías docentes de los Trabajos Fin de Estudios en la formación del profesorado de Educación Física ¿es formativa su evaluación?

Introduction

Final Year Projects in Teacher Education: Undergraduate Degree and Master's Degree

In a context of model change in the European university education, in Spain, Royal Decree 1393/2007 established that university undergraduate and master's degrees would end with the elaboration and defence of a final year project (FYP). It was allocated between 6 and 30 credits, to be completed in the last degree stage (Vicario-Molina et al., 2020), and it should be oriented to the assessment of competencies associated with the studies. It should be further developed and regulated by every university (Rekalde Rodríguez, 2011).

Universities, based on the experience from technical degrees, specified the FYP's (final undergraduate degree projects and final master's degree projects) characteristics very differently (Zornoza Gallego, & Vercher Savall, 2020), which Rekalde Rodríguez (2011) called the second curriculum concreteness level.

It consists in a course whose aim is to allow the student to show the competencies acquired during their education and to evidence global learning outcomes (Rubio et al., 2018) integrating them in an original work, created autonomously, but under the supervision of a professor who guides and advises a production process (Pérez-García, 2021) where the focus does not lie on the results, but on how they have been reached by applying key competencies (Rekalde Rodríguez, 2011). Thus, FYPs acquire certifying importance, since the education body needs to know whether their students are ready to access the labour market in optimal conditions.

Change in Paradigm: Student-Centred Competencies and Teaching

The change in education paradigm introduced competencies, partially, to facilitate graduates' entry into the labour market (Garrote de Marcos, 2015). In fact, the final undergraduate degree project, together with the internship, make up 'the bridge between completed studies and the professional world' (Zornoza Gallego, & Vercher Savall, 2020, 122). Given this scenario, Palacios et al. (2019), after having analysed various theoretical proposals, defined competency as the ability to integrate, mobilise, combine and coordinate knowledge ('knowledge'), skills and abilities ('know-how'), and attitudes, values and rules ('know-how-to-be') in order to respond to a complex situation in a specific context.

General or cross-curricular competencies are considered to be the main competencies to be developed through FYPs (González & Wagenaar, 2010). Nonetheless, teaching practice has allowed us to confirm, in line with García Sanz and Martínez Clares (2012), that rather than integrating content, FYPs allow for the acquisition of new skills and competencies that are specifically associated with planning, elaborating and defending these projects. In the case of Physical Education pre-service teacher education (PSTE-PE), teaching competencies are essential. This is why several studies (Asún-Dieste et al., in press; Salcines et al., 2018; Palacio et al., in press) have considered general teaching competencies (included in all teaching profiles) and specific teaching competencies (specifically related to physical activity) to be specific competencies.

This unique course encounters multiples difficulties (Vicario-Molina et al., 2020), such as an insufficiently defined supervision process, the level of autonomy required, or students with educational shortcomings, e.g. in research techniques. Some of these challenges could be overcome through innovative or existing proposals that are not sufficiently spread across universities, like peer-supervision (Cieza García, 2011), student groups to exchange ideas and information, co-working spaces with shared material resources and educational activities, etc.

This change in paradigm means a qualitative shift towards a student-centred model, whose implementation is still ongoing in Spanish universities. A model focused on competency acquisition that forces to properly align the teaching process through the learning outcomes, the teaching methodology and the assessment system (Zornoza Gallego, & Vercher Savall, 2020), appropriately coordinated as a coherent system and captured in curricula that are visualised through every course's syllabus.

Contract among Teacher, Institution and Student: Syllabus

The first contact of a student with a university course is the syllabus, a public document that contains the basic content, organisational description, requirements, important dates, assessment system and marking criteria (Richmond et al., 2019).

Usually, FYP syllabi are created by the institution, rather than professors or departments (Jiménez-Jiménez et al., 2021), despite the latter being typically in charge of content-related aspect regulation. This is to ensure that all projects from the same centre or university are uniform, since this is a cross-curricular course. It is a contract subscribed between the institution and professors on one side and students on the other; therefore, it needs to be followed and to provide a clear and explicit framework.

Syllabi (Jiménez-Jiménez et al., 2021) can be considered to lie on a continuum, going from a regulatory design, containing mainly organisational aspects, to more content-centred syllabi, which describe and explain the learning process and provide details on assessment indicators and pedagogical principles. The second type would be in keeping with student-centred models (Richmond et al., 2019), which are definitely more consistent with the university model we are referring to.

Need for Element Explanation

If we consider the syllabus to be the contract that supports the teacher-student-course relationship, programme elements should be comprehensive and explicit enough.

Nevertheless, as stated by Lorente et al. (2013), syllabi are brief and they lack accurate enough information, features identified by Richmond et al. (2019) with non-student-centred models. This lack of detail is observed, for example, in the fact that the supervisor's role is hardly mentioned, while Vicario-Molina et al. (2020) found that students considered the relationship with them one of the key aspects to succeed in this course.

Syllabus Elements and Assessment System: Need for Alignment

The most common sections in a syllabus are: competencies, learning outcomes, methodology, learning activities and assessment system. In turn, the assessment system would contain assessment means, instruments, criteria and agents. According to Gallego et al. (2011), means is the evidence used to collect information about the subject under assessment (p. 96); and instruments are the actual and physical tools used to assess the learning highlighted through the assessment means (p. 97). Following Nunziatti (1990), criteria are rules or reference guidelines used to make a judgement or assessment; and agents are the people who assess. The analysis of syllabus design allows for checking the consistency among their elements, especially among competencies, goals, methodology and assessment. As stated by Lorente et al. (2013, 23), the analysis of syllabus components and, especially, assessment allows for identification of what kind of learning prevails and what is given greater importance within a course. Consequently, syllabus analysis is the most accurate aid for optimal decision-making.

It is, therefore, necessary to aim for consistency among syllabus components, based on the constructive alignment mentioned by Biggs (2005), where the set of puzzle pieces is more than just the sum of pieces and it effectively guides the teaching-learning process.

Formative Assessment

Of all programme elements mentioned, learning assessment is undoubtedly one of the most difficult aspects to implement in a model aiming for competency-based education (Villarroel, & Bruna, 2017, 121).

Within that puzzle, formative assessment, defined by Cullen and Harris (2009) as a means to analyse syllabus' student-centredness, is presented as an option that is clearly aligned with the current competency-based programming model.

Research on formative assessment has significantly developed in the past few years in Spain, with several research groups focused on its study and application (Cano, 2021).

This type of assessment was defined by Navarro and Jiménez (2021, 12) based on six core characteristics: (1) learning and teaching assessment integrated in the assessment system; (2) active student engagement in the assessment processes; (3) continuous and shared use of assessment instruments during teaching and learning processes; (4) two-way recurring communication between professor and students regarding the information collected through assessment activities, liable to a planning; (5) assessment criteria that correspond to the knowledge, skills and attitudes that students must jointly mobilise in a competency approach, and that are used as reference to design tasks demanding that type of integrated responses; and (6) assessments activities that are aligned with other teaching design elements (assessment and marking criteria, learning outcomes, competencies and teaching and learning activities).

It is well-known that assessing and informing the student on their assessment promotes learning. Hence, it is worth researching on formative assessment, an assessment for real learning.

We believe the institutional change still needs to be promoted and, in this content, it makes sense to analyse institution documents (in this case, syllabi) as common documents created by universities. In fact, according to Cullen and Harris (2009) and based on learning research, assessment will be the vehicle to change the instruction paradigm into a learner-centred one.

In light of all the above, the main aim of this research was to analyse FYP assessment in PSTE-PE. In particular, the aims were (1) to show an updated map of syllabi's assessment systems, and (2) to design and apply a procedure to analyse assessment systems' formative capacity. This procedure should allow education institutions and professors to assess FYP syllabi's quality.

Method

Design

A sequential mixed-method study (Johnson & Onwuegbuzie, 2004) was designed to analyse the FYP syllabi from Physical Education Teacher Education degrees in Spain. Assessment system elements and other aspects were quantified through document analysis. Subsequently, the analysis focused on the formative capacity of syllabi's assessment systems. An index was created thanks to a panel of experts, which allowed for quantification of assessment systems' formative capacity. Then a MANOVA biplot (Vicente, 1992) for two-way arrays based on multivariate general linear models was applied to graphically represent the elements and variables analysed and to establish and characterise groups or clusters with the degrees under study.

Population

The population was composed of the FYP syllabi of the Primary Education Teacher Education undergraduate degrees, with major in Physical Education, and the Secondary Education Teacher Education master's degrees, with major in Physical Education, of all Spanish universities that met the inclusion criteria. Considering that one quality indicator is related to universal and direct access to information, the inclusion criteria were: (1) FYP (TFG for undergraduate degrees and TFM for master's degrees) syllabi were fully accessible through the corresponding URLs (Rekalde Rodríguez, 2011); (2) one or two degrees with major in Physical Education (PE) were taught at this university; and (3) when a degree was taught in several centres of one university, regular centres would prevail over affiliated ones; if there were more than one regular centre, the one on the university main campus was chosen; finally, if needed, one centre was randomly selected. There are previous studies where syllabi were randomly selected (Zornoza & Vercher Savall, 2020), but also others, like the present one, that involved all syllabi.

Characteristics of Analysed Syllabi. Of the 88 Spanish universities existing in the academic year 2019-2020, 76 offered some type of teacher training degree (Table 1).

Of the 76 universities offering teacher training degrees, 63 offered Physical Education studies: 16 provided only (PE) Teacher Education undergraduate degrees, 11 only (PE) Teacher Education master's degrees and 36 offered both. Thus, in total, 99 syllabi or units of analysis were collected.

Three cases could not be analysed due to different reasons (incomplete, unclear or inaccessible data), so the final population was composed of 96 syllabi.

54.2% of the degrees were undergraduate degrees and 45.8% were master's degrees. The FYP (TFG+TFM) course was allocated 6 credits in 80.2% of the cases. Only 2% of them was below this figure. The highest number of credits was 12, contained in 5.2% of the syllabi analysed. As regards time distribution, the FYP lasted one semester in 72.9% of the degrees.

Instruments

Recording Sheet. A sheet was built to register the assessment system's elements and other FYP aspects. It was based on the protocol for syllabus analysis designed by Romero-Martín et al. (2020) but, given the particularities of the FYP, an ad-hoc document was created. Initially, syllabi's elements were identified on the basis of a specific literature review and a first syllabus review. Once the terms had been identified using both sources of information, they were reduction- and assimilation (synonym)-filtered by four researchers, who were experts in university teaching. Subsequently, a sheet was built to record the following syllabus elements:

1. General Aspects: degree, university, credits and period.

2. Assessment Means and Importance Given in the Mark: process, document (report/poster), presentation and discussion/defence.

3. Assessment Instruments: rubric, check-list, scale, report and others.

4. Assessment Criteria: originality, relevance, engagement, clarity of presentation. Accuracy, compliance with deadlines, formal correctness (oral, written), adequacy of information sources and others.

5. Agents: examination board, supervisor, students and others.

6. Others: topic selection system, supervisor and type of FYP; types of FYP; and type of guidelines provided to the student.

After syllabus analysis, the frequency of every element was quantified and adjusted, so that none of them was given more importance because of having more answering options. As a result, a value was obtained for every element in every syllabus.

Panel of Experts. In order to determine the formative capacity of a syllabus' assessment system, an index was calculated for each of their elements, based on the opinions of a group of experts in formative assessment. This qualitative technique was chosen since there were no historical data that could be used for reference in our analysis. Eight experts (which is within the 5-20 range recommended by Zartha [2014]) gave their opinion, reflected and reconsidered their opinion, taking into account their own and other experts' ideas (Varela Ruiz et al., 2013), before coming to an agreement. The process was divided into four steps:

1. Step 1. The experts were requested to sort the elements based on their importance (5 highest, 1 lowest) in order to detect or reveal their level of alignment with formative assessment, and to explain their responses.

2. Step 2. The research team collected the numerical order and explanations, and built a table that was then sent to the experts for a second round.

3. Step 3. A focus group was organised, where the experts, after having listened to others' ideas, exchanged opinions and agreed on which options (combined or not) were more or less formative within each element (means, instruments, criteria and agents), and assigned them a value between 5 and 1 (Table 2).

1. Step 4. Then the research team calculated the Formative Capacity Index for every element, which was the mean of the values provided by the experts (Table 3).

Every index multiplied by the value of every element in every syllabus yielded a formative capacity value for every element. The sum of all of them was the total value for the syllabus

Procedure

Once the syllabi were found and the aforementioned inclusion criteria were applied, seventeen reviewers were sent the syllabi's URLs, a randomly established list of syllabi to be assessed by every reviewer, the recording sheet and a detailed description of the analysis protocol. The information was recorded. Every one of the 96 syllabi was analysed by two reviewers independently, so a total of 192 assessments were conducted. Once the review was completed, a different group of three reviewers solved the discrepancies in the data.

Simultaneously, eight experts, under guidance of the study coordinator, followed the process described for the panel of experts in order to obtain the formative capacity index of the syllabi's assessment system elements.

Collaborators (eight experts and twenty syllabus reviewers) were selected based on the following criteria: (1) to have published in high-impact journals about formative assessment; (2) to have participated in national or international research projects; and/or (3) to belong to research groups related to this topic; in this case, they were all members of the network for formative and shared assessment in education (Red de Evaluación Formativa y Compartida en Educación, REFYCE); geographical diversity as regards universities of affiliation was sought: Barcelona, Lleida, Murcia, Catholic of Murcia, Valladolid and Zaragoza. Additionally, syllabus reviewers had experience using a similar recording instrument in a previous project.

Data Analysis

Two studies were conducted: (1) a descriptive study of the syllabi's characteristics, based on absolute frequencies; and (2) a study of the assessment systems' formative capacity. Two strategies were used for the second study: an HJ-biplot analysis (Galindo, 1985, 1986), with the aim to examine the relationships between variables, combined with a hierarchical cluster classification through Ward's method using HJ-biplot coordinates, that allowed for syllabus classification according to their formative capacity; and a MANOVA biplot analysis (Vicente, 1992), which graphically showed the differences among those clusters with regard to assessment systems' key elements: agents, instruments, means and criteria, distinguishing by university degree. In the graph corresponding to this second study, every circle represents one university degree cluster, where the centre is the mean value and the radius is the confidence level estimated by a univariate test. The assessment system variables are represented through vectors and the angles between them are directly proportional to the correlation between variables.

Results

Study I. Characterisation of Syllabus Assessment Systems

This section responds to the first aim, consisting in describing the syllabi's characteristics for an overview.

Selection System

In order to determine who chose the project topic, the supervisor or the type of FYP, nineteen combinations of the four options available (students, teachers, coordination team and others) were established.

The most frequent combination to choose the project topic was student and teacher (28.1%). Moreover, the student (alone or with other agents) was involved in this choice in 97.2% of the syllabi, and the teacher in 51% of them.

The type of FYP was also predominantly chosen by the combination of teacher and student (27.1%). The student was involved alone or with others in 61.5% of the syllabi, and the supervisor in 37.5% of the cases.

The supervisor was mostly chosen by the degree coordinator (21.9%). The student made or participated in the decision in 45.8% of the cases, while the teacher was involved in 15.6% of the cases.

Types of FYP

The most frequent types of FYP (Table 4) were: Innovation in education (72.9%), and Research projects on the topics proposed by teachers (70.8%), followed by Teaching interventions in real contexts (64.6%).

Guidelines for Elaboration

Guidelines to help students along the process were included in 95.8% of the syllabi analysed, either as references, appendices or others (Table 5). The most frequent was to have formal aspects (78.1%) defined, followed by the document's table of contents (70.8%) or explanations about the project and its structure, apart from other guidelines, as shown in Table 5. It is noteworthy that a report for the student was created in one third of the cases (32.3%), but it was only used when students did not pass.

FYP Assessment System

What assessment means were applied? The assessment means were stated in 95% of the syllabi. The categories found were: (1) Process; (2) Document (report or poster); (3) Presentation; and (4) Discussion and defence.

The most frequently used means was the presentation (84.4%), followed by the document (report or poster) with 81.2%. At considerable distance, we find the process with 33.3%, and discussion and defence with 15.6%.

Usually, more than one assessment means was applied, so in order to reflect syllabi's reality more accurately, ten profiles were established by combining the different options.

The most common combination of means was document plus presentation (41.7%), followed by these two plus the process (13.5%). The rest of means combinations presented very low percentages.

In formative assessment, it is essential to assess the process that students follow to elaborate their FYP; therefore, a more detailed analysis was conducted. The results revealed that the process was assessed in a higher percentage of TFG syllabi (38.5%) than of TFM syllabi (27.3%).

What mark percentage was assigned to every means? In order to show the percentage value assigned to every means in the reviewed syllabi, they were divided into four quartiles: 1-25%; 26-50%; 51-75% and 76-100%.

Only 2.1% of the one third of the syllabi that used the process followed by the student, assigned more than 50% of the mark to this means.

The presentation was the most frequently used means (84.4% of the syllabi), and in more than two thirds of the occasions it meant between 1 and 50% of the mark.

The document was used in 77.1% of the instances, and in 52.1% of them it was assigned more than 50% of the mark.

Lastly, the discussion/defence was not explicitly taken into account in 84.4% of the cases, and in the rest it belonged to the lowest quartile.

What assessing agents were involved? Students only participated in 2% of the syllabi analysed (1% in the process and 1% in the document), always in collaboration with the supervisor.

The examination board primarily participated in the presentation (77.1%) and, in one third of the syllabi, in the document and in the discussion and defence, which were hardly ever marked. The supervisor predominantly assessed the document (40.6%) and they were almost the only ones to assess the process (30.2%). The presentation was almost exclusively assessed by the examination board (77.1%); only 6.3% was assessed by the supervisor.

In the one third of the syllabi in which the process was assessed, the supervisor was the assessing agent in most of the cases.

Assessment Criteria. Eight assessment criteria were extracted from the first document and literature analysis, plus some others that were minimally present. The subsequent analysis revealed that the most frequently used assessment criterion, considering all instruments it was applied to, was formal correctness, especially for documents (67.7% of the syllabi) and presentations (51%) assessment. The second one was clarity of presentation, especially in presentations (63.5%) and documents (40.6%). The second last place corresponded to relevance, used in 27.4% of the syllabi as a criterion to be considered in the report. Finally, engagement was the least commonly applied criterion, being present in the process in 27.7% of the syllabi and in the report in 17.7% of them.

Assessment Instruments. With regard to the instruments used for assessment, reports (14.6%) and rubrics (12.5%) were primarily used for the process. The same occurred for the document (reports: 33.4%; rubrics: 29.2%) and the presentation, with a slightly higher percentage of rubric use in the latter case. The document was the assessment means for which the largest number of instruments was described. Scales and check-lists were the least frequently used.

Study II. Analysis of the Formative Capacity of Syllabus Assessment Systems

The multivariate HJ-biplot analysis allowed us to organise the syllabi into three large groups (clusters) based on their assessment system's formative capacity. In Figure 1, from left to right, we can see a first group (cluster 2) made of syllabi with high formative capacity regarding means and agent but, in general, low capacity as regards instruments and criteria.

By contrast, the second group (cluster 1) contained syllabi with higher formative capacity in these last two aspects but lower in the first two (means and agent). Lastly, the third group (cluster 3) included those syllabi with the lowest formative capacity, especially regarding means and agent. When analysing the MANOVA biplot (Figure 2) more in depth considering the different academic levels (undergraduate degree and master's degree), differences among the three clusters can be observed in the variable agent, which lay between clusters 1 and 2.

It was clearly shown that, from a multivariate perspective, syllabi separated from each other when they were projected on the directions where the differences between groups were largest. Thus, the syllabi in the first cluster presented high formative capacity regarding instruments and criteria, and moderate formative capacity as regards agent (statistically similar to the second cluster). The syllabi in the second cluster showed high formative capacity with regard to means, while it was moderate as regards agent. By contrast, the syllabi in the third cluster presented the lowest formative capacity in all aspects analysed. When analysed on an individual basis, the degrees of 38.7% of the universities lay in clusters 1 and 2, which were considered to have the highest formative capacity. In universities offering two degrees, a unique pattern in the way of programming was not found: (1) in 35.3% of them, both degrees' syllabi lay in the same cluster; and (2) in the remaining 64.7% of universities, the TFG and TFM syllabi lay in different clusters.

Discussion

This section will discuss the results of the present study related to the two aims proposed: (1) to characterise the assessment systems contained in Physical Education Teacher Education syllabi in Spain, and (2) to analyse the formative capacity of those assessment systems in order to determine their degree of alignment with the current university competency-based educational model in the European Union.

I. Characterisation of Syllabus Assessment Systems

From a general perspective, it was observed that FYPs are not designed to combine knowledge from all courses, but rather to bring into action new general or cross-curricular competencies (Vicario-Molina et al., 2020), in line with Rekalde Rodríguez (2011), who considered general or cross-curricular competencies to be the most frequently required ones. Relevant aspects of assessment systems as a whole and, especially, of their components will be discussed below.

Syllabi's Level of Detail

First of all, the lack of relevant information in the syllabi can be highlighted, since the majority of them do not inform students sufficiently about relevant aspects before enrolment. This affects the degree quality and hinders the creation of a 'genuine assessment culture' (Pérez-Juste, 2000, 266) that would allow for syllabus improvement. As it happened in the study by Lorente et al. (2013) including syllabi prior to the EHEA, the assessment systems lack accurate assessment indicators. The authors claimed a more detailed assessment; however, we have seen that the situation has not substantially changed in the new syllabi. It is very likely that more specifications are included in internal documents, but these can only be accessed by students after enrolment. Syllabi contain public information and should represent the agreement established between the institution and the teaching staff on one hand and the students on the other (Songel & Guimaraens, 2012). One reason may be the fear that the document is too long. Nevertheless, student-centred syllabi are generally longer (Palmer et al., 2016), so this should not become a concern unless it is not just a formal aspect.

Assessing Agents

The participation of various assessing agents enriches the system (Herrero González et al., 2020). In the FYP, no assessment activities were found where the student was involved, or they had minimum relevance. Besides, shared assessment appeared but with very limited presence, and other assessment topics that are aligned with the current educational model, such as dialogued or negotiated assessment, were not detected in the syllabi. It must also be borne in mind that examination boards are composed of teachers who have not followed the student's process, so it is difficult for them to assess their development. As stated by Zornoza Gallego and Vercher Savall (2020, 26), assessment challenges are also related to the examination board's role, since their members have not been present during the project elaboration and can hardly assess general competencies associated with the FYP elaboration process.

Autonomy and Self-Regulation

An element that has turned out of key importance in pre-service education is student's self-regulation capacity related with their autonomy (Cano, 2021). Our analysis revealed that the student usually participated in choosing the project topic; in slightly less than half of the degrees they were involved in choosing the supervisor, and in one third of the cases they were involved in choosing the type of project, the nature of these three aspects being predominantly organisational or formal. Nonetheless, as it has been previously mentioned, student's participation in the assessment was minimum. This implies that syllabi do not show a student-centred context, at least when it comes to important decision making. Furthermore, no detailed information was found in the syllabi about how to programme the necessary process to achieve autonomy, apart from the description of a few organisational aspects such as some guidelines or materials available for document elaboration or other instruments. Therefore, autonomy or self-regulation seems to be a cross-curricular competency that is left to chance, although Zabalza Beraza (2012) argued that cross-curricular competencies should not only be used, but also taught and, therefore, included in syllabi.

Assessment Instruments and Means

Assessment means were mostly common to all syllabi. The main pattern is a written academic document, usually a report, sometimes a poster, which is orally presented to an examination board, with whom it is discussed after the presentation. However, this discussion was only mentioned in a little percentage of syllabi.

This is a result- or product-centred model (Rekalde Rodríguez, 2011; Reyes-García & Díaz-Megolla, 2017). Only one third of the syllabi included process assessment and none of them exclusively assessed the process; of those where it was assessed, only a tiny percentage assigned it more than half of the mark.

Under such conditions, FYP syllabi's formative capacity is very limited. On one hand, the discussion and opinion exchange with the student is insufficiently mentioned. Besides, student's participation in their own process assessment is very low. On the other hand, continuous feedback provision to guide learning is not included, although plenty of studies have pointed out its direct relationship with learning (Cano, 2021). Consequently, little importance is given to the process followed by the student. All these issues should be solved, since the point of the TFG is not the research outcomes, but how they have been reached by applying key competencies from the undergraduate degree (Rekalde Rodríguez, 2011, 190).

Assessment Criteria

As regards the assessment criteria, the limited importance given to student's engagement is noteworthy. This criterion is associated with a proactive attitude towards tasks and the way of engaging in the process. In the present study, it yielded the lowest value of all criteria.

It is especially remarkable when compared with other criteria of formal nature, such as formal correctness or clarity of presentation, which were substantially more frequent. It must also be noted that document's formal correctness was considered more important than relevance and even originality, which are more content-related rather than formal aspects. In agreement with Rekalde (2011), syllabi did not frequently include specific indicators or descriptions accompanying them, for students to know how to use them to guide their learning.

In short, formal aspects are given more importance than content or student's engagement.

Assessment Instruments and Guidelines for TFM Elaboration

The most frequently used assessment instruments (Rodríguez, & Ibarra, 2011) are reports and rubrics. Only one third of the syllabi mentioned some kind of report that was provided to the student and, if used, it was only in case of fail. Consequently, in the majority of cases, the opportunity of providing feedback to the student on their FYP is missed; firstly, because no reports are provided during the process in two thirds of the syllabi and, secondly, because they are not provided at the end either, since the report is linked to a negative final outcome.

In addition, the use of rubrics can guide the student on how to elaborate their project and which criteria will be applied in the assessment (Rekalde Rodríguez, 2011). Nevertheless, according to the present study, this assessment instrument is described, at most, in one third of the syllabi for the document and in one fourth of them for the presentation, although we are unable to know whether they were previously known by students in all cases, or if they were negotiated or agreed on with them.

II. Analysis of the Formative Capacity of Syllabus Assessment Systems

An in-depth study was conducted on the formative capacity of syllabus assessment systems, which allowed for determination of three syllabus groups or clusters. The first one was characterised by high formative capacity regarding instruments and criteria, and moderate as regards agent; the second cluster showed high formative capacity with regard to means and also moderate as regards agent; and the third one presented low formative capacity.

After the study, the existence of syllabi with low formative power, at least in the assessment system, is deemed to devalue a degree, considering the university education paradigm we have been referring to, where the different syllabus elements must be aligned (Biggs, 2005) and must be consistent with student- and learning-centred models. Therefore, the syllabi belonging to the third cluster have large room for improvement in order to bring their educational intervention closer to the current paradigms.

As it was seen in the results, the vector representing the parameter agent lay between clusters 1 and 2, revealing an influence on both of them. This vector was also considerably long, showing a high value, which could indicate either the participation of several agents in the assessment or the presence of the agent student, which, despite its limited presence, was considered to be the option with the highest formative capacity. Consequently, we must interpret that the presence of this component has a positive impact on the formative capacity of both clusters' syllabi. Of the four assessment system elements (variables), the agent is possibly the most stable one, since the prevalence of the report and presentation instruments implies a rather stereotyped agent participation: the supervisor for the report and the examination board for the presentation.

Two main patterns were identified regarding the way universities offering the two degrees designed their syllabi: in one third, both syllabi belonged to the same cluster, while in the rest they lay in different clusters. This reveals that, in the first case, a common criterion was used to design the syllabi for the undergraduate and the master's degrees, while in the second, universities designed FYP syllabi differently depending on the degree. As it was seen in Figure 2, cluster 1 received its high formative value from the strong presence of instruments, either because many were applied or because the one with the best score was used, i.e. the assessment report. Cluster 2 was characterised by the strong presence of means, but also influenced by agents. In any case, it must be admitted that the variable instrument provides a strong functional connotation, in contrast to, for example, the criteria that reveal a specific educational model, which can either be oriented to student-centred paradigms or not.

In summary, there is a group of syllabi with high formative capacity, defined by instruments and criteria and influenced by agents; a second group strongly influenced by means and with participation of agents; and a third group with limited formative capacity.

Definitely, this graphic and interpreting procedure allows us to provide a full image of the study, revealing a global and particular syllabus view.

Conclusions

The first study described the characteristics of the assessment systems of FYP syllabi in PSTE-PE. It allowed us to conclude that there is still a lot of work to be done in order to improve the information provided by syllabi with regard to explaining certain aspects, such as assessment indicators. These would allow for clarification of what is expected from the student or which criteria are considered in their assessment, if we really want to understand syllabi as the contract with students that many authors refer to. Furthermore, more explicit syllabi would be closer to student-centred educational models. In the second study, evidence from the descriptive analysis allowed us to conclude that there is still a long way to go until assessment-related teaching actions acquire actual formative capacity that is strongly aligned with the competency-based programming model.

Furthermore, this study proposes a procedure to analyse syllabus quality in university education that allows us to understand to what extent syllabi are in line with the educational principles that define the university formative model. This is a modern competency-based formative model, focused on the achievement of provable quality standards, which the European community has supported since the beginning of probably the most important reform university has ever undergone, especially regarding programming.

This analysis model is useful to analyse syllabus quality both at the institutional level and in more specific contexts, such as degree quality, assessment and certification commissions. Syllabi should present the information clearly and, thanks to more descriptive analyses of causes and consequences and the participation of the different education agents, they should facilitate decision making for teaching and management teams.

Lastly, the major limitations of this study were the inability to analyse all syllabi due to restricted access in some universities, the difficulty to access the internal documents that universities provide students with after enrolment (although it was already assumed from the design phase), and the fact that the study only involved PSTE-PE degrees. All these are challenges that the research team will try to address and overcome in future studies.

Funding

Grant RTI2018-093292-B-I00 funded by MCIN/AEI/ 10.13039/501100011033 and, by “ERDF A way of making Europe”.

References

Author notes

* Correspondence: Mª Rosario Romero Martín, rromero@unizar.es

Additional information

How to cite this article: Romero-Martín, M. R., Caballero-Julia, D., Ruiz-Lara, E., & Ureña-Ortín, N. (2023) Analysis of Final Year Project Syllabi in Physical Education Teacher Education: Is The Assessment Formative? Cultura, Ciencia y Deporte, 18(55). https://doi.org/10.12800/ccd.v18i55.1955

Cultura, Ciencia y Deporte
ISSN: 1696-5043
Vol. 18
Num. 55
Año. 2023

Analysis of Final Year Project Syllabi in Physical Education Teacher Education: Is The Assessment Formative?

Mª RosarioDanielEncarnaciónNuria Romero-MartínCaballero-JuliaRuiz-LaraUreña-Ortín
University of ZaragozaUniversity of SalamancaCatholic University of MurciaUniversity of Murcia,SpainSpainSpainSpain