(disponible en anglais uniquement)
On 8 September, The Pacific Islands Literacy and Numeracy Assessment (PILNA) trials will begin with several innovations being tested, including a new block design format that accounts for student fatigue, an improved student registration system and more digital support for administration.
Cook Islands’ PILNA coordinator Jane Taurarii says she will be particularly focused in the block format and adaptation for children with special needs. “I am interested in the outcome of how it will be carried out and as well as the results from it,” says Taurarii. “For instance, does the placing of the item in the book make a huge difference?” Taurarii was referring to PILNA’s use of four booklets. Each booklet has two blocks of items or questions that are arranged in different orders. It is envisaged that this block design will allow for a more accurate capture of student learning because it accounts for factors like student fatigue and time constraint. The fact that the booklets’ question blocks are organised in different orders also helps to discourage cheating.
Taurarii was among the 19 education officials from 14 Pacific island countries who were walked through the new PILNA procedures and innovations in workshops that began last week. All, except the Fiji workshop, were held virtually. Two representatives from EQAP’s technical partner, the Australian Council for Educational Research (ACER), were also part of the workshop. The final training session was conducted today (August 18) with education officers from the Republic of the Marshall Islands. The workshops are a necessary support for national coordinators as they prepare for the PILNA field tests next month. Their tasks include preparing the school coordinators and test supervisors to administer the assessments and questionnaires at their respective test schools.
The field trials mark the second part of the first year of administration for PILNA 2021. The trial is an important item because it is an opportunity to test any innovations to ensure that they fit well and will not compromise the main study. “We trial approximately twice as many (assessment) items as are required for the main study so we can pick only the best-performing items to include in the PILNA 2021 instruments next year,” said Dr Michelle Belisle, in her opening remarks at the workshops last week.
Dr Belisle, the director of the Pacific Community’s Educational Quality and Assessment Programme (EQAP), emphasised the importance of the trials in testing the questions and innovations. The preparation process began last December in a workshop to develop new assessment items. These e were refined and workshopped earlier this year to meet a standard of specifications. “The field trial is the final step in the development process where the items are packaged in test booklet format and students respond to them in a testing situation, just as they would in the main study,” said Dr Belisle. “The field trial results will confirm the difficulty levels of items so we can ensure the PILNA instruments have questions that range from fairly easy to quite difficult, allowing us to measure students across multiple levels.”
Fiji’s PILNA coordinator Elena Raboiliku said she was pleased learn how much had advanced in terms of preparation for the field trials. This, she said, bode well for the main study. More importantly, she said the value that every country gained from participating in the PILNA administrations went beyond the findings because it informed their practice. “(PILNA’s item writing skills component) has helped teachers to realise that literacy and numeracy skills need to be developed using a variety of teaching strategies,” said Ms Raboiliku. Taurarii was similarly grateful for the PILNA work. She said it helped them to develop stimulus and items. The coding skills learned from PILNA have also helped them to introduce coding into their national assessments. Taurarii also commended EQAP for the virtual support. She said this was an option they would explore to assist their outer island school coordinators and test supervisors.
The field trials will start on September 8 in Papua New Guinea and the Solomon Islands. Fiji and Tonga will hold their field trials on September 15 and 16, followed by trials in Tokelau and Kiribati. The final trials will be held in Vanuatu from October 20-21. Just like in the main study, the effectiveness of the PILNA field trials is largely dependent on the national coordinators. “It is through you (national coordinators) and your efforts that everything from sampling to budgets to distribution and collection of papers, coding, data entry and results sharing is carried out,” said Dr Belisle when opening the workshops. “Without the support of the NCs across the region, PILNA would not be possible.”
For more information about the PILNA trials please contact: Irene Manarae ([email protected])
Feature Photo: PILNA’s literacy lead Ratieli Kacilala shows Fiji’s national coordinator and education officials one of the four booklets to be used in the field trial next month. The Fiji workshop last week was the only training held in person.
