The Government of Georgia in June 2019 has signed Loan Agreement for financing the implementation of Innovation, Inclusion and Quality Project (I2Q Project) with the World Bank[1]. The Project development objectives are to (i) expand access to preschool education; and (ii) improve the quality of education and the learning environments.
Project – Assessment for the Development is under I2Q project component 2, Subcomponent 2.4 – Development of a National Assessment Framework. It aims to implement census-based e-assessment (fully online mode) for the public schools of Georgia. The goal of assessment is to measure students’ achievements towards the national curriculum and track their progress. About 150 000 students of 4th, 6th and 10th grades will be assessed in literacy and numeracy every year. Students, their teachers, parents, corresponding schools and educational authorities will receive the automated e-reports on the assessment results, in order to support data-driven decision-making processes on individual, classroom, school and national level.
The initial phase of the project has already started in 2019 and draft framework for the assessment has been elaborated, as well as e-tests in literacy and numeracy for 4th and 6th grades. Two pilot phases have been planned for 2020 in order to try-out the content of the tests, to trial the platform and develop business processes for the e-assessment.
NAEC already has hired the staff, exclusively for the Assessment for Development project: project manager, project assistant, TAO administrators, system administrator, developer, literacy and numeracy experts.
The first pilot took place in February 2020. 6 000 students from 59 schools participated in the pilot in order to try-out the items and online delivery platform.
The items were created and assembled in the tests:
- 5 tests for the numeracy in 6th grade. Each test comprised 20 items. The items were developed based on the national curriculum of the 6th grade math;
- 4 tests were developed for the 4th grade numeracy, with 25 items each. The items were developed based on the national curriculum of the 4th grade math.
- 10 texts, with 14 items in each, were developed to test literacy skills in the 4th grade.
- 10 texts were developed to assess 6th grade literacy skills, each text included 8-9 items to assess reading and writing skills for the 6th grade, as defined by the national curriculum of Georgian language and literature.
Students’ responses were analyzed and the items were edited based on the psychometric analysis. The set of the tests were assembled for the second phase of the pilot.
Project team analyzed existing test authoring and delivery platforms and selected OAT (Open Assessment Technologies) TAOTesting open Source eassesement software (TAO). TAO platform enables to create technology-enhanced items (TEIs) and ensures complete content interoperability. It offers: item authoring, test creation, scheduling and delivering tests online on a nation-wide scale, managing test-takers and user access, collecting and saving students responses, automatically grading candidates closed-question responses and exporting the data for further use.
Community edition of TAO platform is installed on the local Education Management Information System (EMIS) servers. EMIS and NAEC provide the hosting and delivery services for the platform.
NAEC became the member of international FLIP+ E-assessment Community in 2019 to discuss and share challenges and success stories of e-assessment project. As a community member, NAEC can share the experiences in e-assessment content development, as well as in TAO open source development by composing extensions for the platform (e.g. portable computer interaction for the development of different interactive technology-enhanced items, reporting features for the platform, offline version for test delivery and etc.). NAEC can work on certain extensions for the platform in cooperation with other countries, thus sharing the budget for the development, or exchange already developed extensions free of charge.
Version 1.0 of e-reporting and e-scoring modules have been developed and integrated with TAOtesting platform. E-reporting module generates the reports for student, teacher and school based on the data collected from the student responses. E-scoring module enables the human scorer to score the student responses collected from TAO testing platform. Both modules are ready for the trial in the 2nd pilot of the project.
The second pilot is planned in autumn 2020, involving 300 public schools of Georgia. It aims to try-out the tests (instead of trying-out the items in the first pilot), to test human scoring and e-reporting system and its compliance with Tao platform.
National Assessment and Examinations Center 