Offline Digital Assessment in Aviation Vocational Education: Architecture, Continuity, and Human Oversight

Yew Yun Xin; Laila Rochmawati; Harunur Rosyid

doi:10.36456/b9sk2k62

Authors

Yew Yun Xin Institut Pendidikan Guru Kampus Bahasa Antarabangsa https://orcid.org/0009-0006-9872-5617
Laila Rochmawati Politeknik Penerbangan Surabaya https://orcid.org/0000-0001-5502-6070
Harunur Rosyid Universitas Muhammadiyah Gresik https://orcid.org/0009-0004-6681-2340

DOI:

https://doi.org/10.36456/b9sk2k62

Keywords:

assessment continuity, aviation vocational education, digital assessment, human oversight, offline-first systems

Abstract

Offline digital assessment can support the continuity and traceability of evidence in vocational education, although technical functionality alone cannot establish assessment validity. This artifact-centered study analyzed ToeicApp v2.0, a local TOEIC-oriented English assessment application in an aviation vocational education setting. A documented technical audit of the executable environment, SQLite database, content resources, media references, session-state functions, and administrative records provided the evidence base. Operational requirements were mapped to implemented functions and separated from pedagogical decisions requiring instructor interpretation. The audited application used .NET Framework 4.7.2 and SQLite and included five packages, 232 question groups, 999 items, 999 answer-key entries, 154 audio paths, and 50 image paths. Timer, autosave, resume, history, and import/export functions showed how local delivery, response persistence, and record access were organized. The findings support a bounded architectural model combining local operability, assessment-state continuity, evidence traceability, and human oversight. The audit did not test recovery reliability under controlled failures, psychometric properties, official TOEIC equivalence, or learning gains. The study offers design considerations for assessment infrastructure in connectivity-constrained vocational settings.

References

Ali, A. D. (2024). Exploring language assessment literacy: A case of perceived needs of two stakeholder groups in Egypt. *International Journal of Training and Development, 28*(3), 335–361. https://doi.org/10.1111/ijtd.12328

Babik, D., Gehringer, E., Kidd, J., Sunday, K., Tinapple, D., & Gilbert, S. (2024). A systematic review of educational online peer-review and assessment systems: Charting the landscape. *Educational Technology Research and Development, 72*(3), 1653–1689. https://doi.org/10.1007/s11423-024-10349-x

Bennett, R. E. (2015). The changing nature of educational assessment. *Review of Research in Education, 39*(1), 370–407. https://doi.org/10.3102/0091732X14554179

Casanova, D., Alsop, G., & Huet, I. (2021). Giving away some of their powers! Towards learner agency in digital assessment and feedback. *Research and Practice in Technology Enhanced Learning, 16*, Article 20. https://doi.org/10.1186/s41039-021-00168-6

Demirdöken, G. (2024). Artificial intelligence in Aviation English testing. *The Literacy Trek, 10*(3), 362–384. https://doi.org/10.47216/literacytrek.1556603

Demirdöken, G., & Atay, D. (2024). Enhancing Aviation English competency: A simulation-based approach for aspiring pilots. *English for Specific Purposes, 76*, 106–121. https://doi.org/10.1016/j.esp.2024.08.001

Douglas, D., & Hegelheimer, V. (2007). Assessing language using computer technology. *Annual Review of Applied Linguistics, 27*, 115–132. https://doi.org/10.1017/S0267190508070062

Duarte, B., da Costa Ferro, M. R., Zarouk, M. Y., Pedro da Silva, A., Martins, M., & Paraguaçu, F. (2025). ALEX (active learning experience): A decentralized, offline-first, student-centered LMS. *International Journal of Information and Communication Technology Education, 21*(1), 1–27. https://doi.org/10.4018/IJICTE.386526

Fajardo-Ramos, D. C., Chiappe, A., & Mella-Norambuena, J. (2025). Human-in-the-loop assessment with AI: Implications for teacher education in Ibero-American universities. *Frontiers in Education, 10*, Article 1710992. https://doi.org/10.3389/feduc.2025.1710992

Fuller, R., Goddard, V. C. T., Nadarajah, V. D., Treasure-Jones, T., Yeates, P., Scott, K., Webb, A., Valter, K., & Pyörälä, E. (2022). Technology enhanced assessment: Ottawa consensus statement and recommendations. *Medical Teacher, 44*(8), 836–850. https://doi.org/10.1080/0142159X.2022.2083489

Grosseck, G., Bran, R. A., & Tiru, L. G. (2024). Digital assessment: A survey of Romanian higher education teachers’ practices and needs. *Education Sciences, 14*(1), Article 32. https://doi.org/10.3390/educsci14010032

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. *MIS Quarterly, 28*(1), 75–105. https://doi.org/10.2307/25148625

Hidri, S., Alblooshi, H., Khemakhem, S., & Kala, T. (2026). Validating a four-factor model of teachers’ online assessment practices in the UAE tertiary education. *Frontiers in Education, 11*, Article 1830811. https://doi.org/10.3389/feduc.2026.1830811

Jacobsen, M., & McKenney, S. (2024). Educational design research: Grappling with methodological fit. *Educational Technology Research and Development, 72*, 2743–2762. https://doi.org/10.1007/s11423-023-10282-5

Jamieson, J. (2005). Trends in computer-based second language assessment. *Annual Review of Applied Linguistics, 25*, 228–242. https://doi.org/10.1017/S0267190505000127

Kane, M. T. (2013). Validating the interpretations and uses of test scores. *Journal of Educational Measurement, 50*(1), 1–73. https://doi.org/10.1111/jedm.12000

Kaya-Capocci, S., O’Leary, M., & Costello, E. (2022). Towards a framework to support the implementation of digital formative assessment in higher education. *Education Sciences, 12*(11), Article 823. https://doi.org/10.3390/educsci12110823

Khan, W., Topham, L., Jones, N., Atherton, P., Al-Shabandar, R., Kolivand, H., Khan, I., Alatrany, A., & Hussain, A. (2026). Auto-assessment of assessment: A human-in-the-loop AI framework addressing policy gaps in academic assessment. *PLOS ONE, 21*(4), Article e0346815. https://doi.org/10.1371/journal.pone.0346815

Lei, W., & Lei, Z. (2026). Formative assessment literacy: A systematic review. *Language Testing in Asia, 16*, Article 7. https://doi.org/10.1186/s40468-025-00418-0

Ma, B., Krötz, M., Deutscher, V., & Winther, E. (2025). Same same yet different: Equivalence of the multidimensional assessment of vocational competence across test formats. *International Journal of Training and Development, 29*(4), 433–448. https://doi.org/10.1111/ijtd.12372

Ma, X., Chen, M., & Diao, J. (2025). Pathways and outcomes of digital transformation in Chinese vocational colleges. *Vocation, Technology & Education, 2*(1), Article e5. https://doi.org/10.54844/vte.2024.0800

Meijer, K., Baartman, L., Vermeulen, M., & de Bruijn, E. (2026). Unpacking assessment literacy: Developing a questionnaire on aspects of assessment literacy teachers consider important. *Cogent Education, 13*(1), Article 2633007. https://doi.org/10.1080/2331186X.2026.2633007

Meijer, K., Grootenboer, C., Baartman, L., Vermeulen, M., & de Bruijn, E. (2025). Illustrating situated manifestations of assessment literacy in higher professional education. *Education Sciences, 15*(12), Article 1644. https://doi.org/10.3390/educsci15121644

Metreveli, A., Chen, X., Hedman, A., & Sergeeva, A. (2025). “Who will be left behind?”: A Swedish case of learning AI in vocational education. *International Journal of Educational Research, 133*, Article 102697. https://doi.org/10.1016/j.ijer.2025.102697

Ockey, G. J. (2009). Developments and challenges in the use of computer-based testing for assessing second language ability. *The Modern Language Journal, 93*(S1), 836–847. https://doi.org/10.1111/j.1540-4781.2009.00976.x

Othman, N., Omar, M., & Abd Majid, M. Z. (2025). Challenges of digitalisation in TVET: A recent comprehensive structured review. *International Journal of Learning, Teaching and Educational Research, 24*(10), 210–229. https://doi.org/10.26803/ijlter.24.10.10

Parua, R., & Yang, W. (2024). The driving logic of digital transformation in TVET. *Vocation, Technology & Education, 1*(2), Article e17. https://doi.org/10.54844/vte.2024.0590

Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. *Journal of Management Information Systems, 24*(3), 45–77. https://doi.org/10.2753/MIS0742-1222240302

Sadeghi, K. (Ed.). (2023). *Fundamental considerations in technology mediated language assessment*. Routledge. https://doi.org/10.4324/9781003292395

Sembey, R., Hoda, R., & Grundy, J. (2024). Emerging technologies in higher education assessment and feedback practices: A systematic literature review. *Journal of Systems and Software, 211*, Article 111988. https://doi.org/10.1016/j.jss.2024.111988

Setiyawan, A., Soeharto, S., Wijaya, T. T., Korenova, L., & Lavicza, Z. (2025). Measuring teachers’ competencies for AI integration: Development and validation of the AI-TPACK in vocational education. *Computers and Education Open, 9*, Article 100319. https://doi.org/10.1016/j.caeo.2025.100319

Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gasevic, D. (2022). Assessment in the age of artificial intelligence. *Computers and Education: Artificial Intelligence, 3*, Article 100075. https://doi.org/10.1016/j.caeai.2022.100075

Treadaway, M., & Read, J. (2025). Assessing the language needs of L2 English student pilots preparing for flight training in English-speaking countries. *English for Specific Purposes, 78*, 125–138. https://doi.org/10.1016/j.esp.2024.12.007

Vogt, K., Bøhn, H., & Tsagari, D. (2024). Language assessment literacy. *Language Teaching, 57*(3), 325–340. https://doi.org/10.1017/S0261444824000090

Widaningsih, L., Sari, A. R., Dwiyanti, V., Maknun, J., & Hayat, J. (2024). How digital competency for Indonesian TVET educators explored in the last decade: A systematic literature review based on the Australian Qualifications Framework. *Jurnal Pendidikan Teknologi dan Kejuruan, 30*(2), 209–221. https://doi.org/10.21831/jptk.v30i2.71436

Witarsyah, D., Almohab, H., Fudzee, M. F. B., & Kocak, S. (2026). Digital transformation in technical and vocational education and training (TVET): A systematic literature review. *ITEGAM-JETIA, 12*(59), 365–375. https://doi.org/10.5935/jetia.v12i59.3450

Xin, Y., Tang, Y., & Mou, X. (2024). An empirical study on the evaluation and influencing factors of digital competence of Chinese teachers for TVET. *PLOS ONE, 19*(9), Article e0310187. https://doi.org/10.1371/journal.pone.0310187

Zhang, J., Yu, G., & Browne, W. J. (2025). Teachers’ language assessment literacy: Exploring its construct and contextual factors. *Studies in Educational Evaluation, 87*, Article 101525. https://doi.org/10.1016/j.stueduc.2025.101525

Zhao, W. (2024). A corpus-based multi-dimensional analysis of the linguistic features of Aviation English. *English for Specific Purposes, 76*, 57–73. https://doi.org/10.1016/j.esp.2024.05.004

Zhong, Z., & Juwaheer, S. (2024). Digital competence development in TVET with a competency-based whole-institution approach. *Vocation, Technology & Education, 1*(2), Article e13. https://doi.org/10.54844/vte.2024.0591