Psychometric Evaluation of an Extended Basic Mathematics Skills Measure
DOI:
https://doi.org/10.28918/circle.v5i1.8923Abstract
Basic mathematics skills (BMS) and mathematics self-concepts are related to achievement in mathematics courses. However, the validity evidence of a BMS measure is limited. Of particular concern are its unidimensionality, the difficulty of the items, and the functioning of the distractors. Therefore, this study attempts to validate a bilingual and extended version of a BMS measure and examine the scores’ correlation with past mathematics achievement, expected grades in a current maths-related course, and a Mathematics Self-Concept (MSC). The BMS was extended by adding five new items and replacing three items. The BME-Ext and MSC questionnaires were given to students at a college taking a quantitative methods course using Google Forms. Data from 123 students were analysed using Winstep and Jamovi. Evidence for unidimensionality is inconclusive while the person’s reliability and separation are not sufficient. Four items are too easy and may need to be replaced with more difficult questions. In six items, there is one distractor that was not chosen at all by the students. The BMS scores have a significant positive correlation, in descending order, with past mathematics achievement, expected grades for the current course, and MSC. Improvements to the BMS and further testing are suggested for measurement and practical purposes.
Keywords:
References
Abd Hamid, H. S., & Sulaiman, M. K. (2014). Statistics anxiety and achievement in a statistics course among psychology students. International Journal of Behavioral Science, 9(1), 55-66.
Adi, N. R. M., Amaruddin, H., Adi, H. M. M., & A'yun, I. L. Q. (2022). Validity and reliability analysis using the Rasch model to measure the quality of mathematics test items of vocational high schools. Journal of Research and Educational Research Evaluation, 11(2), 103-113.
Adlawon, C. M. C., Valdevarona, A. M. G, & Miranda, A. T. (2022). The relationship of self-concept and mathematics performance of college students in the new normal. Asian Journal of Education and Social Studies, 39-47. https://doi.org/10.9734/ajess/2022/v37i3805
Alcívar-Castro, E. J., Tamayo-Batista, M., Bravo-Andrade, B. D., Alcívar-Santander, T. L., & Pinargote-Jiménez, J. A. (2023). Development of mathematical skills in students of basic. International Journal of Social Sciences and Humanities, 7(1), 14–23. https://doi.org/10.53730/ijssh.v7n1.13818
Almubarak, A., Saadi, P., Prayogi, R., & Maldini, P. P. (2023). Assessing students understanding of chemical bonds material by Rasch modeling. Indonesian Journal on Learning and Advanced Education (IJOLAE), 5(3), 217-232.
Azizah, N., Suseno, M., & Hayat, B. (2022). Item analysis of the Rasch model items in the final semester exam Indonesian language lesson. World, 12(1).
Ben-Jacob, M. (2019). The importance of mathematics in STEM education. In EDULEARN19 Proceedings (pp. 2496-2500). https://doi.org/10.21125/edulearn.2019.0683
Chan, C. T., Zulnaidi, H., & Leong, K. E. (2021). Analysis validity and reliability of Self -Efficacy and Metacognitive Awareness instrument toward mathematical reasoning. Türk Bilgisayar Ve Matematik Eğitimi Dergisi, 12(9), 3332–3344. https://doi.org/10.17762/turcomat.v12i9.5739
Cresswell, C., & Speelman, C.P. (2020). Does mathematics training lead to better logical thinking and reasoning? A cross-sectional assessment from students to professors. PLoS ONE, 15.
Delaney, J.M., & Devereux, P.J. (2020). Math matters! the importance of mathematical and verbal skills for degree performance. Economic Letters, 168, 108850. https://doi.org/10.1016/j.econlet.2019.108850
Dzin, N. H. M., & Lay, Y. F. (2021). Validity and reliability of Adapted Self-Efficacy Scales in Malaysian context using PLS-SEM approach. Education Sciences, 11(11), 676. https://doi.org/10.3390/educsci11110676
Hafizi, M.H., & Kamarudin, N. (2020). Creativity in mathematics: Malaysian Perspective. Universal Journal of Educational Research, 8, 77-84.
Hosseini, M., & Toroujeni, S. M. H. (2017). Replacing paper-based testing with an alternative for the assessment of Iranian undergraduate students: Administration mode effect on testing performance. International Journal of Language and Linguistics, 5(3), 78-87.
Ismail, N. A., Nopiah, Z. M., Asshaari, I., Othman, H., Tawil, N. M., & Zaharim, A. (2012). Mathematical performance of engineering students in Universiti Kebangsaan Malaysia (UKM). Procedia-Social and Behavioral Sciences, 60, 206-211.
Jones, T. W., Price, B. A., & Randall, C. H. (2011). A comparative study of student math skills: perceptions, validation, and recommendations. Decision Sciences Journal of Innovative Education, 9(3), 379–394. https://doi.org/doi:10.1111/j.1540-4609.2011.00314.x
Jowinis, A., & Siew, N. M. (2024). Validity and reliability of multiple-choice biology test: a Rasch analysis perspective. International Journal of Modern Education, 6(21), 131-139. https://doi.org/10.35631/IJMOE.621010
Karaali, G. (2016). What's in a name? A critical review of definitions of quantitative literacy, numeracy, and quantitative reasoning. Numeracy ( Journal of the National Numeracy Network), Volume 9, (1), Article 6.
Kania, N., & Juandi, D. (2023). Does self-concept affect mathematics learning achievement? Journal of Education and Learning, 17(3), 455–461. https://doi.org/10.11591/edulearn.v17i3.20554
Kruyen, P. M. (2012). Using short tests and questionnaires for making decisions about individuals: When is short too short. Ridderprint.
Latif, D. A. (2002). The relationship among pharmacy students' basic math scores, traditional preadmission indicators, and performance in a pharmaceutical calculations course. Journal of Pharmacy Teaching, 10(1), 17-29.
Lin, C. H., & Chen, C. H. (2023). Development, testing, and application of a mathematics learning scale of self-direction. Frontiers in Psychology, 14, 1145442. https://doi.org/10.3389/fpsyg.2023.1145442
Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), 328.
Lynch, S. (2022). Adapting paper-based tests for computer administration: lessons learned from 30 years of mode effects studies in education. Practical Assessment, Research & Evaluation, 27, 22.
Mejía-Rodríguez, A. M., Luyten, H., & Meelissen, M. R. (2021). Gender differences in mathematics self-concept across the world: An exploration of student and parent data of TIMSS 2015. International Journal of Science and Mathematics Education, 19(6), 1229-1250.
Mohd Dzin, N. H., & Lay, Y. F. (2021). Assessing the validity and reliability of science multiple choice test using Rasch Dichotomous Measurement Model. Journal of Baltic Science Education, 20(6), 927-941.
Mukhibin, A., Rusyid, H. K., Lutfi, A., Herman, T., & Suryo Utomo, D. A. (2023). Analysis of Students’ Mathematical Self-Efficacy Instruments using Rasch model. Indonesian Journal of Mathematics Education, 6(2), 72–80. https://doi.org/10.31002/ijome.v6i2.994
Núñez-Peña, M. I., Ramon-Casas, M., Cunillera, T., & Campos-Rodríguez, C. (2024). Mathematics self-concept and response pattern in higher education examinations: differences between genders. Assessment & Evaluation in Higher Education, 1–14. https://doi.org/10.1080/02602938.2024.2364024
OECD. (2019). PISA 2018 assessment and analytical framework: Reading, mathematics, and science. OECD Publishing. https://doi.org/10.1787/b25efab8-en
Palenzuela-Luis, N., Duarte-Clíments, G., Gómez-Salgado, J., Rodríguez-Gómez, J. Á., & Sánchez-Gómez, M. B. (2022). International comparison of self-concept, self-perception and lifestyle in adolescents: a systematic review. International Journal of Public Health, 67, 1604954.
Passiatore, Y., Costa, S., Grossi, G., Carrus, G., & Pirchio, S. (2023). Mathematics self-concept moderates the relation between cognitive functions and mathematical skills in primary school children. Social Psychology of Education. https://doi.org/10.1007/s11218-023-09854-3
Rylands, L.J., & Shearman, D. (2022). Diagnostic tests: Purposes and two case studies. MSOR Connections, 20(3). https://doi.org/10.21100/msor.v20i3.1281
Sa’di, D. R., Firdaus, N. P. N., Sinaga, R. D. H., & Yonvitra, N. H. (2023). Kemampuan siswa SMP dalam menyelesaikan persoalan matematika dasar. Jurnal Pendidikan Matematika, 1(2), 10.
Tennant, A., & Küçükdeveci, A. A. (2023). Application of the Rasch measurement model in rehabilitation research and practice: early developments, current practice, and future challenges. Frontiers in Rehabilitation Sciences, 4, 1208670.
Trombley, R. J., & Weiss, D. J. (1993). Measurement of basic skills in mathematics. National Center for Research in Vocational Education
Winarti, A., & Mubarak, A. (2019). Rasch modeling: A multiple choice chemistry test. Indonesian Journal on Learning and Advanced Education (IJOLAE), 2(1), 1-9.
Woldemichael, B., Semela, T., & Tulu, A. (2023). The effect of mathematics self-beliefs on mathematics achievement: the case of undergraduate university students in Ethiopia. Routledge Open Research, 2, 41. https://doi.org/10.12688/routledgeopenres.17995.1
Yuliani, N. A., & Zaenal, R. M. (2023). Analisis kemampuan koneksi matematis ditinjau dari self-concept siswa. Indo-MathEdu Intellectuals Journal, 4(2), 243–254. https://doi.org/10.54373/imeij.v4i2.163
Zhang, X., Kuchinke, L., Woud, M. L., Velten, J., & Margraf, J. (2017). Survey method matters: Online/offline questionnaires and face-to-face or telephone interviews differ. Computers in Human Behavior, 71, 172-180.
Downloads
Published
License
Copyright (c) 2025 Assoc. Prof. Dr. Harris Shah , Shariah Bagam, Siti Masitah, Shafinah
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
