Using the Statistical Process Control to evaluate the learning process in Higher Education / Aplicación del Control Estadístico de Procesos para la evaluación de procesos de aprendizaje en la Educación Superior
DOI:
https://doi.org/10.34140/bjbv3n5-017Palavras-chave:
educational improvement, Higher Education, Statistical Process Control, service qualityResumo
The purpose of this paper is to propose an approach for incorporating Statistical Process Control (SPC) charting technique to monitor and continuously improve the learning processes by monitoring the satisfaction of students who used an interactive computer-based learning material, which was produced to solve problems in the Operation Management course. By applying SPC methods, the authors examine the sources of process variation (common or special causes) and analyse the ability of teaching strategies to ensure the acquisition and development of the competencies and skills demanded in current university studies. A total of 184 students participated in the learning experience. The findings show that the learning process is under control and therefore the variation of process is due to common causes. However, the Capability Analysis carried out, reveals that process has not enough capacity to achieve the specifications required by the teachers involved in the educational project. This quantitative approach will not only allow self-assessment, but can be used for comparative purposes (other teaching strategies, colleges, etc.). Thus, the control charting is a complementary assessment technique that should be included within the current Quality and Learning Assurance Systems at higher education institutions.
Referências
Achcaoucaou, F., Guitart‐Tarres, L., Miravitlles‐Matamoros, P., Nuñez‐Carballosa, A., Bernardo, M., & Bikfalvi, A. (2014). Competence assessment in higher education: A dynamic approach. Human Factors and Ergonomics in Manufacturing & Service Industries, 24 (4), 454-467.
Agencia Nacional de Evaluación de la Calidad y Acreditación (ANECA) (2005). Libro Blanco Título de Grado en Economía y Empresa. Madrid. Retrieved September 13, 2015 from: http://www.aneca.es/var/media/150292/libroblanco_economia_def.pdf
Alfalla-Luque, R., Medina-López, C., & Arenas-Márquez, F.J. (2011). Mejorando la formación en Dirección de Operaciones: la visión del estudiante y su respuesta ante diferentes metodologías docentes. Cuadernos de Economía y Dirección de la Empresa, 14, 40-52.
Aranda, D. A., Domínguez, C. H., & Martínez, M. M. R. (2010). Un enfoque innovador del proceso de enseñanza-aprendizaje en la dirección de empresas: el uso de simuladores en el ámbito universitario. Revista de Educación, 353, 707-721.
Arenas-Márquez, F.J., Machuca, J.A., & Medina-López, C. (2012). Interactive learning in operations management higher education: Software design and experimental evaluation. International Journal of Operations & Production Management, 32 (12), 1395-1426.
Barberá, E. (2004). La educación en la red: actividades virtuales de enseñanza y aprendizaje. Barcelona: Paidós.
Bi, H. H. (2018). A robust interpretation of teaching evaluation ratings. Assessment & Evaluation in Higher Education, 43(1), 79-93.
Cadden, D., Driscoll, V., & Thompson, M. (2008). Improving teaching effectiveness through the application of SPC methodology. College Teaching Methods & Styles Journal, 4 (11), 33-46.
Canton, I., Valle, R.E., & Arias, A.R. (2008). Calidad de la docencia universitaria: procesos clave. Educatio Siglo XXI, 26, 121-160.
Castillo-Manzano, J.I., Castro-Nuño, M., Sanz-Díaz, M.T., & Yñiguez, R. (2016). Does pressing a button make it easier to pass an exam? Evaluating the effectiveness of interactive technologies in higher education. British Journal of Educational Technology, 47, (4), 710-720. doi: 10.1111/bjet.12258
Carlucci, D., Renna, P., Izzo, C., & Schiuma, G. (2019). Assessing teaching performance in higher education: a framework for continuous improvement. Management Decision, 57(2) 461-479.
Chafer, E. (2009). Una introducción a los sistemas de respuesta interactiva. Electrónica y Comunicaciones. Monográfico TICs en las aulas. Elementos Didácticos para la Enseñanza, 242, 56-57.
Coll, C. (2008). Psicología de la educación virtual: aprender y enseñar con las tecnologías de la información y la comunicación. Madrid: Ediciones Morata.
Correa, N. (2003). 6 Sigma Nuevo paradigma para medir la calidad. Mundo Mitutoyo, 6, 5-8.
Cuatrecasas, L. (2005). Gestión Integral de la Calidad Implantación, Control y Certificación. Madrid: Ediciones Gestión 2000.
Daneshmandi, A. A., Noorossana, R., & Farahbakhsh, K. (2020). Developing Statistical Process Control to Monitor the Values Education Process. Journal of Quality Engineering and Production Optimization, 5(1), 33-54.
Debnath, R.M., & Shankar, R. (2014). Emerging trend of customer satisfaction in academic process. TQM Journal, 26 (1), 14-29.
Ding, X., Wardell, D., & Verma, R. (2006). An Assessment of Statistical Process Control‐Based Approaches for Charting Student Evaluation Scores. Decision Sciences Journal of Innovative Education, 4 (2), 259-272.
Djauhari, M. A., Sagadavan, R., & Lee, S. L. (2017). Monitoring the disparity of teaching and learning process variability: a statistical approach. International Journal of Productivity and Quality Management, 21(4), 532-547
EduClick® Educational Software. Sistema de docencia presencial basado en mandos electrónicos de respuesta. Retrieved May, 2014 from: http://www.educlick.es/.
European Commission (2002). European Union Policies and Strategic Change for e-Learning in Universities, Report of the HECTIC Project. Retrieved April 20, 2014 from http://homepages.ed.ac.uk/jhaywood/reports/HECTICREPORT.pdf
González-Zamora, M.D.M. (2006). Algunas necesidades en la enseñanza en administración y dirección de empresas: de la teoría a la práctica a través de las TIC. Pixel-Bit: Revista de medios y educación, 27, 59-77.
Greasley, A., Bennett, D., & Greasley, K. (2004). A virtual learning environment for operations management: Assessing the student's perspective. International Journal of Operations & Production Management, 24 (10), 974-993.
Grygoryev, K., & Karapetrovic, S. (2005a). An integrated system for educational performance measurement, modeling and management at the classroom level. The TQM Magazine, 17 (2), 121-136.
Grygoryev, K., & Karapetrovic, S. (2005b). Tracking classroom teaching and learning: An SPC application. Quality Engineering, 17 (3), 405-418.
Guitart, L., Miravitlles, P., Nuñez, A., Bernardo, M., Achcaoucaou, F., Castan, J.M., Valls, J., & López, J. (2012). Protocol per a la implantació d’eines didàctiques virtuals: competències i habilitats adquirides pels estudiants. REIRE, Revista d’ Innovació i Recerca en Educació, 4 (2), 59-81. Retrieved from: http://www.raco.cat/index.php/REIRE
Guitart, L., Nuñez, A., Miravitlles, P., Bernardo, M., Cruz, C., & Achcaoucaou, F. (2014). Students competences in Business Administration subjects. International Journal on Advances in Education Research, 1 (2), 1-20.
Hoskisson, R.E., Hitt, M.E., Johnson, R.A., & Moesel, D.D. (1993). Construct validity of an objective (entropy) categorical measure of diversification strategy. Strategic Management Journal, 14 (3), 215-35.
Hrynkevych, O. S. (2017). Statistical analysis of higher education quality with use of control charts. Advanced Science Letters, 23(10), 10070-10072.
Hsieh, K.L., & Tong, L.I. (2006). Incorporating process capability index and quality loss function into analyzing the process capability for qualitative data. Advanced Manufacturing Technology, 27 (11), 1217-1222.
Jensen, J.B., & Markland, R.E. (1996). Improving the application of quality conformance tools in service firm. The Journal of Services Marketing, 10 (1), 35-55.
Judson, E. (2002). Learning from past and present: Electronic response systems in college lecture halls. Journal of Computers in Mathematics and Science Teaching, 21 (2), 167-181.
Klein, H. J., Noe, R. A., & Wang, C. (2006). Motivation to learn and course outcomes: The impact of delivery mode, learning goal orientation, and perceived barriers and enablers. Personnel psychology, 59 (3), 665-702.
Kumar, U.D., Crocker, J., & Chitra, T. (2006). Reliability and Six Sigma. New York: Springer-Verlag.
Larson, D., & Chung-Hsien, S. (2009). Comparing student performance: online versus blended versus face-to-face. Journal of Asynchronous Learning Networks, 13 (1), 31-42.
López-Pérez, M. V., Pérez-López, M. C., & Rodríguez-Ariza, L. (2011). Blended learning in higher education: Students’ perceptions and their relation to outcomes. Computers & Education, 56 (3), 818-826.
MacCarthy, B.L., & Wasusri, T. (2002). A review of non-standard applications of statistical process control (SPC) charts. International Journal of Quality & Reliability Management, 19 (3), 295-320.
Manguad, B.A. (2006). Using SPC to assess performance in a graduate course of business. The American Society of Business and Behavioral Sciences Proceedings. 13th Annual Meeting. Las Vegas.
Marín-García, J.A., Miralles-Insa, C., & Marín-García, P. (2008). Oral presentation and assessment skills in engineering education. International Journal of Engineering Education, 24, 926-935.
Marks, N., & O'Connel, T. (2003). Using statistical control charts to analyse data from student’s evaluations of teaching. Decision Sciences Journal of Innovative Education, 1 (2), 259-272.
McCray, G. E. (2000). The hybrid course: Merging on-line instruction and the traditional classroom. Information Technology and Management, 1(4), 307-327.
Milosavljevic, P., Pavlovic, D., Rajic, M., Pavlovic, A., & Fragassa, C. (2018). Implementation of quality tools in higher education process. International Journal of Continuing Engineering Education and Life Long Learning, 28(1), 24-36.
Minitab® 17 Statistical Software.
Montgomery, D.C. (2001). Introduction to Statistical Quality Control, 6th ed. New York: Wiley.
Nandedkar, T., & Bhati, G. (2021). Assessment of Academic Performance through SQC: An Application of Control Charts. In Interdisciplinary Research in Technology and Management: Proceedings of the International Conference on Interdisciplinary Research in Technology and Management (IRTM, 2021), 26-28 February, 2021, Kolkata, India (p. 76). CRC Press.
Nikolaidis, Y., & Dimitriadis, S.G. (2014). On the student evaluation of university courses and faculty members' teaching performance. European Journal of Operational Research, 238 (1), 199-207.
Novo-Corti, I., Varela-Candamio, L., & Ramil-Díaz, M. (2013). E-learning and face to face mixed methodology: Evaluating effectiveness of e-learning and perceived satisfaction for a microeconomic course using the Moodle platform. Computers in Human Behaviour, 29 (2), 410-415.
O’Leary-Kelly, S.W., & Vokurka, R.J. (1998). The empirical assessment of construct validity. Journal of Operations Management, 16 (4), 387-405.
Rao, M. (2015). The effectiveness of a web-based homework and testing software for operations management. International Journal of Services and Standards, 10 (4), 240-252.
Richey, R.C., Fields, D.C., & Foxon, M. (2001). Instructional design competencies: The standards. ERIC Clearinghouse on Information & Technology. Syracuse, NY: Springer Boston.
Rodríguez, M., Álvarez, M., Figuera, P., & Rodríguez, S. (2008). Dels estudis universitaris al món del treball. Barcelona: Publicacions i Edicions UB.
Rubio-Hurtado, M. J., García-Duran, P., & Millet, M. (2010). Evaluación continua a través de Moodle para involucrar al alumnado en su proceso de aprendizaje. REIRE. Revista d'Innovació i Recerca en Educació, 3 (1), 46-65.
Ruiz-Jiménez, A., Ceballos-Hernández, C., & García-Gragera, J.A. (2010a). Utilización de un sistema de respuesta interactiva como herramienta contra el absentismo en la Universidad. In Jiménez-Caballero, J.L. y Rodríguez-Díaz, A. (Coords.), El absentismo en las aulas universitarias: el caso de la Escuela Universitaria de Estudios Empresariales (pp. 179-191). Sevilla: Grupo Editorial Universitario.
Ruíz-Jiménez, A., Ceballos-Hernández, C., García-Gragera, J.A., & Chávez-Miranda, M.E. (2010b). Una experiencia de evaluación continua en un entorno masificado. In Jiménez-Caballero, J.L. y Rodríguez-Díaz, A. (Coords.), Nuevas enseñanzas de grado en la Escuela Universitaria de Estudios Empresariales de la Universidad de Sevilla (pp. 279-294). Sevilla: Grupo Editorial Universitario.
Sacristán-Díaz, M., Garrido-Vega, P., Luque-Alfalla, R., & González-Zamora, M. (2010). De la evaluación sumativa a la formativa a través de las plataformas de enseñanza virtual. In del Pozo Barajas, R. (coord.), Nuevas formas de docencia en el área económico-empresarial (pp. 151-167). Sevilla: Edición Digital@ Tres.
Sangrà, A., Guárdia, L., Williams, P., & Schurm, L. (2004). Modelos de diseño instruccional. In Stephenson, J., Sangrà, A., Williams, P., Schrum, L., Guàrdia, L., Salinas, J., & Chan, M.E., Fundamentos del diseño técnico-pedagógico en e-learning (pp.1-73). Barcelona: FUOC.
Schmidt, S.R., & Launsbyn, R.G. (1994). Understanding industrial designed experiments. Colorado Springs, CO: Air Academy Press.
Skylar, A. A., Higgins, K., Boone, R., & Jones, P. (2005). Distance education: An exploration of alternative methods and types of instructional media in teacher education. Journal of Special Education Technology, 20 (3), 25.
Sulek, J. M. (2004). Statistical quality control in services. International Journal of Services Technology and Management, 5 (5-6), 522-531.
Summers, J.J., Waigandt, A., & Whittaker, T.A. (2005). A comparison of student achievement and satisfaction in an online versus a traditional face-to-face statistics class. Innovative Higher Education, 29 (3), 233-250.
Sun, P., & Cheng, H.K. (2007). The design of instructional multimedia in e-learning: a media richness theory-based approach. Computers & Education, 49 (3), 662-676.
Tuning Educational Structures in Europe II (2005). Final Report. Pilot Project - Phase2. Retrieved from: http://tuningacademy.org/wp-content/uploads/2014/02/TuningEUII_Final-Report_EN.pdf
Utley, J.S., & May, G. (2009). Monitoring service quality with residuals control charts. Managing Service Quality: An International Journal, 19 (2), 162-178.
Webb, H. W., Gill, G., & Poe, G. (2005). Teaching with the case method online: Pure versus hybrid approaches. Decision Sciences Journal of Innovative Education, 3 (2), 223-250.
Wild, R.H., & Griggs, K.A. (2002). Collaborative telelearning: An experiment in remote project management. E-Service Journal, 1 (2), 25-39.
Yang, S., Cheng, T., Hung, Y., & Cheng, S. (2012). A new chart for monitoring service process mean. Quality and Reliability Engineering International, 28, 377-386.
Zolkepley, Z., Djauhari, M. A., & Salleh, R. M. (2018). SPC in service industry: Case in teaching and learning process variability monitoring. In AIP Conference Proceedings 1974 (1), 040026, AIP Publishing LLC.
