Analytical, hierarchical process to evaluate three virtual labs in a higher education setting
DOI:
https://doi.org/10.18041/entramado.2015v11n1.21102Abstract
The purpose of this work was to standardize a protocol using an analytical hierarchical process (AHP) to evaluate the use and relevance of
virtual labs or simulated academic practices in a virtual learning environment for distance higher education. To this end, a quantitative, descriptive
approach was used for evaluating and making a hierarchical categorization of three virtual commercial chemistry labs. Ten students
and seven professors, who are experts in theoretical and practical courses offered in virtual classrooms, participated in the study, using
the Moodle e-learning platform. The following criteria were analyzed: functionality (FUN), reliability (FIA), usability (USAB), efficiency (EFI),
maintainability (MAB), portability (POR), technical aspects (ASPT), psycho pedagogical aspects (ASPS), communicational aspects (ASPC),
and administrative aspects (ASPA) of virtual labs. The review and hierarchical classification of the criteria and virtual labs (VLs) with these
variables were carried out using paired comparison matrices. The consistency ratio of the evaluators’ paired decisions was 0.05885. Since it
was lower than 0.10, it can be concluded that this approach is suitable not only for an objective review and analysis of virtual labs, but also
for a hierarchical classification of virtual labs based on their attributes.
Downloads
Download data is not yet available.
References
1. ALANBAY, O. An enterprise resource planning (ERP) selection using expert choice software. In: ISAHP, 2005. Honolulu, Hawaii.
2. ALONSO, J. Consistency in the analytic hierarchy process: a new approach. In: International Journal for Uncertainty Fuzzines and Knolwledge-Based Systems, 2006, 14 (4), p. 445-459.
3. ALTSCHULD, J. Delphi technique. In: Lecture, Applied evaluation de- sign. The Ohio State University. 2003. [Recuperado el 10 de julio de 2013] disponible en internet: http://link.springer.com/search?facet-au- thor=%22A.+Azadeh%22.
4. AMERICAN EDUCATIONAL RESEARCH ASSOCIATION (AERA). Standards for Educational Psychological Test. In: American Psychologi- cal Association, Washington, D C, 1999.
5. AZADEH, A. y SHIRKOUHI, A. Evaluating simulation software using fuzzy analytical hierarchy process. In: Spring Sim 09 Proceedings of the Spring Simulation Multiconference, 2009. Article No. 41.
6. BELANGER, F. y JORDAN, D. Evaluation and Implementation of Dis- tance Learning: technologies, tools and techniques. En: Idea Group Publishing, London. 2000. p. 246.
7. BOURNE, J., HARRIS, D. y MAYADAS, F. Online Engineering Educa- tion: Learning Anywhere, Anytime. In: Journal of Engineering Educa- tion, 2005, Vol. 94, p.131-146.
8. BRUNER, J. Toward a Theory of Instruction. Harvard University Press, Cambridge, MA, 1966. p. 192.
9. CABALA. P. Using the analytic hierarchy process in evaluating deci- sion alternatives. In: Operations Research And Decisions, 2010. Vol. 1.
10. CABERO, J. Diseño y Evaluación de un Material Multimedia y Tele- mático para la Formación y Perfeccionamiento del Profesorado Uni- versitario para la Utilización de las Nuevas Tecnologías Aplicadas a la Docencia. Universidad de Sevilla. España, 2002, p. 67– 95.
11. CATALDI, Z., DONNAMARÍA M. y LAGE, F. TICs en la enseñanza de la química: Laboratorios virtuales, modelos y simulaciones en cursos universitarios. En: Edutec, 15, 16 y 17 de septiembre, Manaos, 2009.
12. CATALDI, Z., CHIARENZA, D., DOMINIGHINI, C., DONNAMARÍA, L., y LAGE, F. TICs en la enseñanza de la química. En: Revista Iberoame- ricana de Educación en Tecnología y Tecnología en Educación, 2012, Vol. 3, no. 7.
13. COVA, A, ARRIETA X. y AULAR J. Revisión de modelos para eva- luación de software educativos. En: Revista Electrónica de Estudios Telemáticos, 2008. Vol. 7, no. 1, p. 94 - 116.
14. DAVIS, L. y WILLIAMS, G. Evaluation and selecting simulation softwa- re using the analytic hierarchy process. In: Integrated Manufacturing Systems, 1994,. Vol. 51, no.1, p. 23-32. 15. DÍAZ-ANTÓN, G. Uso de software educativo de calidad como he- rramientas de apoyo para el aprendizaje. En: Jornadas educativas: La escuela como instrumento de cambio, IEA, Abril, 2002. Caracas. [Re- cuperado el 11 de mayo de 2013] disponible en internet: http://www. academia-interactiva/articulos.html.
16. GALVIS, A. Ingeniería de software educativo. 1era ed. Sgda reimpre- sión. Universidad de los Andes, Bogotá, 2000.
17. GONZÁLEZ, J. y WAGENAAR, R. Tuning Educational Structures in Europe. En: Informe Final-Proyecto Piloto, Fase 1, 2003. Bilbao: Uni- versidad de Deusto.
18. HERNÁNDEZ, R., FERNÁNDEZ, C. y BAPTISTA, P. Metodología de la investigación. 3th ed. Mc Graw Hill, México, 2006.p. 705.
19. HLUPIC, V. y MANN, A. A. System for Simulation Software Selection. In: Proc. of the 2002, WinterSimulation Conference, Orlando, Florida, p. 720-727.
20. ISO/IEC 9126. Information technology–Software Product Evaluation– Quality Characteristics and guidelines for their use. [Recuperado el 19 de julio de 2013] disponible en internet en: http://www.cse.dcu.ie/ essiscope/sm2/9126ref.html.
21. JADHAV, A. y, SONAR, R. Evaluating and selecting software packages: A review. In: Information and Software Technology. 2009. Vol. 51, no 3, p. 555–563.
22. LANDETA, J. El método Delphi. Una técnica de previsión del futuro. Ariel, Barcelona, 2002.p. 223.
23. LAI, V., WONG, B. y CHEUNG, W. Group decision making in a multi- ple criteria environment: A case using the AHP in software selection. In: European Journal of Operational Research, 2002, Vol. 137, p. 134- 144.
24. LÊ, Q. y LÊ, T. Evaluation of educational software: theory into practice. In: Technology and Teaching. Nova Science Publishers, 2007. 10 p.
25. MARQUÈS, P. Software educativo. Guía de uso y metodología de di- seño. En: Editorial Estela, Barcelona, 1995. 256 p.
26. MARQUÈS, P. Plantilla para la Catalogación y Evaluación Multimedia. 2004 [Recuperado el 8 de septiembre de 2012] disponible en inter- net: http://dewey.uab. es/pmarques/evalua.htm.
27. MARQUÈS, P. Entorno formativo multimedia: elementos, plantilla de evaluación/criterios de calidad. 2005. [Recuperado el 8 de junio de 2010] disponible en internet: http://dewey.uab.es/pmarques/calidad. htm.
28. MCDOUGAL, A y SQUIRES, D. Cómo elegir y utilizar software edu- cativo. Ediciones Morata, Madrid, 2001. 176 p.
29. MENDOZA, L., PÉREZ, M. y GRIMÁN, A. Prototipo de Modelo Sisté- mico de Calidad (MOSCA) del Software. En: Computación y Sistemas, 2005, Vol. 8, no. 3, p. 196-217.
30. OSSADNIK, W. AND LANGE, O. AHP based evaluation of AHP sof- tware. In: European Journalof Operational Research, 1999, vol. 118, no. 3, p. 578-588.
31. OZDEMIR, M. Validity and inconsistency in the analytic hierarchy pro- cess. In: Applied Mathematics and Computation. 2005, vol.161, no. 3, p. 707-720.
32. Office of Technology Assessment (OTA). Power on! - New tools for teaching and Learning. In: Congress of the United States, Office of Technology Assessment, OTA SET-379, Whashington D.C. US Gover- nment Printing Office, 1988.
33. PALCIC. I. y LALIC. B. Analytical hierarchy process as a tool for se- lecting and evaluating projects. En: International journal simulations model. 2009, Vol. 8, no. 1, p. 16-26.
34. RADA, M. Instrumento de análisis y evaluación de software multime- dia educativo. En: Revista educarnos, 2013. [Recuperado el 19 de julio de 2013] disponible en internet: http://www.revistaeducarnos.com/ art%C3%ADculos/educaci%C3%B3n/iaesme-instrumento-de-an%- C3%A1lisis-y-evaluaci%C3%B3n-de-software-multimedia-educati- vos1.
35. RAMOS, B., PELÁEZ, J., GARCÍA, E. y RUIZ, J. Design, Navigation, and Structure of a Hypermedia Application for the Teaching—Learning of Pneumatic Engineering Design Drawing. In: Computer Applications in Engineering Education, 2007, Vol. 15, p. 248-259.
36. RIAÑO y PALOMINO. Diseno y Elaboracion de un Cuestionario Acorde con el Método Delphi, para Seleccionar Laboratorios Virtua- les (Lvs). En: Revista de Investigación en Educaciòn. (s.f.).
37. SAATY, T. The Analytic Network Process. In: Decision Making with the Analytic Network Process. Pittsburg University Press, 2001, p. 1-26.
38. SAATY, T. Extending the Measurement of Tangibles to Intangibles. In: International Journal of Information Technology & Decision Making, 2009, Vol. 8, no.1, pp.7-27.
39. SAMPIERI, COLLADO y BAPTISTA. Metodología de la Investigación. 5ta edición. In: McGraw-Hill, 2010. p. 704.
40. SHERMAN, T. y KURSHAN, B. Constructing Learning Using Techno- logy to Support Teaching for Understanding. In: Learning & Leading with Technology, 2005, Vol. 32. no. 5, p.1-13.
41. VAIDYA, O. y KUMAR, S. Analytic hierarchy process: An overview of applications. In: European Journal of Operational Research, 2006, Vol. 169, p. 1-29.
42. VARGAS, L. An overview of the analytical hierarchy process and its application. In: European Journal of Operations Research, 1990, Vol. 48, No. 1, p. 2-8.
43. VERMA, R., GUPTA, A. y SINGH, K. Simulation Software Evaluation and Selection: A Comprehensive Framework. In: J. Automation & Sys- tems Engineering, 2008, Vol. 2, p. 221-234.
44. VERMA, R., GUPTA, A. y SINGH, K. A critical evaluation and com- parison of four manufacturing simulation softwares. In: Kathmandu University Journal Of Science, Engineering And Technology, 2008, Vol. 5, no. 1, p. 104- 120.
2. ALONSO, J. Consistency in the analytic hierarchy process: a new approach. In: International Journal for Uncertainty Fuzzines and Knolwledge-Based Systems, 2006, 14 (4), p. 445-459.
3. ALTSCHULD, J. Delphi technique. In: Lecture, Applied evaluation de- sign. The Ohio State University. 2003. [Recuperado el 10 de julio de 2013] disponible en internet: http://link.springer.com/search?facet-au- thor=%22A.+Azadeh%22.
4. AMERICAN EDUCATIONAL RESEARCH ASSOCIATION (AERA). Standards for Educational Psychological Test. In: American Psychologi- cal Association, Washington, D C, 1999.
5. AZADEH, A. y SHIRKOUHI, A. Evaluating simulation software using fuzzy analytical hierarchy process. In: Spring Sim 09 Proceedings of the Spring Simulation Multiconference, 2009. Article No. 41.
6. BELANGER, F. y JORDAN, D. Evaluation and Implementation of Dis- tance Learning: technologies, tools and techniques. En: Idea Group Publishing, London. 2000. p. 246.
7. BOURNE, J., HARRIS, D. y MAYADAS, F. Online Engineering Educa- tion: Learning Anywhere, Anytime. In: Journal of Engineering Educa- tion, 2005, Vol. 94, p.131-146.
8. BRUNER, J. Toward a Theory of Instruction. Harvard University Press, Cambridge, MA, 1966. p. 192.
9. CABALA. P. Using the analytic hierarchy process in evaluating deci- sion alternatives. In: Operations Research And Decisions, 2010. Vol. 1.
10. CABERO, J. Diseño y Evaluación de un Material Multimedia y Tele- mático para la Formación y Perfeccionamiento del Profesorado Uni- versitario para la Utilización de las Nuevas Tecnologías Aplicadas a la Docencia. Universidad de Sevilla. España, 2002, p. 67– 95.
11. CATALDI, Z., DONNAMARÍA M. y LAGE, F. TICs en la enseñanza de la química: Laboratorios virtuales, modelos y simulaciones en cursos universitarios. En: Edutec, 15, 16 y 17 de septiembre, Manaos, 2009.
12. CATALDI, Z., CHIARENZA, D., DOMINIGHINI, C., DONNAMARÍA, L., y LAGE, F. TICs en la enseñanza de la química. En: Revista Iberoame- ricana de Educación en Tecnología y Tecnología en Educación, 2012, Vol. 3, no. 7.
13. COVA, A, ARRIETA X. y AULAR J. Revisión de modelos para eva- luación de software educativos. En: Revista Electrónica de Estudios Telemáticos, 2008. Vol. 7, no. 1, p. 94 - 116.
14. DAVIS, L. y WILLIAMS, G. Evaluation and selecting simulation softwa- re using the analytic hierarchy process. In: Integrated Manufacturing Systems, 1994,. Vol. 51, no.1, p. 23-32. 15. DÍAZ-ANTÓN, G. Uso de software educativo de calidad como he- rramientas de apoyo para el aprendizaje. En: Jornadas educativas: La escuela como instrumento de cambio, IEA, Abril, 2002. Caracas. [Re- cuperado el 11 de mayo de 2013] disponible en internet: http://www. academia-interactiva/articulos.html.
16. GALVIS, A. Ingeniería de software educativo. 1era ed. Sgda reimpre- sión. Universidad de los Andes, Bogotá, 2000.
17. GONZÁLEZ, J. y WAGENAAR, R. Tuning Educational Structures in Europe. En: Informe Final-Proyecto Piloto, Fase 1, 2003. Bilbao: Uni- versidad de Deusto.
18. HERNÁNDEZ, R., FERNÁNDEZ, C. y BAPTISTA, P. Metodología de la investigación. 3th ed. Mc Graw Hill, México, 2006.p. 705.
19. HLUPIC, V. y MANN, A. A. System for Simulation Software Selection. In: Proc. of the 2002, WinterSimulation Conference, Orlando, Florida, p. 720-727.
20. ISO/IEC 9126. Information technology–Software Product Evaluation– Quality Characteristics and guidelines for their use. [Recuperado el 19 de julio de 2013] disponible en internet en: http://www.cse.dcu.ie/ essiscope/sm2/9126ref.html.
21. JADHAV, A. y, SONAR, R. Evaluating and selecting software packages: A review. In: Information and Software Technology. 2009. Vol. 51, no 3, p. 555–563.
22. LANDETA, J. El método Delphi. Una técnica de previsión del futuro. Ariel, Barcelona, 2002.p. 223.
23. LAI, V., WONG, B. y CHEUNG, W. Group decision making in a multi- ple criteria environment: A case using the AHP in software selection. In: European Journal of Operational Research, 2002, Vol. 137, p. 134- 144.
24. LÊ, Q. y LÊ, T. Evaluation of educational software: theory into practice. In: Technology and Teaching. Nova Science Publishers, 2007. 10 p.
25. MARQUÈS, P. Software educativo. Guía de uso y metodología de di- seño. En: Editorial Estela, Barcelona, 1995. 256 p.
26. MARQUÈS, P. Plantilla para la Catalogación y Evaluación Multimedia. 2004 [Recuperado el 8 de septiembre de 2012] disponible en inter- net: http://dewey.uab. es/pmarques/evalua.htm.
27. MARQUÈS, P. Entorno formativo multimedia: elementos, plantilla de evaluación/criterios de calidad. 2005. [Recuperado el 8 de junio de 2010] disponible en internet: http://dewey.uab.es/pmarques/calidad. htm.
28. MCDOUGAL, A y SQUIRES, D. Cómo elegir y utilizar software edu- cativo. Ediciones Morata, Madrid, 2001. 176 p.
29. MENDOZA, L., PÉREZ, M. y GRIMÁN, A. Prototipo de Modelo Sisté- mico de Calidad (MOSCA) del Software. En: Computación y Sistemas, 2005, Vol. 8, no. 3, p. 196-217.
30. OSSADNIK, W. AND LANGE, O. AHP based evaluation of AHP sof- tware. In: European Journalof Operational Research, 1999, vol. 118, no. 3, p. 578-588.
31. OZDEMIR, M. Validity and inconsistency in the analytic hierarchy pro- cess. In: Applied Mathematics and Computation. 2005, vol.161, no. 3, p. 707-720.
32. Office of Technology Assessment (OTA). Power on! - New tools for teaching and Learning. In: Congress of the United States, Office of Technology Assessment, OTA SET-379, Whashington D.C. US Gover- nment Printing Office, 1988.
33. PALCIC. I. y LALIC. B. Analytical hierarchy process as a tool for se- lecting and evaluating projects. En: International journal simulations model. 2009, Vol. 8, no. 1, p. 16-26.
34. RADA, M. Instrumento de análisis y evaluación de software multime- dia educativo. En: Revista educarnos, 2013. [Recuperado el 19 de julio de 2013] disponible en internet: http://www.revistaeducarnos.com/ art%C3%ADculos/educaci%C3%B3n/iaesme-instrumento-de-an%- C3%A1lisis-y-evaluaci%C3%B3n-de-software-multimedia-educati- vos1.
35. RAMOS, B., PELÁEZ, J., GARCÍA, E. y RUIZ, J. Design, Navigation, and Structure of a Hypermedia Application for the Teaching—Learning of Pneumatic Engineering Design Drawing. In: Computer Applications in Engineering Education, 2007, Vol. 15, p. 248-259.
36. RIAÑO y PALOMINO. Diseno y Elaboracion de un Cuestionario Acorde con el Método Delphi, para Seleccionar Laboratorios Virtua- les (Lvs). En: Revista de Investigación en Educaciòn. (s.f.).
37. SAATY, T. The Analytic Network Process. In: Decision Making with the Analytic Network Process. Pittsburg University Press, 2001, p. 1-26.
38. SAATY, T. Extending the Measurement of Tangibles to Intangibles. In: International Journal of Information Technology & Decision Making, 2009, Vol. 8, no.1, pp.7-27.
39. SAMPIERI, COLLADO y BAPTISTA. Metodología de la Investigación. 5ta edición. In: McGraw-Hill, 2010. p. 704.
40. SHERMAN, T. y KURSHAN, B. Constructing Learning Using Techno- logy to Support Teaching for Understanding. In: Learning & Leading with Technology, 2005, Vol. 32. no. 5, p.1-13.
41. VAIDYA, O. y KUMAR, S. Analytic hierarchy process: An overview of applications. In: European Journal of Operational Research, 2006, Vol. 169, p. 1-29.
42. VARGAS, L. An overview of the analytical hierarchy process and its application. In: European Journal of Operations Research, 1990, Vol. 48, No. 1, p. 2-8.
43. VERMA, R., GUPTA, A. y SINGH, K. Simulation Software Evaluation and Selection: A Comprehensive Framework. In: J. Automation & Sys- tems Engineering, 2008, Vol. 2, p. 221-234.
44. VERMA, R., GUPTA, A. y SINGH, K. A critical evaluation and com- parison of four manufacturing simulation softwares. In: Kathmandu University Journal Of Science, Engineering And Technology, 2008, Vol. 5, no. 1, p. 104- 120.
Downloads
Published
2015-06-01
Issue
Section
Articles
How to Cite
Analytical, hierarchical process to evaluate three virtual labs in a higher education setting. (2015). Entramado, 11(1), 194-204. https://doi.org/10.18041/entramado.2015v11n1.21102
