Determination of the satisfaction attribute in usability tests using sentiment analysis and fuzzy logic
DOI:
https://doi.org/10.15837/ijccc.2023.3.4901Keywords:
fuzzy logic, satisfaction level, sentiment analysis, usabilityAbstract
With the growth in the number of applications deployed in cloud app stores, usability has become a fundamental attribute to ensure end-user productivity and enterprise competitiveness. According to ISO 9241-11, the three attributes that determine the usability of a product are effectiveness, efficiency and satisfaction, the latter being the subjective attribute of usability. In conventional user tests, the satisfaction attribute is determined using perception questionnaires, being a challenge to determine this attribute more objectively, given the limitations of surveys in terms of veracity and subjectivity. Based on the above, in this article we propose as a contribution, the construction of a system based on fuzzy logic for the estimation of the usability satisfaction attribute in user tests, which has as inputs both the level of satisfaction obtained from the answers to the post-test questionnaire, and the level of satisfaction obtained from the polarities of the opinions of the test users, determined by means of sentiment analysis techniques. The proposed fuzzy system is intended to serve as a reference to be replicated at the academic and enterprise level in the conduct of usability tests and specifically in the more objective determination of the satisfaction attribute based on the advantages of fuzzy logic and sentiment analysis techniques.References
[Online]. Available: https://catalogosiidca.csuca.org/Record/UCR.000174999, Accessed on 8 august 2022.
Ainin, S.; Feizollah, A.; Anuar, N.B.; Abdullah, N.A. (2020). Sentiment analyses of multilingual tweets on halal tourism, Tourism Management Perspectives, 34(apr), 100658, 2020.
https://doi.org/10.1016/j.tmp.2020.100658
Aznar, E.; Royo, J. (2013). Conceptos y aplicaciones de la lógica borrosa, Técnica Industrial, 269, 58-63, 2007.
An Introduction to Fuzzy Logic and Fuzzy Sets, Physica-Verlag HD, 2002.
Chanchí-Golondrino, G.; Gómez-Álvarez, M,; Campo-Muñoz, W. (2020). Propuesta de una herramienta de inspección según los atributos de usabilidad de Nielsen, Revista Ibérica de Sistemas e Tecnologias de Informação, E26, 448-460, 2020.
Chanchí-Golondrino, G.; Sierra-Martínez, L.M.; Campo-Muñoz, W. (2022). Fuzzy Logic-Based System for the Estimation of the Usability Level in User Tests International Journal of Computers Communications & Control, 17(2), 2022.
https://doi.org/10.15837/ijccc.2022.2.4476
Delgado, D.M.; Girón , D.F.; Chanchí, G.E.; Márceles Villalba, K. (2010). Estimación del atributo satisfacción en test de usuarios a partir del análisis de la expresión facial, Revista Ingenierías Universidad de Medellín, 19(36), 13-28, 2019.
https://doi.org/10.22395/rium.v19n36a1
Dinkel, C.; Billenstein, D.; Goller, D.; Rieg, F.(2018). User-oriented optimization of the GUI of a finite element programme to enhance the usability of simulation tools, South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference, SEEDA_CECNSM 2018, Institute of Electrical and Electronics Engineers Inc., 2018.
https://doi.org/10.23919/SEEDA-CECNSM.2018.8544936
Dzulfiqar, M. D.; Khairani, D.; Wardhani, L.K. (2019). The Development of University Website using User Centered Design Method with ISO 9126 Standard, In 2018 6th International Conference on Cyber and IT Service Management, CITSM 2018, Institute of Electrical and Electronics Engineers Inc., 1-4, 2019.
https://doi.org/10.1109/CITSM.2018.8674325
Enriquez, J.G.; Casas, S. (2013). Usabilidad en aplicaciones móviles, Informe Científico Técnico UNPA, 5(2), 25-47, 2013.
https://doi.org/10.22305/ict-unpa.v5i2.71
Finstad, K. (2010). The usability metric for user experience, Interacting with Computers, 22(5), 323-327, 2010.
https://doi.org/10.1016/j.intcom.2010.04.004
Gozález-Sánchez, J.L.; Montero-Simarro, F.; Gutiérrez-Vela, F.L. (2012). Evolución del concepto de usabilidad como indicador de calidad del software, Revista internacional de Información, Documentación, Biblioteconomía y Comunicación, 21, 529-536, 2012.
https://doi.org/10.3145/epi.2012.sep.13
Hering, D.; Schwartz, T.; Boden, A. ; Wulf, V. (2015). Integrating usability-engineering into the software developing processes of SME: A case study of software developing SME in Germany, In Proceedings - 8th International Workshop on Cooperative and Human Aspects of Software Engineering, CHASE 2015, Institute of Electrical and Electronics Engineers Inc., 121-122, 2015.
https://doi.org/10.1109/CHASE.2015.22
Hilletofth, P.; Sequeira, M.; Adlemo, A. (2010). Three novel fuzzy logic concepts applied to reshoring decision-making, Expert Systems with Applications, 126(jul), 133-143, 2019.
https://doi.org/10.1016/j.eswa.2019.02.018
Huang, L.; Dou, Z.; Hu, Y. ;Huang, R. (2019). Textual analysis for online reviews: A polymerization topic sentiment model, IEEE Access, 7,91940-91945, 2019.
https://doi.org/10.1109/ACCESS.2019.2920091
Ikoro, V.; Sharmina, M.; Malik, K.; Batista-Navarro, R. (2018). Analyzing Sentiments Expressed on Twitter by UK Energy Company Consumers, In 2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018, Institute of Electrical and Electronics Engineers Inc., 95-98, 2018. 17] [Online]. Available: https://erika-jacobs.com/post/mood-harry-potter/, Accessed on 8 august 2022.
https://doi.org/10.1109/SNAMS.2018.8554619
Mittal, K.; Jain, A.; Vaisla, K. S.; Castillo, O.; Kacprzyk, J. (2010). A comprehensive review on type 2 fuzzy logic applications: Past, present and future, Engineering Applications of Artificial Intelligence, 95(oct), 103916, 2020.
https://doi.org/10.1016/j.engappai.2020.103916
Nakagawa, F. H.; Felinto, A.; Omori, T. (2013). Inclusion of teaching slides in games: Analysis of the efficiency, effectiveness and satisfaction, IEEE Latin America Transactions, 11(6),1372-1377, 2013.
https://doi.org/10.1109/TLA.2013.6710386
[Online]. Available: https://www.nngroup.com/articles/usability-101-introduction-to-usability/, Accessed on 8 august 2022.
Obermeier, M.; Braun, S.; Vogel-Heuser, B.(2015). A Model-Driven Approach on Object-Oriented PLC Programming for Manufacturing Systems with Regard to Usability IEEE Transactions on Industrial Informatics, 11(3), 790-800, 2015.
https://doi.org/10.1109/TII.2014.2346133
Pai, P.; Liu, Ch. (2018). Predicting vehicle sales by sentiment analysis of twitter data and stock market values, IEEE Access, 6,57655-57662, 2018.
https://doi.org/10.1109/ACCESS.2018.2873730
Park, J. (2020). Framework for Sentiment-Driven Evaluation of Customer Satisfaction With Cosmetics Brands, IEEE Access, 8, 98526-98538, 2020.
https://doi.org/10.1109/ACCESS.2020.2997522
[Online]. Available: http://kpratt ˙net/wp-content/uploads/2009/01/research_methods ˙pdf, Accessed on 8 august 2022.
Radle, K. ; Young, S. (2001). Partnering usability with development: how three organizations succeeded, IEEE Software, 18(1),38-45, 2001.
https://doi.org/10.1109/52.903164
Ramírez, N.; Laguna, M. (2020). La lógica borrosa: conjuntos borrosos, razonamiento aproximado y control borroso, Pistas Educativas, 100, 55-65, 2012.
Rao, S. and Monica, N and Nikhila, P and Tejasri, T and Maram, Balajee (2020). Positivity Calculation using Vader Sentiment Analyser, International Journal of Academic Engineering Research (IJAER), 4(3),13-17, 2000.
Santos, C.;Novais, T.; Ferreira, M.; Albuquerque, C.;De Farias, I.H.; Furtado, A.P.(2016). Metrics focused on usability ISO 9126 based, In Iberian Conference on Information Systems and Technologies, CISTI, IEEE Computer Society, 1-3, 2016.
https://doi.org/10.1109/CISTI.2016.7521437
Sharifirad, S.; Jafarpour, B.;Matwin, S. (2022). How is Your Mood When Writing Sexist tweets? Detecting the Emotion Type and Intensity of Emotion Using Natural Language Processing Techniques, IEEE Access, 10,39313 - 39324, 2022.
https://doi.org/10.1109/ACCESS.2022.3165621
Weichbroth, P. (2010). Usability of mobile applications: A systematic literature study, IEEE Access, 8, 55563-55577, 2020.
https://doi.org/10.1109/ACCESS.2020.2981892
Wu, D.; Zheng, L. ; Olson, D.L. (2014). A decision support approach for online stock forum sentiment analysis, IEEE Transactions on Systems, Man, and Cybernetics: Systems, 44(8),1077- 1087, 2014.
https://doi.org/10.1109/TSMC.2013.2295353
Zadeh, L. (2013). Fuzzy Logic, Neural Networks, and Soft Computing, Communications of the ACM, 37(3), 77-84, 1994.
Additional Files
Published
Issue
Section
License
Copyright (c) 2023 Gabriel Elías Chanchí Golondrino, Manuel Alejandro Ospina Alarcón, Luz Marina Sierra Martínez
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
ONLINE OPEN ACCES: Acces to full text of each article and each issue are allowed for free in respect of Attribution-NonCommercial 4.0 International (CC BY-NC 4.0.
You are free to:
-Share: copy and redistribute the material in any medium or format;
-Adapt: remix, transform, and build upon the material.
The licensor cannot revoke these freedoms as long as you follow the license terms.
DISCLAIMER: The author(s) of each article appearing in International Journal of Computers Communications & Control is/are solely responsible for the content thereof; the publication of an article shall not constitute or be deemed to constitute any representation by the Editors or Agora University Press that the data presented therein are original, correct or sufficient to support the conclusions reached or that the experiment design or methodology is adequate.
