Determination of the satisfaction attribute in usability tests using sentiment analysis and fuzzy logic


  • Gabriel Elías Chanchí Golondrino Faculty of Engineering, University of Cartagena, Colombia
  • Manuel Alejandro Ospina Alarcón Faculty of Engineering, University of Cartagena, Colombia
  • Luz Marina Sierra Martínez Faculty of Electronic Engineering and Telecommunications, University of Cauca, Colombia



fuzzy logic, satisfaction level, sentiment analysis, usability


With the growth in the number of applications deployed in cloud app stores, usability has become a fundamental attribute to ensure end-user productivity and enterprise competitiveness. According to ISO 9241-11, the three attributes that determine the usability of a product are effectiveness, efficiency and satisfaction, the latter being the subjective attribute of usability. In conventional user tests, the satisfaction attribute is determined using perception questionnaires, being a challenge to determine this attribute more objectively, given the limitations of surveys in terms of veracity and subjectivity. Based on the above, in this article we propose as a contribution, the construction of a system based on fuzzy logic for the estimation of the usability satisfaction attribute in user tests, which has as inputs both the level of satisfaction obtained from the answers to the post-test questionnaire, and the level of satisfaction obtained from the polarities of the opinions of the test users, determined by means of sentiment analysis techniques. The proposed fuzzy system is intended to serve as a reference to be replicated at the academic and enterprise level in the conduct of usability tests and specifically in the more objective determination of the satisfaction attribute based on the advantages of fuzzy logic and sentiment analysis techniques.


[Online]. Available:, Accessed on 8 august 2022.

Ainin, S.; Feizollah, A.; Anuar, N.B.; Abdullah, N.A. (2020). Sentiment analyses of multilingual tweets on halal tourism, Tourism Management Perspectives, 34(apr), 100658, 2020.

Aznar, E.; Royo, J. (2013). Conceptos y aplicaciones de la lógica borrosa, Técnica Industrial, 269, 58-63, 2007.

An Introduction to Fuzzy Logic and Fuzzy Sets, Physica-Verlag HD, 2002.

Chanchí-Golondrino, G.; Gómez-Álvarez, M,; Campo-Muñoz, W. (2020). Propuesta de una herramienta de inspección según los atributos de usabilidad de Nielsen, Revista Ibérica de Sistemas e Tecnologias de Informação, E26, 448-460, 2020.

Chanchí-Golondrino, G.; Sierra-Martínez, L.M.; Campo-Muñoz, W. (2022). Fuzzy Logic-Based System for the Estimation of the Usability Level in User Tests International Journal of Computers Communications & Control, 17(2), 2022.

Delgado, D.M.; Girón , D.F.; Chanchí, G.E.; Márceles Villalba, K. (2010). Estimación del atributo satisfacción en test de usuarios a partir del análisis de la expresión facial, Revista Ingenierías Universidad de Medellín, 19(36), 13-28, 2019.

Dinkel, C.; Billenstein, D.; Goller, D.; Rieg, F.(2018). User-oriented optimization of the GUI of a finite element programme to enhance the usability of simulation tools, South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference, SEEDA_CECNSM 2018, Institute of Electrical and Electronics Engineers Inc., 2018.

Dzulfiqar, M. D.; Khairani, D.; Wardhani, L.K. (2019). The Development of University Website using User Centered Design Method with ISO 9126 Standard, In 2018 6th International Conference on Cyber and IT Service Management, CITSM 2018, Institute of Electrical and Electronics Engineers Inc., 1-4, 2019.

Enriquez, J.G.; Casas, S. (2013). Usabilidad en aplicaciones móviles, Informe Científico Técnico UNPA, 5(2), 25-47, 2013.

Finstad, K. (2010). The usability metric for user experience, Interacting with Computers, 22(5), 323-327, 2010.

Gozález-Sánchez, J.L.; Montero-Simarro, F.; Gutiérrez-Vela, F.L. (2012). Evolución del concepto de usabilidad como indicador de calidad del software, Revista internacional de Información, Documentación, Biblioteconomía y Comunicación, 21, 529-536, 2012.

Hering, D.; Schwartz, T.; Boden, A. ; Wulf, V. (2015). Integrating usability-engineering into the software developing processes of SME: A case study of software developing SME in Germany, In Proceedings - 8th International Workshop on Cooperative and Human Aspects of Software Engineering, CHASE 2015, Institute of Electrical and Electronics Engineers Inc., 121-122, 2015.

Hilletofth, P.; Sequeira, M.; Adlemo, A. (2010). Three novel fuzzy logic concepts applied to reshoring decision-making, Expert Systems with Applications, 126(jul), 133-143, 2019.

Huang, L.; Dou, Z.; Hu, Y. ;Huang, R. (2019). Textual analysis for online reviews: A polymerization topic sentiment model, IEEE Access, 7,91940-91945, 2019.

Ikoro, V.; Sharmina, M.; Malik, K.; Batista-Navarro, R. (2018). Analyzing Sentiments Expressed on Twitter by UK Energy Company Consumers, In 2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018, Institute of Electrical and Electronics Engineers Inc., 95-98, 2018. 17] [Online]. Available:, Accessed on 8 august 2022.

Mittal, K.; Jain, A.; Vaisla, K. S.; Castillo, O.; Kacprzyk, J. (2010). A comprehensive review on type 2 fuzzy logic applications: Past, present and future, Engineering Applications of Artificial Intelligence, 95(oct), 103916, 2020.

Nakagawa, F. H.; Felinto, A.; Omori, T. (2013). Inclusion of teaching slides in games: Analysis of the efficiency, effectiveness and satisfaction, IEEE Latin America Transactions, 11(6),1372-1377, 2013.

[Online]. Available:, Accessed on 8 august 2022.

Obermeier, M.; Braun, S.; Vogel-Heuser, B.(2015). A Model-Driven Approach on Object-Oriented PLC Programming for Manufacturing Systems with Regard to Usability IEEE Transactions on Industrial Informatics, 11(3), 790-800, 2015.

Pai, P.; Liu, Ch. (2018). Predicting vehicle sales by sentiment analysis of twitter data and stock market values, IEEE Access, 6,57655-57662, 2018.

Park, J. (2020). Framework for Sentiment-Driven Evaluation of Customer Satisfaction With Cosmetics Brands, IEEE Access, 8, 98526-98538, 2020.

[Online]. Available: http://kpratt ˙net/wp-content/uploads/2009/01/research_methods ˙pdf, Accessed on 8 august 2022.

Radle, K. ; Young, S. (2001). Partnering usability with development: how three organizations succeeded, IEEE Software, 18(1),38-45, 2001.

Ramírez, N.; Laguna, M. (2020). La lógica borrosa: conjuntos borrosos, razonamiento aproximado y control borroso, Pistas Educativas, 100, 55-65, 2012.

Rao, S. and Monica, N and Nikhila, P and Tejasri, T and Maram, Balajee (2020). Positivity Calculation using Vader Sentiment Analyser, International Journal of Academic Engineering Research (IJAER), 4(3),13-17, 2000.

Santos, C.;Novais, T.; Ferreira, M.; Albuquerque, C.;De Farias, I.H.; Furtado, A.P.(2016). Metrics focused on usability ISO 9126 based, In Iberian Conference on Information Systems and Technologies, CISTI, IEEE Computer Society, 1-3, 2016.

Sharifirad, S.; Jafarpour, B.;Matwin, S. (2022). How is Your Mood When Writing Sexist tweets? Detecting the Emotion Type and Intensity of Emotion Using Natural Language Processing Techniques, IEEE Access, 10,39313 - 39324, 2022.

Weichbroth, P. (2010). Usability of mobile applications: A systematic literature study, IEEE Access, 8, 55563-55577, 2020.

Wu, D.; Zheng, L. ; Olson, D.L. (2014). A decision support approach for online stock forum sentiment analysis, IEEE Transactions on Systems, Man, and Cybernetics: Systems, 44(8),1077- 1087, 2014.

Zadeh, L. (2013). Fuzzy Logic, Neural Networks, and Soft Computing, Communications of the ACM, 37(3), 77-84, 1994.

Additional Files



Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.