Deep Learning-Based Facial Emotion Recognition for Detecting Brand Hate
Main Article Content
Keywords
Facial Emotion Recognition, Deep Learning, Brand Hate, Negative Emotions
Abstract
This paper aims to recognise three levels of consumer hate – cold hate, cool hate and hot hate – through facial emotion recognition. It compares the use of transfer learning approaches and custom convolutional neural network (CNN) approaches. Five classes of emotions were considered in this research: fear, anger, disgust and contempt as indicators of a brand with hate, and happy as an indicator of non-brand hate. The databases used are AffectNet and RAF_DB. Multiple facial expressions are included in our dataset, which consists of about 23,529 images belonging to five classes in the training set and 10,088 images belonging to five classes in the testing set. The custom CNN model achieved an accuracy of 78.4% when validated on the testing set, compared to the VGG-16 model, which achieved an accuracy 79%, respectively. The confusion matrix was used to verify the findings, confirming that the custom CNN model outperforms the pre-trained models. Notably, it successfully detected brand hate emotions with an accuracy of 85%. This paper employs facial emotion recognition – a behavioural expression measure – to help companies capture the emotional responses of haters.
References
Abbas, A., Shar, A. H., & Junejo, M. A. (2023). Why brands fail? Antecedents and consequences of brand hate. A study of fashion industry in Pakistan. Journal of Managerial Sciences, 17(1), 1–26. https://journals.qurtuba.edu.pk/ojs/index.php/jms/article/view/696
Abd El-Rahiem, B., Sedik, A., El Banby, G. M., Ibrahem, H. M., Amin, M., Song, O. Y., Khalaf, A. M. M., & Abd El-Samie, F. E. (2023). An efficient deep learning model for classification of thermal face images. Journal of Enterprise Information Management, 36(3), 706–717. https://doi.org/10.1108/JEIM-07-2019-0201
Ahmed, M. U., Woo, K. J., Hyeon, K. Y., Bashar, M. R., & Rhee, P. K. (2018). Wild facial expression recognition based on incremental active learning. Cognitive Systems Research, 52, 212–222. https://doi.org/10.1016/j.cogsys.2018.06.017
Akrout, H., & Mrad, M. (2023). Measuring brand hate in a cross-cultural context: Emic and Etic scale development and validation. Journal of Business Research, 154, 113289. https://doi.org/10.1016/j.jbusres.2022.08.053
Akhand, M. A. H., Roy, S., Siddique, N., Kamal, M. A. S., & Shimamura, T. (2021). Facial emotion recognition using transfer learning in the deep CNN. Electronics, 10(9), 1036. https://doi.org/10.3390/electronics10091036
Alam, I. N., Kartowisastro, I. H., & Wicaksono, P. (2022). Transfer learning technique with EfficientNet for facial expression recognition system. Revue d’Intelligence Artificielle, 36(4). https://doi.org/10.18280/ria.360405
Alfred Daniel, J., Chandru Vignesh, C., Muthu, B. A., Senthil Kumar, R., Sivaparthipan, C. B., & Marin, C. E. M. (2023). Fully convolutional neural networks for LIDAR–camera fusion for pedestrian detection in autonomous vehicle. Multimedia Tools and Applications, 82(16), 25107–25130. https://doi.org/10.1007/s11042-023-14417-x
Alhussan, A. A., Talaat, F. M., El-kenawy, E. S. M., Abdelhamid, A. A., Ibrahim, A., Khafaga, D. S., & Alnaggar, M. (2023). Facial expression recognition model depending on optimized support vector machine. Computers, Materials & Continua, 76(1). https://www.techscience.com/cmc/v76n1/53096
Ali, S., Attiq, S., & Talib, N. (2020). Antecedents of brand hate: Mediating role of customer dissatisfaction and moderating role of narcissism. Pakistan Journal of Commerce and Social Sciences (PJCSS), 14(3), 603–628. https://hdl.handle.net/10419/224953
Alsharif, A. H., Salleh, N. Z. M., & Baharun, R. (2021). Neuromarketing: The popularity of the brain-imaging and physiological tools. Neuroscience Research Notes, 3(5), 13–22. https://doi.org/10.31117/neuroscirn.v3i5.80
Anaza, N. A., Saavedra, J. L., Hair Jr, J. F., Bagherzadeh, R., Rawal, M., & Osakwe, C. N. (2021). Customer-brand disidentification: Conceptualization, scale development and validation. Journal of Business Research, 133, 116–131. https://doi.org/10.1016/j.jbusres.2021.03.064
Arnade, E. A. (2014). Measuring Consumer Emotional Response to Tastes and Foods through Facial Expression Analysis. (Doctoral dissertation, Virginia Tech). https://hdl.handle.net/10919/54538
Atwal, G., Bryson, D., & Kaiser, M. (2022). The chopsticks debacle: How brand hate flattened Dolce & Gabbana in China. Journal of Business Strategy, 43(1), 37–43. https://doi.org/10.1108/JBS-07-2020-0160
Baer, J. (2016). Hug your haters: How to embrace complaints and keep your customers. Portfolio, Penguin Publishing Group. https://archive.org/details/hugyourhatershow0000baer
Bhavana, V., Dhanalakshmi, M., & Aishwarya, S. (2023, December). A brief survey on facial expression detection. In 2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON) (Vol. 10, pp. 110–115). IEEE. https://doi.org/10.1109/UPCON59197.2023.10434419
Bond Jr, C. F., & DePaulo, B. M. (2006). Accuracy of deception judgments. Personality and Social Psychology Review, 10(3), 214–234. https://doi.org/10.1207/s15327957pspr1003_2
Bryson, D., Atwal, G., Hultén, P., & Heine, K. (2021). Antecedents of luxury brand hate: A quantitative study. Strategic Change, 30(1), 35–43. https://doi.org/10.1002/jsc.2387
Bustamam, A. (2023). Arrhythmia classification using the deep learning visual geometry group (VGG) model. Nusantara Science and Technology Proceedings, 7–18. https://nstproceeding.com/index.php/nuscientech/article/view/1080
Buroon, J. K., Bulkr, D. B., Ebesu, A. S., White, C. H., & Rockwell, P. A. (1996). Testing interpersonal deception theory: Effects of suspicion on communication behaviors and perceptions. Communication Theory, 6(3), 243–267. https://doi.org/10.1111/j.1468-2885.1996.tb00128.x
Cleeren, K., Van Heerde, H. J., & Dekimpe, M. G. (2013). Rising from the ashes: How brands and categories can overcome product-harm crises. Journal of Marketing, 77(2),
58–77. https://doi.org/10.1509/jm.10.0414
Crist, C. A., Duncan, S. E., Arnade, E. A., Leitch, K. A., O’Keefe, S. F., & Gallagher, D. L. (2018). Automated facial expression analysis for emotional responsivity using an aqueous bitter model. Food Quality and Preference, 68, 349–359. https://doi.org/10.1016/j.foodqual.2018.04.004
Curina, I., Francioni, B., Cioppi, M., and Savelli, E. (2021). Traits and peculiarities of different brand hate behaviours, Journal of Strategic Marketing, Vol. 29 No. 3, 227–246. https://doi.org/10.1080/0965254X.2019.1676293
Dada, E. G., Oyewola, D. O., Joseph, S. B., Emebo, O., & Oluwagbemi, O. O. (2023). Facial emotion recognition and classification using the Convolutional Neural Network‐10 (CNN‐10). Applied Computational Intelligence and Soft Computing, 2023(1), 2457898. https://doi.org/10.1155/2023/2457898
Danner, L., & Duerrschmid, K. (2018). Automatic facial expressions analysis in consumer science. In Methods in Consumer Research, Volume 2 231–252. Woodhead Publishing. https://doi.org/10.1016/B978-0-08-101743-2.00010-8
Dong, X., Veloutsou, C., & Morgan-Thomas, A. (2024). Negative online brand engagement: conceptualisation, scale development and validation. Journal of Research in Interactive Marketing, (ahead of print). https://doi.org/10.1108/JRIM-09-2023-0303
Durga, B. K., Rajesh, V., Jagannadham, S., Kumar, P. S., Rashed, A. N. Z., & Saikumar, K. (2023). Deep learning-based micro facial expression recognition using an adaptive tiefes FCNN model. Traitement du Signal, 40(3). https://doi.org/10.18280/ts.400319
Ezati, A., Dezyani, M., Rana, R., Rajabi, R., & Ayatollahi, A. (2024). A Lightweight Attention-based Deep Network via Multi-Scale Feature Fusion for Multi-View Facial Expression Recognition. arXiv preprint arXiv:2403.14318. https://doi.org/10.48550/arXiv.2403.14318
Febrian, R., Halim, B. M., Christina, M., Ramdhan, D., & Chowanda, A. (2023). Facial expression recognition using bidirectional LSTM-CNN. Procedia Computer Science, 216(2022), 39–47. https://doi.org/10.1016/j.procs.2022.12.109
Fetscherin, M. (2019). The five types of brand hate: How they affect consumer behavior. Journal of Business Research, 101, 116–127. https://doi.org/10.1016/j.jbusres.2019.04.017
Generosi, A., Ceccacci, S., & Mengoni, M. (2018, September). A deep learning-based system to track and analyze customer behavior in retail store. In 2018 IEEE 8th International Conference on Consumer Electronics – Berlin (ICCE-Berlin) (pp. 1–6). IEEE. https://doi.org/10.1109/ICCE-Berlin.2018.8576169
Gunvantray, T. D., & Ananthan, T. (2024, March). Sign language to text translation using convolutional neural network. In 2024 International Conference on Emerging Smart Computing and Informatics (ESCI) (pp. 1–5). IEEE. https://doi.org/10.1109/ESCI59607.2024.10497209
Gusnadi, F., Efendi, M., & Apriyana, N. (2024). Analisis brand hate terhadap brand avoidance pada merek Starbucks konteks issue Israel-Palestina. Jurnal Cahaya Mandalika ISSN 2721–4796 (online), 5(1), 265–277. https://doi.org/10.36312/jcm.v5i1.2726.
Hamelin, N., El Moujahid, O., & Thaichon, P. (2017). Emotion and advertising effectiveness: A novel facial expression analysis approach. Journal of Retailing and Consumer Services, 36, 103–111. https://doi.org/10.1016/j.jretconser.2017.01.001
Hashim, S., & Kasana, S. (2019). Antecedents of brand hate in the fast food industry. Spanish Journal of Marketing – ESIC, 23(2), 227–248. https://doi.org/10.1108/SJME-10-2018-0047
Hegner, S. M., Fetscherin, M., & Van Delzen, M. (2017). Determinants and outcomes of brand hate. Journal of Product & Brand Management, 26(1), 13–25. https://doi.org/10.1108/JPBM-01-2016-1070
Hess, U., & Thibault, P. (2009). Darwin and emotion expression. American Psychologist, 64(2), 120. https://psycnet.apa.org/buy/2009-01602-003
Huang, Z. Y., Chiang, C. C., Chen, J. H., Chen, Y. C., Chung, H. L., Cai, Y. P., & Hsu, H. C. (2023). A study on computer vision for facial emotion recognition. Scientific Reports, 13(1), 8425. https://doi.org/10.1038/s41598-023-35446-4
Husnain, M., Syed, F., Hussain, K., Zhang, Q., Usman, M., & Javed, M. (2023). Explaining the mechanism of brand hate: a mixed-method investigation using moderated mediation in emerging markets. Kybernetes, 52(10), 3857–3877. https://doi.org/10.1108/K-12-2021-1246
Iranzo Barreira, Á., Kuster, I., & Ruiz Mafe, C. (2024). Inward negative emotions and brand hate in users of snow-sports’ brands. Journal of Product & Brand Management, 33(6), 745-760. https://doi.org/10.1108/JPBM-05-2023-4541
Kaur, J., & Singh, W. (2024). A systematic review of object detection from images using deep learning. Multimedia Tools and Applications, 83(4), 12253-12338. https://doi.org/10.1007/s11042-023-15981-y
Khaydarova, N. (2024). Some features of verbal and non-verbal types of communication. XALQARO ILMIY-AMALIY KONFERENSIYALAR, 1(1), 288–291. https://tkti.uz/uz/events/detail/xalqaro-ilmiy-amaliy-konferentsiya/
Kopalidis, T., Solachidis, V., Vretos, N., & Daras, P. (2024). Advances in facial expression recognition: A Survey of methods, benchmarks, models, and datasets. Information, 15(3), 135. https://doi.org/10.3390/info15030135
Kucuk, S. U. (2019a). Consumer brand hate: Steam rolling whatever I see. Psychology & Marketing, 36(5), 431–443. https://doi.org/10.1002/mar.21175
Kucuk, S. U., (2019b). What is brand hate? . In: Brand Hate. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-00380-7_2
Kumar, S., Sagar, V., & Punetha, D. (2023). A comparative study on facial expression recognition using local binary patterns, convolutional neural network and frequency neural network. Multimedia Tools and Applications, 82(16), 24369–24385. https://doi.org/10.1007/s11042-023-14753-y.
Leong, S. C., Tang, Y. M., Lai, C. H., & Lee, C. K. M. (2023). Facial expression and body gesture emotion recognition: A systematic review on the use of visual data in affective computing. Computer Science Review, 48, 100545. https://doi.org/10.1016/j.cosrev.2023.100545
Li, J., & Lam, E. Y. (2015, September). Facial expression recognition using deep neural networks. In 2015 IEEE International Conference on Imaging Systems and Techniques (IST) (pp. 1–6). IEEE. https://doi.org/10.1109/IST.2015.7294547
Markowitz, D. M. (2024). Deconstructing deception: Frequency, communicator characteristics, and linguistic features of embeddedness. Applied Cognitive Psychology, 38(3), e4215. https://doi.org/10.1002/acp.4215
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and emotion, 23(2), 209–237. https://doi.org/10.1080/02699930802204677
Mednini, L., & Turki, M. D. (2022, May). What leads customer to create and participate in anti-brand community: a netnographic approach. In International Conference on Digital Economy (pp. 159–169). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-17037-9_11
Mednini, L., Noubigh, Z., & Turki, M. D. (2024). Natural language processing for detecting brand hate speech. Journal of Telecommunications and the Digital Economy, 12(1), 486–509. https://search.informit.org/doi/abs/10.3316/informit.T2024040300011091695498163
Mednini, L., & Damak Turki, M. (2024). How to transform brand haters into forgivers through emotional intelligence? Management Decision, 62(1), 183–199. https://doi.org/10.1108/MD-06-2022-0819
Mednini, L., & Hmida, I. C. B. (2023, May). Antecedents and outcomes of brand hate: A case of anti-brand community. In International Conference on Digital Economy (pp. 222–234). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-42788-6_14
Meena, G., Mohbey, K. K., Indian, A., Khan, M. Z., & Kumar, S. (2024). Identifying emotions from facial expressions using a deep convolutional neural network-based approach. Multimedia Tools and Applications, 83(6), 15711–15732. https://doi.org/10.1007/s11042-023-16174-3
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2017). AffectNet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, 10(1), 18–31. https://doi.org/10.1109/TAFFC.2017.2740923
Mushtaq, F. M., Ghazali, E. M., & Hamzah, Z. L. (2024). Brand hate: a systematic literature review and future perspectives. Management Review Quarterly, 1–34. https://doi.org/10.1007/s11301-023-00402-z
Naga, P., Marri, S. D., & Borreo, R. (2023). Facial emotion recognition methods, datasets and technologies: A literature survey. Materials Today: Proceedings, 80, 2824–2828. https://doi.org/10.1016/j.matpr.2021.07.046
Naidoo, S. W., Naicker, N., Patel, S. S., & Govender, P. (2022). Computer vision: the effectiveness of deep learning for emotion detection in marketing campaigns. International Journal of Advanced Computer Science and Applications; 13(5). doi: https://doi.org/10.14569/ijacsa.2022.01305100
Noubigh, Z., Mezghani, A., & Kherallah, M. (2020, November). Transfer learning to improve Arabic handwriting text recognition. In 2020 21st International Arab Conference on Information Technology (ACIT) (pp. 1–6). IEEE. https://doi.org/10.1109/ACIT50332.2020.9300105
Noor, U., Mansoor, M., & Rabbani, S. (2022). Brand hate and retaliation in Muslim consumers: Does offensive advertising matter? Journal of Islamic Marketing, 13(6), 1395–1413. https://doi.org/10.1108/JIMA-10-2020-0316
Noubigh, Z., Mezghani, A., & Kherallah, M. (2021). Densely connected layer to improve VGGnet-based CRNN for Arabic handwriting text line recognition. International Journal of Hybrid Intelligent Systems, 17(3–4), 113–127. https://doi.org/10.3233/HIS-210009
Odoom, R., Kosiba, J. P. B., & Odoom, P. T. (2024). Brand hate experiences and the role of social media influencers in altering consumer emotions. Journal of Brand Management, 1–14. https://doi.org/10.1057/s41262-024-00356-x
Pantano, E. (2020). Non-verbal evaluation of retail service encounters through consumers’ facial expressions. Computers in Human Behavior, 111, 106448. https://doi.org/10.1016/j.chb.2020.106448
Pereira, J., and Loureiro, S. M. C. (2020), The different shades of brand hate: A qualitative approach, 2020 Global Marketing Conference at Seoul Proceedings: 632-636 (November 2020), pp. 632–636. https://db.koreascholar.com/Article/Detail/401143
Pinto, O., & Brandão, A. (2021). Antecedents and consequences of brand hate: Empirical evidence from the telecommunication industry. European Journal of Management and Business Economics, 30(1), 18–35. https://doi.org/10.1108/EJMBE-04-2020-0084
Rachman, R., Hamid, M. A., Wijaya, B. K., Wibowo, S. E., & Intan, D. N. (2024). Brand storytelling in the digital age: challenges and opportunities in online marketing. Jurnal Ekonomi, 13(01), 355–364. https://ejournal.seaninstitute.or.id/index.php/Ekonomi/article/view/3748
Rahimah, A., Dang, H. P., Nguyen, T. T., Cheng, J. M. S., & Kusumawati, A. (2023). The subsequent effects of negative emotions: From brand hate to anti-brand consumption behavior under moderating mechanisms. Journal of Product & Brand Management, 32(4), 618–631. https://doi.org/10.1108/JPBM-12-2021-3778
Ren, C., Moisieiev, D., Rodrigo, P., & Johnson, E. (2024). Hot and cold: How do consumers hate and forgive offending charity brands? Journal of Philanthropy and Marketing, 29(3), e1875. https://doi.org/10.1002/nvsm.1875
Rozin, P., Haidt, J., McCauley, C., Dunlop, L., & Ashmore, M. (1999). Individual differences in disgust sensitivity: Comparisons and evaluations of paper-and-pencil versus behavioral measures. Journal of Research in Personality, 33(3), 330–351. https://doi.org/10.1006/jrpe.1999.2251
Sameeni, M. S., Qadeer, F., Ahmad, W., & Filieri, R. (2024). An empirical examination of brand hate influence on negative consumer behaviors through NeWOM intensity. Does consumer personality matter? Journal of Business Research, 173, 114469. https://doi.org/10.1016/j.jbusres.2023.114469
Scherer, K. R. (2009). The dynamic architecture of emotion: Evidence for the component process model. Cognition and emotion, 23(7), 1307–1351. https://doi.org/10.1080/02699930902928969
Shahzad, F., Qazi, T. A., & Shehzad, R. (2023). Framing of Israel and Palestine conflict in RT news, Al-Jazeera, CNN & BBC News. Global Digital & Print Media Review, 6(II), 1–14. https://doi.org/10.31703/gdpmr.2023(VI-II).01
Sharma, A., Fadahunsi, A., Abbas, H., & Pathak, V. K. (2022). A multi-analytic approach to predict social media marketing influence on consumer purchase intention. Journal of Indian Business Research, 14(2), 125–149. https://doi.org/10.1108/JIBR-08-2021-0313
Siomkos, G., Triantafillidou, A., Vassilikopoulou, A., & Tsiamis, I. (2010). Opportunities and threats for competitors in product‐harm crises. Marketing Intelligence & Planning, 28(6), 770–791. https://doi.org/10.1108/02634501011078156
Solbu, A., Frank, M. G., Xu, F., Nwogu, I., & Neurohr, M. (2024). The effect of synchrony of happiness on facial expression of negative emotion when lying. Journal of Nonverbal Behavior, 48(1), 73–92. https://doi.org/10.1007/s10919-023-00447-4
Sternberg, R. J. (2003). A duplex theory of hate: Development and application to terrorism, massacres, and genocide. Review of General Psychology, 7(3), 299–328. https://doi.org/10.1037/1089-2680.7.3.299
Sternberg, R. J. (2005). Understanding and combating hate. In R. J. Sternberg (Ed.), The Psychology of Hate, 37–49. American Psychological Association. https://doi.org/10.1037/10930-002
Taye, M. M. (2023). Theoretical understanding of convolutional neural network: Concepts, architectures, applications, future directions. Computation, 11(3), 52. https://doi.org/10.3390/computation11030052
Ten Brinke, L., & Porter, S. (2012). Cry me a river: identifying the behavioral consequences of extremely high-stakes interpersonal deception. Law and Human Behavior, 36(6), 469. https://psycnet.apa.org/doi/10.1037/h0093929
Valenzuela, C., Castellucci, L., Mena, M. T., & Bianchi, C. (2022). Consumer brand hate: The role of ambivalence. The International Review of Retail, Distribution and Consumer Research, 32(1), 100–125. https://doi.org/10.1080/09593969.2021.2018017
Wang, C. Y., & Lin, F. S. (2022, June). Assessing the effectiveness of digital advertising for green products: A facial expression evaluation approach. In International Conference on Human-Computer Interaction (pp. 221–232). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-05544-7_17
Wang, W., & Sun, L. (2024). Brand marketing strategy based on user emotion recognition model of consumer. Computer-Aided Design and Applications, 21(S12), 115–129. https://doi.org/10.14733/cadaps.2024.S12.115-129
Webb, T. L., Miles, E., & Sheeran, P. (2012). Dealing with feeling: a meta-analysis of the effectiveness of strategies derived from the process model of emotion regulation. Psychological bulletin, 138(4), 775. https://psycnet.apa.org/doi/10.1037/a0027600
Xue, Y., Sun, J., Liu, Y., Li, X., & Yuan, K. (2024). Facial expression-enhanced recommendation for virtual fitting rooms. Decision Support Systems, 177, 114082. https://doi.org/10.1016/j.dss.2023.114082
Xu, X., Ruan, Z., & Yang, L. (2020, July). Facial expression recognition based on graph neural network. In 2020 IEEE 5th International Conference on Image, Vision and Computing (ICIVC) (pp. 211–214). IEEE. https://ieeexplore.ieee.org/document/7832107/
Yadav, A. (2024). When personalities collide: Examining the impact of consumer and brand personalities’ interplay on brand hate development. Journal of Product & Brand Management. https://doi.org/10.1108/JPBM-01-2023-4307
Zloteanu, M., Bull, P., Krumhuber, E. G., & Richardson, D. C. (2021). Veracity judgement, not accuracy: Reconsidering the role of facial expressions, empathy, and emotion recognition training on deception detection. Quarterly Journal of Experimental Psychology, 74(5), 910–927. https://doi.org/10.1177/1747021820978851
Abd El-Rahiem, B., Sedik, A., El Banby, G. M., Ibrahem, H. M., Amin, M., Song, O. Y., Khalaf, A. M. M., & Abd El-Samie, F. E. (2023). An efficient deep learning model for classification of thermal face images. Journal of Enterprise Information Management, 36(3), 706–717. https://doi.org/10.1108/JEIM-07-2019-0201
Ahmed, M. U., Woo, K. J., Hyeon, K. Y., Bashar, M. R., & Rhee, P. K. (2018). Wild facial expression recognition based on incremental active learning. Cognitive Systems Research, 52, 212–222. https://doi.org/10.1016/j.cogsys.2018.06.017
Akrout, H., & Mrad, M. (2023). Measuring brand hate in a cross-cultural context: Emic and Etic scale development and validation. Journal of Business Research, 154, 113289. https://doi.org/10.1016/j.jbusres.2022.08.053
Akhand, M. A. H., Roy, S., Siddique, N., Kamal, M. A. S., & Shimamura, T. (2021). Facial emotion recognition using transfer learning in the deep CNN. Electronics, 10(9), 1036. https://doi.org/10.3390/electronics10091036
Alam, I. N., Kartowisastro, I. H., & Wicaksono, P. (2022). Transfer learning technique with EfficientNet for facial expression recognition system. Revue d’Intelligence Artificielle, 36(4). https://doi.org/10.18280/ria.360405
Alfred Daniel, J., Chandru Vignesh, C., Muthu, B. A., Senthil Kumar, R., Sivaparthipan, C. B., & Marin, C. E. M. (2023). Fully convolutional neural networks for LIDAR–camera fusion for pedestrian detection in autonomous vehicle. Multimedia Tools and Applications, 82(16), 25107–25130. https://doi.org/10.1007/s11042-023-14417-x
Alhussan, A. A., Talaat, F. M., El-kenawy, E. S. M., Abdelhamid, A. A., Ibrahim, A., Khafaga, D. S., & Alnaggar, M. (2023). Facial expression recognition model depending on optimized support vector machine. Computers, Materials & Continua, 76(1). https://www.techscience.com/cmc/v76n1/53096
Ali, S., Attiq, S., & Talib, N. (2020). Antecedents of brand hate: Mediating role of customer dissatisfaction and moderating role of narcissism. Pakistan Journal of Commerce and Social Sciences (PJCSS), 14(3), 603–628. https://hdl.handle.net/10419/224953
Alsharif, A. H., Salleh, N. Z. M., & Baharun, R. (2021). Neuromarketing: The popularity of the brain-imaging and physiological tools. Neuroscience Research Notes, 3(5), 13–22. https://doi.org/10.31117/neuroscirn.v3i5.80
Anaza, N. A., Saavedra, J. L., Hair Jr, J. F., Bagherzadeh, R., Rawal, M., & Osakwe, C. N. (2021). Customer-brand disidentification: Conceptualization, scale development and validation. Journal of Business Research, 133, 116–131. https://doi.org/10.1016/j.jbusres.2021.03.064
Arnade, E. A. (2014). Measuring Consumer Emotional Response to Tastes and Foods through Facial Expression Analysis. (Doctoral dissertation, Virginia Tech). https://hdl.handle.net/10919/54538
Atwal, G., Bryson, D., & Kaiser, M. (2022). The chopsticks debacle: How brand hate flattened Dolce & Gabbana in China. Journal of Business Strategy, 43(1), 37–43. https://doi.org/10.1108/JBS-07-2020-0160
Baer, J. (2016). Hug your haters: How to embrace complaints and keep your customers. Portfolio, Penguin Publishing Group. https://archive.org/details/hugyourhatershow0000baer
Bhavana, V., Dhanalakshmi, M., & Aishwarya, S. (2023, December). A brief survey on facial expression detection. In 2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON) (Vol. 10, pp. 110–115). IEEE. https://doi.org/10.1109/UPCON59197.2023.10434419
Bond Jr, C. F., & DePaulo, B. M. (2006). Accuracy of deception judgments. Personality and Social Psychology Review, 10(3), 214–234. https://doi.org/10.1207/s15327957pspr1003_2
Bryson, D., Atwal, G., Hultén, P., & Heine, K. (2021). Antecedents of luxury brand hate: A quantitative study. Strategic Change, 30(1), 35–43. https://doi.org/10.1002/jsc.2387
Bustamam, A. (2023). Arrhythmia classification using the deep learning visual geometry group (VGG) model. Nusantara Science and Technology Proceedings, 7–18. https://nstproceeding.com/index.php/nuscientech/article/view/1080
Buroon, J. K., Bulkr, D. B., Ebesu, A. S., White, C. H., & Rockwell, P. A. (1996). Testing interpersonal deception theory: Effects of suspicion on communication behaviors and perceptions. Communication Theory, 6(3), 243–267. https://doi.org/10.1111/j.1468-2885.1996.tb00128.x
Cleeren, K., Van Heerde, H. J., & Dekimpe, M. G. (2013). Rising from the ashes: How brands and categories can overcome product-harm crises. Journal of Marketing, 77(2),
58–77. https://doi.org/10.1509/jm.10.0414
Crist, C. A., Duncan, S. E., Arnade, E. A., Leitch, K. A., O’Keefe, S. F., & Gallagher, D. L. (2018). Automated facial expression analysis for emotional responsivity using an aqueous bitter model. Food Quality and Preference, 68, 349–359. https://doi.org/10.1016/j.foodqual.2018.04.004
Curina, I., Francioni, B., Cioppi, M., and Savelli, E. (2021). Traits and peculiarities of different brand hate behaviours, Journal of Strategic Marketing, Vol. 29 No. 3, 227–246. https://doi.org/10.1080/0965254X.2019.1676293
Dada, E. G., Oyewola, D. O., Joseph, S. B., Emebo, O., & Oluwagbemi, O. O. (2023). Facial emotion recognition and classification using the Convolutional Neural Network‐10 (CNN‐10). Applied Computational Intelligence and Soft Computing, 2023(1), 2457898. https://doi.org/10.1155/2023/2457898
Danner, L., & Duerrschmid, K. (2018). Automatic facial expressions analysis in consumer science. In Methods in Consumer Research, Volume 2 231–252. Woodhead Publishing. https://doi.org/10.1016/B978-0-08-101743-2.00010-8
Dong, X., Veloutsou, C., & Morgan-Thomas, A. (2024). Negative online brand engagement: conceptualisation, scale development and validation. Journal of Research in Interactive Marketing, (ahead of print). https://doi.org/10.1108/JRIM-09-2023-0303
Durga, B. K., Rajesh, V., Jagannadham, S., Kumar, P. S., Rashed, A. N. Z., & Saikumar, K. (2023). Deep learning-based micro facial expression recognition using an adaptive tiefes FCNN model. Traitement du Signal, 40(3). https://doi.org/10.18280/ts.400319
Ezati, A., Dezyani, M., Rana, R., Rajabi, R., & Ayatollahi, A. (2024). A Lightweight Attention-based Deep Network via Multi-Scale Feature Fusion for Multi-View Facial Expression Recognition. arXiv preprint arXiv:2403.14318. https://doi.org/10.48550/arXiv.2403.14318
Febrian, R., Halim, B. M., Christina, M., Ramdhan, D., & Chowanda, A. (2023). Facial expression recognition using bidirectional LSTM-CNN. Procedia Computer Science, 216(2022), 39–47. https://doi.org/10.1016/j.procs.2022.12.109
Fetscherin, M. (2019). The five types of brand hate: How they affect consumer behavior. Journal of Business Research, 101, 116–127. https://doi.org/10.1016/j.jbusres.2019.04.017
Generosi, A., Ceccacci, S., & Mengoni, M. (2018, September). A deep learning-based system to track and analyze customer behavior in retail store. In 2018 IEEE 8th International Conference on Consumer Electronics – Berlin (ICCE-Berlin) (pp. 1–6). IEEE. https://doi.org/10.1109/ICCE-Berlin.2018.8576169
Gunvantray, T. D., & Ananthan, T. (2024, March). Sign language to text translation using convolutional neural network. In 2024 International Conference on Emerging Smart Computing and Informatics (ESCI) (pp. 1–5). IEEE. https://doi.org/10.1109/ESCI59607.2024.10497209
Gusnadi, F., Efendi, M., & Apriyana, N. (2024). Analisis brand hate terhadap brand avoidance pada merek Starbucks konteks issue Israel-Palestina. Jurnal Cahaya Mandalika ISSN 2721–4796 (online), 5(1), 265–277. https://doi.org/10.36312/jcm.v5i1.2726.
Hamelin, N., El Moujahid, O., & Thaichon, P. (2017). Emotion and advertising effectiveness: A novel facial expression analysis approach. Journal of Retailing and Consumer Services, 36, 103–111. https://doi.org/10.1016/j.jretconser.2017.01.001
Hashim, S., & Kasana, S. (2019). Antecedents of brand hate in the fast food industry. Spanish Journal of Marketing – ESIC, 23(2), 227–248. https://doi.org/10.1108/SJME-10-2018-0047
Hegner, S. M., Fetscherin, M., & Van Delzen, M. (2017). Determinants and outcomes of brand hate. Journal of Product & Brand Management, 26(1), 13–25. https://doi.org/10.1108/JPBM-01-2016-1070
Hess, U., & Thibault, P. (2009). Darwin and emotion expression. American Psychologist, 64(2), 120. https://psycnet.apa.org/buy/2009-01602-003
Huang, Z. Y., Chiang, C. C., Chen, J. H., Chen, Y. C., Chung, H. L., Cai, Y. P., & Hsu, H. C. (2023). A study on computer vision for facial emotion recognition. Scientific Reports, 13(1), 8425. https://doi.org/10.1038/s41598-023-35446-4
Husnain, M., Syed, F., Hussain, K., Zhang, Q., Usman, M., & Javed, M. (2023). Explaining the mechanism of brand hate: a mixed-method investigation using moderated mediation in emerging markets. Kybernetes, 52(10), 3857–3877. https://doi.org/10.1108/K-12-2021-1246
Iranzo Barreira, Á., Kuster, I., & Ruiz Mafe, C. (2024). Inward negative emotions and brand hate in users of snow-sports’ brands. Journal of Product & Brand Management, 33(6), 745-760. https://doi.org/10.1108/JPBM-05-2023-4541
Kaur, J., & Singh, W. (2024). A systematic review of object detection from images using deep learning. Multimedia Tools and Applications, 83(4), 12253-12338. https://doi.org/10.1007/s11042-023-15981-y
Khaydarova, N. (2024). Some features of verbal and non-verbal types of communication. XALQARO ILMIY-AMALIY KONFERENSIYALAR, 1(1), 288–291. https://tkti.uz/uz/events/detail/xalqaro-ilmiy-amaliy-konferentsiya/
Kopalidis, T., Solachidis, V., Vretos, N., & Daras, P. (2024). Advances in facial expression recognition: A Survey of methods, benchmarks, models, and datasets. Information, 15(3), 135. https://doi.org/10.3390/info15030135
Kucuk, S. U. (2019a). Consumer brand hate: Steam rolling whatever I see. Psychology & Marketing, 36(5), 431–443. https://doi.org/10.1002/mar.21175
Kucuk, S. U., (2019b). What is brand hate? . In: Brand Hate. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-00380-7_2
Kumar, S., Sagar, V., & Punetha, D. (2023). A comparative study on facial expression recognition using local binary patterns, convolutional neural network and frequency neural network. Multimedia Tools and Applications, 82(16), 24369–24385. https://doi.org/10.1007/s11042-023-14753-y.
Leong, S. C., Tang, Y. M., Lai, C. H., & Lee, C. K. M. (2023). Facial expression and body gesture emotion recognition: A systematic review on the use of visual data in affective computing. Computer Science Review, 48, 100545. https://doi.org/10.1016/j.cosrev.2023.100545
Li, J., & Lam, E. Y. (2015, September). Facial expression recognition using deep neural networks. In 2015 IEEE International Conference on Imaging Systems and Techniques (IST) (pp. 1–6). IEEE. https://doi.org/10.1109/IST.2015.7294547
Markowitz, D. M. (2024). Deconstructing deception: Frequency, communicator characteristics, and linguistic features of embeddedness. Applied Cognitive Psychology, 38(3), e4215. https://doi.org/10.1002/acp.4215
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and emotion, 23(2), 209–237. https://doi.org/10.1080/02699930802204677
Mednini, L., & Turki, M. D. (2022, May). What leads customer to create and participate in anti-brand community: a netnographic approach. In International Conference on Digital Economy (pp. 159–169). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-17037-9_11
Mednini, L., Noubigh, Z., & Turki, M. D. (2024). Natural language processing for detecting brand hate speech. Journal of Telecommunications and the Digital Economy, 12(1), 486–509. https://search.informit.org/doi/abs/10.3316/informit.T2024040300011091695498163
Mednini, L., & Damak Turki, M. (2024). How to transform brand haters into forgivers through emotional intelligence? Management Decision, 62(1), 183–199. https://doi.org/10.1108/MD-06-2022-0819
Mednini, L., & Hmida, I. C. B. (2023, May). Antecedents and outcomes of brand hate: A case of anti-brand community. In International Conference on Digital Economy (pp. 222–234). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-42788-6_14
Meena, G., Mohbey, K. K., Indian, A., Khan, M. Z., & Kumar, S. (2024). Identifying emotions from facial expressions using a deep convolutional neural network-based approach. Multimedia Tools and Applications, 83(6), 15711–15732. https://doi.org/10.1007/s11042-023-16174-3
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2017). AffectNet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, 10(1), 18–31. https://doi.org/10.1109/TAFFC.2017.2740923
Mushtaq, F. M., Ghazali, E. M., & Hamzah, Z. L. (2024). Brand hate: a systematic literature review and future perspectives. Management Review Quarterly, 1–34. https://doi.org/10.1007/s11301-023-00402-z
Naga, P., Marri, S. D., & Borreo, R. (2023). Facial emotion recognition methods, datasets and technologies: A literature survey. Materials Today: Proceedings, 80, 2824–2828. https://doi.org/10.1016/j.matpr.2021.07.046
Naidoo, S. W., Naicker, N., Patel, S. S., & Govender, P. (2022). Computer vision: the effectiveness of deep learning for emotion detection in marketing campaigns. International Journal of Advanced Computer Science and Applications; 13(5). doi: https://doi.org/10.14569/ijacsa.2022.01305100
Noubigh, Z., Mezghani, A., & Kherallah, M. (2020, November). Transfer learning to improve Arabic handwriting text recognition. In 2020 21st International Arab Conference on Information Technology (ACIT) (pp. 1–6). IEEE. https://doi.org/10.1109/ACIT50332.2020.9300105
Noor, U., Mansoor, M., & Rabbani, S. (2022). Brand hate and retaliation in Muslim consumers: Does offensive advertising matter? Journal of Islamic Marketing, 13(6), 1395–1413. https://doi.org/10.1108/JIMA-10-2020-0316
Noubigh, Z., Mezghani, A., & Kherallah, M. (2021). Densely connected layer to improve VGGnet-based CRNN for Arabic handwriting text line recognition. International Journal of Hybrid Intelligent Systems, 17(3–4), 113–127. https://doi.org/10.3233/HIS-210009
Odoom, R., Kosiba, J. P. B., & Odoom, P. T. (2024). Brand hate experiences and the role of social media influencers in altering consumer emotions. Journal of Brand Management, 1–14. https://doi.org/10.1057/s41262-024-00356-x
Pantano, E. (2020). Non-verbal evaluation of retail service encounters through consumers’ facial expressions. Computers in Human Behavior, 111, 106448. https://doi.org/10.1016/j.chb.2020.106448
Pereira, J., and Loureiro, S. M. C. (2020), The different shades of brand hate: A qualitative approach, 2020 Global Marketing Conference at Seoul Proceedings: 632-636 (November 2020), pp. 632–636. https://db.koreascholar.com/Article/Detail/401143
Pinto, O., & Brandão, A. (2021). Antecedents and consequences of brand hate: Empirical evidence from the telecommunication industry. European Journal of Management and Business Economics, 30(1), 18–35. https://doi.org/10.1108/EJMBE-04-2020-0084
Rachman, R., Hamid, M. A., Wijaya, B. K., Wibowo, S. E., & Intan, D. N. (2024). Brand storytelling in the digital age: challenges and opportunities in online marketing. Jurnal Ekonomi, 13(01), 355–364. https://ejournal.seaninstitute.or.id/index.php/Ekonomi/article/view/3748
Rahimah, A., Dang, H. P., Nguyen, T. T., Cheng, J. M. S., & Kusumawati, A. (2023). The subsequent effects of negative emotions: From brand hate to anti-brand consumption behavior under moderating mechanisms. Journal of Product & Brand Management, 32(4), 618–631. https://doi.org/10.1108/JPBM-12-2021-3778
Ren, C., Moisieiev, D., Rodrigo, P., & Johnson, E. (2024). Hot and cold: How do consumers hate and forgive offending charity brands? Journal of Philanthropy and Marketing, 29(3), e1875. https://doi.org/10.1002/nvsm.1875
Rozin, P., Haidt, J., McCauley, C., Dunlop, L., & Ashmore, M. (1999). Individual differences in disgust sensitivity: Comparisons and evaluations of paper-and-pencil versus behavioral measures. Journal of Research in Personality, 33(3), 330–351. https://doi.org/10.1006/jrpe.1999.2251
Sameeni, M. S., Qadeer, F., Ahmad, W., & Filieri, R. (2024). An empirical examination of brand hate influence on negative consumer behaviors through NeWOM intensity. Does consumer personality matter? Journal of Business Research, 173, 114469. https://doi.org/10.1016/j.jbusres.2023.114469
Scherer, K. R. (2009). The dynamic architecture of emotion: Evidence for the component process model. Cognition and emotion, 23(7), 1307–1351. https://doi.org/10.1080/02699930902928969
Shahzad, F., Qazi, T. A., & Shehzad, R. (2023). Framing of Israel and Palestine conflict in RT news, Al-Jazeera, CNN & BBC News. Global Digital & Print Media Review, 6(II), 1–14. https://doi.org/10.31703/gdpmr.2023(VI-II).01
Sharma, A., Fadahunsi, A., Abbas, H., & Pathak, V. K. (2022). A multi-analytic approach to predict social media marketing influence on consumer purchase intention. Journal of Indian Business Research, 14(2), 125–149. https://doi.org/10.1108/JIBR-08-2021-0313
Siomkos, G., Triantafillidou, A., Vassilikopoulou, A., & Tsiamis, I. (2010). Opportunities and threats for competitors in product‐harm crises. Marketing Intelligence & Planning, 28(6), 770–791. https://doi.org/10.1108/02634501011078156
Solbu, A., Frank, M. G., Xu, F., Nwogu, I., & Neurohr, M. (2024). The effect of synchrony of happiness on facial expression of negative emotion when lying. Journal of Nonverbal Behavior, 48(1), 73–92. https://doi.org/10.1007/s10919-023-00447-4
Sternberg, R. J. (2003). A duplex theory of hate: Development and application to terrorism, massacres, and genocide. Review of General Psychology, 7(3), 299–328. https://doi.org/10.1037/1089-2680.7.3.299
Sternberg, R. J. (2005). Understanding and combating hate. In R. J. Sternberg (Ed.), The Psychology of Hate, 37–49. American Psychological Association. https://doi.org/10.1037/10930-002
Taye, M. M. (2023). Theoretical understanding of convolutional neural network: Concepts, architectures, applications, future directions. Computation, 11(3), 52. https://doi.org/10.3390/computation11030052
Ten Brinke, L., & Porter, S. (2012). Cry me a river: identifying the behavioral consequences of extremely high-stakes interpersonal deception. Law and Human Behavior, 36(6), 469. https://psycnet.apa.org/doi/10.1037/h0093929
Valenzuela, C., Castellucci, L., Mena, M. T., & Bianchi, C. (2022). Consumer brand hate: The role of ambivalence. The International Review of Retail, Distribution and Consumer Research, 32(1), 100–125. https://doi.org/10.1080/09593969.2021.2018017
Wang, C. Y., & Lin, F. S. (2022, June). Assessing the effectiveness of digital advertising for green products: A facial expression evaluation approach. In International Conference on Human-Computer Interaction (pp. 221–232). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-031-05544-7_17
Wang, W., & Sun, L. (2024). Brand marketing strategy based on user emotion recognition model of consumer. Computer-Aided Design and Applications, 21(S12), 115–129. https://doi.org/10.14733/cadaps.2024.S12.115-129
Webb, T. L., Miles, E., & Sheeran, P. (2012). Dealing with feeling: a meta-analysis of the effectiveness of strategies derived from the process model of emotion regulation. Psychological bulletin, 138(4), 775. https://psycnet.apa.org/doi/10.1037/a0027600
Xue, Y., Sun, J., Liu, Y., Li, X., & Yuan, K. (2024). Facial expression-enhanced recommendation for virtual fitting rooms. Decision Support Systems, 177, 114082. https://doi.org/10.1016/j.dss.2023.114082
Xu, X., Ruan, Z., & Yang, L. (2020, July). Facial expression recognition based on graph neural network. In 2020 IEEE 5th International Conference on Image, Vision and Computing (ICIVC) (pp. 211–214). IEEE. https://ieeexplore.ieee.org/document/7832107/
Yadav, A. (2024). When personalities collide: Examining the impact of consumer and brand personalities’ interplay on brand hate development. Journal of Product & Brand Management. https://doi.org/10.1108/JPBM-01-2023-4307
Zloteanu, M., Bull, P., Krumhuber, E. G., & Richardson, D. C. (2021). Veracity judgement, not accuracy: Reconsidering the role of facial expressions, empathy, and emotion recognition training on deception detection. Quarterly Journal of Experimental Psychology, 74(5), 910–927. https://doi.org/10.1177/1747021820978851
Article Sidebar
Published
2025-04-01
Article Details
How to Cite
Mednini, L., & Noubigh, Z. (2025). Deep Learning-Based Facial Emotion Recognition for Detecting Brand Hate. Journal of Telecommunications and the Digital Economy, 13(1), 244-267. https://doi.org/10.18080/jtde.v13n1.1067
Issue
Section
Special Issue: Driving the Digital Economy
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright Telecommunications Association Inc.
How to Cite
Mednini, L., & Noubigh, Z. (2025). Deep Learning-Based Facial Emotion Recognition for Detecting Brand Hate. Journal of Telecommunications and the Digital Economy, 13(1), 244-267. https://doi.org/10.18080/jtde.v13n1.1067
