Facial Expression Decoding based on fMRI Brain Signal

Benchun Cao, Yanchun Liang, Shinichi Yoshida, Renchu Guan

Abstract


The analysis of facial expressions is a hot topic in brain-computer interface research. To determine the facial expressions of the subjects under the corresponding stimulation, we analyze the fMRI images acquired by the Magnetic Resonance. There are six kinds of facial expressions: "anger", "disgust", "sadness", "happiness", "joy" and "surprise". We demonstrate that brain decoding is achievable through the parsing of two facial expressions ("anger" and "joy"). Support vector machine and extreme learning machine are selected to classify these expressions based on time series features. Experimental results show that the classification performance of the extreme learning machine algorithm is better than support vector machine. Among the eight participants in the trials, the classification accuracy of three subjects reached 70-80%, and the remaining five subjects also achieved accuracy of 50-60%. Therefore, we can conclude that the brain decoding can be used to help analyzing human facial expressions.

Keywords


Brain-computer interface, machine learning, extreme learning machine, fMRI, image processing

Full Text:

PDF

References


Arlot, S.; Celisse, A.(2010). A survey of cross-validation procedures for model selection, Statistics Survey, 4, 40-79, 2010.
https://doi.org/10.1214/09-SS054

Blum, A.; Kalai, A.(1999). Beating the hold-out:bounds for K-fold and progressive crossvalidation, Proceedings of the twelfth annual conference on Computational learning theory. ACM, 203-208, 1999.
https://doi.org/10.1145/307400.307439

Davis, M; Whalen, P. J.(2001). The amygdala: vigilance and emotion, Mol Psychiatry, 6(1), 13-34, 2001.
https://doi.org/10.1038/sj.mp.4000812

Desikan, R.S.; Segonne, F; Fischl, B.(2006). An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest, Neuroimage, 31(3), 968-980, 2006.
https://doi.org/10.1016/j.neuroimage.2006.01.021

Ekman, P.(1992). An argument for basic emotions, Cognition & Emotion, 6(3-4), 169-200, 1992.
https://doi.org/10.1080/02699939208411068

Forman, S D; Cohen, J.D.; Fitzgerald, M. et al.(1992). Improved Assessment of Significant Activation in Functional Magnetic Resonance Imaging (fMRI): Use of a Cluster-Size Threshold, Magnetic Resonance in Medicine, 3(5), 636-647, 2010.
https://doi.org/10.1002/mrm.1910330508

Gilead, M.; Liberman, N.; Maril, A.(2013). The language of future-thought: An fMRI study of embodiment and tense processing, Neuroimage, 65(2), 267-279, 2013.
https://doi.org/10.1016/j.neuroimage.2012.09.073

Huang, G.B.; Zhu, Q.Y.; Siew, C.K.(2004). Extreme learning machine: a new learning scheme of feedforward neural networks, IEEE International Joint Conference on Neural Networks, 2, 985-990, 2005.

Huang, G. B.; Zhu, Q. Y.; Siew, C.K.(2006). Extreme learning machine: Theory and applications, Neurocomputing, 70(1), 489-501, 2006.
https://doi.org/10.1016/j.neucom.2005.12.126

Kearns, M; Ron, D.(1997). Algorithmic stability and sanity-check bounds for leave-one-out cross-validation, Neural Computation, 11(6), 1427-1453, 1997.
https://doi.org/10.1162/089976699300016304

Logothetis, N.K.; Pauls, J.; Augath, M.(2001). Neurophysiological investigation of the basis of the fMRI signal, Nature, 412(6843), 150-157, 2001.
https://doi.org/10.1038/35084005

Lehmann, T.M; Gonner, C.; Spitzer, K.(1999). Survey: interpolation methods in medical image processing, IEEE Transactions on Medical Imaging, 18(11), 1049-1075, 1999.
https://doi.org/10.1109/42.816070

Litvak, V.; Mattout, J.; Kiebel, S. et al. (2011). EEG and MEG Data Analysis in SPM8, Computational Intelligence and Neuroscience, 2011(3), 852961, 2011.
https://doi.org/10.1155/2011/852961

Michel, C.M; Murray, M.M.; Lantz, G.(2004). EEG source imaging, Clinical Neurophysiology, 115(10), 2195-2222, 2004.
https://doi.org/10.1016/j.clinph.2004.06.001

Rodriguez, J.D.; Perez, A.; Lozano, J.A.(2010). Sensitivity Analysis of k-Fold Cross Validation in Prediction Error Estimation, IEEE Computer Society, 32(3), 569-575, 2010.
https://doi.org/10.1109/TPAMI.2009.187

Schalk, G.; Mcfarland, D.J.; Hinterberger, T.(2004). BCI2000: a general-purpose braincomputer interface (BCI) system, IEEE Transactions on Bio-medical Engineering, 51(6), 1034-1043, 2004.
https://doi.org/10.1109/TBME.2004.827072

Suykens, J.A.K.(1999). Least squares support vector machine classifiers: a large scale algorithm[ C]. European Conference on Circuit Theory and Design, 1999, 839-842, 1999.

Tugui, A. (2014). GLM Analysis for fMRI using Connex Array International Journal of Computers Communications & Control, 9(6), 768-775, 2014.
https://doi.org/10.15837/ijccc.2014.6.1482

Ziegel, E.R.(2012). An Introduction to Generalized Linear Models[J]. Technometrics, 44(4), 406-407, 2012.
https://doi.org/10.1198/tech.2002.s91

Zilles, K; Amunts, K.(2010). Centenary of Brodmann's map-conception and fate, Nature Reviews Neuroscience, 11(2), 139-145, 2010.
https://doi.org/10.1038/nrn2776




DOI: https://doi.org/10.15837/ijccc.2019.4.3433



Copyright (c) 2019 Benchun Cao, Yanchun Liang, Shinichi Yoshida, Renchu Guan

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

CC-BY-NC  License for Website User

Articles published in IJCCC user license are protected by copyright.

Users can access, download, copy, translate the IJCCC articles for non-commercial purposes provided that users, but cannot redistribute, display or adapt:

  • Cite the article using an appropriate bibliographic citation: author(s), article title, journal, volume, issue, page numbers, year of publication, DOI, and the link to the definitive published version on IJCCC website;
  • Maintain the integrity of the IJCCC article;
  • Retain the copyright notices and links to these terms and conditions so it is clear to other users what can and what cannot be done with the  article;
  • Ensure that, for any content in the IJCCC article that is identified as belonging to a third party, any re-use complies with the copyright policies of that third party;
  • Any translations must prominently display the statement: "This is an unofficial translation of an article that appeared in IJCCC. Agora University  has not endorsed this translation."

This is a non commercial license where the use of published articles for commercial purposes is forbiden. 

Commercial purposes include: 

  • Copying or downloading IJCCC articles, or linking to such postings, for further redistribution, sale or licensing, for a fee;
  • Copying, downloading or posting by a site or service that incorporates advertising with such content;
  • The inclusion or incorporation of article content in other works or services (other than normal quotations with an appropriate citation) that is then available for sale or licensing, for a fee;
  • Use of IJCCC articles or article content (other than normal quotations with appropriate citation) by for-profit organizations for promotional purposes, whether for a fee or otherwise;
  • Use for the purposes of monetary reward by means of sale, resale, license, loan, transfer or other form of commercial exploitation;

    The licensor cannot revoke these freedoms as long as you follow the license terms.

[End of CC-BY-NC  License for Website User]


INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL (IJCCC), With Emphasis on the Integration of Three Technologies (C & C & C),  ISSN 1841-9836.

IJCCC was founded in 2006,  at Agora University, by  Ioan DZITAC (Editor-in-Chief),  Florin Gheorghe FILIP (Editor-in-Chief), and  Misu-Jan MANOLESCU (Managing Editor).

Ethics: This journal is a member of, and subscribes to the principles of, the Committee on Publication Ethics (COPE).

Ioan  DZITAC (Editor-in-Chief) at COPE European Seminar, Bruxelles, 2015:

IJCCC is covered/indexed/abstracted in Science Citation Index Expanded (since vol.1(S),  2006); JCR2018: IF=1.585..

IJCCC is indexed in Scopus from 2008 (CiteScore2018 = 1.56):

Nomination by Elsevier for Journal Excellence Award Romania 2015 (SNIP2014 = 1.029): Elsevier/ Scopus

IJCCC was nominated by Elsevier for Journal Excellence Award - "Scopus Awards Romania 2015" (SNIP2014 = 1.029).

IJCCC is in Top 3 of 157 Romanian journals indexed by Scopus (in all fields) and No.1 in Computer Science field by Elsevier/ Scopus.

 

 Impact Factor in JCR2018 (Clarivate Analytics/SCI Expanded/ISI Web of Science): IF=1.585 (Q3). Scopus: CiteScore2018=1.56 (Q2); Editors-in-Chief: Ioan DZITAC & Florin Gheorghe FILIP.