Decision Rule Ensemble Formation Via a Multicriteria Evolutionary Algorithm for the Problem of Human Emotion Analysis in Audio Data

One of the most important problems at the current stage of social informatisation is development of human-machine interaction systems, including automated human emotion recognition systems. It is possible to describe human emotions using a combination of two parameters: Valence, which represents how attractive an emotion is (referring to positive and negative emotions), and Arousal, denoting the strength of the emotion (that is, degree of agitation). These parameters are real numbers. We propose to employ ensemble learning methods to improve prediction accuracy. We evaluate the accuracy of an ensemble decision via its congruence coefficient. We used a multicriteria evolutionary algorithm to select agents (algorithms) for the ensemble. Employing a multicriteria evolutionary algorithm made it possible to automate the ensemble formation process, which enabled us to save time and physical resources. Ensemble formation depended on two criteria: maximising accuracy and minimising the number of agents in the ensemble. We used the following ensemble decision-making methods: majority voting, weighted average, weighted average in proportion to the agent trust, and a fuzzy logic system. We present a modification to the fuzzy logic system that improves solution efficiency for the data mining problem. We analysed and investigated how efficient a multicriteria evolutionary algorithm is when solving the problem of predicting emotional behaviour in humans. Our experiments showed that using a multicriteria evolutionary algorithm to automate ensemble formation improves the solution accuracy The study was supported by the Ministry of Education and Science of the Russian Federation as part of basic state funding of project no. 2.1680.2017/PCh

References

[1] Kuncheva L. Combining pattern classifiers. Methods and algorithms. Wiley, 2004.

[2] Hansen L.K., Salmon P. Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell., 1990, vol. 12, no. 10, pp. 993--1001. DOI: 10.1109/34.58871

[3] Yamaguchi T., Mackin K.J., Nunohiro E., et al. Artificial neural network ensemble-based land-cover classifiers using MODIS data. Artif. Life Robotics, 2009, vol. 13, no. 2, pp. 570--574. DOI: 10.1007/s10015-008-0615-4

[4] Ridgeway G. The state of boosting. Proc. 31st Symp. Interface, 1999, pp. 172--181

[5] Breiman L. Bagging predictors. Mach. Learn., 1996, vol. 24, no. 2, pp. 123--140. DOI: 10.1023/A:1018054314350

[6] Deb K., Agrawal S., Pratap A., et al. A fast elitist non-dominated sorting genetic algorithm for Multi-objective optimization: NSGA-II. Parallel Problem Solving from Nature PPSN VI. Springer, 2000, pp. 849--858. DOI: 10.1007/3-540-45356-3_83

[7] Zitzler E., Laumanns M., Thiele L. SPEA2: improving the strength Pareto evolutionary algorithm 2. TIK Report 103. Computer Engineering and Networks Laboratory, Swiss Federal Institute of Technology (ETH). Zurich, Switzerland, 2001

[8] Gunes H., Pantic M. Automatic, dimensional and continuous emotion recognition. IJSE, 2010, vol. 1, no. 1, pp. 68--99. DOI: 10.4018/jse.2010101605

[9] Drucker H., Burges C.J., Kaufman L., et al. Support vector regression machines. Adv. Neural Inf. Process. Syst., 1997, vol. 9, pp. 155--161.

[10] Tipping M.E. Sparse Bayesian learning and the relevance vector machine, JMLR, 2001, vol. 1, pp. 211--244.

[11] Kwok T.-Y., Yeung D.-Y. Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Trans. Neural Netw., 1997, vol. 8, no. 3, pp. 630--645. DOI: 10.1109/72.572102

[12] Williams R.J., Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Comput., 1989, vol. 1, no. 2, pp. 270--280. DOI: 10.1162/neco.1989.1.2.270

[13] Tian L., Moore J.D., Lai C. Emotion recognition in spontaneous and acted dialogues. Proc. ACII, 2015, pp. 698--704. DOI: 10.1109/ACII.2015.7344645

[14] Nicolaou M.A., Gunes H., Pantic M. Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Trans. Affect. Comput., 2011, vol. 2, no. 2, pp. 92--105. DOI: 10.1109/T-AFFC.2011.9

[15] Polyakova A., Lipinskiy L. A study of fuzzy logic ensemble system performance on face recognition problem. IOP Conf. Ser.: Mater. Sci. Eng., 2017, vol. 173, no. 1, art. 012013. DOI: 10.1088/1757-899X/173/1/012013

[16] Ringeval F., Sonderegger A., Sauer J., et al. Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. 10th IEEE Int. Conf. Workshops on Automatic Face and Gesture Recognition (FG), 2013, pp. 1--8. DOI: 10.1109/FG.2013.6553805

[17] Russell J.A. A circumplex model of affect. J. Pers. Soc. Psychol., 1980, vol. 39, no. 6, pp. 1161--1178. DOI: 10.1037/h0077714

[18] Ringeval F., Eyben F., Kroupi E., et al. Prediction of asynchronous dimensional emotion ratings from audio-visual and physiological data. Pattern Recogni. Lett., 2015, vol. 66, pp. 22--30. DOI: 10.1016/j.patrec.2014.11.007