GRNTI 50.07 Теоретические основы вычислительной техники
BBK 3297 Вычислительная техника
The paper is devoted to using Internet of Things technologies for hardware human-machine interfaces development. Thanks to these technologies, it may be possible to improve the capabilities of visual analytics systems with multiple modalities: movements, audio, etc. It can speed up semantic data filtering and interpretation, increasing the efficiency of analytics. We suggest using ontology engineering methods and tools to automate both the programming of custom hardware human-machine interfaces and connecting them to the third-party software. The proposed concept is tested by solving the real-world tasks of discovering the relationships between the psychological characteristics of the native speakers and their verbal behavior.
visual analytics, Internet of Things, human-machine interface, ontology engineering
1. Aravena P., Delevoye-Turrell Y., Deprez V., Cheylus A., Paulignan Y., Frak V., Nazir T. Grip Force Reveals the Context Sensitivity of Language-Induced Motor Activity during «Action Words» Processing: Evidence from Sentential Negation // PLoS ONE. – 2012. – V. 7, I. 12. DOI: 10.1371/journal.pone.0050287
2. Barsalou L. Perceptual Symbol Systems // Behavioral and Brain Sciences. – 1999. – V. 22. – PP. 577–609.
3. Belousov K., Erofeeva E., Leshchenko Y., Baranov D. «Semograph» Information System as a Framework for Network-Based Science and Education // Smart Education and eLearning. – Springer, 2017. – PP. 263–272. DOI: 10.1007/978-3-319-59451-4_26.
4. Chang Sh.-F., Ellis D., Jiang W., Lee K., Yanagawa A., Loui A., Luo J. Large-Scale Multimodal Semantic Concept Detection for Consumer Video // Multimedia Information Retrieval. – 2007. – P. 255–264. DOI: 10.1145/1290082.1290118.
5. Gallese V., Lakoff G. The Brain Concepts: the Role of the Sensorymotor System in Conceptual Structure // Cognitive Neuropsychology. – 2005. – V. 22, I. 3. – PP. 455–479. DOI: 10.1080/02643290442000310.
6. Hoffman P., Lambon Ralph M. Shapes, Scents and Sounds: Quantifying the Full Multi-Sensory Basis of Conceptual Knowledge // Neuropsychologia. – Elsevier, 2013. – V. 51, I. 1. – PP. 14–25. DOI: 10.1016/j.neuropsychologia.2012.11.009.
7. Ishii H., Ullmer B. Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms // CHI ’97 Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. – ACM, 1997. – PP. 234–241. DOI: 10.1145/258549.258715.
8. Laeng B., Sulutvedt U. The Eye Pupil Adjusts to Imaginary Light // Psychological Science. – 2013. – V. 25, I. 1. – PP. 188–197. DOI: 10.1177/0956797613503556.
9. Lockwood G., Hagoort P., Dingemanse M. How Iconicity Helps People Learn New Words: Neural Correlates and Individual Differences in Sound-Symbolic Bootstrapping // Collabra: Psychology. – 2016. – V. 2, I. 1. – PP. 1–15.DOI: 10.1525/collabra.42.
10. Meteyard L., Cuadrado S., Bahrami B., Vigliocco G. Coming of Age: a Review of Embodiment and the Neuroscience of Semantics // Cortex. – Elsevier, 2012. – V. 48, I. 7. – PP. 788–804. DOI: 10.1016/j.cortex.2010.11.002.
11. Pulvermüller F. Words in the Brain’s Language // Behavioral and Brain Sciences. – 1999. – V. 22. – PP. 253–279.
12. Rose K., Eldridge S., Chapin L. The Internet of Things: an Overview // The Internet Society (ISOC). – 2015 [Elektronnyy resurs]. URL: https://www.internetsociety.org/ resources/doc/2015/iot-overview (Data obrascheniya 17.07.2019).
13. Ryabinin K.V., Belousov K.I., Chuprina S.I., Shchebetenko S.A., Permyakov S.S. Visual Analytics Tools for Systematic Exploration of Multi-Parameter Data of Social Web-Based Service Users // Scientific Visualization. – M.: National Research Nuclear University "MEPhI", 2018. – Q. 3, V. 10, No. 4. – PP. 82–99. DOI: 10.26583/sv.10.4.07.
14. Ryabinin K., Chuprina S. High-Level Toolset For Comprehensive Visual Data Analysis and Model Validation // Procedia Computer Science. – Elsevier, 2017. – V. 108. – PP. 2090–2099. DOI: 10.1016/j.procs.2017.05.050.
15. Ryabinin K., Chuprina S., Belousov K. OntologyDriven Automation of IoT-Based Human-Machine Interfaces Development // Lecture Notes in Computer Science. – Springer, 2019. – V. 11540. – PP. 110–124. DOI: 10.1007/978-3-030-22750-0_9.
16. Ryabinin K., Chuprina S., Kolesnik M. Calibration and Monitoring of IoT Devices by Means of Embedded Scientific Visualization Tools // Lecture Notes in Computer Science. – Springer, 2018. – V. 10861. – PP. 655–668. DOI: 10.1007/978-3-319-93701-4_52.
17. Sanfelice R. Analysis and Design of Cyber-Physical Systems. A Hybrid Control Systems Approach // Cyber-Physical Systems: From Theory to Practice / Rawat D., Rodrigues J., Stojmenovic I. – CRC Press, 2015. – PP. 3–31. DOI: 10.1201/b19290-3.
18. Shchebetenko S. Reflexive Characteristic Adaptations Explain Sex Differences in the Big Five: but not in Neuroticism // Personality and Individual Differences. – 2017. – V. 111. – PP. 153–156. DOI: 10.1016/j.paid.2017.02.013.
19. Staats A., Hammond O. Natural Words as Physiological Conditioned Stimuli: Food-WordElicited Salivation and Deprivation Effects // Journal of Experimental Psychology. – 1972. – V. 96, I. 1. – PP. 206–208. DOI: 10.1037/h0033508.
20. Zhang P., Zhou M., Fortino G. Security and trust issues in Fog computing: A survey // Future Generation Computer Systems. – Elsevier, 2018. – V. 88. – PP. 16–27. DOI: 10.1016/j.future.2018.05.008.
21. Durov A.V. Konspekt lekciy po kursu «Teoreticheskaya i prikladnaya lingvistika» / A. V. Durov. – 2018 [Elektronnyy resurs]. – URL: www.durov. com/study/bilety-1718.docx (Data obrascheniya 17.07.2019).
22. Ryabinin K.V., Baranov D.A., Belousov K.I. Integraciya instrumentariya nauchnoy vizualizacii SciVi s informacionnoy sistemoy Semograf // Trudy 27-y Mezhdunarodnoy konferencii GraphiCon 2017. – Perm', 2017. – S. 138–141.
23. Ryabinin K.V., Chuprina S.I., Belousov K.I., Permyakov S.S. Metody vizual'noy analitiki variativnosti rechevogo povedeniya pol'zovateley social'nyh setey v zavisimosti ot psihologicheskih chert lichnosti // Trudy 28-y Mezhdunarodnoy konferencii GraphiCon 2018. – Tomsk, 2018. – S. 163–167.
