Оntological approach to aggregated evaluation of the work of teams with multiple levels of hierarchy





ontological approach, competence, hierarchical team, aggregated evaluation, military unit, semantic proximity, machine learning


Evaluation of organizations with multi-level hierarchical structure (project groups, groups of researchers, military units, expert commissions, medical rapid response teams) is an urgent problem today, the solution of which will allow to organize their rapid dynamic adaptation in a new way to perform important operational tasks, as well as training team members in the absence of sufficient competencies, skills and experience. The assessment also reveals the strengths and weaknesses of both the team as a whole and each participant, which in the future provides opportunities for their further growth. Evaluation of the work of teams is complex, including both professional knowledge in a certain field of activity and team management skills (so-called leadership skills, the ability to organize the execution of tasks and the achievement of goals and obtaining positive work results). In order to improve the evaluations of hierarchical aggregated teams we propose the use of ontological approach: domain ontology defines knowledge about relevant combinations of competencies for hierarchical team positions. Information about competencies of applicants is acquired by portfolio analysis (resume, certificates, diplomas, merits, etc.). More complex tasks can use a hierarchical set of ontologies and Web-services for obtaining evaluation results and recommendations for competence improving for various sublevels. We describe the step-by-step method of team evaluation that use elements of semantic similarity between different information objects for the matching of applicants and equipment with team positions. Proposed approach to hierarchical team evaluation is a component for integrated multi-criteria decision-making oriented on some special case of user tasks: the set of evaluation criteria is determined by task and can be built on the basis of domain knowledge, but importance of particular criterion is defined by state of environment in different points in time. This approach can be expanded by means and methods of knowledge management, Data mining and machine learning used for acquisition of competence knowledge.

Author Biographies

Anatoly Gladun, Institute of special communication and information protection of National technical university of Ukraine “Igor Sikorsky Kyiv polytechnic institute”, Kyiv

candidate of engineering sciences, associate professor, associate professor at the cybersecurity and application of information systems and technology academic department

Julia Rogushyna, Institute of software systems of National academy of sciences of Ukraine, Kyiv

candidate of physical and mathematical sciences, associate professor, senior researcher

Martin Lesage, University of Québec at Montreal, Montreal

doctor of philosophy in education, course instructor and course administrator for the canadian military engineers, associate professor at the education and pedagogy academic department


M. Lesage, G. Raîche, M. Riopel, F. Fortin, D. Sebkhi, and Y. Otis, “Teamwork Assessment with Several Levels of Hierarchy in Education: The Hierarchical Aggregate Assessment Process”, in New Horizons in Education and Social Studies. V. Hus, Eds., vol. 7. London, United Kingdom: Book Publisher International, 2020, pp. 124-164). [Online]. Available: http://www.bookpi.org/bookstore/product/new-horizons-in-education-and-social-studies-vol-7/. Accessed on: Aug. 10, 2022.

S. Ghosh, S. Thomke, and H. Pourkhalkhali, The Effects of Hierarchy on Learning and Performance in Business Experimentation. Harvard: Harvard business School, 2020. Paper 20-081. [Online]. Available: https://www.hbs.edu/ris/Publication%20Files/20-081_9d2608f9-5a49-4bc3-a902-307d3427477b.pdf. Accessed on: Aug. 10, 2022.

M. Lesage, “Hierarchical Aggregate Assessment (HAA): An Assessment Process of Teams with Several Levels of Hierarchy in Education”, Creative Education, vol. 7, no. 14, pp. 1974-1994, 2016, doi: http://dx.doi.org/10.4236/ce.2016.714200. Accessed on: Aug. 10, 2022.

G. Sammour, A. Al-Zoubi, A. Gladun, K. Khala, and J. Schreurs, “Semantic web and ontologies for personalisation of learning in MOOCs”, in Proc. IEEE 7th International Conference on Intelligent Computing and Information Systems, Cairo, 2015, pp. 185-190, doi: https://doi.org/10.1109/IntelCIS.2015.7397219. Accessed on: Aug. 10, 2022.

W. D. Nance, “Improving information systems students’ teamwork and project management capabilities: Experiences from an innovative classroom”, Information Technology and Management, vol. 1, iss. 4, pp. 293-306, 2000, doi: https://doi.org/10.1023/A:1019137428045.

A. Gladun, and J. Rogushina, “Use of ontological analysis for evaluation of expert competences in the domain of national standards development”, Systemic research and information technologies, no. 3, pp. 19-32, 2016, doi: https://doi.org/10.20535/SRIT.2308-8893.2016.3.02.

S. M. Pryima, J. V. Rogushina, A. Y. Gladun, and O. V. Strokan, “AdvisOnt: Semanticization of agricultural advisory services for validation of the results of non-formal and informal education”, Control Systems and Computers, no. 1, pp. 62-70, 2021, doi: https://doi.org/10.15407/csc.2021.01.062.

A. Hladun, Y. Rohushyna, and I. Subach, “An ontology modelling human resources management for innovational domains”, Information Technology and Security, vol. 6, iss. 1, pp. 15-25, 2018, doi: https://doi.org/10.20535/2411-1031.2018.6.1.153125.

A. Ya. Gladun, and K. O. Khala, “The use of ontological models for formalized knowledge assessment”. Computer tools, networks and systems, no. 18, pp. 5-10, 2019.

M. Lesage, G. Raîche, M. Riopel, F. Fortin, and D. Sebkhi, “The internet implementation of the hierarchical aggregate assessment process with the “Cluster” Wi-Fi E-learning and E assessment application: A particular case of teamwork assessment”, in E-Learning: Instructional Design, Organizational Strategy and Management, B. Gradinarova, Eds. Rijeka, Croatia: InTech Europe, 2015, pp. 83-125, doi: http://dx.doi.org/10.5772/60850.

K. C. Laudon, and J. P. Laudon, Management information systems: Organization and technology in the networked enterprise, Harlow, England: Pearson Education Limited, 2014.

K. Willey, and M. Freeman, “Improving teamwork and engagement: The case for self and peer assessment”, Australasian Journal of Engineering Education, vol. 12, iss. 2; pp. 1-19, 2006.

H. Van Zyl, and L. Massyn, “Integrated assessment: A learning adventure and growth opportunity for adult learners”, American Journal of Business Education, vol. 1, iss. 2, pp. 95-104, 2008, doi: https://doi.org/10.19030/ajbe.v1i2.46282.

R. W. Lingard, “Teaching and assessing teamwork skills in engineering and computer science”, Journal of Systemics, Cybernetics and Informatics, vol. 8, iss. 1, pp. 35, 2010.

D. Gijbels, P. Dochy, P. Van Den Bossche, and M. Segers, “Effects of problem-based learning: A metaanalysis from the angle of assessment”, Review of Educational Research, vol. 75, iss. 1, pp. 29-33, 2005, doi: https://doi.org/10.3102/00346543075001027.

A. Ya. Gladun, and Ju. V. Rogushina, Data Mining – finding knowledge in data. Kyiv, Ukraine: LLC “VD “ADEF-Ukraine”, 2016.

D. V. Lande, I. Yu. Subach, and Yu. E Boyarynova, Fundamentals of the theory and practice of intelligent data analysis in the field of cybersecurity. Kyiv, Ukraine: ISCIS of Igor Sikorsky Kyiv Polytechnic Institut, 2018.

D. V. Lande, and I. Yu. Subach, Visualization and analysis of network structures. Kyiv, Ukraine: ISCIS of Igor Sikorsky Kyiv Polytechnic Institut, 2020.

Yu. V. Rogushina, A. Ya. Gladun, and V.M. Shtonda, “Development of ontological terminological systems of Internet information resources and their cognitive models in scientific research”, Problems of programming. no. 2-3. pp. 114-122, 2010.

J. Rogushina, and S. Priyma, “Use of competence ontological model for matching of qualifications”, Chemistry: Bulgarian Journal of Science Education, vol. 26, no. 2, pp. 216-228, 2017.

The European Multilingual Classifier of Skills, Competences, Qualifications and Occupations. [Online]. Available: https://ec.europa.eu/esco/portal/home. Accessed on: Aug. 10, 2022.

A. Ya. Gladun, and Yu. V. Rogushina, “Organizational ontologies and knowledge management in decision-making support systems”, in Proc. International scientific conference Intelligent decision-making systems and applied aspects of information technologies, vol. 1. Kherson, 2006, pp. 363-366.

A. Y. Gladun, and K. A. Khala, “Ontology-based semantic similarity to metadata analysis in the information security domain”, Prombles in programming, no. 2. pp. 34-41, 2021, doi: https://doi.org/10.15407/pp2021.02.034.



How to Cite

Gladun, A., Rogushyna, J., & Lesage, M. (2022). Оntological approach to aggregated evaluation of the work of teams with multiple levels of hierarchy. Collection "Information Technology and Security", 10(2), 126–140. https://doi.org/10.20535/2411-1031.2022.10.2.270284