DOI: 10.17587/prin.12.283-294

Algorithm for Automated Generation of a Training Sample for Solving the Problem of Determining Semantic Similarity between a Pair of Keywords using Machine Learning Methods

K. V. Lunev, kirilllunev@gmail.com, Moscow State University, Moscow, 119192, Russian Federation

Corresponding author: Lunev Kirill V., Postgraduate Student, Moscow State University, Moscow, 119192, Russian Federation, E-mail: kirilllunev@gmail.com

Currently, machine learning is an effective approach to solving many problems of information-analytical systems. To use such approaches, a training set of examples is required. Collecting a training dataset is usually a time-consuming process. Its implementation requires the participation of several experts in the subject area for which the training set is collected. Moreover, for some tasks, including the task of determining the semantic similarity of keyword pairs, it is difficult even to correctly draw up instructions for experts to adequately evaluate the test examples. The reason for such difficulties is that semantic similarity is a subjective value and strongly depends on the scope, context, person, and task. The article presents the results of research on the search for models, algorithms and software tools for the automated formation of objects of the training sample in the problem of determining the semantic similarity of a pair of words. In addition, models built on an automated training sample allow us to solve not only the problem of determining semantic similarity, but also an arbitrary problem of classifying edges of a graph. The methods used in this paper are based on graph theory algorithms.

Keywords: semantic similarity, natural language processing, graph algorithms, graph theory, machine learning

pp. 283–294

For citation:
Lunev K. V. Algorithm for Automated Generation of a Training Sample for Solving the Problem of Determining Semantic Similarity between a Pair of Keywords using Machine Learning Methods, Programmnaya Ingeneria, 2021, vol. 12, no. 6, pp. 283—294.