How intent recognition works?

📘

NLU is the acronym for Natural Language Understanding.

The NLU algorithm works by detecting similarities between statements. It is a similarity at the sentence or utterance level, hence, the similarity between elements of the phrases issued to the helpline are checked againsts elements of phrases within the training set for different intents.

What we are dealing with here is called CLASSIFICATION (in other words: categorization), that is, comparing and "putting phrases into drawers with similar phrases", that is, intents, or for the model terms - classes.

Hence, the more diverse the classes are (here: intents), the greater the probability of their unambiguous classification. Conversely, the more similar the phrases within different intents are, the less likely the model is to distinguish them from each other.

Similarly, if we have too many intents, e.g. a few hundred (many drawers) with a small number of phrases, we reduce the chance of building an effective model compared to a situation where we have fewer intents (from 30 to 100), but homogeneous in terms of phrases.

Therefore, it is worth ensuring that similar or identical sentences do not end up in two processes at the same time, as this will cause the model to be confused while classifying phrases to intents.