Hypernyms
We are mainly interested in the hypernym/hyponym relationships between synsets for the Q&A task. A hypernym is a more general sense of a meaning—the hypernym of the synset {plant, flora} is {living thing, organism} whose hypernym is {thing, entity}. There are several such hierarchies of synsets that represent the specificity of a particular synset. For example, the word "horse" is found lower in the hierarchy than the more general word "animal."
Hypernyms enable matches between questions and answers without exact word comparisons. A hypernym ("animal") in a question matches a specific word ("horse") from an answer. You identify a set of hypernyms with every specific word. However, if you include hypernyms that are too specific, then words will have long lists of associated hypernyms, unnecessarily expanding the size of the index. On the other hand, if the hypernyms are too general, then you may have "noisy" matches for cases where the question and answer words match at a high level. Therefore, you select hypernyms such that the number of words assigned to the hypernym is within a range, and the depth of the hypernym is not too high.
Every hypernym is also assigned a score. This score is computed using a training set of answers and questions. If the hypernym is seen often in many question and answer pairs, then it is assigned a higher weight than a hypernym that occurs more broadly across diverse questions and answers. The hypernym score is used to more precisely generate a query from a question. A hypernym with a higher score has a proportionately higher keyword boost in the generated query.
The use of inflected words is another problem in matching words. A search engine may not match words such as "buy" and "bought" or "purchased" and "purchase". WordNet uses a dictionary and a set of rules to strip suffixes to find the root form of any given word. The three word usage problems (synonyms, hypernyms, and inflected words) are the leading causes of word mismatches between potential answers and questions.
Query Transformation
The best answer for a question may not be the one that contains the most keywords in common with the question. For example, the answer to the question "Where is the main office of UNIDO?" may contain phrases such as "located in" or "is located."
A simple question categorizer looks for the occurrence of question words such as "who," "when," "where," "which," "what," "how," and "why." A set of phrases commonly associated with a question category can be found using a collection of training questions and answers. The list of questions and answers are grouped by a question category. We identify a set of phrases that are frequently seen in answers for a particular question category. For example, a "when" question will have phrases such as "on the" and "will be." These phrases are used to supplement the query generated from the question. The selection of phrases excludes occurrences with nouns, since such phrases are usually specific to a question. A technical question and answer training set may contain many occurrences of the word "computer" or "database" that are not seen in general answers. Therefore, all phrases containing a noun are excluded.
A weight is computed for every phrase depending on the number of occurrences in all answers, the set of relevant answers (for the question category), and the set of irrelevant answers (for all other question categories). The n top-ranked phrases are assigned to a question category and become part of the generated query for a question.
