TOPICScore is a new metric for evaluating Text Summarization by creating a trade-off between Extractiveness and Abstractiveness. This is done by using Embeddings, Word Occurrences, and Topic Detection in order to compute a global score. This score is designed according to three aspects: Compression, Overlapping, and Semantic Similarity. By incorporating these three dimensions, TOPICScore is more agnostic to the used approach.

The intuition behind TOPICScore is built on the following heuristic idea : an ideal ATS system should generate a summary that shares the same topic with the original text and the reference. For instance, the summary of a text describing how to cook a dish needs to keep the same topic, which is cooking, and avoid including sentences that deal with a different subject.

The computing algorithm of TOPICScore is provided in the following figure:

For more details about the background, the experiments, the limitations and the improvements ; please refer to the paper, here is the link : https://ieeexplore.ieee.org/document/10548450