I have N documents.
I have found out cosine similarity between all documents. T = (N*(N-1)/2)
Because docSim between a document with itself is 1.
Now, I have one query and I want to find out all documents which are similar to it. I don't want to rank it but cluster it based on threshold value.
Here is how I calculated threshold value.
- Average: Mean of all(T) cosineSimilarities
- Average(T) + alpha*standardDeviation(T)
How do you find the parameter alpha?
I found this formula here
https://www.researchgate.net/post/Determination_of_threshold_for_cosine_similarity_score
Is there any other way to find out threshold?
Some suggestions on formula might improve my cluster.
Thank you