I am trying to get a cleaned address from a bunch of addresses.
These are different addresses I have for Harvard University. What I want is to convert all these addresses to "Harvard University".
1) Division of Renal Medicine, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Harvard Institutes of Medicine Suite 550, 4 Blackfan Circle, Boston, Massachusetts 02115, USA
2) FAS Center for Systems Biology, Harvard University, Cambridge, Massachusetts 02138
3) Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, Massachusetts, United States of America.
Just simple text matching doesn't work. So, I tried difflib.
from difflib import SequenceMatcher
def similar(a, b):
retu SequenceMatcher(None, a, b).ratio()
a = "Division of Renal Medicine, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Harvard Institutes of Medicine Suite 550, 4 Blackfan Circle, Boston, Massachusetts 02115, USA"
b = "Harvard University"
print(similar(a, b)) # gives 0.11981566820276497
print(similar(a, "Toronto University")) # gives 0.04608294930875576
But I think this approach won't give correct results for my data set. How can I set a threshold for the similarity? Can anyone recommend a better approach?