I am trying to crawl some Urls with Apache nutch 1.11. There are 7 Urls in my seeds.txt file, and I run the command :
bin crawl -i urls crawl 22
My problem is that with depth 22, I expect it to fetch quite a few number of pages. But today, all it does is fetching the exact same Urls that exist in my seeds.txt file and nothing more. And as weird as it sounds, yesterday the exact same files and properties ended up fetching 313 Urls. I didn't change anything since yesterday. Anyone knows what's going on? The only thing that has changes is that yesterday I was using another computer. But since I am ruing the crawl command on a remote computer, I don't think it has anything to do with it. Does it?