fetcher.max.crawl.delay  默认是30秒，这里改为 5秒 修改nutch-default.xml 
      
      
       fetcher.max.crawl.delay
       
      
       5
       
      
        If the Crawl-Delay in robots.txt is set to greater than this value (in seconds) then the fetcher will skip this page, generating an error report. If set to -1 the fetcher will never skip such pages and will wait the amount of time retrieved from robots.txt Crawl-Delay, however long that might be.