I'm trying to get the multiprocessing.Pool method to work. It initiates and shows all the cores are active, although ends up just stalled without completing.
To troubleshoot the problem, I first tried using a simple for loop and that worked (Method 1 below). I also tried using the multiprocessing.Process method, which also worked (Method 2 below).
Method 3 (below), however, just hangs. Where might I be going wrong in implementing the multiprocessing.Pool method?
import arcpy, os, glob, multiprocessing
# Method 1 works
inws = r'C:tempraster_data'
rasters = glob.glob(os.path.join(inws, "*.tif"))
def worker(raster):
arcpy.arcpy.BuildRasterAttributeTable_management(raster)
arcpy.AddField_management(raster, "Cover", "TEXT", "", "", 20)
for raster in rasters:
worker(raster)
# Method 2 works
def mp_worker(raster):
arcpy.arcpy.BuildRasterAttributeTable_management(raster)
arcpy.AddField_management(raster, "Cover", "TEXT", "", "", 20)
if __name__ == '__main__':
inws = r'C:tempraster_data'
rasters = glob.glob(os.path.join(inws, "*.tif"))
for raster in rasters:
p = multiprocessing.Process(target=mp_worker(raster))
p.start()
p.join()
# Method 3 not working
def mp_worker(raster):
arcpy.arcpy.BuildRasterAttributeTable_management(raster)
arcpy.AddField_management(raster, "Cover", "TEXT", "", "", 20)
def mp_handler():
p = multiprocessing.Pool(8)
p.map(mp_worker, rasters)
if __name__ == '__main__':
inws = r'C:tempraster_data'
rasters = glob.glob(os.path.join(inws, "*.tif"))
mp_handler()
