0





150

Advertisement

The documentation for doMC seems very sparse, listing only doMC-package and registerDoMC(). The problem I'm encountering is I'll spawn several workers via doMC/foreach, but then when the job is done they just sit there taking up memory. I can go and hunt their process IDs, but I often kill the master process by accident.

library(doMC)
library(foreach)

registerDoMC(32)

foreach(i=1:32) %dopar% foo()

##kill command here?

I've tried following with registerDoSEQ() but it doesn't seem to kill off the processes.

Question author Patrick-mccarthy | Source

Advertisement


0


The doMC package is basically a wrapper around the mclapply function, and mclapply forks workers that should exit before it returns. It doesn't use persistent workers like the snow package or the snow-derived functions in the parallel package, so it doesn't need a function like stopCluster to shutdown the workers.

Do you see the same problem when using mclapply directly? Does it work any better when you call registerDoMC with a smaller value for cores?

Are you using doMC from a IDE such as RStudio or R.app on a Mac? If so, you might want try using R from a terminal to see if that makes a difference. There could be a problem calling fork in an IDE.

Answer author Patrick-mccarthy