Hi everyone!
We are working to upload our quarkyonic module to MUSES, but I have a quick question. I am currently parallelizing the python script using joblib, like this:
from joblib import Parallel, delayed
def compute_point(i, N, ann, bnn, apn, bpn, para, steps, res, delta):
binding_en, quark_fract, charge_fract, press = total_en_slope(N, ann, bnn, apn, bpn, para, steps, res, delta)
return i, binding_en, quark_fract, charge_fract, press
results = Parallel(n_jobs = -2)(
delayed(compute_point)(i, denmin + i\*delt, x0\[0\], x0\[1\], x0\[2\], x0\[3\], x0\[4\], res, fr_acc, der_step)
for i in range(incr))
Here the “n_jobs = -2” tells the system to use all but 1 CPU. However, I am suspicious that this won’t mesh with you celery worker implementation. Can I get a bit of guidance here on how to assign jobs to specific workers?