Python Site-Packages Pysspark
import os
def myfun(x):`
os.system("pip install shapely")
return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect()
## call each cluster to run the code to import the library
Successful Stag