Error Installing Pandas Library into Spark pool

Copper Contributor

Hi Team, 


I need a bit of help understanding an issue we are facing while trying to install a pandas library into a spark pool cluster that we have on synapse.


Steps followed,


1. Create a new requirements.txt file with the following text



also tried below



Step 2: Upload this requirements.txt file to the sparkpool under Packages and restart cluster. 


Step 3: Post successful application of settings, run a notebook under the cluster.





Notebook returns the below error, 




Basically this change makes the spark pool unusable.


Would you be able to provide any insights on why this keeps happening ?



1 Reply
Try creating a spark v3.2 runtime spark pool and run the import in a notebook with the new spark pool. Let me know how it goes.