I left an unsafe assignment once and never looked back. There was nothing the company did to me because they could not.
Since then, I did more research on the hospital where I was considering a travel job, that included talking to other travel nurses that I requested from the travel company..
You have rights and if you feel that uncomfortable then you do, no big deal , just give the notice to the job and call your company and say its unsafe, they cannot do anything to you.
I heard years ago , that Cape Fear was awful.
Go and don't look back.. You will be free .