I understand it is not too far from where you are now, but is the cost of living in that area significantly different?
If not, or if it is for the positive, I think I would follow my passion, HOWEVER I would not lose sight of the pay cut you took and work towards righting the course as soon as you land. I took a “pay cut” after a layoff in an industry I was certainly happy to find myself in, but I think I let my pay slide with little to no discussion for a too long and it eventually served as a contributing factor to leaving a job I and a company I really loved.