Portfolio definitely matters and posting it might help us give better advice.
Especially if it’s a barnburner f a 'folio, then there might be something else at play here that can be traced back to your interview style.
But I assume that you had shared your portfolio perviously with those employers, right?
So in that case I’d assume they have already a good understanding of your strengths and abilities.
My general advice might be to not look for a junior position but rather a good, solid and paid internship.
I’d definitely would recommend that over freelance work. The experience these employers are looking for, you will most likely gain from your peers, good direction and management and great serious clients. This is something that is very hard to come by through freelance work.
I tried the freelance thing too when I was fresh out of school and had trouble finding work but it really didn’t work for me. I needed a nurturing and inspiring environment with great designers around me in order to evolve. Sitting at home and doing some renders for a client I had barely met didn’t help me progress.