It’s more a question of what was happening during that period. I’ve interviewed a few people that traveled the world for a year after graduation. Nothing wrong with that as long as you are ready to settle down and give me a couple years of great work now. I also know a designer who would work for 2-3 years and then travel for 6-12 months at a time. He did that for at least a decade and always managed to find a job when he was ready.
Basically, have a great portfolio and don’t worry about this until the interview. Then, have a better answer ready than, “I’m a horrible designer”.