ID is very much a show and prove field, without clearly displaying an ability to do the work through an ID oriented portfolio it may be VERY difficult to land ID work. 2 years is not too late, particularly early on. I would make the switch if ID is what you want to do.