Yes there is a level of truth to it, especially when it comes to design patents (vs utility patents) since there is a lot of subtlety between direct knock-off and “so close to knocking you off but you can’t do crap without a courtroom of expert witnesses”. There’s also the ability to enforce patents across country lines which is harder if not impossible these days with China willing to knock off anything.
It really comes down to your idea, what the level of novelty is, and being able to get a level of market success so that patents are a fallback, or a warhead against competitors.
I wouldn’t advise specifically not getting a patent because of legal fees (the concept of a patent in itself means your idea has commercial value, which means if you were selling it as a product you’d be making money that could be used to enforce the IP) and it is still a definite deterrent even before you go to court, but patents in themselves are not foolproof.
Without making a ton of generalizations your best bet would be to discuss it with a patent lawyer, since you probably don’t want to disclose it on a public forum if you’re worried about it.