Get an EIN. Fast and easy on the federal website. Put your EIN on your invoices. Your clients will send you a W9 in March, but you should know those numbers anyway.
You need to make quarterly payments to the IRS (income and FICA [the full 15.6% for SS]). Forms on their website.
Most states also require quarterly payments (income only). Forms on their website.
Obviously those payments will depend on how much you make. You can adjust from quarter to quarter as needed. I always gave them a touch too much. Never got audited, but ymmv.
That is the bare minimum to get up and running.
You can setup an LLC, or you can save a few bucks, do nothing and go sole proprietor. The difference is as a sole proprietor your personal finances are open to litigation. As an LLC, your business and personal finances are separate in terms of liability. For a long time I worked as a sole proprietor as doing sketches and prototypes opened me to zero risk. My clients had engineering departments and they carried any risk. How risk-adverse you are is your choice and I have been told the annual paperwork for an LLC doesn’t have to be a pain in the ass as it was for me when I switched to being an LLC.