Voltage, amperage, output, heat and cost.
The size of power bricks has to do with how much power it has to output. A cell phone typically charges on a USB standard which is only 5 Volts and ~1 amp. By comparison, if you read your power brick you’ll see it’s 19V and probably ~7-8 amps. That means it pushes out a lot more power, and since power transformers are only so efficient, some of that results in heat.
Heat is naturally bad, so usually everything in the power supply grows to compensate for it.
For example I have the same HP laptop and I use 3 power supplies. The recommended power supply is rated at 150 Watts since for the big workstations they can draw a lot of power.
Most of the time when I travel, I’ll use the smaller 75 Watt power supply and it still works, because it still provides enough power - but if I use it for a few hours of gaming (peak power draw) the thing gets scorching hot - almost burns you to the touch.
Now if you look at a Macbook adapter it’s still generally much smaller than a typical laptop. That’s because companies like HP and others will find a power supply that meets their needs for multiple products (usually that means your power supply is actually designed for the largest 17" notebook with all the fixins, plus some extra margin on top) and then use that for anything below that. This way they don’t need to stock a unique power supply for every laptop.