First, owning an airplane has far greater benefits compared to renting when it comes to 1) access to the aircraft, 2) a known maintenance history, and 3) motivation to go fly on the good days when everyone else wants to do the same (partially related to reason 1). However, it is not without responsibilities, and some of them can be very expensive.
First piece of advice: DO NOT buy an airplane just to get Private Pilot. Purchase an airplane that fits your particular mission in aviation at the current moment. If you see yourself traveling for business on 500 nm trips, get an airplane that can reasonably do that. If you want to take 1-2 passengers on a vacation regularly then get a 4-seat airplane. To be 1 seat and quite a bit of cargo room short of what you need means you will not use your airplane for that trip. Get something that meets your needs for the highest level of certificate you intend to go after now (meaning a real IFR airplane if you want your instrument rating). Upgrades can be VERY expensive.
Second piece of advice: If you disobey the first piece of advice DO NOT upgrade the airplane. We own a 1966 Piper Cherokee 140, and had I known then (when we purchased) what I know now I would have purchased an Archer. We spent more than the purchase price on upgrades and the first annual. The inital upgrades were all necessary: paint (the original 1966 paint was still in place and falling off rapidly, leaving bare aluminum to corrode), shoulder harnesses (necessary for safety, IMHO), new wingtips with HID lights (for added safety at night), thicker windshields (the old ones needed replacement anyway). Then it came to the first annual with some unforseen engine trouble and a few other issues that either appeared after or were not caught at the pre-purchase inspection. In a little over 1 year a $23,000 Cherokee 140 became a $50,000 Cherokee 140 (see #6 below) and we are not talking about glass cockpit/ new panel, 1 nav-comm without glideslope, portable intercomm, "shotgun" panel with the VSI above my left knee obscured by the yoke.
3rd: Get someone who knows about purchasing and owning airplanes to help you out.
4th: Insurance is cheap in something simple, my wife is a student pilot with 2 hrs logged time at purchase and our premiums with $42,000 hull coverage are around $1,000 per year. Our cars are more expensive to insure, at least in New Jersey.
5th: Depending upon where you live (particularly in the midwest for hail/ tornadoes or the southwest for severe heat that cooks avionics/ interiors) you may NEED a hangar to protect your investment. It is always nice to have one regardless, but factor that into the cost analysis and the economics might not make much sense.
6th: Owning an airplane CANNOT be financially justified unless you own your own business and plan to use it to get to and from your non-aviation work sites. However, if you have the resources and you really want an airplane go for it but realize that the care and feeding of an airplane can be very expensive if you put safety as a top priority.
In the end, the direct costs for our Cherokee 140 can, some years, go above $200/ hr. Would I still own knowing I can rent for half of that anywhere in the country? Absolutely! The piece of mind is well worth it. Now I just need a faster airplane for my new mission, and something that I can finish my instrument rating in.