Assuming that a golfer would want to skip golf fitness and exercise they would find themselves faced with one major problem. Increasingly they would find themselves frustrated and would be faced with a situation where it would seem that the standard of their game is dropping.
There are few things that are as infuriating and frustrating as seeing your opponent effortlessly run rings around you on the course just because they have taken golf fitness and exercise seriously and you have not.
In all likelihood the golfer who is determined to do without golf fitness and exercises, just like in the old days, would increasingly find themselves in a position where they enjoy their game less and less.