I don't agree with Elon on many aspects, and like you, I both respect and dislike him, but this particular quote can actually be used positively in the right context. First of all we also need to look at what Elon calls work and what the rest of the world calls work. Elon works because he likes working on the things he builds. When you build your own empire, it's not seen as work (and it's not felt as work either), however when you help build someone else's empire, it is work. When you build your own dreams, your pay is not by the hour but by the experience and the outcome.

Unfortunately a lot of people complain about their circumstances in life and the pace of their success, but I also see that many of them also want to put in as little effort as possible or none at all. I am not saying that's bad, I am saying that in the context of a capitalist society, traditionally the higher the effort, the higher the reward. Just quite recently I calculated for a friend that if they worked 1h extra as a part-time gig, every day, they could save an extra $10K/year. Do just that for 25 years, and you're looking at the cost of an entire house in many places in the world (in their case it would actually be 5 nice houses, not just one).