In spite of the difficulty of predicting the future and that unforeseen technological inventions can completely upset the most careful predictions, you must try to foresee the future you will face. To illustrate the importance of this point of trying to foresee the future I often use a standard story.

It is well known the drunken sailor who staggers to the left or right with n independent random steps will, on the average, end up about √n steps from the origin. But if there is a pretty girl in one direction, then his steps will tend to go in that direction and he will go a distance proportional to n. In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance √n. In a sense, the main difference between those who go far and those who do not is some people have a vision and the others do not and therefore can only react to the current events as they happen.

One of the main tasks of this course is to start you on the path of creating in some detail your vision of your future. If I fail in this I fail in the whole course. You will probably object that if you try to get a vision now it is likely to be wrong—and my reply is from observation I have seen the accuracy of the vision matters less than you might suppose, getting anywhere is better than drifting, there are potentially many paths to greatness for you, and just which path you go on, so long as it takes you to greatness, is none of my business. You must, as in the case of forging your personal style, find your vision of your future career, and then follow it as best you can. No vision, not much of a future.

- Richard Hamming

Hamming stresses that without a vision, one is locked into a state of relatively passive reaction. A compass can be spun, but always wrenches itself back into the Northern direction. Living a life that doesn't automatically wrench itself back into a particular direction dooms you to a fraction (that is: the square root) of the distance your steps could have otherwise taken you.

For some, this is framed as a good thing. Action is scary: the more steps in a continuous direction that you take, the more stuff you affect. The more stuff you affect, the more likely your plans are to backfire and get you blamed for it. But this is a cowardly way to face life. If you continue picking smallness over largeness and inaction over action, you'll end up inviting awful demons into your head, like resentment and hatred of greatness—you won't even have any courage left to kick them out!

So don't refuse to have a vision "because it could backfire". It's not that it can't backfire—it's that ensuring it doesn't backfire is a matter of planning, not having a vision in the first place. To pass up on ambition because it could hypothetically be dangerous is horribly short-sighted and ensures only that you have no say in how dangerous Earth becomes, instead of some say.

"It's too ambitious" or "wow that sounds hubristic" or "god-complex alert" are not arguments. They remain in the meta, and the meta's only bearing on reality is in object-level patterns, so you'd better point those out instead or the conversation between us ends.

1: Hubris is object-level, not meta-level.

Often this voice will tell you that picking humility over hubris is good. This is besides the point: setting a vision is not hubristic! Hubris operates at object-level, not meta-level!

Icarus' mistake was not thinking he could brush the gods with his wings—it was refusing to heed his father's warnings. Had Deadalus coated his son's wings in reflective foil so as to cast off the sun's rays, Icarus would have been in his rights to fly as high as he could. Had Zeus shot him down in a fit of temper, Icarus' hubris would have been a failure to properly predict the sky god's whims—not to have passed some sort of magical threshold after with one's benevolent ambition becomes "bad" in some sense.

Similarly, hubris was not man's desire to build a tower that could reach the heavens: hubris was failing to read the clause about construction regulations in God's instruction manual, or forgetting to ask Noah to ring up divine technical support and ensure they could procure a permit. There's no fit of emotion or grand ideal to be regretted in the episode of Babel, only plain incompetence.

Ambition does not automatically shift from safe to dangerous: it's completely orthogonal to the real culprits: lack of wisdom, ethics, or respect for Chesterson's fences. When Sam Altman accidentally drives humanity extinct after building o2-preview-turbo-28eo, the blame won't rest on him for being too power-seeking. It'll be for ignoring safety research for too long, or rushing a model along with suboptimal RLHF for some moronic reason like "Anthropic was going to release Requiem and steal the market for therapist companions"

2: Ambition and explosive failure are correlated though.

Human psychology is messy, and the correlation between "visibly power-seeking" and "unwise" is probably significant. One is thus justified in worrying about negative consequences when one sees a person act with ruthless ambition, even when that doesn't immediately bear on the specific quality of their work. But truly, it is the quality of their work which ultimately matters, not their temperament, even when the two are closely intertwined.

To criticize another's temperament—whether implicitly or explicitly—is justified only on object-level grounds (i.e.: "this leads or will lead to problems" not "confidence is bad"). I mean this mostly as a response to the type of person who will automatically condemn someone for expressing a desire to change too much of the world at once, despite no convincing argument (i.e. mere rationalizations) as to why this is a bad idea. The tall-poppy-cutters are either willfully conflating ambition with contemptible evil, or operating on a heuristic which cannot possibly be used in all cases heuristics that almost always work (ambition = dangerous).