HPC doesn’t care about the cloud. They want the cloud, but they don’t want the cloud.
When I left NERSC in May 2022, I speculated that the future of large-scale supercomputer centers would be one of two things:
- They develop, adopt, steal, or squish cloud technologies into their supercomputers to make them functionally equivalent to cloud HPC deployments.
- They find better overall economics in eventually moving to massive, long-term, billion-dollar deals where flagship HPC systems and their “more than just batch jobs” features are colocated inside cloud datacenters sited at economically advantageous locations in the country.
People matter, and culture matters.
If you want to go fast, go alone; if you want to go far, go together.
Being good at things doesn’t always map to a job
People tell me that I’m pretty good at a bunch of stuff: figuring out how things work, explaining complex concepts in understandable ways, taking a critical look at data and figuring out what’s missing. And I enjoy doing these things; this is why I post to my blog, maintain my digital garden, and love getting on stage and giving presentations. But people also say that, because I’m good at these things, there’d be no shortage of opportunities for me in the HPC industry should I ever go looking. However, a job has to be an amalgamation of responsibilities that need to be taken, and connecting those to aptitudes is not always straightforward.
If I like to learn things and share them with others, what kind of jobs actually involve a responsibility that includes that? Well, I’ve learned that
- Developers don’t really do this at all. Their job is really to keep those git commits coming. Sometimes this requires learning new things, but it also requires getting nagged by product managers just as much.
- Product managers do a little of this. I had to learn a few things and then repeat them a lot. Over and over. And nagging people to act on what I taught them, maintain Gantt charts, and repeating the same basic things to different people.
- Curiously, salespeople also do a little of this. They have to stay current on customer needs and product features, then repeat them a lot. They also have to nag people. But they have less agency than product managers about what things they learn, because they are handed a portfolio with sales targets. They also have to work hard to get an audience, since the people who might benefit from being taught don’t even know it.
- System architects do a fair amount of this. I had to learn about what technologies are on the horizon, figure out how to piece them into an idea that could be implemented, then explain why it’d all be a good idea to others.
- Researchers do a lot of this. They have to stay abreast of the state-of-the-art, then push things forward through research. And once a discovery is made, they have to explain its importance to others. But they have to always be generating new ideas, and they can’t linger on any single one for too long. They also need to compete with other researchers for an audience.
- Educators do a lot of this. The technology industry is always moving, so learning is required to stay up to date. But they also get to focus on the ideas worth sharing and downplay the rest, and they have a captive audience in students.
Happiness costs money.
Money definitely buys happiness, to a point. Beyond then, there’s calculus to be done to figure out what price you’re willing to pay for incremental happiness.