Episode: Kelsey Hightower - Don't Outsource Your Core Skills!
Links und Kontaktmöglichkeiten
- Kelsey Hightower on LinkedIn: https://www.linkedin.com/in/kelsey-hightower-849b342b1/
- Kelsey Hightower Bluesky: https://bsky.app/profile/kelseyhightower.com
Das Transkript der Episode
Hi, hello and welcome to the Beyond Code Podcast, the podcast with the makers and experts from tech scene. My name is Felix Becker. Nice to you here. As I on social media, today a for you haven't heard it it's time to channels. can the podcast on LinkedIn, on X, Twitter,
Blue Sky and on Instagram. And yes, today will be a special episode for me because it will first international episode of the podcast. have a guest from the USA in the interview and for that I will be the language to English the rest of the podcast will be in English. Today's guest hardly needs an introduction for anyone in the tech industry. He started his journey in IT support and went.
on to become one of the most influential in cloud computing and open source. As a former engineer at Google Cloud and a passionate advocate for human side of technology, he has proven that create engineering is much more about empathy and storytelling, it's about code. I am more than delighted to welcome Kelsey Hightower to the podcast. Kelsey, great that you joined the podcast. Welcome to Beyond Code.
Happy to be here.
Thank you so much. have a every asked. And I want you also was the last time you wrote and what code was it?
⁓ the last time I wrote code was during a keynote three days ago in Amsterdam and it was just a hello world program and it was in Golang. So the purpose of that program though was to show the audience that even the simple hello world program can be made much smaller and ask them, how would you do it? So package main, import the fmt package, fmt main, fmt print line, hello world.
And I did an object dump so they can see some of the assembly and the machine code generated underneath it. And that's how we could make it smaller because the megabyte was, know, binary was 2.3 megabytes. And so you can just kind of see people just stuck there. Maybe they were shy, but I don't think people knew what to change. And so I removed the fmt package, removed the fmt print hello world in exchange for the built-in print line, which is built into Golang.
that removes a lot of the flexibility that you get from the format package and ran it again, compiled it, and it reduced the binary to like, I don't know, 1.6 megabytes. And it probably removed eight instruction sets or instruction calls from it. And you can see the audience like, hey, okay, sometimes we do forget that we can just remove things to make simple changes to our applications and reduce them to the perfect size when we focus on what they need to do versus what more we can add.
So that was the last bit of live code. So I did it live, not recorded, not a video, just open the editor and even writing that live is very different than having the privacy of just you and your computer and access to all the things on the internet to experiment. But when you do it live, you almost got to do it right the first time. And it almost feels like no matter how long you've been doing this, that people are judging you in some way.
true, absolutely. And you basically give a raw stream and out to the output stream or what did you do to actually remove the fmt package?
Hello, just open the code again, main.go and remove the front package and delete that line of code and replace it with a more simple one. Rebuild the app and print to standard out again. And so it's just the development process where everyone was thinking of like, you can add something or you can strip the binary using some compiler optimization. But sometimes you should just remove it from your code base if it does the same thing. So that was, that was that case. So you asked the question of technically that was the last piece of code I wrote.
But the current thing that I'm working on is a SQLite binding written in CGO. So I love the Go programming language. Go has a lot of abstractions for interacting with the databases. So there's a SQL database package where no matter what database you interface with, you can use it in a common way. It's not quite an ORM where it does all these high level object things for you. It's more like bindings to run a query, process the rows, that kind of thing.
But there's cases where I just don't like the go away of dealing with databases. And sometimes I just want to use the SQLite functions to just open a database, construct a prepared statement, store it, and then use it over and over again by substituting parameters, execute the query, get the results, and then just close the database. Like really simple, no abstractions, almost a one-to-one mapping. But in order to do that and go,
Either you find a library and there's some good ones out there, but just to keep my skills sharp, I do maintain my own private SQLite binding. So you have to use cgo. This is your foreign function interface in C for go. And basically you import the package, you download the algorithmation file where all of SQLite is there in one big file. And you just basically wrap the functions that you want. So you can call them in a native go way. The reason why I like doing that is because in the C world, you know, the memory
management is different than Go. So have to be very careful where you're passing memory between returning from a function, do things stay on the stack, do they get garbage collected? And so it just kind of reminds me that Go does a lot of work for you, but that is not the case in all languages. So I keep that thing maintained and I use it just enough to interact with SQLite using the raw functionality without all the abstractions.
That's great to hear. And I guess that's why they call Go the all batteries are included language. that was basically a great seque How important is writing code for you today when you're not really at Google Is it still a thing that you on a daily weekly base? Or is it something that came not as frequent as it was before?
Yes, I don't code as much as before. Last year, I didn't code as much before. Even five years ago, I didn't code as much as before. But I think if you were a chef and you worked in the restaurant for 20, 30 years and you learn how to make all these amazing dishes for other people, you've really mastered your craft, your world-class chef. And look, there are situations where you come home and the last thing you want to do is cook. So you may just order out or enjoy someone else cooking for you. But I think even chef.
when they were to retire, what they would want to never cook again. And for me, the answer is no. It's a skill that takes a long term to master. If you knew how to cook, could you live a world where you can only taste what other people put together for you, knowing that there's a situation where you can remove an ingredient and make it better. And so I still get that joy that I do use off the shelf stuff as much as can. But when I have an idea, there are just times where I just want to make it myself.
to my own liking, even if I'm not cooking for other people these days, I don't mind making my own food from time to time.
Great. And once you see it, you see the world in a different right? If you're a chef and you see how others are prepared the food and what ingredients you use, you have a whole different view on it. And I think that's the same with code as well. How did you start in tech? What was your first steps to become the role as an distinguished engineer at Google?
So Distinguished Engineer comes what, the path to Distinguished Engineer comes way towards the end, right? And so in the beginning, I just wanted to get a job in this exciting field, know, 1998, 1997, I'm in high school. And my first introduction to like this world of tech, right? I played video games on my computer before, but the concept of making the computer do things in a technology class.
where you had to learn things like AutoCAD. So you would have a specification and you would go into Adobe's AutoCAD and you would build out the specification, whether it was a blueprint for a house or you're going to build a bridge and you have to pick the materials and then you have to print it out on the dot plot. And even then I was like, this is amazing that I actually understand a little bit of what's behind the buildings and the bridges.
And then I remember one of the classmates, we all used to have the large calculators, the TI 81 or the TI 82. And they're like, you can write programs on it. I was like, no, no, no, no, wait a minute. Like how are you playing Tetris on your calculator? And it's like, not only can you play Tetris, you can build your own games. I remember learning TI basic and making the snake game. And when I say learn TI basic, mean copying and pasting.
the code from somewhere else and you're on this little calculator trying to punch in everything and then hitting save and then watching it run. And that's when I think it kind of clicked for me at that moment that even though the calculator was designed to do what calculators do, they made it programmable. I finally understood what a programmable calculator meant. I see people can add new mathematical functions, but you could also make it do other things like in this case, play games.
So this is what I was like, okay, now I understand that computers aren't just a tool for consumption. They can be as well. And so this is around 10th grade. I think it kind of seeded in my mind. So when it was time to graduate, I didn't go to college like some people would have done, computer science kind of thing. I ended up doing IT certification. So A plus certifications. This is where you learn about motherboards, memory, graphics cards. And the other half of that certification is like,
Windows and eventually Linux became part of the certification criteria. And so now I'm learning about hardware and software and how all of these components fit together to make these things we call computers. And at that point, I still didn't have the confidence to apply for like a mainstream job, right? If you go to these hiring websites, this is around 1999. They wanted things like AIX experience. You need to know Java, Cobolt, and then college was preferred.
Not a lot of places had things like it's not required. And so I just didn't have the confidence to even apply, even though open source was making its way up. And so I opened a computer store. So I did a little bit of contract work. This is around the time. So now we're in like 2000, about 18 years old, turning about to turn 19. And I got a contract job during the era where people were removing dial-up and replacing it with DSL. So giving people high speed internet access, they had
contractors like me, would show up with our tool bag and we would, if we needed to, we would put in a network ethernet card, make a ethernet cable, plug it in, and then get you onto this high speed so no more dial-ups. So we were swapping people out from dial-up to DSL. It was a big deal for a lot of people, right? Because the speed jump from 56K modems to, I forget what the speed was, let's call it 190, you know, it was a little bit better than T1.
for over DSL and people were just like, my God, it's a game changer. And so that was my first foray into kind of the professional side of everything, right? We were going to businesses and sometimes people's homes. And in the case of a business, you would have to run long cat five runs, right? You have to go through walls. So you learn a little bit of carpentry cut into the drywall, put a ethernet jack there. And that turned into a computer store where I started selling.
computers in the neighborhood in which I grew up in or in high school. And so this is a case where people say, ⁓ I have this list of parts. I want this Nvidia GPU. I want this Intel processor. I want this much memory. I want this much hard drive. And we would build those PCs, but also we would also service those companies. So we would do things like bring in network switches, network the computers, window services that connect to multiple offices, that kind of thing. So that was kind of like my start, which was
get the IT certifications, gain the skills, and I became an entrepreneur out of necessity just so that I can keep doing this work without necessarily a permission from a hiring manager at one of the large companies.
And what I hear is that you cross-functional. So wherever the work was, you did it. You learned it and you provided the service help your clients. And this is something that we also want today, but it's really hands-on. It's not that educational part where most of the people go through. It's just, I see there's work. I see there's an opportunity to make money. I go for it. And if I don't know it, I learn a skill.
You know what? Now that I think about it, I am so glad that I started my career that way because as an entrepreneur, I wasn't in position to go get a loan or borrow a bunch of money to kick the company off. I had to be profitable on day one. And that means if I ran into a new customer and they asked me like, Hey, can you network a Xerox printer or can you replace Novell network with Outlook? The answer is always yes. And in my mind, I may give you a price and
Sure.
I may lose because it's going to take me so long to do it the first time, but I always thought about it from an entrepreneurial mindset of if I learn how to install Outlook and migrate email between these two platforms, then I can do it much faster next time. And then it's now a new profitable skill. And so it's almost like I'm going to learn everything I can because the more skills, the more business that I could do. So I always saw it as a worthy investment with an immediate payoff.
Fantastic. Sounds great. And from your computer store, so you went from installation to selling hardware, what was your next step?
The next step after that is I remember applying to Google, right? They were hiring contractors to work in a data center. This is right outside of Atlanta, Georgia, Southeast of the United States. And it was like, must know Linux. You must be proficient at networking and all of these things. And I'm like, you know, I don't know if I'm ready, but I had been running this business for three or four years. I did a really good job as an entrepreneur.
And contrary to what most people believe, just because you own a business doesn't mean you make a lot of money. I did pretty well. I had other things I was doing because part of that business, I was also setting up music studios because during that time people were transitioning from analog gear. the big mixing boards and tape reels to digital. And so that means they were getting computers in the studio for the first time. We were installing tools like pro tools and then rerouting the audio to throw through the computer instead.
And so, you know, through those endeavors, I did pretty well, but you have to do a lot of work to make money. And so I'm seeing how much these jobs are paying. It's like, wow, you don't have to be, you don't have to stock inventory. You don't have to hire people. You just get paid every two weeks. Oh my God. That would be a dream to not worry about how to make money, but just do the work part. And so at some point it made sense when I was starting my family to like reduce the amount of work I was doing there and jump into the field that I was aiming for.
And so there was a contract job. So the way it works typically, you the work contractor job for three months, and if they like you, you can become full-time. So lots of companies, especially where I was living, they preferred that route. And so I went to the interview and there's three people sitting there from Google. And just like anyone else, I'm admiring Google's like, wow, this is like one of the most innovative companies of our time. This is like, you know, early 2000s.
And I was going to work in a data center as a data center technician. I didn't even care what that was. I was just looking like, Hey, they pay pretty decent. And I would love to just figure out what it's like to work at a place like Google. And I remember they were asking me a lot of what I would consider trivia based questions, right? Like, you know, what does this command output? What does this particular thing does? How's the Linux kernel do XYZ? Tell me about the execution of a process from the Elf binary all the way through runtime. And I'm thinking to myself is like, why is this so important to you all? Cause
While it's great knowledge to have, it doesn't reflect the real world in terms of what you actually do every day. But I get it. They just wanted to see if people had at least done the work to understand how these systems work, because I think they wanted people's creativity, not for them just to follow, you know, the same script every day. And I did, I would say I did pretty okay in that interview, but there were some Linux things I didn't do well on. And I remember one of the interviewers.
He was asking me like, you you may not know Linux that well, is there another operating system? Something came to the point where I had this opportunity to say, you know what? I actually use FreeBSD for everything. And just my luck, this person had a FreeBSD tattoo on their leg. And if you've never seen the FreeBSD mascot, it's the beast. He's like a little demon rare with this little pitch fork in his hand. And I saw that I was like, ⁓ my God, luck is on my side.
my god.
And we just started talking about FreeBSD and how its kernel architecture is a little different, how the software distribution model is a little different than Debian and RPM, you can do make world. And we're just geeking out on these things. I was like, oh my God, I have found a friend on the interview panel. And I got that job. And that type of job is these are huge data centers. We're talking hundreds of thousands of servers. And when you go in there at Mechland, everything is well organized. There's a hot and cold side. They're designed.
The fibers run separate from the ethernet, power runs on the ground. There's ramps, there's hills. There's even scooters that people were riding inside of the data center because it was so big to get from one place to another. And we had maps. We had all of these things. And I remember as a data center technician, our job was to help automate the provision of the system. So you can imagine a rack, maybe if like 30 to 40 computers will come in off of a truck. We would put them into burn-in.
So we would run tools to make sure the hardware wasn't having any potential immediate failures. And then we would plug them up. And then all the Mac addresses will automatically get put into a machine database. And then they will get provisions to join. It wasn't called Borg at the time. I think for me it was called Isadore, some other name. And this thing would join the cluster. And then any other team can have it. So it can go to Gmail, it can go to whatever team. And then we would have to troubleshoot these things. So at that time, Google was starting to build.
custom network switches. Google's building custom SATA cards so we can connect 12 drives on the disk tray that then feed up to the top of the rack. And then I remember these things would catch like on fire. Like you would have a smell where these things are obviously overheating. And then we would try to find these things. So sometimes we had scripts that can actually comb through thousands of servers to say, ⁓ something is off. could be temperature reading. It could be.
⁓ slow disk writes, we can kind of tell that something's about to fail. And then we would make a decision. So you as a system administrator, it's almost like this mark and sweep concept. I think this server is going to have a bad disk W and we need to replace that drive. Now you get there and you pull the rack out and what you see is that, ⁓ the SATA controller has that little Burton mark on it.
So then you may upgrade your diagnosis to it's a SATA problem. And so you switch out the SATA card and go ahead and switch out the drive just because it showed up in your tooling and then you slide it back. And you will repeat this process for top of rack switches. You repeat this for servers and you will get this score. And this thing you can log in, it will tell you how accurate your diagnosis was, the fixes you put in. So you can imagine if you swap drive W.
and then it comes back that drive A fails next week, there's a chance that you were off in your diagnosis. Maybe you should have swapped the different drive instead. But this feedback loop was really good to help us refine our tooling, our scripts, and the way we thought about solving problems. And when I really got good at this, if someone was able to repair, let's call it 10 machines a day, I think I got to the point where I was competitive and can do like 40 in a day and keep a high.
scoring in terms of my ability to execute and diagnose. And the ending to this was there was this process like to help you determine if you go full time. And part of it was like remembering names of engineering leaders, some of the values, some of the processes. And I was pretty stubborn at that time. I like, I am, I'm not doing this, right? You see my performance. I don't want to remember all this trivia. And if you didn't pass, then there's a chance you would never make it to full time.
And as an entrepreneur, thought to myself, like, I would just go get another job. It's just not that serious. So after three months, I never made it to full time. And I didn't feel bad about it because I ended up getting another job in voiceover IP, making more money. And so I was like, I'm fine. It's like going to the next customer. So it didn't feel bad, but yes, I was not at Google my entire career, but that's kind of where it started officially in terms of the tech sector we think about today.
Yeah, fantastic. Great insights. Thank you so much for that. But I hear you're kind of a competitive guy, right? You wanted to double and want to be the best in this. So how did that help you in your career? can elaborate.
So I played sports,
you know, in high school and middle school and it is competitive. That's what makes sports kind of fun. It's not even if you win all the time. It's just this idea that I go work out. I learned the game and I try to get better at the game and you get this really nice feedback loop when you're on the court or we're on the American football field. And this feedback loop up, you didn't score or you couldn't get the first down or you didn't catch the ball. It tells you, you need to go work on your craft.
And then you're also playing with other people. So you're also seeing at what performance level they're at. Sometimes you're the best player on the field. Sometimes you're just can't compete. And so growing up that way, it was a healthy competitive spirit knowing that if I can, and look, there's limits to it. I never made it to the pros. I didn't have the height. didn't have, I didn't have the dedication, but I was competitive in the areas that I chose to compete in. And so when I got into tech,
That mindset doesn't leave. As for when I'm on a job, I'm looking at what average means. So if everyone else is doing 10 of something, what would it take to do 40 of something? What kind of training is required? So that means you got to learn the game first, see what the rules are, see how people play the game. What are their VIM bindings? What scripting skills do they have? How much of networking do they understand? Why are these people proficient? Who is the best person?
at the company, I want to meet this person so I can understand what great looks like. But then I'm asking myself, well, why can't you be great too? What's required to be great? Well, you're to have to learn about networking. You're to have to learn scripting deep. You're going to have to learn mini-com. You got to learn why all of these tools exist. And then you can be creative and then you can bring a new style to the process. And that's how you sometimes have the ability to go from doing 10 of something to 40 of something. Maybe sometimes you do 80 of something.
And then people say, well, how are you doing 80? It's like, well, what I didn't need to do, I modified the scripts a little bit. And if you do this instead, you're taking a risk here that your scoring could go down, but I can do way more than I can do. So if I take a 5 % on the score hit, but I can do double, I think it's a good trade off. And so that's been my mentality. So every job I've went to, and also imagine having a feedback loop where your salaries continue to increase. So during that time period, every three months,
I will learn as much as I can on any job. I don't care what it was. When I worked in VoiceOver IP, I learned how the VoiceOver IP stack work. We were using this open source, ⁓ open PBX system. I learned everything about it. learned just enough Java to troubleshoot. I learned how to pixie boot Cisco IP phones so that they can be connected to this thing. I learned everything I can about that one industry. And then when it time to move to the next company, I had this data center tech experience.
voice over IP tech experience, also support experience, and the ability to write scripts to automate things. And so I was prepared for the next job. So that was kind of my formula.
Yeah.
Very great. And think that comparison to athletics and sports top on here because if you're on the court, you look at other players, what do they do? How much do they practice? How can I challenge them? How can I beat them? And applying this to your daily job is just working the same way. Do you see right now in our industry, the same motivation, the thing that I that learning is on my own responsibility? I have nobody providing me.
⁓ the educational stuff, it's not like free lunch. ⁓ I have to work for it
I think the caveat here is that not everyone's gonna be fortunate to have their body or their mind in shape to do this. So with that caveat understood, right? Because we should also figure out a world where we can take care of the people who don't necessarily have the means to take care of themselves, right? So that's obvious. Hopefully it's going to be obvious. But for the people who do have the means, there is something about the challenge.
and the responsibility of meeting that challenge. And I don't think it's very relevant because throughout my IT career, I was very surprised, especially when there was a big hiring boom in tech. It was almost like if you were willing to even try to do the work, there was a place for you. And I do remember people saying, well, that's not my job. I'm a Linux system administrator. That's a database. You have to go talk to the DBAs. And so for someone coming from the entrepreneurial world, I'm like,
why would you not want to learn how to do the database stuff? I'm not saying you have to be an expert. I'm not saying you got to learn everything at the same time. I'm just saying that, you know the value of being able to do Linux system administration and a little bit of the DBA work? And it's kind of natural, right? Because the DBAs will ask you to install the database, tune the kernel, give them more disk space. It feels natural to then understand how to create a database, structure a table, diagnose whether index is helping or hurting.
finding slow queries, it feels like a natural progression in order to add more value and expand your toolbox. And I will watch time and time again where people are just like, no, no one's asked me to do that. No one's trained me to do that. So I just don't do that. And then 10 years later, they still can't do that. So in those scenarios, I'm like, you find yourself in this weird place when the technology moves and we don't need as many Linux system administrators anymore.
You're now caught off guard because, and now instead of having maybe 10 years of Linux experience and three years of DBA experience, you now have 10 years of Linux experience and zero years of DBA experience. So you making the pivot is a little harder for you because you never had the warmup period. So you're a senior Linux administrator and a junior, maybe not even a junior DBA. And so I just saw that from time and time again, and I don't want to blame people as if the layoffs are their fault or.
getting caught up in these industry shuffles are their fault. But when you put on the competitive hat, you're trying to make sure that you evolve as the game evolves. And that means you can't rely on your coach to tell you everything you must do. Even your trainer, right? There's a lot of teams that have practice, but that's not the only practice you've got to do if you want to be competitive. Like that is structured practice. But if you got to work on some skill gaps, you have to sometimes then train additionally on your own.
to make sure that you're getting targeted time with yourself. So that way, when you join the team practice, the whole team makes progress at the same time. So I encourage a lot of people that talk about getting into IT, I have no magic formula for you. But I think, I don't want to say just working hard, but having this commitment to yourself to evolve and put the onus on yourself to get better over time. I don't know if you lose or you have a better chance of winning if you go that route.
Great. And lots of the things are related to deferred, so it's not an instant feedback loop that you get. You work on your skills and you do repetition and then it pays I see a lot of people right now or a lot of things where they expect immediately, how important do you think is being patient in that industry? mentioned something like
get three years and three years is huge time span and how important is being patient and work for a longer goals instead of want to have it immediately.
I know it's kind of easy to ask people to be patient, especially if they've never, if you've experienced and they haven't. And I also think there's a part where if you just appreciate even the small wins, like when you learn how to write a show script for the first time and it works, you have to enjoy progress, that success. Some people don't.
And so when you ask them to be patient when they're learning something new, if they won't even enjoy the small wins, they don't have the patience for the big wins to pay off. And so for me, I can think about when I was learning networking, I was a Linux system administrator and I just like, you know what, I'm going to learn the CCNA. I'm just so interested in networking. I want to learn everything from encapsulation. How does a network packet turn into electricity, move across the wire?
and then get decapsulated on the other side and then built back up to the application layer, what can then be processed by UDP or TCP and then depending on the protocol, HTTP. I was so intrigued by just learning that. And I remember seeing that diagram that showed the process of encapsulation and then showing the process of like, then here's where it turns that into an electric signal to send it. And then here's the rules around the speed of light and how to measure latency. And going through that, people are like,
But why you need that as a Linux system administrator? I have config and you're done. It's like, I just want to learn it. And I remember years later, I went to go work at Puppet Labs and we had to put the Puppet agent on Juniper devices. So the good news is Juniper operating system is based on FreeBSD. So I was so happy that 10 years prior, I learned FreeBSD because it felt very familiar to me. Also,
how you have to encapsulate things. Like you had to just learn, like this is what the device is actually doing. There was a benefit in doing this. I actually understood what it meant, right? I didn't understand what a VLAN was, but reading the documentation about what a VLAN does and where it puts the VLAN tag in the header. And it's like, ⁓ okay. Yeah, I understand this model. So a VLAN is a tag that goes into header because when it's being decapsulated, it can read the header, read this bit of information, see that it's a VLAN and then virtually switch it.
to another port or to another device based on that. If I didn't do the homework years prior, a VLAN would be a foreign concept that I couldn't either grasp because I will feel the need to understand encapsulation first. And so to me, that was like a three year, four year payoff. And I've learned to just trust myself that these investments tend to pay off in a way that you can't understand.
until the time comes. So one way I would say to encourage people to really feel this, think about all the things you can't do. An incident happens and say, hey, this problem is with the database. And then ask yourself, why can't I help right now? Why can't I offer any value during this critical moment? And then that will tell you like, ⁓ so then I think that creates a sense of urgency. Ideally, it creates a sense of urgency. And that's like the...
Other side of patience. So now you're urgent to learn the DBA stuff. So you go get the book, you start from the hello world, you create basic things, you're just training. Maybe you go get an IT certification so you can be a certified DBA, even though you don't have the real world experience. Then you start signing up for real world experience. But that's the part that requires discipline and that discipline and time equals patience. So the next time there's an outage or the next time there's an opportunity to work in a database space.
you at least now have the skills because now you're prepared. So that's the way I like to think about it. If you can't think about patience, then think about the urgency to create the starting point and then be patient so you can get there.
⁓ I also the idea to compare to your previous self, ⁓ two or three weeks ago, or two or three months ago, not compared to others, ⁓ just monitor your own success. And, also you mentioned it briefly, what is my impact at this moment? So lots of things are just done because we used to do it, but, measuring the impact or having focus on the impact is, it's, it's a huge winner, right?
Can we break that down for the audience just quickly because I love this phrase a lot, the idea of activities versus impact. So an activity would be learning about the Postgres database. An activity would be learning how to create a table, maybe even a good schema design. These are all great activities you could do. And even doing this for your company, someone wants you to adjust a column. Hey, we want to turn this column from a string
to an integer, right? And there's a lot that goes into that activity. But then you may not understand the impact of that activity. It may turn out that changing it from a string to an integer improves the database performance by 30%, reducing costs by millions of dollars, reducing latency and response time on the application side, leading to happier customers, ultimately leading to saved infrastructure costs and maybe more revenue because it turns out customers were leaving
because it took way too long to process the orders. That's the impact. And I think once you get good enough to learn how your activities are associated with impact, then it makes you want to learn how to do more activities to increase the number of times you can actually make an impact.
That's wonderful. I love that. Thank you. you eventually landed most position in tech at a huge You were distinguished you share a little bit what a distinguished engineers? Because most of the companies, they force the engineers to go into management positions to go towards a higher position in their career. But you managed to get
from my point of view, the highest position at one of the largest companies in the world.
Google was best job because it was my last job. And it was also the best version of who I was, right? I had been developing at CoreOS, Puppet Labs, big and small companies in between. And when I got to Google, it wasn't about can you write code. It wasn't about how fast can you write code. Do you know what a database is? Do you know what a network is? That's how.
When I got to Google, was all about, can you make an impact with those so joining Google at, maybe it was at L5 or L6, let's just say L5. And so a layer five, L5, you expect someone to have experience. And if you assign them something to do, they could deliver it. When you get to L6, you kind of want them to be part of what work should the team be doing? This could be a team lead.
Right? We should be doing this work as a priority and maybe not this work and you'll work directly with something like a PM. And so in those worlds, L5 and L6 is about that execution and maybe even showing leadership on what to execute on. And then L7, which is like a senior engineer and a senior engineer's goal is to be able to maybe get other people to improve over time. You're probably in leadership meetings.
where the decisions are getting made about what to work on before it even gets to the team lead. And so that senior engineer is going to be expected to create work for others, hopefully meaningful work, maybe not a bunch of bugs and breaking everything, but like meaningful things that lead to really big outcomes and impact. And at that level, remember a senior engineer is required in many ways to become a principal engineer first before they get promoted to a principal engineer.
So the title comes last. And so as a senior engineer, you will become a principal engineer when you're starting to show the ability to influence your team and other teams maybe. So let's say you're a principal engineer and you work on the networking team. Kubernetes comes out. And the Kubernetes team wants you to create container-friendly networking abstractions. And so as a principal engineer, you understand that you just can't add abstractions to
something as big as a cloud provider without thinking about the cost of doing that, right? Can you even handle that scale? So give you a concrete thing. Cloud providers have VPCs. VPCs are designed around virtual machines. Virtual machines have virtual adapters that connect to VPCs. And the whole fabric is optimized for this. When containers come into the mix, those containers want IP addresses, but they are not a virtual machine. They now have to share the interface with the virtual machine. And of course, you can do that with
bridges and natting and all kind of weird things. So if you're a network engineer, a principal engineer, you would suggest, hey, we already have a stack. Here's how you use it. And maybe bridges don't work with the firewalls and we will add that. So, all right, we will make bridges become a first-class thing and we can now surface pod IPs in the firewall stack. So the principal engineer knows enough about this to suggest to the other teams, this is your way forward with what we have.
but they may also kick off a five year project that will make it possible that a pod can have its own IP and we will change the entire network fabric to support pod IPs at the same level as machine IPs. But a principal engineer knows that you're not going to do this in one sprint. You're going to have to think across the entire company. How does this affect the billing model? Do we even have enough private IPs?
in a VPC to hand out to all of these pods. And so you're going through all of this strategy to come up with a plan that may take years to execute. And so a principal engineer will think about all of these things and come up with something that they can manage from start to finish over a multi-year period and influence all the other teams that have to work together, billing.
the UI teams, the VM teams, the networking teams to come together and execute over this plan after years. And they can do the difference between launches and landing. So launching is the first prototype. Maybe you shipped it. Landing is when it just works and no one's complaining anymore. And that can take five years. All right, so that's principal engineer. So now you're principal engineer, you've gotten promoted. You've demonstrated the ability to lead these complex.
Great.
impactful efforts, typically that involve multiple teams. So what is a distinguished engineer doing? And in many ways, you're doing more of that. You may do that across not just Kubernetes and networking. You may be able to influence what happens on the observability side. Maybe you inspire the other engineers to do a Prometheus API on top of the native system.
without actually having to implement Prometheus so that all the Kubernetes clusters can use that instead. You might suggest to the Spanner database team that a Postgres interface would make it more friendlier to new people that want to use our globally consistent database, but they don't like the SQL dialect. And so you're not going to write that code. You're not the principal engineer, but you may inspire that principal engineer, the director, all the people that have to really commit to this because it's not a
we ship it and see what happens. You ship this, you have to own it for years because you can't deprecate it that easily. And so a distinguished engineer has this unique ability to inspire other people to move in a certain direction, even when they have no authority over those people. And at Google, there was another premium put on the concept of external influence, industry influence. What does that look like concretely? Industry influence would be
You see Kelsey on the stage and you say, what is he going to talk about today? And I said, we're to talk about the value of serverless even for the Kubernetes community. I've worked on serverless just a little bit at Google cloud. And here's what I've learned. Serverless is more like a framework for writing applications and assuming that the infrastructure will do the heavy lifting. And for most people using Kubernetes, that is also should be the goal. So if you think writing YAML files and Helm charts is the end game, you're missing something here.
It's supposed to be low level infrastructure. So here's what we can learn from serverless. And you start showing them a container and then you do this live demo of using Amazon's Lambda framework that they have, where you can actually take a container, put the shim in there. And then that container can now run on Lambda and showing people moving from between these two platforms. What did we learn? And I remember, and I'm pretty sure they had their prior thoughts, but I remember someone watching that keynote.
and going to build one of the first serverless platforms for Kubernetes. And it's that nudging at the industry level where people say, Kelsey may be right. Maybe there is a way to bring that type of semantics to Kubernetes. Maybe people start changing their behavior. I remember the first time I'm watching myself being quoted in people's, like these are like company off engineering off sites where they're like, Kelsey said XYZ.
Here's what we're doing to our internal infrastructure to match some of the ideas he's gotten there. Now, look, I don't think I'm that special, but I think when you can offer clarity and insight that is reasonable and be willing to answer questions to back up why you've come to this conclusion, not only is it inspiring, it can actually give people a roadmap for the way forward. And to me, you do that long enough internally and externally. That is a distinguished set of attributes that a company will then say, look,
you have to demonstrate that over time consistently. And then maybe we'll decide that your impact has led you to become a distinguished engineer. And every distinguished engineer that I've met, they all arrived at this in a different way. Some of them did create the world's best search algorithm. Some people did build a new networking stack that reduced latency by order of magnitude. In my case, that was the approach that I've taken. Taking this
this engineer with empathy, the ability to see the big picture and to inspire people to move in a certain direction, that has been my contribution to earn that title.
That's great. ⁓ And certainly you have some superpower that means ⁓ having complex topics explained in a very easy and understandable way. And I think that's why most of the people enjoyed your keynotes and listen to you when you speak. That's certainly something, if you ask me, that helped you a lot on that. But if I would ask you, what was the number
one single superpower or skill set that helped you the most throughout your career.
I'll probably say that it's still along the lines of communication, but the number one skill was helping other people believe it was possible. And so I'm going to give a very concrete example. Let's say the company isn't using CI-CD. Everyone complains, we're just building stuff randomly. This guy's building on laptop. There's some scripts running somewhere. We should have used something more formal. And so you say, OK, everyone has a lot of reasons why this won't work.
Right? ⁓ no CI CD system is perfect. We had to train everyone to do it. It's going to take years. And so then they just said, let's just stick with what we're doing. So then it's like, okay, how do you inspire this people to believe that that's possible? And so then what you do is number one, you learn the current process. I think this part is so important. Learn what people are currently doing. So that way you're not talking past anyone and you actually understand the unique challenges that team has. And it's also a sign of respect.
I'm not going to come in and blindly tell you that you're doing it wrong and then just suggest something new that may not even work for with this situation at hand. So the first thing I do is just learn the thing. So now my storytelling becomes a lot more relatable. So you start by saying, here's how you're currently doing it. This is the machine. It's Brad's machine. We actually do most of our bills off of Brad's machine. Brad went on vacation. It's not Brad's fault. And we couldn't do a build. And so we should probably fix that.
So we have a couple of solutions. We can take Brad's computer. Don't worry, Brad, we'll get you a new one. But we can take Brad's computer and we can move it to a central location so that anyone can build on it. But then we have to have a way for people to log into Brad's computer without Brad being here or sharing his password. So we could create accounts for everyone on Brad's computer. ⁓ So that could be a way we can do that. Or we can move what Brad has been doing
to a different tool. Now, I don't really care about the tool, Jenkins versus this other thing, not important, but I'm gonna show you what I've managed to do on my own as researching one of the possible outcomes. So now you're in front of everyone. So the first part of the story feels good. He does understand what we're doing. I even laughed at the Brad joke part. So now I'm ready to hear where the story is going.
And so look, this is what I did. Brad has a bunch of scripts under this directory. This is where most of the builds happen. Somebody was like, I didn't know what Brad was doing all of this time. I can finally see the scripts. The first thing I did was sit with Brad and I documented all of the scripts. Hey, Brad, what do all these scripts do? And I learned how to do what Brad does. So now that I have all this stuff documented, you can find it here. I've translated this into a CI CD pipeline. So these scripts are just here. I just copy and paste them in this box.
And now you guys ready? I click the button and I let all the scripts that Brad used to run, run in the CI CD platform. So we're roughly doing the same thing with a different tool. And if any of you all want to try this, you all have logins, we've connected the authorization to the company's active directory. So if you work here, you have a login, you can totally log in. If it doesn't work for you, come talk to me. So now people are watching this like, so, so it's stood up already. Yep. It's totally installed.
⁓ does it work with our stuff? A hundred percent. Me and Brad did it together. So I got co-founders. So now me and Brad on the same team, Brad feels respected. Brad is the expert. Brad is the one that helped kick this thing off and Brad deserves a lot of credit. So now the team is like, so you and Brad did this like, yes, a hundred percent. So can I do it too? It's like, yeah, a hundred percent. So if you want anyone that's ready to write a pipeline, we can do it together before you know it. People are now porting their own scripts to the pipeline.
and the people that are feeling uncomfortable, then I hold a training for them. Hey, we're to have a training next week. Everyone's invited and I'll show you how to do this. Have a tutorial and you can ask any question you want. Six months goes by, all of the scripts are in the CI CD platform. And so the next time I say there's something we can do, I go through the same process and then they just believe it again. They say, we can totally do this. So that to me has always been a superpower, whether it's on a small team or at the industry level.
And if you do it well, and it does take a lot of work because I have to get my hands dirty, I have to learn the details of these things. So I'm not just talking without experience. But when it looks and when it's real and when it's true, a lot of people get inspired, then they respond.
inspire people, not do it on yourself, just take them with you. have a saying that you don't want to serve them the fish. You want to show them how to fish and go out on the sea and experience the good feelings. And this, pretty much feels like that. But, from the content perspective, what you just mentioned was basically something like we call today platform engineering. that.
something how good platforms work because platforms from my point of view are not only a technical solution, they also something about communication and ⁓ getting people involved. Do you see that the same way or you have another perspective from platforms and platform engineering?
So again, I think I've been practicing this stuff for so long that when these phrases come about, I look at them almost as a vocabulary list. I look at them almost as a label to describe a subset of the work or a subset of the attributes a person would need to do in order to be more productive for a particular set of outcomes. So when I hear platform engineering, some people are describing this concept where there's this goal where developers...
work with the business to figure out what to build. What's the right thing to build? Either what customers need, want, or what's required by law. But either way, there's going to be software being built. What's required to take that software and get it in the hands of the customers that can use it? That's it. it. And so the thing that underpins that is the platform. So it could be the platform you use to build and compile the tools, security scans.
configure the tools, keep them running, distribute them, secure them, whatever the thing is, that is the high level goal. So when I hear platform engineering, I'm asking myself, what is required to get to that level? So let's say you're using Heroku, right? There's the PaaS, you can just do Heroku push and the app is running. So if you're a platform engineer in that context,
Your job isn't to make Heroku run because that's their platform engineer's job. They made a platform called Heroku, has an API, and has tools. But in order to get things into Heroku, you have to go from source code to a build pack or some type of config that Heroku requires. You may also have key values that are configuration, so you've got to derive those from the team. So in that instance, a platform engineer's job will be maybe the CI CD pipeline.
and capturing all the things that needed to be expressed to turn it into inputs for Heroku. That could literally be your job. And then maybe performance, maybe you have a high Heroku bill and it turns out using languages like Ruby costs more money because they're single threaded typically. So a request comes in and as that request is being processed, you have to spin up another instance on Heroku costing a lot of money. There's a premium there. So as a platform engineer,
you may implement observability and observe that, hey, nothing's wrong with Ruby, but if we're going to use Ruby to process long-running requests, then it's going to get very expensive. So as a platform engineer, I'm trying a different language, whether it's Java, something on the JVM, or maybe Golang that can actually do some of this kind of parallel or concurrent processing. What happens to the build? So let's try it. And then you notice that you can do like 10x more requests for the same price.
That is good platform engineering. So your suggestion to the developers would be, Hey, we can stick with Ruby, but maybe we have to move to JRuby so we can benefit from the JVM. And if we did that, here's my prototype, we can actually process 10 times more requests without the price going up. Again, that's good platform engineering. Now I'm going to give you one more because I think it's so important because we're not always starting from such a nice starting point when it comes to platform engineering. And in many cases.
We're doing platform engineering across eight, nine different platforms at the same time. So the same person or the same team is going to have to do different things for different parts of the stack. So let's say the next platform you have is virtual machines. my God, you can imagine the contract you get from VMware or a cloud provider is essentially compute memory storage, maybe SSH. That's it. So in order to get software on this machine,
You may need puppet for configuration management. You may need Terraform. You may like Kubernetes. So you layer that on top. Now you have this API, you Helm charts, you got to layer that on top. And then you may layer something else on top. So no one has to learn Helm or Terraform. So platform engineering there again, is filling in the gray area between what the platform provides out of the box and what the team needs to use the thing underneath. And that gray area to me is platform
it's basically the integration part. I buy something from the market and I make it run in my company. And what I also heard is the impact thing again. So if you measure how much it would cost to have Ruby and replace it with JRuby, that's some business impact that we have. And that's also a job of the platform engineering team. Yeah, that's...
Exactly. It's very
fluid and needs to be very flexible. And there's no one size fits all because depending on your starting point, you may need a whole different set of skills than the other. And the mistake I see people make is they build sometimes out of necessity. So they did a good job. They build all of this custom tooling to make their existing platform works. Ten years later, a much better industry standard shows up. And you got to ask yourself a question. Wow, do we keep maintaining this platform?
Or do we pivot to where the industry is now? And I think that's the hardest thing for a platform engineer to do because that deprecation process isn't easy. You can't just go from one platform typically to another one. There's usually a period of time we're to have to maintain both. And that could be years. So there's a lot of discipline required.
And it's also the sunken cost fallacy that we've put in millions and we can't switch because we invested it. on the same time, it's important to find out the right abstraction. If your abstraction is too tight, then you're not able to innovate when cloud provider or other huge companies move on.
Yeah, you may not be able to switch quickly, but you may be able to transition over time. And I think that nuance is still worthy of being explored.
Yeah, excellent. We were talking previously about experience and what I see is a lot of engineering teams, they suffer cognitive load because they have to know so many tools. Then we come in and we build a platform and a lot of stuff away from them. But on the same time,
they work on that platform and on the abstraction, but they don't know the things below that. And do you see that is a dangerous thing because if you know what a root user is and you put a file on that server and you can operate them, that's if you know the basics, then the abstraction makes sense. But if you started an extraction and you don't even know the basics, that's maybe a recipe for disaster.
What's your take on that? If I have like an implemented platform that abstracts a lot away from me, call it a Heroku or a fly or something, and people are just used to it. And is this something where we miss out?
So this is where I contradict myself. On one hand, I'm always telling myself and others, learn the fundamentals as far down as you can. You should do that your entire career at your own pace or at the pace necessary based on the work you're doing. So if you're writing PHP, you might want to understand what PHP bytecode does, what the PHP virtual machine is doing, and then how that relates to the kernel and the related system calls.
But don't really need to learn that in all in one day. And then when you think about it, it goes deeper than that. Then how do those things map onto the processor that you're using? All of these things are just like a lot. It could take a lifetime to learn all of these things. So I'm always going to advocate for going up and down the stack. But then there's this other part. In a professional setting where we're on the team, we're teammates. And as teammates,
We are working as a unit to try to do something better than we could do by ourselves. And this is where I specialization is okay. So if I look at the soccer field, there's a goalie, there's full backs, there's strikers, there's all of these things. There's referees, there's the manager. There's all of these people who know that they could probably play goalie if they had to. Doesn't mean they'd be good at it, but the goalie got injured and the backup goalie got injured and it's like, we have no choice.
But now that we have to have a defensive back play goalie, okay, but that's not where we're optimizing for. We're going to optimize for a scenario where you knowing what a goalie does can help you be a better defensive back. But I need you to be the world's best defensive back to compliment everyone else on the team. And so to me, when I think about what we do in software engineering, I think silo is okay as long as you have great APIs in between them.
So if I'm a developer, it will be great to have someone that says, I check my code in, without fail, I will always get an artifact. I don't even have to think about it. As soon as I check this in and I have my pom.xml or I have my go modules file, no matter what, there's always a binary that works. And if it doesn't work, it's because my pom.xml isn't right. And there's going to be a UI or some type of logging.
to tell me there is a problem with your POM XML, please fix that, test it on your local machine. If it builds on your local machine, it will definitely build in CI. And if it doesn't, you will not have to worry about it because our team is going to take care of it 100%. Just like the internet and the people behind it take care of the internet 100 % and the networking and the power people take care of the power 100%. And so I think if you have a collection of people, especially at the larger enterprises,
where we're all doing the best job we can, being transparent when we fail, aka logging and dashboards, and then having some type of SLA. And people are like, Kelsey, that won't work. It's like we have a whole industry that does this. It's called the cloud. At the cloud, you don't get to interact with all the teams that provide everything AWS offers. They do exactly what I'm saying. It's a big silo. And the reason why it works is because they tend to give you APIs that allows you to spin up a virtual machine or tear down a virtual machine.
the kernel has APIs for getting information about how the kernel is performing. So I think when you look at all of these abstractions and the role of a platform engineering team is to put in those APIs that are missing from the stack. So as a collective, if I'm a developer and I have a very strong platform engineering team, they're going to give me these APIs backed by support and SLAs that say, listen, something's went wrong. Our team has it. Now I may ask you for help from time to time.
But the everyday idea of like writing software and deploying it to the same servers over and over again, come on, we got to have better workflows and better abstractions for doing the most common things that we do. And then we collaborate on the things we don't do very often until they also become common.
Wonderful. And that's basically some of the cost savings you can have with a platform. If you don't have to do the base things, the common layer over and over again, you just add the business value on top of it and you rely on your folks which have their own business value providing the platform. Yeah, that's great. I'm switching from one hype topic, the platforms to another. And I know you...
You have a strong opinion on that and I would like to touch on AI with you. And what ⁓ I heard in one of your talks, you had your daughter as a guest and you were having a Q &A at the end. And suddenly the moderator hands her a microphone and she was asking you a question that was pretty deep and she was asking with all the AI.
And I paraphrasing here, what's the future for me when I start out in tech? And this resonated very deep with me because I a small daughter on my own. And we have the good thing that we started in the tech career. So we have some background knowledge and now the things are going to change. But what perspective can you give the engineers for the future from your point of view?
Yeah, so I'm not just an engineer, I'm a human first. I think we forget that when we're talking about technology and the work that we do. So from a human, I have a philosophy on like life and like everyone's in constant search of purpose. And so typically we're building all of this stuff because we're trying to, you know, certify some purpose, right? You don't build a bridge until you have to cross two points that are so big that a bridge is required.
And software is like that in many ways. So as a tool, I actually have very little opinion about AI. I tell people I'm a retired person. And so I don't really need AI to have an incredible outcome or not. It's just not as important as it would have been 10 years prior. So AI is, know, whether it's a good tool or not, I have actually a very weak opinion about any of that. It's almost short of, I don't care.
But in a way that's like, there's so many innovations that happened in the world that I don't pay attention to. Medical breakthroughs, there's all kind of amazing things happening in the world. AI is just one of them. Now, some would argue it's the biggest one of them all. And that's the part where I say, I don't know, man, like artificial is great, but I also like the real thing, right? Real is also amazing. The people who have created the training set from our experiences and recording of our actions,
have become data sets that we're using to record the machines. And look, machines are really fast and powerful. They have access to all this data that they can draw and pull from. So I get why people are excited. But there's a few narratives that I worry about. So one of the narratives is that you can reduce costs by using AI. And then you say, could you explain the details? How am I reducing costs?
Does AI cost less? no, no, no. You will pay us more for the AI enabled tools than you were paying before. Okay, so I'm not going to save money there. no, no, no, no. We are definitely going to get our money. ⁓ Does the hardware cost less? no, no, no, no. It's exponentially more. Okay, so I'm not going to save money on the hardware. Absolutely not. You're to pay more. So where am I going to save all of this money with your people? And this is the tough discussion to have because
as a society, we were really hard to govern how we live with each other, right? If a person can't fish, if they can't learn to fish, we found a way to fish for them using technology so that they can also have fish. That's the nice thing about technology. It changes that saying to, you know, it's better to teach someone, you know, instead of fish or giving someone a fish, you want to teach them to fish. But if they can't learn to fish.
that we use technology to feed millions of people with one person. That is amazing. That's what the beauty of technology is. When you come with that mentality, technology is amazing and we need more of it. But what happens when you say, well, we don't need people to fish anymore. So, okay, let's dig into that. We don't need anybody to fish. But what happens when people forget how to fish? Or...
that people selling the technology that does all the fishing decides to raise the price. Or maybe because your country doesn't like this other country that they decide that they need to turn off the algorithm that does the fishing. And so now people are starving. And in that moment, the people will say, we don't have the robots to fish. Luckily, there's a hundred million of us. We will fish ourselves. Does anyone remember how to fish?
And he said, no, we don't even know how to do it. I think you have to have a pole and a string and I think there's something called bait. And you're like, my God, we don't even know how to fish anymore. This is bad. And I know that sounds like extreme that most people in the world can't remember phone numbers. So if you get lost and someone says, hey, you've lost your phone, you can use mine, just call for help. Imagine a world where you can't remember any phone numbers.
That is a reality right now today for the majority of people. So it's not far fetched that if you leverage technology in a way that reduces the biggest, you one of the best muscles you have, which is your brain. So you know, you've had a practice. So when I think about AI back to your original question, all of this discovery that we've done, all the stack overflow searches, all the trial and error, that is how we've built up ourselves. This is how we've trained our own models.
to recognize good code and bad code. And look, I think the other argument is that most people have never done what we talked about earlier, which is this idea of training, figuring out what the best is, pushing themselves to be the best. Maybe many people have decided to just be mediocre. I'm just going to do the bare minimum code. I don't care what it looks like as long as it compiles. I don't care. Just ship it. And if that's where we are, if we revert it to that,
then of course people are going to be excited about a thing generating mediocre output. But it doesn't answer the question, what do we do with all the people? So that's just where I feel like I have to put my attention. What do we do with all the people when the system is designed to get its efficiency by reducing number of people, but not being very clear about what do they do next? Right? It's not clear that these well-paid individuals that get to be creative all the time, that have all these opportunities, will get that again.
Mm-hmm.
And of course, I'm going to be saddened by the potential that my daughter may never get to experience that world. I don't know if it's going to be as immediate as that, so I don't have that kind of fear. But maybe I fear for the next generation after that, that may never get to learn how to fish.
It's really, really great insights. I had an interesting discussion with a coworker of mine who was really into AI. And he said, imagine this is the same thing where back in the years we hadn't had any hand lettering or any writing. And we were all memorizing the stuff in our heads and our heads and our memory was like perfect. But all of a sudden somebody
Hmm.
Hmm.
explored and invented the written format so we could persist something over legacies and we can be more precise. We wrote it down and he compared this to the age of AI. said, basically right now from our skill sets, we hold everything in our memory, even if we're typing, but metaphorically. And ⁓ what we learn right now is something like
writing and persisting things over generation. Is this something which relates or is this something that could be true that we just avoiding things to have like a bigger progression in the future?
So as long as we don't ignore the human responsibility in this equation, if you read something and it says this is the law, you can only drive 30 miles per hour in this area. You read this. And then you go out there and you drive 30 miles an hour and you get pulled over by the police and they say, hey, here's a ticket for speeding. And you say, hey, listen, nope, right here, it says that was written that I can drive 30 miles an hour. And the police officer says, well, that's wrong.
Here, you can drive only 10. Now you're going to get the ticket. Doesn't matter that you read it somewhere because it's a judgment call. You've ingested this information. How do know it's accurate? How do you know that it was written for where you live? How do you know that that thing didn't say, well, you can drive 30 miles an hour after 4 o'clock PM? So the context is missing. So the responsibility is that
There will be enforcement by society in this analogy, the police officer. And even if you've read it, you're still wrong. And so now it doesn't matter what's written. Now you have to adjust the peers. no, no, no, I don't care what's written there. I got a ticket last time, so I'm never doing that again. So you will always continue to hold the real rules of reality in your mind regardless, because guess what? We're also in an age where we have conflicting information. One thing says this.
One thing says that who decides you have to decide what I'm watching happen is people. Let's just take code generation. You don't know the language. You don't know how to do it. So you say, build me an application and it spits one out. And you're like, OK, I don't know. I don't know if this is a good app. I don't know if it's secure. Hell, I'm not even a developer. I've never written an app before, but this thing has put out the app.
and it seems to work. I can click on everything and then you launch it. And then before you know it, you're hacked. You don't even know how to fix it because you're like, why is it doing this? You don't even have the vocabulary to describe the prompt on what to do next. You can say, hey, people are using my app and putting things on the homepage that I don't like. Remove the things. And they say, well, all right, I'm removing the pictures. Oh, thank you. AI is so intelligent.
You wake up the next morning, things are back. Hey, can you prevent these things from ever happening again? It's like, okay. And then it rewrites the app, but this time in a different programming language, right? Because it didn't have the context that it had to do with any other one. The model is optimized for security stuff using a whole different framework. And then you launch that, but now your bill is five times more expensive before, because you're using this framework that is not as efficient, but you don't know.
Hey, why is my bill higher? So now you're arguing with this thing. You still don't understand, but I promise you, you will have to pay the bill because the bill has nothing to do with your understanding of whatever. So if we treat AI like things being written down, that is still your responsibility to read it and own the consequences of believing what you've read. Absolutely. We should bring in tools that can help. Hell, I've copied and pasted lots of stuff from Stack Overflow.
and tweaked it until it worked for me. Like that's, that makes sense. Why would, that's like sharing knowledge. But I think people are doing more than that sometimes. I think they're outsourcing thinking. That's the part I would worry about.
Yeah, and I hear when I speak to people that ⁓ they treat AI as a solution and not AI as a tool for the solution. And this is something that's a small word in a sentence, but it's a huge difference. And you mentioned something like...
Hmm.
knowing what's right and what's wrong and getting understanding about the information that you get How is your opinion on if today's large models know basically the whole internet because they were scraping all the data, feeding it into the model and training it. What would be the next model because the internet information is already there and maybe we produce synthetic data and
But this is not something new. We still want innovation in some part. And the innovation comes usually from the humans that don't rely on, sometimes they rely on existing information and extend them. But how do you see this? If you have like people at some point, the truth is dictated by data engineers. So whenever company trains the model decides what's the truth and not the truth.
And on the other hand side is that no new, really new information is presented to the model. It's generated synthetically from the other data. Is this something that you are concerned or do you just, you know.
So I was watching a podcast and the guest was, I think his name was Richard Sutton. This is the guy that won the Nobel Prize for reinforcement And like many people in the AI space, these are credible people, there's a list of them, that said the LLMs are dead end. You all could be happy with this result. You all can find utility in it even, right? Like the ability to...
generate a document that other people read that could be amazing, right? There's some utility in it. But the reason why they don't think it's going to get us to where humans are, like what we call real intelligence or whatever your definition of AGI, but just say, I like real intelligence, like you as a human. And he is big on reinforcement learning. And the thing that I tuck away from him describing why an LLM is a dead end, and that's very different than an LLM being useful.
And I think that's the thing people have to come to terms with. There are a lot of useful things that can make no additional progress. A pencil is useful, but that's kind of the end of the road. We know it's lead wrapped around wood and you can write with it. That's it. We're done. If you want something else, we got to go to a different architecture. And so that's what they're describing is that there's a need to go to a different architecture. Then they're trying to remind people why they spend so much time in the reinforcement learning.
And they were like, think about it. A bird learns to fly without reading anything. Right? It observes the world. It's engineered to do these things. And then it flies. And there's lots of physics involved that the bird knows how to do. The data scientist does not decide the truth of flight. They can only observe it, describe it, and then a machine can repeat it. The machine has never flown before. The machine has never saw a bird.
and had chills run down its spine because the bird was so beautiful. The machine has never taken a photo just at the right time to be so amazed that they could do that. And then shared that photo and it became the front page of a magazine cover and the experience of knowing that they tuck in that photo and inspired someone to have a great day. The machine has never experienced any of these things. All they can do is synthesize the fact that someone said, this is a great looking photo.
If enough people say it's a great looking photo, the machine is probably going to statistically say it's a great looking photo. Why is it a great looking photo? And all they could do is like, well, lots of people have said it's a great looking photo. Is it a great looking photo to you? It has no frame of reference. If it's a good photo, it doesn't know what that is. And I think the fact that it cannot understand, because I think if you believe that understanding
comes from experience. And a machine can never have experience. And maybe you can put it in a robot form, and then we get into a new architecture that then uses experience-based learning to create its own model and evolve from there. But that is very different than the data scientists collecting random data that could be conflicting with each other, deciding where to tune the weights, creating bias in a model, therefore creating this synthetic truth.
And that's just not how society works. You can say whatever you want to be true. People will decide and then we have courts and we have ways of selling disputes. So I do think the LLM architecture from that reasoning alone is a dead end. It's not going to replicate you, but it can parrot you. can imitate you. It can mimic you. But I will never believe it because it's never had that experience. And that's what we look for in humanity. We like to listen to people who have done it.
And then we want to hear their perspective, not a, I've read the most books about winning World Cups. Yeah, okay. I think I want to talk to someone who's played in the World Cup. Even if they lost, I'd rather hear their perspective about the World Cup because it's going to be very different from someone who just read about what it's like to be in the World
Amazing. And I think what you just described is know the technology. Even if you use the technology, know what are the strengths, how to use it, how to leverage the best of it. And then you're working with the tool and you're not becoming the tool at the end. So this could also sometime happen if you're just run blindless, be steered by the AI, you become the tool.
know the basics and control the tool and use it for something useful. That's great. I have another thing that this is a German or European podcast and you worked at a cloud provider. Right now we have all the discussion about digital sovereignty because people recognize that all their stuff is running in data centers that's owned by US companies.
On the same site, when I was at KubeCon, I saw that map from the contributors there and there was a lot of stuff happening in Europe. I would love to hear your opinion. How sovereign should we be? Is this something that we should be concerned of and what could we do?
I mean, sovereign is tricky because I don't think every country wants to live on an island, You want people to buy BMW cars and Mercedes cars and Volkswagen's, Kinepix pliers and tools, right? So you make a lot of things, but you're not making them only for Germans. You're making these things for the world. So in that case, the rest of the world would have concerns like, maybe we're buying too many Mercedes cars. We're relying on Germany for luxury cars.
Maybe we should make our own luxury cars, right? And compete in that arena. And I think that question is always healthy. So in that scenario, given that we do like to trade ideas, we do like to trade product and services, it's great. So when it comes to automotive sovereignty, Germany should not be afraid, right? Now you might be afraid economics wise, will people buy enough of your cars to offset the number of cars you're allowed to buy from other places?
So you will always have that situation. But I don't think Germans are worried about their ability to make cars. They make some of the world's best cars at every scale. Even myself, I drive an ID4. I drive a German car that happens to be made in America, actually. And so that's a beautiful outcome. So when we get into other parts of the economy or other industries like cloud,
Maybe we shouldn't skip cloud because maybe we talk about some things that have come from Germany in particular. ⁓ When I was in Berlin a couple of weeks ago, I spent time with Leonard Pottering. This is the founder of System D, which has found its way deep into Linux operating systems. And then you have Linus Tover, which is Finnish, he's from Finland, and he created Linux and created a whole community around this. So just right there, you know, in the case of System D,
Germany is leading where Linux has largely gone. The init system has largely been replaced. A lot of the tooling we use has now been rolled into system D itself. It's a big part of the user land when it comes to interacting with the operating system. So in many ways, Germany has led the way, if you want to count Leonard party as one of your own in this direction. So this is amazing. You've already demonstrated it, but something has happened around, I think building tech companies.
you know, the capital required to build a cloud provider, the type of things you need to do globally, because a cloud provider that only dominates in one country isn't that great for people who want to be global. So now it's like, wow, is there an appetite for Germany to take the same things they've done with SAP, Volkswagen, Mercedes? Can they apply the same thing to cloud? And I think that's what it requires. So I think the Germans know how to do this.
Right? This is very clear for a long time. They know how to build very complex businesses at scale. You already have enough proof points. The question is, can Germany now just rely on other countries for things like cloud? What's the drawbacks of doing this? And we kind of talked about it before. If you stop learning how to fish, there's going to come a time where it's almost feel impossible to build a cloud platform in Germany because cloud keeps moving.
It's almost moving faster than you can start learning today. So the most expedient thing to do is to do what has already been done, which is use one of the major cloud providers. Maybe you can push for open source to help democratize your ability to float between the other cloud providers. And then if you take something from like China's book, they have Huawei cloud, they have Alibaba cloud.
And those things have evolved over the years, right? They're pretty good offerings where people in China have a really strong competitive offering for cloud. So now we have to ask the question is like, is that something Germany should do? Is it even big enough to make this a valuable outcome? Some would argue the question is yes. So you got to start now. So in 10 years, you can compete. Or you do what I've seen in practice, which is you got to have data centers built in Germany.
And then in those data centers, a lot of the hyperscalers have mixed the employee base to be part German or mostly German that are operating these data centers. And that's a good example of them learning how to fish. How do you operate hyperscale based data centers? How will you tune our laws and compliance and match the capabilities of those hyperscalers? And if you do a good job long enough, when the time comes, you're now going to have a lot of people who have worked for Google, Amazon, and Azure.
that can now be the people who pioneer whatever cloud efforts that Germany does in the future. So in many ways, you're already on the right path, in my opinion, of doing things that match reality. You got to do what you got to do for today. But I think the investment around making sure your employees are actually a part of the equation. And at the time, whatever come, hopefully there's just peace and we don't need to do this out of an emergency. But at the time ever came for any reason.
You now have thousands of people who can do a really good job of doing this.
That's an amazing take. And I think you're referring on the sovereign cloud offering of AWS and Azure. I'm sure Google offers something like this as well. And how trustworthy would you see those offerings? I mean, they have the skills and you don't write software at one day to another. And they also introduced something like, there's a Germany company.
where people are actually hired at the state and they hired at AWS. And whenever something happens, they say, I'm still hired by the state so I can proceed with our procedures. Is this something, I mean, I'm totally, love globality and I'm not hoping for it, but these are serious questions the community ask around here.
It's the same question that's being asked of you. Can I trust Mercedes? I have my family in this car I'm driving and one software update can make this car be dangerous for me. Can I really trust Mercedes or should I only drive Ford? Right? So it's the same trust that we're giving that you're going to build a quality car. If something bad were to happen, you will own it. You will improve it. I'll give you good example.
The reason why I bought the Volkswagen is because they had three years of free car charging. So before I bought this house, we were living in an apartment. And so I was like, I want to buy an electric vehicle, but not having somewhere where I can charge it reliably isn't great. And so when I was shopping around for cars, Volkswagen ID4 came with three years of free public charging, Electrify America. And in my brain, I was like, how much is it costing Volkswagen to pay
Electrify America, which I'm thinking at the time was a different company to give all of their customers free charging. How did they negotiate this deal? And it turns out Electrify America is a byproduct of the diesel gate scandal where Volkswagen was caught cheating on a mission test. And part of the restitution is that they will create this charging network for EV. So we're going to not only get better at energy efficiency,
by not doing that anymore for those cars, but we're also going to invest heavily in electric vehicles and to accelerate this move towards green energy, we're going to provide free charging to people who buy our cars. And so to me, that's a good example of what happens when you ask this question. What if something bad happens? Because we're not talking about this because everything good is going on. We're talking about this because something bad may happen. So the Volkswagen example is an example of something bad happened.
the company fixed the problem and actually improve things for the future. And I think that's what we expect from the cloud provider. trust, but verify. Are you getting what's being promised? And when it doesn't happen, then you have to have some form of accountability. And that accountability, we know how this works, like, you know, fines, percentage of your revenue. There are many ways to make companies do the right thing. And I think given those consequences are on the table,
It's one of the things that motivates companies like Google and Amazon, Azure to not violate that trust in the same way that BMW and Mercedes and Volkswagen are going to build reliable cars because the moment those things are not reliable is the moment people stop buying these things. So trust, but verify. And of course, learn how to do these things yourself. So when the time comes, you can.
Thank you very much for this and I think this is a very good conclusion on that And even the pressure to think about it more right now and to find out your options and be more ready to participate more in the game, not just being a fence guest from watching the game, just be more inside and play with it. I really like that.
I just want to segue a little bit before we close to your current situation. You left Google, you retired, but you're not sitting at home. I guess you were for a couple of time, but you're speaking on public stages and you also be an advisor to a lot of startups. So what's your day now since you left Google?
So I'm retired, not tired. Lots of experience, lots of energy. So where do I put my energy? So I do things like an average person would do. You clean your house, you have more time to appreciate the things that you own. I've learned how to do my own maintenance. I'm actually in the process of replacing the carpets with hardwood floors. I'm doing it myself. I learned all the skills.
So I'm still acquiring skills and knowledge. still think it's powerful, whether it's software or in other areas and disciplines. But the advisory stuff kind of started while I was at Google, know, Google Ventures, Google's investment arm. They were investing startups. I would come on as an advisor to kind of help them either go to market, build and design products, especially in my knowledge domain, like Kubernetes and containers and infrastructure in general. So I kept doing that. So I usually advise 10 companies at a time. So right now.
That's going to be companies like Docker, Q Labs, is based in Europe, Sivo, which is a small cloud provider based out of the UK, focused on sovereignty. ⁓ And in those cases, I'm working with the founders, I'm working with the founders, product teams or engineers, on just how to build things. Crossplane also comes to mind, UpBound. So this array of companies, they all kind of have similar challenges. You want to build something that people want to pay for.
And you do believe on the success of openness, like open APIs, extension points, and open source to some degree. That has been my sweet spot. also am pretty good at thinking about like business and go to market and pricing and how you structure these things. And that led into doing due diligence for a venture fund. So it's a company called Trenchers Capital. These are two pro NFL players. One of them retired, Larry Fitzgerald. He's probably going to the hall of fame this year.
The other one, Kelvin Beacham, he still plays for the Arizona Cardinals. And when I met them, these are pro athletes. How do they have time to do anything else? But they've been investors for a number of years. We call them limited partners. So they give big checks to investment funds and they make investments on their behalf or the collective, the limited partners that put money in and they started their own fund. So from scratch, they raise money from others so that we could be the investors.
And that means we have to do our own due diligence. We have to look at the business model, rewrite the memos. And then we're the ones cutting the check, putting liquidity into these companies, hoping that we're making good investments so that long-term we can return a benefit to the people in the fund. And so I've learned a lot about venture capital, all the slang and terminology that they throw around. I've also learned that a good company isn't just based on the good technology of the best engineers. Sometimes you just want a company that has a strong founder base.
that has known to be able to pivot and execute and do what's necessary to build a business. Even if that means throwing away the piece of technology or not being great on day one. I've seen dumb ideas be successful. Like if I reveal, ⁓ no one's gonna ever buy this. And then they have an amazing exit. So I learned to really humble myself when I'm evaluating these companies and really think about the big picture and what potential lies there and then how to align that with your investment thesis. So it's one thing to judge a company based on the product and services that you touch.
It's another to judge them based on would you put money into them to see them grow even further.
That's very interesting.
Are there common patterns that you see on companies that are not as successful as they could be?
I think the number one pattern that I see is companies not understanding that they're building a business. So some of the founders I worked with over the years, maybe going on almost four or five years now, they may be really excited about the product or the project, like open source project, lots of users, lots of GitHub stars. But then they realize they have to build a business. So now you have to think about pricing.
marketing, selling to people, telling a story, good documentation, even the logo and what the website looks like. And a lot of people didn't really want to run a business. They wanted to create a project. And so when they raised their seed round, maybe they were like, oh, okay, we're going to hire really strong engineers. We're to build all of these features. We need to listen to users. There's a distinction between user and customer. They're mostly the same except for one of them pays you money.
And so a lot of times they over index on the user and that's nothing wrong with that. But if you're in charge of building a business, then the customers are probably a bit more important because without them, the business doesn't work. And that's the part where I think a lot of people don't like the business of software. They want to just build cool things, ship cool things.
And many times, some people are very afraid to ask for money. Think about this, you built a service that you think is valuable, but then you're uncomfortable asking for money.
But the thing is, when you start a business, that's literally the point. You're building something because you want to charge money for it. And remember, this is a business. I'm not saying that every open source project needs to be a business, but I'm saying every business needs to be a business. And that's typically what a startup is designed to do to become a business.
So maybe a more holistic view, looking beyond to the technology and just also think about making money because that keeps the engine running. You mentioned about open source and we see some of the open source product. They were really solid and they were really good. People appreciate them and the people weren't making money off it because they started free. You mentioned Docker. think Docker is an example. had an
and very good product, but it was free for years. And now it sometimes feels that I'm coming with additional extensions to finally make the money. And I understand they need to pay the engineers and the same thing with what happens with Terraform. I think Elastic was also some of the cases. Is this a bad movement for open source or is it just some, I don't know.
The business needs to be more thorough where to find the sweet spot, where to provide the impact to make money because usually they rely on really solid technology that's already fulfilled their plan. It doesn't have to be fancy bells and whistles. It just works. And now there's the part we need to make money. What's your take on that?
think sometimes since software seems to be so easy to build and distribute, we think it should be zero dollars in terms of monetary value. If you're a software engineer and you work at a company, then you probably want to be paid. There's probably no way that you're go to work every day and do it for free. So think we understand that part of the equation. But when we see software that someone else has written, especially if it has the open source label,
we think it should cause zero dollars. And in many ways, if you think about the core essence of some of this open source licenses, it's kind of the point in some cases, right? Like if I put this on GitHub, so I'm kind of sharing the code and I put a license that says you can also take this code and do whatever you want with it. And to me, when I think about that definition of like open source, let's say you pick Apache 2.
If you don't share the code, then it's not open source because like, it doesn't matter if I have an Apache two license software project that is sitting over here in the corner that no one can see. It has an open source license, but I don't know if that counts. Is it an open source project if no one can see it? I would probably say no, I don't think so. So I think, and I could be wrong, but I think once you share the code,
and then apply a license to the shared code, do you enter the realm of open source? The end to end realm, meaning other people can enjoy the benefits of open source, not just you. And then the license dictates. So you have GPL, Apache 2, MIT, and maybe there are a few others. So now we are at this point where that feels free, because the license kind of dictates the terms. The license doesn't say you can use this if you pay $10.
Now, you could make a license that way. That would be traditional software licensing. And you could probably put a license file that says, you can use this software, but you have to send me $5 first. And then you don't have a way to enforce that. So people could use it and not pay you. There are lots of people who violate open source licenses like GPL. Maybe they're not redistributing changes. So it's not just monetary ways that people violate these licenses. So I understand why people have associated open source with being zero cost.
So then let's talk about the next phase that creates this illusion. If I were to build binaries for you, this is version 1.0 and it's a compiled binary. I think it's probably safe to say that the majority of people who use open source have never looked at the source code.
They probably are just using the free binary. I know a lot of people use VS Code. They've probably never seen the VS Code code base.
So if that's the case, then some people then associate open source to this free binary that you can just use or free software that you can use. And not necessarily about the source code because they don't care about the source code. They care about the utility of the application. So some people think that's open source. And for a lot of people, that's the only relationship they have with open source, a free thing that's useful.
If there ever came a time where a company were to form around those two things, the source code and the binary or some artifact, and they say you can have it for zero dollars, then you say, well, what's the point of your company? Well, you can call us if there's any problem. ⁓ so this doesn't work. I can call you. Say, huh? But don't you have to pay the people that I'm calling? Yeah, I do. So is support free? And the first thing a lot of open source companies will do is say, no, no, no, you have to pay for support.
If that price is too high, then I will just do my own support or I will just go to the community to see if someone will help me for free.
And so you might say, okay, that's fair. So you may lower your support or there'll be big companies that will pay for support, but it may not be enough money. Or maybe your investors don't think it's enough money. So you got to do something else. Hey, wait a minute. All of these companies are using your database to run even bigger companies that make more money like they do. Well, what do you think is fair? Say, well, we think it's fair to say $100 a year. So you see, put a price tag on it. This thing is now $100 a year.
And the community says, can I use it for free? You can, but we would really like you to pay us $100 a year. So I'm not paying if I don't have to. So most people elect not to pay. Years go by and a company is like, cannot, we can't even run this business this way. everyone finds it still useful. So we have a good product market fit, but we don't have the revenue to match the market fit.
So we're going to require you to pay. How are you going to do that? Well, we're going to take away the binaries. We will no longer compile the binary. So we're not going to have version you want the next thing, you got to go to master. And you got to take all the bugs and all the new fixes. And it may be unstable. But you can have it for free. But if you want stability, AKA version releases, then you have to pay for those.
and you can no longer download this for free. Now I understand the reaction in the community. It makes sense. If I've been using this thing for free for 10 years and all of a sudden you say I can no longer use it for free, I feel like you've done something bad to me. So imagine you took something from me. Now the fact that you gave me something for 10 years doesn't matter right now. You've taken from me. And so now I'm upset.
You took something. No, you just realized there's something exactly.
And the only way to make this right is you have to give it back. But I'm telling you, I cannot afford to give it back. And so you tell that group of people that you were not paying us anyway, and we already had product market fit thanks to maybe you using it. Maybe you did some bug reports. Maybe you even contributed a little, but that was then. This is now. And I think we have to be OK with this. So if Mercedes gave me a free car for 10 years and tomorrow they say, hey, look, you've driven that car for free. We even paid for the gas.
Tomorrow you now have to pay for the Mercedes. I'm not gonna go online and say, I hate Mercedes as a company. You cannot trust them. They let you drive a car free 10 years, they pay for all your gas, and all of a sudden they want you to pay for the car. Now some of the people say, Mercedes was wrong for setting that expectation that I can drive a car for free. And I've built my entire life around free transportation. So I understand that part and I think it's fair to have criticism in both ways. But when it comes to me personally,
I will tell the companies like Elastic and Hashtag Corp. Thank you for the previous 10 years. In the case of Hashtag Corp. I have code that I've contributed to Terraform and to Packer. And I'm okay with them now saying that they want to try something new. We can all use the old version. We can all choose to fork it and maintain it ourselves. So I'm not as angry at them, even though I respect the people who are. I'm just more thankful than I am angry about that. So I think going forward, if you're an open source company.
I think it's smart not to give away the entire product. If your goal is to build a business, then I think a good way to start would be what we're seeing now.
You can post the source code. You can guarantee transparency and collaboration, but then you may say, we will never publish a binary. There will be no concept of version one and version two and backporting bug fixes. That will never be free. if you can deal with the top of the tree and work out what version is which and cherry pick and patch it yourself, you can do that. We're not going to stop you from doing that. So you now have to do the work and you will understand the value.
If you want us to do that kind of work, you can buy the binaries from us. And I think that will be a fair way to look at it. So open source gives you transparency, maybe even a way to contribute, or you can fork it and go in a different direction if you disagree with the current direction. And the company can now keep the value and sell the binaries and sell any additional value added service on top. I think that's what we will actually get to if we're talking about a business. But if you're open source project, then your idea is to try to reduce friction.
⁓
so the community can use things. So of course you're to give away the binaries if you're not trying to build a business.
I think that's really great advice. Thank you for that. But sometimes I feel if you brought the analogy with a car and you give the car for free, I have the feeling that people, they can touch something, if they have like a physical object that's even more associated with worth, then have something virtual, but they just can download. this is something also
What, what's the mindset thing? And I, I like your analogy always there has to be someone who's actually working on it and making it happen that you can use that tool and we should all be more aware of that and should pay for that if we can afford it. And otherwise there's always a choice to either fork it or use another tool.
And I also think, just speaking for myself, I'm not telling the entire open source community what to do. But for me, I just try to practice gratitude. Getting these things for free, thank you. Sharing your idea, even if you don't give us the source code and you just choose to write a white paper, thank you. And I think that gratitude, for example, everyone relies on air to breathe. It's just there. You don't really think about air until you don't have it.
then air is the most important thing in the world. And open source can be like that for a lot of people. So if you don't practice gratitude, some people even do breathing exercises where they remind themselves that breathing is important. And so I think if we were just practice gratitude, it will create a little bit of empathy. And you can see this sometimes in open source projects when you go to the issues that are open. Some people are demanding new features, demanding bug fixes for their situation. It's like, hey, whoa.
You know, how are you demanding so much from people that have given so much?
Yes, of course. And we also should think about people on the other side. They are doing it on a free time and they are not paid for it. be more thankful and humble that you can use the software. That's, really cool. Thank you. I want to revert a little bit back where you said you were basically get money to invest for other people into the companies. And you also give the companies at strong advisors to
steer in one order in the other direction. And this is some responsibility and sometimes maybe some pressure, especially if it not turns out the way you expected it to be. How do you handle that responsibility and how do you work with that? What's your on that?
So the key is to always remember the founder is always in charge. Just like we talked about when you get an answer from a machine learning model or chat GPT or something like this, or you read something, doesn't make it true. It may not make it true for your situation. So as an advisor, what are they really asking? Given your unique experience in the world, so Kelsey has his own model, his own worldview. And some cases they're going to ask, based on that, what would you do in this scenario?
And the founder knows that it's their ultimate responsibility to take my input as just one stream of information. And ideally they're combining it with all the real things that they're seeing in reality. And ideally if I'm doing a good job, when I give input, and remember one big part of this is the reasoning. Kelsey, why are you saying we should do this instead of that? And you say, well, here's the reasoning. And then you give them, this is what people are currently doing.
Here's what they would like to do. And here's what your opportunity is. And if you can help them do that thing, then I think it results in success for you. And if you do it the right way, you can capture the sum of that success and turn it into monetary gain. So then there's this reputation component. You're not going to be great for every company all the time. You may be great for six months and then you've gone as far as you can take the company.
and they should go get new advisors to give them new ideas and different perspectives. So there's been cases where I make a suggestion like, I don't think you should build that. I think you should just use this open thing instead and just build on top of it. Or I might say, should open source this part, but it won't make you any money, but it will be a good way to fill the top of your funnel so you have a chance at converting people to actually use the paid product. And so then you ship these things.
And then I remember sometimes, well, we'll agree to do something. job is to also explain myself. And the team says, all right, I think I understand it now. And they do it and they execute. So all credit goes to them for the execution. And then we share the idea. So another thing I can bring to the table is given my reputation, given my following, if I actually believe in something, then I will say it in public. And then that will create demand.
Right? That's that external influence. The same thing that helped me become a distinguished engineer at Google is the same thing I can do to help bring good ideas, at least to people's attention. And then they have to decide if they want to follow that or not. So there's also a trust component with those people. If I'm just posting things that are just nonsense and they look like ads, then people will say, I can't trust Kelsey anymore. Like, let's not even click on any link that he shares. So I have to make sure that I'm involved in the work. I believe in the work and I can explain the work.
So if people show up and they start using it, it turns out that our collective idea was great. A lot of the founders were like, wow, Kelsey, we were not going in that direction. Your input that nudged us that way in our ability to execute, it turned out well this time. And it's so hard to do that, that if you do it four, five, six times, that makes its way to other venture capitalists.
It makes its way to other founders. They've seen this themselves. They call to other founders. Hey, is Kelsey helpful? How has he been helpful exactly? And they say, look, remember this thing we shipped? A lot of that had to do with Kelsey getting us to go in that direction, explain to the team why he made us understand why we did it. Then he helped us bring it to people's attention. And now people are using. He's definitely earned his equity and his pay.
And so that's the balance that I try to walk and I won't be great fit for everyone, but that is the outcome that I'm wishing for each and every time that I do these things. So that's it. That's how the accountability works. I'm never just like, Hey, do this and it should just work. And if not, you can just let me go. That's not what I'm trying to do here.
⁓ That's really powerful. And what I hear was that you also lending the people you experience on the one hand side, but on the other hand side, sharing your thought process. by thinking out loud and sharing your thought process, you make the founders think in the same way ⁓ of a certain period of time. They might not have your experience, but they tried to get the way you were thinking. And this is really powerful. Thank you for sharing.
And sometimes
you find yourself as a mentor. Sometimes it's not even about products and it's about the stress of building a company. Sometimes it's the stress of like shipping something that didn't work. And then the founder or the product team wants to sit down and just talk through it. Like Kelsey, you know, we made this mistake. Now what do we do? Have this ever happened to you in the past? And you say, yeah, you've made a mistake. Sometimes you could write a blog post about the mistake. Sometimes you can say, well, all right, we put the wrong thing out.
Do we even have the feedback loop to tell us what the right thing is? And do we have the ability to move in that direction? And if that's the case, then it's just part of the process. And that experience, that easing of the mind, in some ways you could be like the therapist for that small window of time. ⁓ Because even the people who've done this several times, they're still gonna come into a situation where they just need someone to talk to about things and just kind of be a sounding board.
and just be human for a moment before they get back to being founder and CEO.
And that's, I can believe that you cannot underestimate this part of the job that people just need someone who can talk to and just be themselves at one point, because I imagine I'm not a founder, but I can imagine that sometimes you're really alone and you're fighting against the whole market. And if you have somebody with experience at your side, so ⁓ thank you very much for sharing that. We spoke a long time and I really appreciate taking the time.
I have one more question for you before we wrap up is you retired, but you're not tired. That's what you said. And you basically, I heard another interview that you said you were content with your, with enough in the industry. But on the other hand side, I saw a mentality in the beginning where you said, I want to be the greatest in one thing. was looking always to who's the next one who can I compete with and.
This is something what, what's, it's a mindset that you can't take it from someone. So if you want to share what's your thing right now, where you want to be the best and what is on your mind right now where you want to be the best in.
So let's explain the best. I want to put in the work. I want to be competitive. I want to be able to earn being the best in that scenario because of the training and the preparation. But if you lose the game and that day you are not the best, I'm okay with that. That day I'm like, hey, I'm okay because I know I put the work in. I know I did the very best that I could do. So in many ways you're just kind of competing with yourself in that regard like, ey
I did a really, really good job. Today, I just wasn't the best, but I'm able to be. There are days where I do feel like I was the best. I was the best version of myself. I was the best on the playing field. And just having a taste of what that feels like makes me want to continue to grow, continue to learn, continue to kind of exercise, which then leads to you proving that it's paying off. then you have, so it's not really just like the best.
and getting the trophy is like being able to say, I put in the best effort. what does that come to? What is that for? If like, ⁓ I want it to
have bidets in my bathroom. I wanted those. And so that means you have to have an electrician, you need a plumber, you have to switch out the toilets, you have to run electricity into every bathroom so they can have a dedicated circuit. You need 20 amp circuit and not 15 because there's not a power because these things use 1200 watts of power. So you call the electrician and say, hey man, this is going to be thousands of dollars. It's a brand new house and you sure you want to do this because we got to go through the wall, through the attic.
If it's good, go to code. We got to go through the garage until the breaker box. It's a lot of work. It's going to be expensive. And. You got to do this for two bathrooms and these are like brand new toilets. You sure you want to remove them? And I thought about us like, you know, I really want this, but I get what they're saying. It's a lot of work and it's going to be expensive because it's their business. But I was like, what if I was able to do it myself? Is I just watching YouTube videos? And I'm like, what is the best?
plumber doing, how do you remove a toilet the best? And you watch a few videos and one guy says, slow down, clean everything, get a shot back, drain the water, cut the water off, remove the seal, and then the toilet rocks. You pick it up, put a towel to stop the smell. You don't have to use wax strings anymore. They have these rubber ones now. And what's the best toilets? Toto. You get these Toto toilets, the portion, the way it drains, the low profile, the way it heats the water, all of these things. I'm like learning.
What's the best? And if I can afford the best ones, then I give them. And so, well, then how do I install it? But I want to install it the best. I want to learn from the people who have been doing this a long time. So now watching the videos, like, what's the best way to put it in? You clean the area, get everything balanced. You got to cut the thing so it's not sticking over the top and make sure you twist a hole and all of these things. So I'm going to do it that way. So I go slow. What tools do the best use? So you're watching the video and you pause.
And now I'm just looking at the tool bag. Hmm. Is that a Kinepix? What's Kinepix? These are the best pliers in the world. I was like, mean, all the pliers look the same. Why is the Kinepix one's the best? And now I'm watching videos about pliers and the history in pliers because I want to know how they determine that this is the best. And so now that the toilets are in, everything looks great. I posted videos online. I showed my family members.
I gained the skills. have the days now on dedicated circuits. I passed inspection because you have to get a permit to run electrical wires all the to the breaker. I did everything right. And inspector came and said, you did a great job. Here's the sticker. Put on the breaker box. This is certified done correctly. I'm like, yeah, that's what I wanted to do. I wanted to do the best way possible. And since it's my house, I don't have to rush. I'm not trying to get to the next job. That means I can go so slow to make sure I'm doing every technique.
without worrying about spending too much time on one job site. And when it really came to mind, the new skill, I remember visiting my mom the following summer and I went to one of the bathrooms and it was rocking just a little, so this is unacceptable. We have to fix this right away. She was like, what are you going to do? Call the plumber? No, I ain't going to fix this. So I went to the store. We got a new toilet. I pulled the wax ring. I saw that the flange was broken. I replaced the flange because I watched enough videos.
that shows you what happens when you encounter something you have to troubleshoot. Everything's going to be a clean install. Fix the flange, put the wax ring on, put the new toilet, everything is solid. Now it's right. And that's the moment of like, ⁓ I've acquired the skills that in this moment, in that house, at that moment, I was the best plumber in that house. So that's what I mean by this. So now that I'm not necessarily doing work at a software company or things like this, I'm now able to try to be the best person.
the best person at doing things in real life. So that's what I'm doing. So it's not like I'm trying to go and compete at the plumber competition. I'm not trying to be the best plumber in the world, but I am trying to learn from the best plumbers so that I can have their skills and techniques so that I can do stuff for myself and my family.
that's really great. And it also says, just get things done, just get started and learn the basics and be responsible for your own knowledge. think this is some of the great key takeaways for me. We have so much information out there. YouTube is basically a treasure.
of all the information that we can all use and we don't have to wait for education. can just start ourselves and whatever we are curious on, can go on and tackle it basically. Kelsey, thank you very much for being at the podcast, for taking a lot of time and sharing your valuable wisdom with all the community.
Where can people find you if they want to reach out? What is your preferred social media or what's the way they can find or learn the best about you?
So I'm on Blue Sky, I'm KelseyHightower.com. You can actually use your domain there. So I use my own domain, so KelseyHightower.com on Blue Sky. And then you can find me on LinkedIn, where I actually post a little bit more than I used to sharing ideas and just learning from other people. So Blue Sky and LinkedIn is where you can find me these days.
Excellent. We'll share this in the show notes in the links. Are there any final words or wisdom you want to share before we close?
I hope the big takeaway here is train your own model. And one thing that humans get to do is we get to learn from experience. And LLMs are amazing and artificial intelligence is cool. But I still think real intelligence is great. And you actually get to go experience the world. So reinforcement learning, that means you got to go outside and experience new things and just be open to tuning your own model along the way. So hopefully.
people will take advantage of still continuing to invest in training their own model.
Wow, powerful words. Thank you so much. to all the others out thank you so to this episode. Share with me if you more international guests. I found it super exciting, I learned look forward to seeing you the next episode. Until then, thank you and bye.