Hi! My name's Kat, and I've been an FOSS dev for over a decade.
I had the opportunity of being, and eventually tech-leading, the NPM CLI team which I was a part of from 2015 until a couple of months ago. I'm the one who wrote things like npx, npm ci, and I helped design and add package-lock.json and much of its behavior to the CLI, and I was the main author of the ~40x+ speedup between npm@4 and npm@5 and later that made the CLI catch up (and often surpass) Yarn and PNPM in performance.
I also do a lot of Rust these days, and I'm currently on the NuGet client team at Microsoft, the package manager for the .NET ecosystem. I'm also on the core dev team for the Entropic client, ds, trying to build a new, distributed package manager for the JavaScript community!
AMA!
Top comments (50)
I use your software everyday! (and never knew who was behind it :) ) - so thanks!
I suppose I could ask: what are your feelings on yarn vs npm? I'm ashamed to say I've never bothered to dive deep into the differences between the two, so I just end up using whichever is more convenient at the time... is that something I should look more into?
Thanks for the ama!
They're both fine. I'm obviously biased towards the thing I worked on myself, even if my relationship with my former employer soured a bit.
Yarn is fine software and it works for a lot of people, and if you find that the way they do things works better for you, by all means, use it! That's also the policy we had on Yarn since its inception.
Once npm@5 came out, I think most differences disappeared, with the major exception of Workspaces, which I'm sure will eventually come to NPM as well. We had them on the roadmap, until they fired my team for trying to labor-organize, and then me and Rebecca (the OTHER former architect of the CLI and my long-time colleague) decided to leave. I have no idea about current plans anymore. :)
Yikes! I hadn't heard that story before; thanks for the link.
And thanks for the answer - makes me feel better about not knowing the difference, since there appears to not be much of a difference since npm@5 :)
Thank you for all the effort you have put into the npm and the ecosystem. I consider
npmone of the most important parts of today's web development ecosystem 😄Putting any work drama aside, and any budget/time constraints, if you could re-write part or all of
npmwhat would you change and why?The tree builder, because it would add Workspaces support and simplify some fairly complex code. I hope
I am usually doing backend programming on the JVM with gradle.org and when I do front-end development, the npm CLI seems to need a lot more of baby sitting. For example:
$ ./gradlew :myTaskwill auto-bootstrap it for me by downloading the right version ofgradle-xx.jar. Node and npm on the other hand will just fail on me with a weird error message$ ./gradlew :testwill detect it and install them.$ npm testwill fail on me because I didn't run$ npm installfirst.$ ./gradlew :myTaskwill complete almost immediatly with a messageTask UP-To-DATE, while npm will gladly waste my time by actually doing everything twice.Ever considered implementing at least some of those features?
I don't really work on NPM anymore, but I do work on a client for Entropic.
I think the first and third features are out of scope for how I generally think of package managers, but I might need to think more on that.
The second one, though, is something that
ds(the Entropic client) will actually take care of for you. We'll see how it goes! Thanks for sharing!For the first feature, I think the package manager should at least document which version of node is required and fail with an explicit error message if it's not present.
The third feature was already present in Makefiles - although obviously in a crude implementation compared to what Gradle is doing.
docs.gradle.org/current/userguide/...
Update: maybe my questions are more about Webpack than about npm/entropic
Thanks for that AMA! Use your software every day :)
Care to talk about the backgrounds that made you move to leave NPM?
I mean, I read some people were let go in a bad way, but I don't understand how a diverse powerhouse where all are biggest friends could lead to that outcome.
Basically, the CEO and COO are horrible human beings, and I strongly believe they fired my colleagues for trying to organize. There's a lot that happened internally that I can't talk about that completely eroded my faith in the company's mission to the community, as it was when I arrived.
I decided I didn't have to keep taking a pay cut for the sake of a culture that was no longer something I could stand behind, and was not what I had been promised.
Hey Kat,
I was wondering how you managed to become Tech Lead for NPM and how you got your job at Msft. Please don't get me wrong, I'm just asking you this because I've recently switched jobs but I'm constantly being unmotivated and I can't figure out why, also, I find that the impostor syndrome is always making me feel that I won't be as good as developers like yourself.
I got hired back in July 2015 the same way anyone would: applying and getting through the application process. From there, it was mostly a process of attrition. I was the second-oldest member of the team, and the oldest (and former architect), Rebecca, moved on to full-on management and product management, leaving an open spot for me once we started hiring more folks into our team. A big reason for giving me that tech lead role was that by then, I'd done major refactors, rewrites, and rearchitecting that reached into all parts of the CLI, so I was intimately familiar with the shape and function of the code. That took about 3 or 4 years to happen, though.
I think a lot of things are simply a matter of time.
As far as joining MSFT, I was introduced to the team through professional connections (both a colleague in TC39, and through twitter). The messages bounced around until I got in touch with someone on the NuGet team, and they got me started with the application process from there!
It's funny: I almost turned Microsoft down because I really really didn't want to do the interview gauntlet. I was deeply burnt out, depressed, and just not braining right after the chaos at NPM, and I didn't think I could perform well enough to get hired at a major company. I even cancelled my initial on-site interview, but one of the contacts messaged me a while later after finding out and talked me into giving it a shot anyway. I still wasn't sure I wanted to do all this, but I went for it, and after a couple of months, I ended up getting an offer!
MSFT wasn't the only place I applied to. I had a spreadsheet of about 40 different places, most of which I at least had first-contact with, and that resulted in about 3 final offers (most of them dropped off because of technicalities, constraints, or me simply deciding not to continue). Of all three, MSFT's ended up being the best combination of the requirements I was looking for in a new job (plus the compensation package), so that's what I picked.
As far as "being as good as developers like myself": I think it's important to note that I consider myself pretty mediocre for someone who's been doing this for 10 years, and any perception of ability is simply a function of experience. The rest is simply luck, and access to opportunity -- I moved to the SF Bay Area about 6 years ago, and the fact is the opportunities here were way more numerous (and lucrative) than anything I experienced living in several other places after I moved out of home.
I kinda take issue with the cult of personality that forms around people in visible position and so-called thoughtleaders on the internet. We're all actually fairly average, normal people who happened upon fame by sheer luck. Please don't evaluate your ability and competence based on where visible folks landed. So much of it has more to do with privilege and luck as opposed to what you or I can actually do, as developers. Keep at it!
Such a great and motivating answer for all of us. Thank you!
Was the v5 release of the npm difficult for you personally? Did the GitHub issues get to you ever? Was there a technical part of the project that was the biggest issue? Was improving the tool to such a great degree hard to do for other reasons (step on anyone's toes?)? #npm5 #theBehemoth #suchGood #wow #muchLove
Sorry if this is a bit long, but I guess it's story time?
npm@5was simultaneously one of the most satisfying and one of the most destructive things I've done in my career...So, flash back to late 2016, and Yarn comes out, putting an IMMENSE amount of pressure on my team. The main reason the CLI had seemed to stagnate was because our team had decided that stability and reliability and making sure what was there worked well were the primary goals for our project, so we'd spent the better part of that entire year fixing bugs, improving Windows support, and just generally making sure that the CLI was a nice, reliable tool.
Yarn came in and kind of flipped our table (and included a lot of FUD about it that I felt at the time, and still do, was very unfair), and our priorities had suddenly shifted: we were bleeding users, and we didn't have a plan to improve the CLI as much as it needed, on the timeline it needed to improve. The only clue we had that might do it was the so-called "cache rewrite" that had been in our backlog for literally years, but we didn't even have data on how much it would help. It was just clear that whatever it is we were doing was not actually what the community wanted from us, and we needed a new plan.
And then, my boss gets fired. Abruptly.
And my new boss tells me to go whole-hog on the cache rewrite.
Now, you have to keep in mind that working on the CLI was my day job, and I worked at a startup, so the emotional subjective experience of seeing Yarn take market share felt like these big Facebook bullies were putting my job and livelihood at risk. I was kinda panicking. I was genuinely scared and was ready to do whatever I needed to do to keep my job. My manager getting summarily executed (figuratively, I promise) heightened that sense of survival, I thought "I'm next", and such.
What followed was one of the most productive periods I've had in my entire career. I was working 12-14 hour days on npm@5, mostly on my own (while Rebecca took care of the stability of
npm@4and generally supported me in my work). I didn't really tell people this was what I was doing. I just did it because I needed to seem useful and that seemed like the most important thing. I needed to try to save the project that paid my bills.Out of that, frankly, herculean effort came libraries like cacache, and then ssri, and then make-fetch-happen and pacote for the network bits, which I worked on in complete isolation from the main CLI project, benchmarking and testing them thoroughly along the way to make sure they were the fastest and most stable things I could write. I essentially rewrote the entire networking and caching layer of the CLI, from scratch, mostly on my own, in about 6 months.
We weren't expecting to release as early as we did, but the Node Core project threw us a curve ball towards the end, saying that we needed to have a semver-major version of NPM ready by early May in order to get it into the new Node release, or we'd be SOL, and we really couldn't risk missing that deadline.
So, I integrated everything in a couple of weeks, and we threw together
package-lock.jsonin only a day or two, in a super-rush. We really didn't have time to test out the new semantics, the new format (which was largely based onnpm-shrinkwrap.json, or most of the integration (remember, the ENTIRE networking layer had been replaced, down to the low-level http client).So yeah, the
npm@5release was bumpy, but I was mostly numb by then from all the work I'd been doing the past several months, and I was relieved to get it out the door. I'm pretty proud of it, honestly! It turned out pretty damn good considering the constraints!You did so great! I loved
npm@5and was so impressed by how much work and love you poured into the project, and your story here just confirms a lot of my suspicion as a casual user of npm.Keep doing awesome stuff; and take care of yourself and do cool things at a sustainable pace.
You are an inspiration <3 👍
You can barely see the guillotine scars anymore!
Also big <3 for all the work you put into npm@5. I really wish you hadn't had to do it under the gun. I can't help but feel that casts a pall over what is really and truly an impressive achievement by pretty much any standards, and regardless of the material conditions of its creation.
No questions, just a huge thank you for what you have done! 🤘
I've been doing javascript for quite some time now and planning to learn Rust, so are you liking the Rust so far? also I would like to know what projects you working on in rust? also since JS abstracts a lot of internal things, does that make learning rust difficult?
I love Rust! I've never done systems programming, and I've just been really enjoying doing new things in it. The community is incredibly friendly, and there's a lot of People Like Me (queer, non-man, etc) actively participating and having a voice in the project, which makes me feel less alone!
Right now, I'm working on a new client for Entropic in Rust, and I've ported cacache, ssri, and srisum over to Rust, just to learn it, and also as backing code for the new Entropic client.
I think the fact that Rust is a systems language adds some necessary complexity, and having to deal with such an intricate type system, and the new borrow checker, means that there's plenty to learn for a plain old JS dev. Once you get over the initial bump, though, Rust gets super fun and interesting to work on! I think Rust is a surprisingly high-level language, considering its systems lang status, and I think it's a great way for JS devs to get into systems dev.
You are the real deal!
Thank you!
I hope you're having fun! That's what matters, in the end, for learning a new thing, I think.
Not yet, but I'm really looking forward to finding something I can wasm!