Just briefly wanted to talk about how this built and how it works, of course the entire codebase is open-source so feel free to poke around yourself
So this package works by wrapping around the esbuild go api to add this extra functionality, so yes technically we built your whole code base to parse what dependencies you are using. This might sound incredibly stupid, but considering esbuild speed and flexibility, it is still significantly quicker than most solutions written in javascript
I realize this is not a great benchmark but considering most other tools don't have good monorepo support it was the best I can do. I compared depp (this tool) vs depcheck (a popular javascript tool with almost 3k stars) on a previous project https://github.com/wei/socialify
depp 0.42s user 0.36s system 135% cpu 0.576 total
depcheck 1.69s user 0.13s system 116% cpu 1.567 total
This tool was about 2.7x faster than depcheck while practically building the entire codebase
Using esbuild like this also allows me to build this tool, without writing an entire javascript/typescript parser
Setup
First we need to get the paths of all the package.json files and then all the source files,
I did this using something I built for an older project which is a modification of go standard library's [Glob](https://github.com/CryogenicPlanet/depp/blob/master/utils.go#L79-L87) This modification allow me to easily find all nested package.json and source files
Now we can read the package.json files and get all the packages in the repository, we can store these as a hash map or dictionary to compare against in the future.
Finally, we can pass the source files to esbuild as entry points and have it do its magic
Callbacks
First we use Resolve and Load callbacks from the esbuild api to check for packages as they are being imported.
There are some caveats here tho, we don't want to go down a rabbit hole of every import. Any import from node_module we don't care about, these are generally nested imports of modules importing other modules. So we can mark these files as external
Another optimization we can add is, any non .ts, .tsx, .js,.jsx file can be completely ignored. This is a bit painful in go because it does not support regex lookaheads (?)
Now with any actual import, we can check the kind of resolve it is, whether it is a import-statement , require-call or something else we don't care about. Finally we can check if the imported path contains any of our packages
Metafile
Additionally we use the esbuild meta file to do a second pass on any import we might have missed or that slip through or resolve callbacks. The esbuild meta file contains a bunch of information about inputs and their imports which we can use to see if a given package has been used or not
Duplicates
While the process of finding unused packages is quite involved, finding duplicates is quite easy on the other hand. It is just about correctly inputting the parsed package.jsons and checking for duplicates and storing their versions. There is a bit more work for check unused @types packages but it mostly similar to duplicates
Not going to go into it much here, that said it probably constitutes its own post of how to bundle go binaries with an npm package and how to release them
All that said this was a fairly fun project to make in the 8 or so hours it took, and it has already been fairly useful internally. Hope you find some use in the tool itself or maybe how I built it
If you want to try out the tool checkout
Top comments (0)