I built a simple JavaScript bundler and it turned out to be much easier than I expected. I'll share all I learned in this post.
When writing large applications, it is good practice to divide our JavaScript source code into separate js files, however adding these files to your html document using multiple script tags introduces new problems such as
pollution of the global namespace.
race conditions.
Module bundlers combine our source code from different files into one big file, helping us enjoy the benefits of abstractions while avoiding the downsides.
Module bundlers generally do this in two steps.
- Finding all the JavaScript source files, beginning from the entry file. This is known as dependency resolution and the map generated is called a dependency graph.
- Using the dependency graph to generate a bundle: a large string of JavaScript source code that can run in a browser. This could be written to a file and added to the html document using a script tag.
DEPENDENCY RESOLUTION
As previously mentioned, here we
- take an entry file,
- read and parse its content,
- Add it to an array of modules
- find all its dependencies (other files it imports),
- Read and parse contents of dependencies
- Add dependencies to array
- Find dependencies of dependencies and so on and so forth till we get to the last module
Here’s how we would do that (JavaScript code ahead)
Create a bundler.js file in your text editor and add the following code:
const bundler = (entry)=>{
const graph = createDependencyGraph(entry)
const bundle = createBundle(graph)
return bundle
}
The bundler function is the main entry of our bundler. It takes the path to a file (entry file) and returns a string (the bundle). Within it, it generates a dependency graph using the createDependencyGraph function.
const createDependencyGraph = (path)=>{
const entryModule = createModule(path)
/* other code */
}
The createDependencyGraph function takes the path to the entry file. It uses the createModule function generate a module representation o this file.
let ID = 0
const createModule = (filename)=>{
const content = fs.readFileSync(filename)
const ast = babylon.parse(content, {sourceType: “module”})
const {code} = babel.transformFromAst(ast, null, {
presets: ['env']
})
const dependencies = [ ]
const id = ID++
traverse(ast, {
ImportDeclaration: ({node})=>{
dependencies.push(node.source.value)
}
}
return {
id,
filename,
code,
dependencies
}
}
The createAsset function takes the path to a file and reads it’s content into a string. This string is then parsed into an abstract syntax tree. An abstract syntax tree is a tree representation of the content of a source code. It can be likened to the DOM tree of an html document. This makes it easier to run some functionality on the code such as searching through, etc.
We create an ast from the module using the babylon parser.
Next with the help of the babel core transpiler we convert convert the code content to a pre-es2015 syntax for cross browser compatibility.
Afterwards the ast is traversed using a special function from babel to find each import declaration of our source file(dependencies).
We then push these dependencies (which are strings text of relative file paths) into a dependency array.
Also we create an id to uniquely identify this module and
Finally we return an object representing this module. This module contains an id, the contents of our file in a string format, an array of dependencies and the absolute file path.
const createDependencyGraph = (path)=>{
const entryModule = createModule(path)
const graph = [ entryModule ]
for ( const module of graph) {
module.mapping = { }
module.dependencies.forEach((dep)=>{
let absolutePath = path.join(dirname, dep);
let child = graph.find(mod=> mod.filename == dep)
if(!child){
child = createModule(dep)
graph.push(child)
}
module.mapping[dep] = child.id
})
}
return graph
}
Back in our createDependencyGraph function, we can now begin the process of generating our graph. Our graph is an array of objects with each object representing each source file used in our application.
We initialize our graph with the entry module and then loop it. Although it contains only one item, we add items to the end of the array by accessing the dependencies array of the entry module (and other modules we will add).
The dependencies array contains relative file paths of all dependencies of a module. The array is looped over and for each relative file path, the absolute path is first resolved and used to create a new module. This child module is pushed to the end of the graph and the process starts all over again till all dependencies have been converted to modules.
Also each module is giving a mapping object which simply maps each dependency relative path to the id of the child module.
A check for if a module exists already is performed on each dependency to prevent duplication of modules and infinite circular dependencies.
Finally we return our graph which now contains all modules of our application.
BUNDLING
With the dependency graph done, generating a bundle will involve two steps
- Wrapping each module in a function. This creates the idea of each module having its own scope
- Wrapping the module in a runtime.
Wrapping each module
We have to convert our module objects to strings so we can be able to write them into the bundle.js file. We do this by initializing moduleString as an empty string. Next we loop through our graph appending each module into the module string as key value pairs, with the id of a module being the key and an array containing two items: first, the module content wrapped in function (to give it scope as stated earlier) and second an object containing the mapping of its dependencies.
const wrapModules = (graph)=>{
let modules = ‘’
graph.forEach(mod => {
modules += `${http://mod.id}: [
function (require, module, exports) {
${mod.code}
},
${JSON.stringify(mod.mapping)},
],`;
});
return modules
}
Also to note, the function wrapping each module takes a require, export and module objects as arguments. This is because these don’t exist in the browser but since they appear in our code we will create them and pass them into these modules.
Creating the runtime
This is code that will run immediately the bundle is loaded, it will provide our modules with the require, module and module.exports objects.
const bundle = (graph)=>{
let modules = wrapModules(graph)
const result = `
(function(modules) {
function require(id) {
const [fn, mapping] = modules[id];
function localRequire(name) {
return require(mapping[name]);
}
const module = { exports : {} };
fn(localRequire, module, module.exports);
return module.exports;
}
require(0);
})({${modules}})`;
return result;
}
We use an immediately invoked function expression that takes our module object as an argument. Inside it we define our require function that gets a module from our module object using its id.
It constructs a localRequire function specific to a particular module to map file path string to id. And a module object with an empty exports property
It runs our module code, passing the localrequire, module and exports object as arguments and then returns module.exports just like a node js module would.
Finally we call require on our entry module (index 0).
To test our bundler, in the working directory of our bundler.js file create an index.js file and two directories: a src and a public directory.
In the public directory create an index.html file, and add the following code in the body tag:
<!DOCTYPE html>
<html>
<head>
<title>Module bundler</title>
<meta name="viewport" content="width=device-width, initial-scale=1" />
</head>
<body>
<div id='root'></div>
<script src= ‘./bundler.js> <script>
</body>
</html
In the src directory create a name.js file and add the following code
const name = “David”
export default name
also create a hello.js file and add the following code
import name from ‘./name.js’
const hello = document.getElementById(“root”)
hello.innerHTML = “hello” + name
Lastly in the index.js file of the root directory import our bundler, bundle the files and write it to a bundle.js file in the public directory
const createBundle = require(“./bundler.js”)
const run = (output , input)=>{
let bundle = creatBundle(entry)
fs.writeFileSync(bundle, ‘utf-8’)
}
run(“./public/bundle.js”, “./src/hello.js”)
Open our index.html file in the browser to see the magic.
In this post we have illustrated how a simple module bundler works. This is a minimal bundler meant for understanding how these technologies work behind the hood.
please like if you found this insightful and comment any questions you may have.
Top comments (0)