The State of JS Build Tools 2015
My starting point in this process was the “Gruntfile.js” created for me by the useful scaffolding tool Yeoman. Out of the box it came configured to use Sass with SCSS, while I prefer Less. So, I looked into what it would take to switch. The Grunt configuration, for what I considered a fairly typical build, was hefty, weighing in at 450+ lines. Digging in further, I discovered a number of things I didn’t like. That led me to evaluate other options. Moving on from Grunt, I looked at the next most popular option, Gulp. Then, I investigated the less common Broccoli.js and Brunch. I briefly considered the obscure Mimosa and Jake, but didn’t see anything compelling enough on their websites to make me think they needed to be considered over and above the others. This posts details my findings.
Being a professional web developer, I also expect that a web focused build tool will have lots of support for the complex asset pipelines of modern web apps. This is an area where Visual Studio definitely falls down. This is also an interesting and unique requirement for build tools. Traditional tools like “make” rarely had to deal with situations like this. Compiling a C++ app is mostly about finding all the source code, compiling and linking. There was rarely much more to it than that. For web applications, there are many different kinds of files that must each pass through their own long pipeline of transformations and combinations. Additionally, the result isn’t a single executable, but a large assemblage of files. That makes the need for good file clean-up much more important. Finally, developer expectations have risen since the original days of make tools. Then we were happy to get an executable when we ran the make command. Now, we expect file watching with incremental rebuilds and live browser reloads.
- Bundling/Concatenation: combining of scripts and styles into multiple files
- Minification: scripts, styles and html
- Source Maps: for both scripts and styles
- CSS Preprocessor: Less, Sass, Stylus etc.
- Style Transforms: Autoprefixer, PostCSS etc.
- Cache Busting: renaming files with a hash to prevent incorrect caching
- Image Optimization
- Compiling Templates: Mustache or HandlebarsJS, Underscore, EJS, Jade etc.
- Copying Assets: html, fav icon etc.
- Watching for Changes
- Incremental Rebuild
- Clean Build: deleting all files at start or preferably cleaning up files as needed
- Injecting References: generating script and style tags that reference the bundled files
- Build Configurations: separate Dev, Test and Prod configuration, for example to not minify html in dev build
- Serve: running a development web server
- Running Unit Tests
- Dependencies: handle dependencies on npm and Bower packages, Browserfy etc.
So what did I find? There were some common failings in all the build tools. One was the presumption that npm and Bower package restore were outside the scope of build. Unlike with an IDE such as Visual Studio that automatically runs Nuget package restore as needed, none of the JS build tools attempts to. This was often impossible to work around due to the fact that the build tool required it be one of the npm dependencies of your project and would not run until it was restored. Several of the tools did have plug-ins supporting Bower package restore. However, almost all examples I saw assumed one would first run
npm install then
bower install before building. They didn’t even setup
bower install as an action triggered by
Gulp is the hot new tool. It is rapidly gaining in popularity and there are some justifiable reasons for this. It recognizes and attempts to handle the primary unique requirement of web build tools, namely the long pipelines. In Gulp all plug-ins work on streams (well as of today there are still a number that don’t, which is very annoying). This streaming model allows Gulp tasks to be chained together in streams without worrying about or even producing intermediate temporary files. Gulp touts its high performance due to both streams and parallel task running. Yet, many users have complained that the parallel task execution is confusing and there are plans to extend the configuration to give more control over this. Personally, I found the parallel task execution very intuitive and feel the proposed cure is worse than the problem.
While streams are a compelling base for a build tool, I ultimately found Gulp lacking. Since the core configuration options are minimal, the attitude seems to be that little documentation is needed, but I found the documentation inadequate. Like Grunt, it was necessary to rely on the usually poor documentation of the plug-ins. This was somewhat mitigated by the greater consistency in plug-in usage versus Grunt. The Gulp authors believe in opinionated software and have some strong opinions when it comes to Gulp plug-ins. Normally, I appreciate opinionated software, but in this case I have to disagree with the authors’ opinions. They believe that a tool that already supports streaming should not be wrapped in a plug-in, but used directly. This runs counter to making builds as easy as possible to setup because it requires one to learn and adapt the tools’ unique APIs to the Gulp system. For a peek into the mess this causes, check out the GitHub issue over the blacklisting of the gulp-browserify plug-in. In it, dozens of users attempt to figure out the correct way of using Browserify. I don’t think any quite hit upon the difficult to find official recipe. But note, even that fails to support the important case of streaming files into Browserify. The whole issue is complicated enough that it takes an excellent but lengthy blog post to fully explain. Furthermore, the streaming model becomes confusing for things that don’t fit into it, such as cleaning. Source maps in particular are odd. I attempted to combine external source maps with cache busting with a manifest file and cleaning up old revisions and while the final config was quite short, it took me several hours and additional plug-ins to get working. Watching still requires manual configuration and incremental building is quite confusing to setup. Figuring out how to do anything is made worse by the plethora of plug-ins for doing the same task. Finally, while I don’t understand the issue myself, I found many discussions of needing to do extra configuration to get reasonable error messages out of Gulp.
Despite all these excellent attributes, Broccoli is still very immature and in beta. It is lacking in the area of documentation. The read me boldly states windows support is spotty. The fact that there is little to no source map support was a big issue for me. It seemed to me that Broccoli wasn’t quite ready for prime time, but I’d really like to see it mature and supplant Grunt and Gulp.
So which tool would I recommend? Personally, I am still working with Brunch to see if it will work for my project. It would be at the top of my list right now (if you like the convention over configuration approach of Brunch, you could also check out Mimosa). However, if you don’t mind the large awkward configurations then you might want to select Gulp or even Grunt just for the size of the community around them and the selection of plug-ins (though I haven’t had a problem finding the Brunch plug-ins I need). I’m really interested in watching the development of Broccoli and hope it becomes a viable option in the future. The ultimate answer is likely to be a tool that doesn’t even exist today. Once package dependencies are ironed out correctly, I think we are likely to find all the existing tools don’t quite make sense anymore. Of course, there is no telling how long it will take to get to such a state.