When we started working on Cloudify's new version, 3.0 which is a multi-module project based on Python and YAML that will have an entirely new UI, our dev team needed to interact with to many new technologies, which required entirely new skills. Before setting out on this project, we realized that with all this coming-to-life of many new modules in the product, the constant flow of new features and requirements, and the accumulating set of specs, we would need to come up with solutions to simplify the project management. So, we decided to align our development workflows so that we could better work across teams and environments. This workflow automation, essentially would enable us to occupy ourselves with the work at hand, rather than the ramp up of getting acquainted with new methodologies each time a new project rolls around, eventually letting us do what we do best: programming.
Since we found ourselves writing new projects for nearly every component in the new product, we had many configurations. Before aligning our workflows, we spent too much time just becoming familiar with new environments and configurations--sometimes spending more time on that than the actual task. This cost us valuable development time, so all these activities really needed to be standardized to help us remain in focus as a team. In this short article, I'm going to dive into the process, the workflows we installed, and the tools we use to get the job done, ultimately to make our whole dev teams as portable as possible.
Working the Workflow
Having chosen this stack, the first thing a developer does to create a new project is go to the command line and type
After a project is set up, a developer typically needs to do two things: fetch external dependencies and assemble it all together. To handle dependencies in our environment, we use the NodeJS Package Manager (NPM) to pull in third-party libraries for the backend, and Bower for client-side components. As Bower, NPM, and Grunt already have their own configuration files included in each project, this means that after we've decided on our components and defined them in these configuration files, we then simply call an NPM install (as simple as running
npm install), a Bower install (similarly easy), and then call a Grunt build and just like that, all our components are included into the project automagically. This enables us to start working immediately.
At our site, we hate to reinvent the wheel. So we use what other developers, like us, have written in the past to solve issues just like ours. For instance, if a developer needs some sort of a slider component to be used in a Web page, she can just look up the Bower repository with Bower search, find an appropriate slider (let's say, angular-slider), and then all she'd have to do is Bower install angular-slider and it's integrated into the project. Of course, some of the time, the developers who write those components are ourselves. We have our very own repository of reusable components we built for generic use. We introduce them into a project in the exact same manner they're all published into Bower!
Running the Client
While developing in the client, we use preview servers provided by Grunt's LiveReload and Watch plugins, in our case, so we really don't have to refresh the browser page at any point, which is annoying for developers. Once Grunt's development server is up and running, it will watch for any change in the project files and reload that specific resource in the Web app using websockets. This allows us to work continuously. Grunt rewires the app into the browser as we go along, and there's no clicking the refresh button. Ever. It's all very tidy.
Once we finish developing a given feature, we commit it to the source control. But before the check-in, it needs to be validated and tested. Again, we use Grunt, which provides some helpers out of the box. To verify our code, we go to the command line, type
grunt jshint, and it runs validation on the code. To run tests, just run
grunt test. We could do all of this with a single command,
grunt, which runs all these steps for us. Once the code passes locally, we're ready to commit to Github and have the continuous integration (CI) tools do their job. Needless to say, the CI tools also use Grunt to create the distribution (
dist is the Grunt task responsible for that).
There's a lot to be said about continuous integration that I won't dive into here, but I'll just share that we use TravisCI and QuickBuild to manage our tailored/nightly builds.
Just a Bit More
We also use profiles. A profile colors an environment with certain configuration flavor for a specific need. When developing, our environment has two flavors of configuration: one for production, and the other for development, (the latter is also used for testing). This separation provides us with convenient mocking abilities when testing or developing (Why use a full-blown backend server when all you need is a fixed response? Mock it up with Jasmine!), and it also protects us from incorporating any sensitive data into the production code and possibly having it checked-in into the repository. Such data items might include passwords, secret keys, API keys, and so forth. Those all reside in the development profile; the production profile will be built with the real credentials. Again, once developers form a habit of working with profiles, it becomes second nature. Just agree on a private place for personal configuration files, switch the dev profile on, and roll with it.
A Final Note
When we started using unified workflows and tools, we ourselves became more portable: we were able to hop from one team to another easily and work seamlessly across diverse projects. From developing a UI component, to working on a feature at the backend, shifting projects was easy. This means we can always concentrate on the important things, increasing productivity and speeding up delivery of projects significantly. Aligning workflows means less friction. Less friction means fewer worries. Fewer worries means more coding. And more coding means happy developers. Well, at least that's been the case for us.
Eliran Malka is a Web developer at GigaSpaces working on Cloudify. He's passionate about problem solving and always looking for new ways to do so.