Our Development Process
Increasing Reliability with Modern Development Practices
At Duo we've been developing websites and online applications since 2002. In that time the size and complexity of the work we undertake has grown massively. In this article I'd like to outline some of the practices, tools and methods we use to keep producing reliable and effective solutions.
Local Development Servers
There was a time when editing code on live servers was common practice. To get an update live, you would fire up an ftp client, download a file, edit it and throw it back online. Make a mistake? Whoops your site would be dead and until you got it working again your customers would think you'd gone out of business. Worse they'd be in the middle of an e-commerce transaction and the basket would be lost and they'd paid you for a product they'd never receive!
For everything but the most basic static site, editing in this way just isn't feasible.
For many years now we have run local development servers. These are virtual machines which run on our desktop machines which mimic the live environment very closely. We utilise synchronisation scripts to pull down copies of all the website content and assets to create an exact facsimile of the live website. These 'dev' servers allow us to update and refine websites without modifying the live site until the work is complete and ready to go.
In addition to the development team being able to check the updates, we also make our development environments available to our clients via a different domain name, so they can check over our changes before pushing anything live.
Docker and Containerisation
Getting a reproducible environment where your website works reliably can be challenging. Years ago, it would be straight forward to upload some HTML files to any old hosting. With the myriad of features now contained within websites for e-commerce, auto image resizing, email sending, and third party api integrations the server your site runs from has to be configured to run an often vast array of code with a whole range of specific dependencies.
To streamline this process at Duo we utilise Docker Containers. These run both within our local development servers and on our cluster of cloud servers.
Utilising containers allows us to script and configure very specific environments on a per project basis. As these containers are portable and reproducible on any hardware which can run Docker, we have lots of flexibility on the hardware and with which providers we choose to run specific websites.
Version Control
Creating a modern websites requires a team of designers and developers to work on a codebase which often stretches across many files, and often thousands of lines of code. Keeping all these changes in sync, knowing who did what when, tracking down and fixing mistakes would be impossible without a version control system.
At Duo we use git the modern standard for version control. Every line of code change is logged and updates can be reviewed both by team members and in many cases by our clients via a secure web application which forms part of our development process.
We utilise git to develop our code, move code from one development machine to another and also to manage deployment of code updates to live servers.
Bug Tracking and Feature Additions
We utilise an issue tracking system which allows us to tag both bugs and tag our code update in git with the fixes to these bugs. Many of our clients have logins to our issue tracking system, allowing them to not only report bugs but see their progress to resolution, even clicking through to see the work done, seeing code both before and after the bug fix.
This same system works well for feature additions. In combination with our development servers it is possible for clients to see progress on features being added and give feedback prior to going live. The issue tracker acts as a record of feedback given at each stage, allowing us to use it as a reference point when making decisions on implementation details.
Automated Integration Tests
To prevent bugs in the way web apps and websites work it is important to be able to test the software as often possible in as many environments as possible. Ideally you would test your software after every change, but to do this manually would be expensive and time consuming.
At Duo, any complex application is accompanied with a suite of tests which run though the functionality of each piece of code and test it does what it is intended to do. If a bug is reported we aim to recreate the bug with a test, once the test is 'failing' we put a fix in place to make the test pass, both ensuring the fix works and preventing the bug from reoccurring.
These tests are commonly run on every file save while the developer is working, giving them feedback as they work so they know if they break any existing functionality as early as possible.
Continuous Integration and Deployment
Whenever code changes are pushed to git, a separate continuous integration server runs the tests against the code. If the tests pass it is possible to then deploy it across our servers to update all sites running the code. If it fails a notification is sent to the development team and the developer is able to fix the issue and re-push. At any time it is possible for anyone in the development team to see the current state of the code, when it last failed and what code was changed to make it fail. This gives us a great deal of insight into the development of our software and to push new features and fixes with confidence and regularity. Our content management system has around 200 versions deployed a year.
And there's more
The above are just some of the tools we utilise to make our websites and online apps work as reliably as possible, providing a professional service to our clients. We also utilise:
Automated Regular Off Server Backups
Realtime automated error collection from our sites
Automated tests across multiple browsers and devices
R&D into anything and everything which can improve our services on the modern web
and many more...