Post Mortem Discussion
Check out photos of this project
What was the problem to be solved?
RadPad is an end-to-end rental marketplace focused around people. Renters use RadPad to find a place, sign their lease and pay rent. Landlords use RadPad to list places, sign leases electronically and accept rents.
Jonathan Eppers, Tyler Galpin and Tim Watson founded RadPad in 2012 after attempting to find an apartment in Los Angeles proved to be a terrible experience.
Development began in July 2012 in Venice, California, and was released to friends on Facebook in October.
In January 2013, RadPad had reached over 10,000 downloads and was invited to join the startup accelerator Amplify.LA.
It was around this time that RadPad realized it had a Scalability problem - both in terms of listings scraping and traffic through the app.
As a startup, RadPad needed an infrastructure that was highly available, redundant, Secure and would automatically scale without human intervention.
And it needed to be done on a minimal budget.
What was the proposed solution?
Gunner Technology proposed to move RadPad's infrastructure to Heroku a Platform-as-a-Solution that was relatively easy to use and could be configured as part of the development process in a DevOps mold.
What challenges arose during the project?
The biggest challenge was getting the memory footprint of the application down.
Gunner inherited a Ruby on Rails application running Mongrels that consumed way more memory than the, at the time, allowable 512 mb on Heroku.
The app leaked like a sieve, too.
To combat this in the short term, the team migrated to Unicorn application servers instead of Mongrel and used NewRelic to knock off some low-hanging fruit in terms of memory consumption.
The developers then wrote a script that would constantly monitor and kill Unicorn instances as they approached 100% memory usage.
Once the app was stable, the team went to work tracking down the source of leaks in the application code and fixing those as well.
What was the technical approach to the project?
Heroku allowed the team to script deployments and easily clone the Production environment for staging and demonstration purposes.
The app would also run background dynos that would automatically increase in number to parse rental listings from various sources and replicate the data to a Postgres Database.
If, for example, there were thousands of listings to process, the app could run 20 dynos and parse the listings in minutes and then scale back down to a single dyno.
This allowed Gunner to keep cost low as it was only billed for the hours the dynos were running.
The team did something similar on the front end to handle spikes in traffic which were occurring frequently as RadPad had begun to be featured in trade blogs and news media.
What was the project management approach to the project?
This was a trail-by-fire project as RadPad was an existing app with growing demand that needed fixing asap.
The team started with Agile approach, but ended up having to put out a lot of fires mid-sprint on the existing infrastructure while working on migrating to the new infrastructure.
Gunner had at least one on-call engineer 24/7 who handled the fires while the other engineer worked on the stories in each one-week sprint.
What platform was built for this project?
Before Gunner got its hands on the project, the app was running on a single instance.
After the migration, the application was separated into multiple environments each running a number of front end and background dynos that varied in number from one to 100 depending on the overall load of the application.
What did you learn from working on this project?
Gunner had used New Relic before but fell in love with it on this project as it was a godsend for chasing down the memory issues.
How did this project benefit the client?
After the migration, RadPad no longer had a Scalability issue and managed 99.999% uptime.
Additionally, while usage increased over 1,000%, cost only increased by 32%.
Why was Gunner selected for this project?
While at ESPN, Dary and Cody had migrated a large social network to Heroku when the platform was still young.
That gave the team the expertise necessary for tackling this project while under fire.
Who worked on this project?
What are similar projects Gunner has worked on?
Red Carpet Database, like Rad Pad, was a Ruby on Rails application built on the Heroku platform.
What tools, techniques and methodologies were used on this project?
Agile software development refers to a group of software development methodologies based on iterative development, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams
Ruby is a dynamic, interpreted, open source programming language with a focus on simplicity and productivity.
Rails is a web application development framework written in the Ruby programming language
Scrum is a framework for project management that emphasizes teamwork, accountability and iterative progress toward a well-defined goal.