Expert interview: How to scale Django

Eventbrite's John Shuping and Simon Willison reveal all about scaling Django, the Python Web framework developers love

1 2 3 4 5 Page 3
Page 3 of 5

Willison: This is another difference between Lanyrd and Eventbrite. At Lanyrd we can throw the site into read-only mode for an hour and it doesn't cause problems, whereas Eventbrite sells tickets. An hour of not being able to sell tickets to people is not good for us and not good for the event organizers that rely on our platform.

Shuping: One random, supertechnical anecdote you guys reminded me of is that I think Django was historically more oriented toward PostgreSQL. We learned we needed to know how the replication is done: Is it row-based replication or statement-based replication? PostgreSQL operates by default in a row-based mode, and with our investigating those slave-lag issues, we found that MySQL defaults to statement-based replication when in fact Django prefers row-based. By switching MySQL to row-based replication, we minimized the lag issues.

Freeman: What else helps you scale?

Willison: I'd say one of the most crucial techniques, and the technique that certainly gives you the most payback for time invested, is to have a good feature-flag system. We built our own at Lanyrd, and Eventbrite uses an open source feature flag system (Gargoyle). This gives you an enormous amount of power and flexibility in managing a complex site.

Feature flags essentially are a way of saying features can be turned on or off across the whole site, or they can be turned on for specific users. You can say our internal users can access this feature at the moment while we're testing it, or our outer testing group, or turn a feature on for these couple of large-scale events. We use both for Lanyrd and Evenbrite, and we use these everyday. Almost every piece of code that we write ends up hidden behind a feature flag at one point or another.

It's fantastic for scaling your development process as well. It means that rather than your developers working in month-long feature branches and having a complete nightmare when they try and merge that into master, feature branches tend to exist up until the point at which a feature flag is in place. Then they can be merged back into the main code base. Your code is constantly being exercised, and it's visible to all the other engineers in the company.

It also means that launches are much less stressful. With a big launch of a new feature, you have to do a big deploy to turn the feature on and you have all the normal worries around making a major code deploy. With feature flags, you deploy your code the night or the week before. Then on launch day, all you have to do is flick the feature on, and it becomes available to your audience. It also means that rolling back is much easier because you can turn the feature off; you don't have to do a major rollback procedure if something goes wrong.

Shuping: We've added something on top of that called the "experiments framework." You can enable a feature flag for some percentage of users, so we've instrumented this in a way that we can conduct A/B tests behind feature flags. If we have a new feature and want to test it against an old feature, we enable this flag in such a way that it does an A/B test for us and allows us to go and see the result later.

Freeman: You can then adjust the feature or roll it out to more users, based on the results.

Willison: From talking to people who run large-scale websites, this technique is increasingly common. If someone's not running feature flags on their project, it's one of the first things I suggest they do. It makes such a huge difference to your productivity, your teamwork, and your level of confidence that you're able to control the code you're rolling out.

1 2 3 4 5 Page 3
Page 3 of 5