We have an interesting challenge here at Buyometric. The number of emails we send out on a daily basis is increasing rapidly, but the timeframe we have to deliver them is remaining constant. Imagine increasing the quantity of breakfast cereal that you pour into your bowl by 5% each morning. One of these days, you’re going to end up in a mess.
Our goal is to deliver an individually tailored email to every one of our users by 10am at the very latest. The email contains some deals that are only available for one day, so the earlier you receive the deals the longer you have to ponder a purchase before time runs out – or the deal sells out. Our classification and email creation processes is completed at around 8am, which gives us a 2 hour window to deliver the emails.
That’s no problem when your sending 10 emails, or 100, or 1,000 or 10,000. But what happens when you are sending hundreds of thousands of emails?
We’ve been finding out. Recently our 10am deadline has been slipping. Some days our mail server would crash altogether, and on others it would grind to a halt after chewing through all the available memory. Sometimes emails can’t be delivered and our server needs to keep trying. On these days, the extra load of repeated delivery attempts would cause something of a nuclear reaction and kill our sending processes dead.
The problem peaked about two weeks ago, where the last emails to be delivered were being received at around 2pm. This was totally unacceptable for us – something had to be done.
The fine folks at Catn, who provide and maintain our infrastructure, worked with us to come up with a solution. First we tried switching to a memory based mail server, but that didn’t cut it, so we ended up moving the server to a machine with a faster disc and much higher memory allocation. We’ve also started compressing our emails prior to dispatch to reduce the overall file size and eliminate any unnecessary overhead.
The result are looking good. We’re back within our target window, with email currently being fully dispatched shortly after 9am. That enables us to roughly double in size before we need to look at a further upgrade. At this rate we’d better book that in right around Christmas time…
TrackBack URL :