If you tried to purchase parking passes for San Diego Comic-Con in the Ace Parking sale today, the odds are that you left empty handed. However, that outcome was definitely not for lack of trying, either on the part of attendees or on the part of Ace Parking.
This morning at 9AM PT, attendees and Ace Parking both seemed excited that for once, a San Diego Comic-Con sale was going to go off without a hitch. Ace’s Chief Information Officer Jon Gjerset and his team had been working for months to solve the problems that plagued last year’s sale, and had been thoroughly testing and vetting the new system. And unlike with other companies running SDCC sales, the Ace team had been holding an open dialogue with attendees about the sale — answering questions on Twitter, giving a walk-through of the new system, being honest about problems from 2013. So when the site went live, Ace seemed ready.
However, for all of the planning that the team at Ace Parking put into this year’s sale (and they put a lot of planning into it), something went wrong — attendees were greeted with pages that loaded very slowly, green spinning buttons, and multiple error messages (gotta catch ’em all?).
Ace made the decision to take the site offline, at least temporarily, while Ace worked to find out what was causing the problems.
Eventually, the decision was made to bring the site back up at 2:45PM PT, when the team thought they had increased the server capacity enough to handle the onslaught of San Diego Comic-Con attendees.
However, when the second sale went live, the same thing continued to happen — more errors, more time outs, and more disgruntled attendees.
Only a handful of attendees reported being able to complete all the steps necessary in either sale, within the five minute time window they were allotted before the tickets in their carts timed out (though during the second sale, this was bumped up to 30 minutes), and actually walk away with a parking pass.
So what went wrong? We now know, in a message that has since been removed from Ace Parking’s Facebook page, that the site was the victim of a DDoS attack. Essentially, this means that when the site was live, someone was sending millions of simultaneous requests to the server at once. The server then got overloaded in handling the requests, and it soon failed. The system had been tested for the onslaught of San Diego Comic-Con attendees, but not for this.
Ace is continuing to work through the issue now, and reported on Facebook that they expect to have the site back up and running by 8AM tomorrow morning. However, throughout the evening, some of the testing may open the site back up and cause it to go live again — and anyone who is lucky enough to purchase a badge during these times will have their purchase honored.