If you’ve used the site in the past couple of weeks, you may have noticed that PadMapper recently got nominated for a Webby Award for best Real Estate site! We were pretty excited about this since our team is tiny compared to most real estate sites (it’s me and a summer intern), and the Webbies are one of the more prestigious awards that a website can win. At first we didn’t think we could actually win the People’s Voice popularity contest against a big company like Zillow, but at 13.5 days into a 14-day voting period, we’d gotten 37% of the votes in the Real Estate category, 5 percentage points ahead of the second place contender Zillow, and 12 percentage points ahead of the third place contender Apartment List. We’d gotten an outpouring of love from users and things were all around pretty great (thanks everyone).
Then at 12PM on Thursday, with 12 hours of voting to go, the Webby Awards decided close the doors and hide the percentages. The stated reason was to keep the anticipation high until they officially announced the winners 5 days later.
We were bummed that we wouldn’t get to see the end of the race, but then around 4PM, we figured out how to see the percentages again! Whether intentional or not, the voting site was set up such that if we logged out and logged back in again in the Chrome browser using Facebook login, we could see the voting breakdowns again. Logging in and out to look at voting results is a pain, though, so when we saw the percentages were basically the same as 4 hours before, we went back to work on more productive projects. Since the voting had basically stabilized to a ~1-2% fluctuation per day due to the large number of votes from the long voting period and because there were only 8 hours left for voting, we were feeling good about PadMapper’s chances of winning the popular vote. We were in such a good mood that we made ourselves lamb chops for dinner. With ice cream!
Then at 10:40PM, we got curious and we peeked again at the voting results. We were still in first place, but Apartment List had pulled up dramatically. We smelled something fishy. We’re not usually the suspicious type, but we’d had a bad experience dealing with Apartment List before, so we decided to track the voting closely.
Here is where things get crazy. Watch closely.
Below are the voting breakdowns for the Webby Awards Real Estate Category at various points on the last day of voting for the top 3 contenders, all times in PDT:
|Previous day||PadMapper (37%), Zillow (32%), Apartment List (~25%)|
|Just before the numbers disappeared at 12PM||PadMapper (37%), Zillow (32%), Apartment List (25%)|
|When we found a way to see the numbers at 4PM||PadMapper (37%), Zillow (32%), Apartment List (25%)|
|10:40PM||PadMapper (35%), Zillow (30%), Apartment List (30%)|
|10:55PM||PadMapper (34%), Zillow (29%), Apartment List (31%)|
|11:10PM||PadMapper (34%), Zillow (28%), Apartment List (32%)|
|11:32PM||PadMapper (32%), Zillow (27%), Apartment List (35%)|
|11:32PM – 12:00AM||No fluctuation in votes|
Needless to say, we were a little emotional for the 50 minutes between 10:40 and 11:30. First crossing our fingers because the race was so tight, then calling “Foul!” on the couch in our living room. By the end, it was like getting to the end of the first Game of Thrones book and, all of a sudden, [redacted] lost his [redacted]! Then as now, we were shocked and unhappy.
Math says that new votes for Apartment List alone in those 50 minutes accounted for at least 7% of all votes in the contest over the 2-week contest. That’s 7% of the votes in 0.2% of the contest time. Aside from this sheer volume of votes, 5 other things seemed fishy:
A) The movement observed happened after 10:40PM on the West Coast. That means it was 1:40AM on the East Coast, on a weeknight.
B) Throughout this flurry of activity, Apartment List was the only site to gain in percentage, and PadMapper and Zillow remained 5% apart at all times.
C) The pace of Apartment List’s gains was linear over the course of the 50 minutes.
D) The movement suddenly stopped at 11:30PM with Apartment List at a 3% lead, 30 minutes before voting officially closed.
E) A quick check on Apartment List’s Twitter and Facebook pages showed no activity in the last 6 hours. No promoting, no calls for votes, nothing.
The total rise from their previous days’ numbers represent a roughly 50% increase in the total number of votes over the number they received in the previous two weeks, despite having next to no social media buzz on the last day.
So here are some guesses at how this could have happened:
1) Lots of avid Apartment List fans suddenly discovered the contest in its final hour and voted at 2AM on the East Coast and 11PM on the West Coast on a Thursday night, at a constant linear pace. Then suddenly no more votes for the last half hour of the contest.
2) Lots of people halfway across the world where it was daytime suddenly discovered the contest in its final hour and voted, at a constant pace, for a website that only serves the US.
3) Someone created a bot that pre-registered for a bunch of fake Twitter accounts and used those to vote for Apartment List.
4) Someone did the human equivalent and used Mechanical Turk or another form of cheap labor to buy votes for an hour or two using fake accounts.
The first two don’t seem very likely.
Only the people at the Webby Awards have the voting logs, but based on the data points we showed above, our best estimate is that the voting distribution over time probably looked something like this (with minor fluctuations):
Sadly, the voting is over, and the results may be set the way they ended up above. We ate a too-early celebratory meal, then we prayed that we could still win, then it seems that a bot or something else came and bulldozed over us with a truckload of fake votes. It’s been an interesting exercise to study the voting pattern of this situation. Whereas most social network-driven activity probably builds in a Poisson-shaped distribution, the votes for Apartment List were added linearly, with a sudden halt at 11:30PM. We hope that the administrators of the Webby Awards will disqualify the almost certainly fake votes and we’ll regain first place, but we haven’t received a response from the admins about this yet.
What makes this whole voting mess more troubling to people who don’t give a fig about our winning an award is that decision-making is increasingly based on internet reviews or votes. The aggregate opinion of the masses is often viewed as more accurate or trustworthy than that of a single reviewer in a magazine, since we expect that we may have different preferences from a single reviewer, but if everyone likes something, we probably will too. This event shows that an aggregate opinion online may actually be easier to fake than a single detailed review. This other story posted less than a week ago about a man being duped by fake reviews for a moving company on Yelp demonstrates that this isn’t always so harmless.
We are left to ponder on the broader question of the pitfalls of Internet voting and the necessity of transparency in things involving voting. If the Webby Awards had not decided to hide the voting distribution in the last hours of voting, would the perpetrator have risked exposure and still pulled this off? Or would s/he have simply done it in a more subtle way?
As a final note, if you decide to write about this, please don’t link to Apartment List as a result of this post. This event shouldn’t lead to an SEO windfall.
UPDATE: Thanks to those of you who pointed me to a blog post from a few days ago by Apartment List – the explanation they provide is that they asked people not to confirm their email addresses after voting, and instead forward the confirmations to them so they could confirm them near the end. That would explain this strange voting pattern, but I’m skeptical, since their public voting campaigns didn’t mention anything about this convoluted step, and I think the number of people they would have had to get to do this privately would have needed to be very large. The Webby admins doubtless have a record of when the votes were originally cast, though, and if it is legitimate, it’s a pretty neat trick to avoid counter voting drives.
UPDATE #2: PadMapper won the Webby for best Real Estate site! Still no response from them about the People’s Voice side of things on whether they checked out the votes, though.
UPDATE #3: Thanks to commenter Mike, who dug up a contest they ran for votes in exchange for a chance to win an iPad. Not as shady as running a bot or hiring people directly, but still basically buying votes. It doesn’t seem to have been extremely successful, though (39 comments from people who voted), and their directions don’t mention not clicking the confirmation email, so I don’t think this explains the vote spike in the last two hours. It seems to have been taken down, which is a bit suspicious. Here’s the cached copy: Cached contest page