Fighting spam is really an arms race. As technology evolves, new threats emerge and keeping up is an absolute necessity. The Site Integrity team at Facebook works to protect people from spam, scams, account compromises, and other forms of abuse by building real-time classification systems that process millions of interactions per second and investigate and respond to new threats.
At the beginning of 2011, I joined a new sub-team on the very front line of the war against spam—Recon & Response (SI-RAR). This team is constantly fighting against the biggest and most pressing threats out there, and when things are quiet, we build more defenses. Here's a look at some of the attacks we worked on last year and the new systems we built to combat emerging threats.
When I first joined SI-RAR, I immediately started battling a fake account attack. Typically, spammers pose as attractive women to have their friend requests accepted and then use this friendship to send spam. Spammers will go to great lengths to create accounts that look as real as possible, generating fake backgrounds, fake photos, and even fake conversations. In this particular case, a website for Russian brides created fake accounts to promote its services. We had already done a considerable amount of work to identify and shut down these accounts before a single friend request could be sent, but this case was different because these accounts would not send friend requests, but rather comment on some public threads in hopes of receiving a friend request. I learned very quickly that it is very common for attackers to react promptly to the protection we put in place—sometimes within hours. So, I cut my SI-RAR teeth by tweaking our systems often and disabling all of these fake accounts.
Social engineering attacks
As you can imagine, attacks do not spike on Wednesdays at 2:00 p.m. when everyone is at their desks, ready for the smallest incident—that would be too simple. They usually start on Friday night, and by the time you open your computer on Saturday, the attack is well under way. Spammers are located all over the world, especially in countries where it is hard to prosecute them. They have day jobs, very often in computer security, and they are always planning strikes around times we are less likely to be at work. Data and automation are our best friends in this world and we use these tools to constantly monitor all negative interactions happening on the site (friend requests being declined, messages reported as spam, etc.). On top of that, an important part of my job is to research and predict what is going to come next and plan for the attacks that we don't know about yet.
Last month, we noticed a significant change from self-XSS to malicious browser extension. The upside is that we blocked self-XSS well enough that the attackers had to find a new vector for spam and changed to something with a lot more friction. This browser extension is essentially a virus that people install on their computer under the illusion they are installing a video plugin. We have been working with the browser vendors to fight these extensions and are deploying more advanced counter measures all the time.
And so this was 2011. We saw old attacks declining (drive-by download) and new attacks showing up (self-XSS), and we've put new protections in place that we will continue to iterate on this year. However, a team of security engineers will never be as effective as millions of users aware of security issues.
My team works hard to build spam protection that will secure everyone's account, and with more than 800 million people using Facebook, this is a considerable challenge. But everyone on my team is always on-call, ready to fight. I joined Site Integrity because I liked ML problems and thought I could have a big impact working on security, and two years later, every day still brings a new set of challenges.
Want to help us protect the internet? Apply here.