Data Scraping Is Becoming More Prevalent And More Of An Intrusion On Your Privacy

The Wall Street Journal posted a troubling article today on how the incidence of companies scraping data from websites (i.e. obtaining the data by pulling it from the user side instead of my accessing a site’s database or other private/proprietary repositories) is increasing.

Media research firm Nielsen Co. was recently caught scraping data from the forum at by signing in as a regular forum member, then systematically copying/scraping the data on the forums, including user’s profile information and posts. While not technically illegal, the practice is pretty creepy.

According to the WSJ article, Nielsen isn’t the only company doing this. It’s just one of many.

“Customers for whom we were regularly blocking about 1,000 to 2,000 scrapes a month are now seeing three times or in some cases 10 times as much scraping,” says Marino Zini, managing director of Sentor Anti Scraping System. The company’s Stockholm team blocks scrapers on behalf of website clients.

This kind of data has always been used for marketing and advertising purposes, and many people don’t see that as a big deal. However, the more disturbing stuff comes when scrapers use that information for more targeted, personal reasons. Employers aren’t just relying on Google to tell them about you, but using scraping firms to dig deeper. Your tweets, status updates, and other social networking chatter may not be safe even if it doesn’t show up on your public profile. And then there are companies like PeekYou LLC, which is attempting to patent a method that is meant to match people’s real names to their  screen names/handles/pseudonyms they use on blogs, social networks, etc. Classy.

Is there any way to protect yourself from scraping? There’s not likely a completely fool-proof way, and perhaps the best you can hope for is that whatever data these companies mine can’t be linked to you if you don’t want it to be. Right now there’s no law on this here in the U.S., and apparently international law is fuzzy and varies. Looking at this diagram, taken from PeekYou’s patent application, it may be possible to thwart their efforts to link you to your screen name by changing up the details you give different websites. Don’t always use the same email, don’t give your real birthday, leave out your middle name and initial, and avoid giving your address or even city.

There is probably very little you can do to completely protect yourself if you spend time on the Internet at all, but minimizing the amount of data these companies can scrape and use to their benefit is a good first step.

Email* (will not be published)
*Indicates required field
Submit Comments

  1. aftermath Says:

    Thank you for posting this. Even as somebody who is well aware of the practice, I hadn’t realize that it’s incidence had was on the rise. Even consumers who don’t care about this practice are better of just for knowing that it’s out there and on the rise.

All Product Types Accessories Cars Digital Camcorders Digital Cameras eReaders GPS Laptops MP3 & Video Players Projectors Smartphones Software Storage Tablets / MIDs VoIP Wi-Fi
All Subcategories
All Subcategories All-Purpose Budget Business Desktop Replacement Gaming Multimedia Netbook Nettop Rugged Student Tablet PCs Ultraportable
Acer Alienware Apple Archos ASUS Averatec BenQ CTL Corp. Dell Digital Storm eMachines Emtec Everex Fujitsu GammaTech Gateway General Dynamics Getac Gigabyte Hercules HP HTC iBuyPower Intel Lenovo MSI Nokia Nvidia OCZ OLPC OQO Origin Panasonic Sager Samsung Sony Sylvania Systemax TabletKiosk Toshiba Verizon Viewsonic Viliv VooDoo Workhorse PC ZT Systems
Minimum Rating
Any Rating 4.5 Stars 4.0 Stars 3.5 Stars 3.0 Stars
Screen Size
10 11 12 13 14 15 16 17 18 20 4 5 6 7 8 9
1024x576 1024x600 1024x768 1200X800 1280 x 720 1280x1024 1280x768 1280x800 1366x678 1366x768 1440x1050 1440x900 1600x768 1600x900 1680x1050 1680x945 1920x1080 1920x1200 800x400 800x480
Weight Range
10.1 - 12.0 pounds 12.1 - 14.0 pounds 14.1 - 16.0 pounds 2 lbs 2 pounds and under 2+ lbs 2.1 - 4.0 pounds 4.1 - 6.0 pounds 6.1 - 8.0 pounds 8.1 - 10.0 pounds Over 16 pounds Under 2 pounds
more options