• micka190@lemmy.world
    link
    fedilink
    English
    arrow-up
    9
    ·
    edit-2
    6 months ago

    There’s actually legal precedent against scrapping a website through unofficial channels, even if the information is public. But basically, if you scrape a website and hinder their ability to operate, it falls under “virtual trespassing”.

    I’m assuming it would be even worse now that everyone is using the cloud and that scrapping their site would cause a noticeable increase in resource cost (and thus, directly cost them more money because of cloud usage fees).

    It’s why APIs are such a big deal. They provide you with an official, controlled, entry point to a platform’s data.

    • Dr. Moose@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      arrow-down
      1
      ·
      edit-2
      6 months ago

      It’s the opposite! There’s legal precedence that scraping public data is 100% legal in the US.

      There are few countries where scraping is illegal though like Japan and China. European countries often also have things called “database protection” laws that forbid replicating public databases through scraping or any other means but that has to be a big chunk of overal database. Also there are personally identifiable info (PII) protection laws that protect storing of people data without their consent (like GDPR).

      Source: I work with anti bot tech and we have to explain this to almost every customer who wants to “sue the web scrapers” that lol if Linkedin couldn’t do it, you’re not sueing anyone.