Bridgy Fed made a splash earlier this week by announcing its latest progress in connecting the Fediverse to Bluesky and Nostr. Sadly, not everyone was welcoming.

  • cosmic_slate@dmv.social
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    5 months ago

    You know that it is comically easy to scrape stuff off of the fediverse, right?

    I’d wager the vast majority of instances aren’t utilizing any meaningful rate limits, and even if there are rate limits, just distribute your scraping across several instances.

    Or just set up your own new instance and subscribe to literally everything you can find. You don’t even have to scrape, it gets pushed to you!

    If you are worried about scraping, use Facebook. Facebook has teams of people who combat bot/scraper activity.

    • rglullis@communick.news
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      1
      ·
      5 months ago

      The largest Mastodon instance (mastodon.social) has 360k MAU. This means that one can crawl all of its activities with less than 5 requests per second, every day.

      Even with rate limits, the Fediverse is still so small that I could crawl the top 10 mastodon instances in less than a day.

      From my desktop PC.

      On my shitty DSL.

      Anyone thinking that bullying one developer into a well-meaning project will be enough to keep their “secret clubs” away from malicious actors are in for a sad realization.