• diyrebel@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      1 year ago

      Countless bots. E.g.

      • a bot that scrapes real estate sites & public transport sites, then finds you a home within your parameters that’s a short commute to work/school on public transport. (I did this)

      • a bot that scrapes travel sites and finds you the cheapest itinerary, which could even combine buses, trains, planes, and bla bla car. Most travel sites don’t even give a quote calendar to quickly show you which days the cheapest fare is available – and those that do often lack a way to specify criteria like time of day ranges. They pretty much all suck.

      • a bot that scrapes the site of your snail mail service so you never have to login manually… grabs envelopes, orders scans, grabs PDFs, etc.

      • a bot that scrapes dating sites & finds your ideal mate. A woman who rejected the narrow search criteria of a dating site did this and her own algorithm found her a date who eventually ended up becoming her husband.

      • a bot that scrapes 2nd-hand goods sites and finds what you’re looking for

      • a bot that scrapes your bank & logs in on a monthly basis to fetch your statements with no manual labor on your part.

      • a bot that manages your email forwarding service by resetting counts of allowed msgs for particular addresses.

      • a bot that scrapes bike sharing networks and uses openstreetmaps to work out which portion of your journey is flat or downhill so you don’t have to pedal a heavy rental bike uphill.

      (edit) And generally, any website that has a lousy or useless search criteria needs the help of a beneficial bot. Any site that is manual human labor intensive needs a beneficial bot.

      • Whirlybird
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Ok yeh some good examples, thanks.

        I’m not a fan of content scrapers like that, but I get why people like them.

        • diyrebel@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          1 year ago

          You probably benefit from bots without realizing it. Even if you don’t code or install a bot yourself, you might visit a site that runs with scraped content. I discovered that one of the real estate sites that I scraped was itself showing scraped content. Most likely at some point you’ve bought airfare from a consolidation site where the data was not only fed by the shared db but also showed flights from some of the more protectionist small airline sites.

          I wish the general public were more aware of this. The masses unwittingly support the anti-bot measures without realizing the harvested data is often made available to everyone via another (improved) UI.

          Cloudflare works hard to demonize bots.