• whaleross@lemmy.world
    link
    fedilink
    arrow-up
    2
    ·
    6 months ago

    There is probably a neat wget oneliner that could crawl everything on the open web. The real challenge is how to index all the information. That might be a neat Perl oneliner.