It looks like the lack of persistent storage for the federated activity queue is leading to instances running out of memory in a matter of hours. See my comment for more details.

Furthermore, this leads to data loss, since there is no other consistency mechanism. I think it might be a high priority issue, taking into account the current momentum behind growth of Lemmy…

  • Gashole711@lemmy.worldEnglish
    3·
    1 year ago

    Using Couchbase as your eventual consistency database is perfect for this scenario. It’s designed for this type of thing. Even if systems are offline for a few days they will queue up and replicate when they come back online. Cruise ships use it for this very reason.

    • King@vlemmy.netEnglish
      2·
      1 year ago

      Can you provide a link? Only thing I see on Couchbase is for NoSQL databases.

      • Gashole711@lemmy.worldEnglish
        1·
        1 year ago

        It is a NoSQL database but the SQL syntax is ANSI SQL compliant. If you moved the queues to Couchbase and let it handle the replication and consistency, you wouldn’t have to code for it.

      • Gashole711@lemmy.worldEnglish
        1·
        1 year ago

        Actually XDCR is not available in the community edition so that option is gone. Sorry about that.