I think it remains to be seen. The rapid growth of .world has been the first real production test of how the platform handles more users and content. Amazing work by the team, but there are a lot of rough edges and it is a new platform with a lot of unknowns.
The things that spring to mind for me are:
Sign up needs to be streamlined and made more simple, and find a way to not overload individual servers without just randomly assigning people to instances.
Live defects, bugs and things feeling rough around the edges.
Back-end build and scaling.
Duplicate communities across instances.
Account migration between instances.
Data retention past x period - how will various instances handle this with a large number of users.
GDPR and data request compliance from individuals, governments, etc.
Funding the costs and resources associated with rapid, large growth. How do people know what their money is going to fund? I think there needs to be real transparency, public roadmaps and backlogs and understand how / if admins are accountable.
How the platform and users will respond to large corporations or even individual admins on instances adding adverts, using / selling user data in ways the userbase do not expect.
The biggest issue would be data retention. Reddit serves as a real world database that stores all the historical content and search engines like google make it searchable.
We’re talking about petabytes, and lemmy hardly has a few gigabytes.
Who is going to store all this data, even in a distributed environment, the bigger instances would have to store a few hundred terrabytes (per year).
Text is very light and compresses very well. While instances may risk having scaling issues with photo and video, text should be very easy to archive forever.
On the other hand, do we really need to store it? Sure some posts will remain relevant, but many and even most posts on reddit, forums etc are outdated. Maybe communities and mods should decide what posts are relevant and make them permanent, where the rest just get erased after a set period of time that the community sets.
If stuff is deleted it can end up as a DenverCoder9 suituation where you search the tntire internet to find a solution to yer specific problem, find someone who had iot a decade ago, solved it and either never posted it or it was wiped.
I specifically use Reddit for the data retention and ease of finding “old” information, unlike basically other social media which scrubs it from any search even seconds after you looked at it (even if they still store the data)
Personally I think its ok for instances to delete older posts to save space provided that there are means to archive threads that users find valuable.
For fediverse to thrive it should be as easy as possible for people to setup and manage instances without having to think about the storage space too much.
Archival of historical content is something that I feel should be handled separately.
That’s really interesting, what are the benefits to duplicate communities beyond one server going offline or not retaining data? Is it better to have a lot of smaller communities with duplicate posts in them and having a quite splintered user base, or having everyone in one place but have the risk of having all the eggs in one basket!
The resiliance to opresion. Because federation it is wanted to have smaller communities. As everything is splintered into small communities. And smaller communities are better managable.
I think it remains to be seen. The rapid growth of .world has been the first real production test of how the platform handles more users and content. Amazing work by the team, but there are a lot of rough edges and it is a new platform with a lot of unknowns.
The things that spring to mind for me are:
Sign up needs to be streamlined and made more simple, and find a way to not overload individual servers without just randomly assigning people to instances.
Live defects, bugs and things feeling rough around the edges.
Back-end build and scaling.
Duplicate communities across instances.
Account migration between instances.
Data retention past x period - how will various instances handle this with a large number of users.
GDPR and data request compliance from individuals, governments, etc.
Funding the costs and resources associated with rapid, large growth. How do people know what their money is going to fund? I think there needs to be real transparency, public roadmaps and backlogs and understand how / if admins are accountable.
How the platform and users will respond to large corporations or even individual admins on instances adding adverts, using / selling user data in ways the userbase do not expect.
The biggest issue would be data retention. Reddit serves as a real world database that stores all the historical content and search engines like google make it searchable.
We’re talking about petabytes, and lemmy hardly has a few gigabytes.
Who is going to store all this data, even in a distributed environment, the bigger instances would have to store a few hundred terrabytes (per year).
Text is very light and compresses very well. While instances may risk having scaling issues with photo and video, text should be very easy to archive forever.
Media is only stored by the instance of the user that’s uploading it, if you want to upload tons of data you’re going to end up having to self-host.
…and it’s not like links don’t break on reddit all the time. Don’t worry about archiving that’s what archive.org is for.
On the other hand, do we really need to store it? Sure some posts will remain relevant, but many and even most posts on reddit, forums etc are outdated. Maybe communities and mods should decide what posts are relevant and make them permanent, where the rest just get erased after a set period of time that the community sets.
If stuff is deleted it can end up as a DenverCoder9 suituation where you search the tntire internet to find a solution to yer specific problem, find someone who had iot a decade ago, solved it and either never posted it or it was wiped.
I specifically use Reddit for the data retention and ease of finding “old” information, unlike basically other social media which scrubs it from any search even seconds after you looked at it (even if they still store the data)
Wouldn’t that take even more resources?
Personally I think its ok for instances to delete older posts to save space provided that there are means to archive threads that users find valuable. For fediverse to thrive it should be as easy as possible for people to setup and manage instances without having to think about the storage space too much.
Archival of historical content is something that I feel should be handled separately.
Duplicate Communities are wanted.
That’s really interesting, what are the benefits to duplicate communities beyond one server going offline or not retaining data? Is it better to have a lot of smaller communities with duplicate posts in them and having a quite splintered user base, or having everyone in one place but have the risk of having all the eggs in one basket!
The resiliance to opresion. Because federation it is wanted to have smaller communities. As everything is splintered into small communities. And smaller communities are better managable.
For sure! It will be interesting to see how it all works out as Lemmy hopefully grows and scales
Yeah we will see :)
There are still a bunch of UI bugs that remain to be solved. That one is also a big hurdle to becoming mainstream