RE: Improving the Steem platform for long-term content
@markrmorrisjr, @anyx & @theoretical
Reedited this post after I went back and read what a hot mess it was because I tried to merge content from notepad, github and content related to the OP's post regarding this thread.
Sorry if things got a bit ampped up on Github. Going back though the ticket thread I realized that a lot of us had almost turned it into a free forum for debate rather than on topic discussion regarding the issue of payouts. In actuality I don't think I've ever seen a Github issue thread get that derailed in a very long time and I'm sorry if I was also contributing to that as well.
To stay a bit more on topic I'd like to first turn things down a notch first by apologizing to everyone who had to dig through that thread and all of the posts that turned into long rants. I think a lot of the heat in the debate was because the issue of voter nullification took on a big life of its own on in the past day or so. While the 2 issues at hand were unrelated they somehow got jumbled together in regards to the posts about them and they became almost one and the same, regarding upcoming changes to be made in HF 14.
Since that all started, the voter nullification issue has been tabled for the time being and I've had a chance to sleep on it to calm down a bit. Hopefully everything will stay a bit more on topic from here on out. Sorry if my anger over one issue overflowed into the ticket about the payout times. The last thing in the world I want to see is Steemit have the same kind of split in the user base that did irreparable damage to the Ethereum Foundation.
With keeping on topic about payouts and resource use, I'm also running a Witness & Mining node as well but it's not taking up anywhere near 7 GB of RAM for some reason and I'm running windows 10 which is known for it's terrible method of allocating resources. My total RAM usage is around the 3 GB mark which is still a bit high for a simple P2P client but I find it pretty manageable having 16 GB in total right now. Even with 7 GB being in use 16 GB would be more than enough for right now.
I'm upgrading to a total of 32 GB in the next week or so but it does sound like a bit of code optimization may be in order a full a node is eating up 7 GB now and was only eating 5 GB a few months ago. I'm not sure if the major difference is stemming from the fact that I'm only running 5 threads right now without hyper threading on an AMD processor, if it's an OS issue or what's going on there, but 7GB for a P2P client seems high to me. If it was using about 2/3rds of that a month back then this model will obviously be unsustainable in the long term when the user base grows to a much larger size. Regardless of payout times, more users means higher post volumes.
If the assertion about the RAM allocation problems in relation to posts pending payout is the indeed case, then that would definitely have to be addressed before the post volumes get to be unmanageable on a standard PC. In terms of Reddit's archival policies, the time until archive is 6 months which is well beyond the current 30 day proposal for Steemit.
I've done quite a bit of homework on how the Graphene protocol works and it seems it was optimized to handle a massive amount of small transactions per second and that metric was deemed to be the most important part about Graphene. TPS speeds rather than overall resource utilization could become a very serious issue if this platform starts to grow outside of that framework, and begins to be comprised of much larger data transactions meant to be sustained for a much longer period of time. It may not be what Graphene was designed for, but it is how many other social media platforms in existence seem to operate today. In regards to the life of an active thread, (paid out or not) 30 days is a very small window of time until a post is locked and archived.
Long term data accessibility will be paramount to Steemit's success if it's going to be a game changer and truly in it for the long haul. Having a "stop grave digging up posts" mentality would put Steemit in the same class with platforms like twitter (an almost purely Type I platform), and less so into the arenas of Facebook and Reddit (much more in the realm of a Type II bias). I would almost classify wiki as more of a Type III because not only is its past referenced regularly, but it's also updated and changed frequently which is what gives it value over both Type I and Type II style platforms. A Type III situation would make little sense for a platform like Steemit. Steemit is not a virtual encyclopedia but there's no reason it could not be a hybridized Type I/Type II kind of platform in which it would act as more of a library with a reference section. There is always value in having new and fresh content in a library, but no one would debate the merits of also having a section for classic materials written long before many of us were born. Such a library wouldn't last very long in the real world, (in keeping with the metaphor) it would instead become more of a trendy bookstore.
I'm not sure what specs have been recommended or required for the top tier of witnesses running full nodes but it sounds like multiple SSD's in Raid 0 may almost become a necessity if people start topping out on RAM and have to rely on virtual memory to pick up the slack. While that solution isn't nearly as cost effective or ideal as popping in a bit of extra RAM, when speed counts it's about the only way to go without the need for a full on conversion to rack mounted server blades or off site hosting in a data center.
It's hopefully possible that a combination of the 2 solutions (both code optimization and SSDs in a RAID 0 config) will become more plausible in the near future. Hardware prices are continuing to drop rapidly as SSDs become more of the norm rather than the exception, and during that time more data can be extrapolated in terms of the future rate of growth for this platform. I'm hoping that more data on what is necessary in terms of long term scalability can mitigate the need to archive old blocks for the sole purpose of saving on system resources. This kind of platform has the potential to be a total game changer and I hope time will afford both Steemit and Graphene the means to become the best go to platforms for long term, high speed, P2P social media.
For my part, I saw a thread and joined it, not aware that github was primarily being used to discuss the technical aspects. My apologies. I'm a writer and publisher, not a tech.
@markrmorrisjr , No worries. We really don't have much of a better way to directly address technical concerns and their likely ramifications to the Dev's without hopping on to Github. The original posts that drew attention to this very issue even had links pointing people to Github because the flow of information around here is a little... Uhm... Disorganized at times?
We don't have sub's like Reddit we only have #tags so the next best thing was to point people at Github I guess? (Doh... I doubt that mistake will be made again in the future.)
It was kind of like watching a scene from Family Guy where Peter just keeps on fighting with the chicken regardless of the venue or the collateral damage. The debate just kept moving from place to place only to end up right back where it started. We even managed to carve a path of collateral damage in our wake as well that shall forever be attached to the Steemit Git Repo.
Yup, we're an awesome bunch sometimes. You may not have known any better but plenty of us did and I'm sure we all realized this morning that it was too late to take it back. I honestly doubt anyone was proud to have taken place in the cross platform brawl, but as @anyx said on his thread from yesterday this place is very much the Wild West of social media.
In light of that revelation here's my peace offering of a parody for taking part in such an incident considering many of us did this very thing right here :
Steemit, gave me a bad coupon...