Performance and Scalability Updates

Over the past few months, the Steemit development team has been working hard on several major performance improvements to the condenser code (steemit.com) as well as the steemd blockchain API. These developments include improved speed, efficiency, and reliability of the blockchain as well as the Steemit website. This post is to update you all on our progress.

Jussi

Historically, all calls to steemit.com have retrieved data directly from the blockchain, which was not the most efficient way to process requests. Therefore, we developed a new backend service called jussi which creates a layer between the website and blockchain (as well as the other backend services we are developing), allowing queries to be cached and processed in a much more effective way. It also creates an additional layer of resilience against attacks.

These changes have been deployed to production for several days now, and we have seen a dramatic improvement in the efficiency of read requests from the Steemit website to the blockchain to retrieve data. Jussi is built to leverage additional opportunities for improving efficiency by increasing modularity through the new AppBase backend architecture.

AppBase

Originally, a single steemd blockchain instance could use multiple threads to process different tasks, but only a single thread would handle API requests. This created a bottleneck on the amount of queries a single steemd node could process at once, as they were processed one by one on the API thread.

AppBase is a new architecture that creates distinct “modules” which enable separate processor threads to process different API requests at the same time, which is significantly more efficient. Modularizing steemd enables it to take full advantage of the parallel nature of modern processors. Many additional improvements can be built on top of this, but the foundation for a modular blockchain has now been laid.

The development team has been working around the clock to get AppBase ready for production, and we plan to launch it very soon.

Peer-to-Peer (P2P) network changes

The P2P code is what keeps all of the steemd blockchain nodes in sync. Efficiently transmitting operations (such as posts, votes, etc.) to other nodes in the network allows transactions sent from websites like Steemit to be quickly processed and included in one of the blocks by the witnesses before the transaction expires.

We have completed multiple performance improvements to the peer-to-peer communication code, and we may make additional improvements to this system over the coming months.

Communication

Many of these improvements take significant time and resources to investigate, design, develop, test, and deploy. We understand that you all want to be kept up to date with the latest progress, but there are many competing factors when providing the community with frequent updates. Our top priority remains improving the platform as rapidly as possible, and we do not want to distract from that. We also want to avoid drawing unnecessary attention to anything that a potential attacker could exploit until we have fully evaluated the situation.

The Steemit development team takes any issue that impacts the usability of the website very seriously. We have multiple performance monitors in place, which are running on the website 24/7, and we are alerted instantly if the site ever goes down. We track many metrics that inform us of how long things are taking as users are interacting with the website. We know when there are issues. We also appreciate the many alerts that we receive from users when they notice something that could be wrong. We may not always communicate publicly about every challenge we face, but you can rest assured that we are actively working to resolve all issues as quickly as possible.

These improvements we are making have us very excited about the future of the platform because they will enable us to handle the exponential user growth that we are expecting. We appreciate your patience as we continue to scale and build a world-class website.