DiscoSteem-Bot contribution: node switcher implementation

in #utopian-io7 years ago (edited)

A couple of days ago, I was chatting with @planetenamek and talking about a little project that I started thanks to her idea (more about that in its introduction tomorrow). Looking at the code, she told me she liked the way I handled node switching to avoid getting too many errors and asked if I could implement it for DiscoSteem-Bot. This post documents the steps I have taken to add a node switcher to this bot. If you don't know about DiscoSteem-Bot already, it's a bot that allows for a multitude of interactions with the Steem blockchain through Discord. It messages wallet informations, rankings, created content and way more. If you want to see what it's capable of, a complete list of its commands is available on its GitHub page. The part I changed to implement my node switcher is the stream function, at the time of writing this it means that this node switcher should impact the post streaming and the "tracker voter" streaming for the better.

My implementation of the node switcher

The node switcher needed a JSON file to store a list of available nodes. Previously only the 'rpc.steemviz.com' node was used, now the six following nodes can be used by the app. This was definitely needed because you can't trust a RPC node to be working 24/7, no matter how reputable it is. If you are looking to fork DiscoSteem-Bot and change the nodes to some you prefer, make sure that those nodes aren't low memory ones or the app won't work. Low memory nodes return some properties as empty like the json metadata and the body of posts, that's something I learnt the hard way.
Now about the actual implementation of the node switcher. The idea is to have a recursive stream function that would call itself whenever its stream crashes, in this case whenever steem.api.streamOperations crashes. My way of putting that idea into code is as follows (the screenshot doesn't show the full function but only its relevant part, click on it if you want to see the full function).
First, out of the function, we get the nodes array from the nodes.json file I already talked about and we declare an index variable with an assigned value of 0. When the stream function is called, we set the RPC node url to the value of the nodes array at the current index. Then we return a new promise that doesn't actually have a way of being resolved but that can reject errors. Some may say that it's bad practice but I strongly disagree, promises that don't resolve anything can be really useful, you just need to be cautious when using them and not expect any value to get resolved. In the promise, the steem.api.streamOperations function is called. If any error is encountered while streaming (bad gateway, unable to acquire database lock...), an error is rejected and catched. The error message is then logged as well as the node that caused it. Then, the index gets incremented (and set to 0 if its new value is equal to the nodes array length) and the stream function is called. Since setting the RPC node to be used is done at the beginning of the stream function, it will change node before calling steem.api.streamOperations. This process repeats over and over again until the app execution is ended manually or an unhandled error is thrown (which shouldn't happen). This implementation may not work as expected if you want to stream all the operations that happen since you will probably miss some while switching between nodes, but I'm not sure there is a way to prevent that with any node switcher implementation anyway and switching nodes is really fast so you would have to be extremely unlucky for that to happen.

Contributions

Don't hesitate to contact @planetenamek on Discord or GitHub if you would like to contribute to DiscoSteem-Bot or if you need help setting up the bot.



Posted on Utopian.io - Rewarding Open Source Contributors

Sort:  

Thanks for your contribution, I think with this switcher we should be very efficient with the stream!

The solution to really not miss anything would surely be to save all the information in the files and do a check operation to see if the last recorded operation corresponds to the last operation of the blockchain.

I think it's pretty heavy to implement and especially in our it's not useful at all, so your switcher will do the trick!

Thanks again and see you soon

Hi @ragepeanut, unfortunately your contribution cannot be accepted.

Of course we appreciate everyone who contributes to open source projects, but for contributions submitted via Utopian to be accepted, and in turn rewarded, we expect a little bit more work.


Need help? Write a ticket on https://support.utopian.io.
Chat with us on Discord.

[utopian-moderator]

Yeah I get it, I was myself not sure if it was worthy of a vote. Thanks for taking the time to explain though. Hopefully my next post will be deemed worthy of a vote because it took me way more work to make it. See ya !