SteemRadar.js - An event-driven lightweight library for streaming Steem Blockchain Update 2.0.0

in #utopian-io5 years ago

Repository

https://github.com/gigatoride/steemradar.js

line1.png

What is this?

Commit

This is version 2.0.0 a major version for this library to be an event-driven with more flexibility instead of streaming methods.

Some difficulties
  • The old version was based on streaming from steem-js library and it wasn't stable for a long time running
  • Steem-js is taking a lot of space sometimes causing difficulties in using the library
  • It's hard to customize each stream for example if you want to edit transfers senders accounts for different cases
  • For the old version, each method needs creates an instance which sometimes leads to increased consumption of server resources

line2.png

A quantum leap!

  • No longer based on any other Steem library client like Steem-js
  • Directly based on RPC web sockets nodes
  • Just simple & lightweight
  • Totally event-driven library
  • Set options for more flexibility and customization
  • All events are flexible
  • Easy error handling
  • A lightweight and stable solution for streaming Steem blockchain
  • Git rid of old unnecessary methods and take the purest ones
  • The library has become very reliable
  • Totally recoded from scratch based on the old methods
  • Settings for each operation type
  • 100% coverage for all blockchain operation types thanks to the dynamic algorithm.

line1.png

Package size

the version 2.0.0 is 87% less than the old versions with extremely better performance and smarter code

Annotation 2019-06-28 051518.png

line2.png

WebSocket Highlights

The following is a part code from the launch promise which is defined as this._start after it resolves

this.ws
  .on("open", () => {
    this.isWebSocketReady = true;
    this.ws.addListener("error", this.onError);
    this.ws.addListener("close", this.onClose);
    resolve();
  })
  .on("error", err => {
    this._start = null;
    reject(err);
  });

Now each time when need to use call method or check the socket ready status it should return this._start

   if (this._start) return this._start;

So we don't have to resolve it each time or create a new instance.

line1.png

Settings Highlights

I've made these settings to be flexible and based on extended Map class the user will be able to call them more than once unlike the old version, also all Map features, e.g. .has() .get() .set()
All events will directly change the stream results according to the last setting

The following method for setting transaction.operation as the event callback instead of transaction object

  setStreamOperations(enabled) {
    this.settings.set('streamOperation', enabled);
  }

The following utility is being used for this method after some researches I've found that some Steem nodes use different names so I've put all possible scenarios

setOperation: trx => {
  return trx.operation || trx.op || trx.operations[0];
};

The rest of the settings is the same based on an extended map class from a prototype

First I will start with the validator after creating a new map instance for each operation type:

new Proxy(Map.prototype.set, { apply: validator });

With a Proxy, I've made a validate for the passed value for the map built-in prototype set

Here is a validator example

_validateComment(target, that, args) {
    const [option, value] = args;

   // Some switch statement for validation

   // Apply the settings if all validations have passed
    target.apply(that, args);
  }

Target object to wrap with the Proxy. It is including the map set prototype function then pass the arguments.

line2.png

Events highlights

The following function is the yield for streaming blockchain operations

async * getOperations() {
    for await (const num of this.getSequentBlockNum()) {
      const operations = await this.getBlockOperations(num).catch(console.error);
      for (const op of operations) yield op;
    }
  }

It's only called once and from it, we create formatted emits to pass all transactions arguments
by settings for operation types as well as the main transaction event.

The command above is handling the sequent block number and yield for of operation

The following is an emit example code a custom operation

if (!this.settings.comment.has('authors') || this.settings.comment.get('authors').includes(author))
 this._emitTransaction(trx, type, getCommentType(data));

So each transaction emit is being handled by _emitTransaction() function which formats each emit by its parameters.

line1.png

Usage

Settings

Set transaction as txType,txData

 // true or false
client.blockchain.setStreamOperations(true);

For event: transaction:comment

 // an object has [authors] property
client.blockchain.setCommentOptions({ authors: ["steem"] });

For event: transaction:transfer

 // an object has [senders] [receivers]
client.blockchain.setTransferOptions();

For event: transaction:transfer:funds:track

 // an object has parentSender as string
client.blockchain.setFundsTrackOptions();

line2.png

Events

Event: transaction

Event: transaction:<transactionType>

Event: transaction:<transactionType>:<customParent>

Event: transaction:<transactionType>:<customParent>:<customChild>

Custom events parent/child types

TypeCustom ParentCustom Child
transactiontransferfundstrack
transactiontransfermemoprofane
transactioncommentparent
transactioncommentchild
transactioncommentbodyprofane
transactioncommentauthorblacklisted
transactionvotepositive
transactionvotenegative

the table above is just for the custom client methods built-in that is not included in the blockchain

line1.png

Examples

Transaction event

client.blockchain.on("transaction", trx => {
  console.log(trx);
});

A custom child/parent example from the table above

client.blockchain.on("transaction:comment:author:blacklisted", trx => {
  console.log(trx);
});

A general transaction type event, you can use any operation type you want from the blockchain.

client.blockchain.on("transaction:feed_publish", trx => {
  console.log(trx);
});

Error handling:

client.blockchain.on("error", err => {
  console.log(err);
});

Close that will remove all listeners and reset all settings

client.blockchain.close();

line1.png

Testing

I've also written the tests from scratch to cover all possible cases.

Use the following command:

npm test

Ci Test Result

line2.png

What's next

  • Support for the HTTP protocol
  • Support operations from, to blocks

GitHub Account

https://github.com/gigatoride

Sort:  

Thank you for your contribution. A great update indeed and it's always nice to refactor the code as much as possible. Today morning I was reading an article on dev.to where the third and important part was Deleting code as much as possible. I think this contribution sums up that article ver nicely.

Coming back to the contribution, nothing much to say, a great post with lot of details, +1 for writing tests and +1 for giving examples.


Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Chat with us on Discord.

[utopian-moderator]

Thank you for your review, @codingdefined! Keep up the good work!

Hi @gigatoride!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

Hey, @gigatoride!

Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!