EOS.IO Transaction Structure - Developer's Log, Stardate 201707.9

in #eos7 years ago (edited)

Today I would like to take a moment to explain the current structure of an EOS.IO transaction so that developers can better understand the concurrency model. Below is a JSON representation of a transaction that will transfer currency from sam to alice. In this case, currency, sam, and alice are all account names; however, they are used in different ways.

{
  "refBlockNum": "12",
  "refBlockPrefix": "2792049106",
  "expiration": "2015-05-15T14:29:01",
  "scope": [
    "alice",
    "sam"
  ],
  "messages": [
    {
      "code": "currency",
      "type": "transfer",
      "recipients": [
        "sam",
        "alice"
      ],
      "authorization": [
        {
          "account": "sam",
          "permission": "active"
        }
      ],
      "data": "a34a59dcc8000000c9251a0000000000501a00000000000008454f53000000000568656c6c6f"
    }
  ],
  "signatures": []
}

When serialized to binary with a single signature, this transaction is about 160 bytes in size which is slightly larger than a Steem transfer which is about 120 bytes or a BitShares transfer which is about 94 bytes. Much of the extra size comes from having to explicitly specify recipients, authorization, and scope which collectively add 51 bytes to the message.

TaPoS - Transactions as Proof of Stake

Those of you familiar with Steem & BitShares will recognize the first 3 fields of the transaction; they remain unchanged. These fields are used by TaPoS (Transactions as Proof of Stake) and ensure that this transaction can only be included after the referenced block and before the expiration.

Scope

The next field, "scope", is new to EOS.IO and specifies the range of data that may be read and/or written to. If a message attempts to read or write data outside of scope then the transaction will fail. Transactions can be processed in parallel so long as there is no overlap in their scope.

A key innovation of the EOS.IO software is that scope and code are two entirely separate concepts. You will notice that the currency contract is not referenced in the scope even though we are executing a transfer using the currency contract's code.

Messages

A transaction can have one or more messages that must be applied in order and atomically (all succeed or all fail). In this case there is exactly one message, so lets look closer at the message:

code:

Every message must specify which code it will be executing, in this case the currency contract's code will be executing resulting in the following method being called:

currency::apply_currency_transfer(data)

type:

The type field defines the type of message (and implicitly the format of data). From an object oriented programming perspective you could view type as a method "name" on the "currency" class. In this example the type is "transfer" and hence explains the naming of the method being called:

${namespace}::apply_${code}_${type}( data )

In case the "namespace" is the currency contract; however, this same method apply_currency_transfer may also be called in other namespaces.

recipients:

In addition to calling currency::apply_currency_transfer(data), the method apply_currency_transfer(data) will also be called for each recipient listed. For example, the following methods would be called sequentially in this order:

currency::apply_currency_transfer(data)
alice::apply_currency_transfer(data)
sam::apply_currency_transfer(data)


The notation account:: specifies the contract which implements the method. alice and sam may choose to not implement this method if they don't have any special logic to perform when currency::apply_currency_transfer is executed. However, if sam was an exchange, then sam would probably want to process deposits and withdraws when ever a currency transfer is made.

The person who generates the transaction can add any number of recipients (provided they all execute quickly enough). In addition some contracts can require that certain parties be notified. In the case of currency both the sender and receiver are required to be notified. You can see how this is specified in the currency contract.

void apply_currency_transfer() {
   const auto& transfer  = currentMessage<Transfer>();
   requireNotice( transfer.to, transfer.from );
   ...
}

authorization:

Each message may require authorization from one or more accounts. In Steem and BitShares the required authorization is implicitly defined based on the message type; however, with EOS.IO the message must explicitly define the authorization that is provided. The EOS.IO system will automatically verify that the transaction has been signed by all of the necessary signatures to grant the specified authorization.

In this case the message is indicating that it must be signed by sam's active permission level. The currency code will verify that sam's authorization was provided. You can view this check in the example currency contract.

void apply_currency_transfer() {
   const auto& transfer  = currentMessage<Transfer>();
   requireNotice( transfer.to, transfer.from );
   requireAuth( transfer.from );
   ...
}

data:

Every contract can define it's own data format. Without the ABI the data can only be interpreted as a hexadecimal data; however, the currency contract defines the format of data to be a Transfer struct:

struct Transfer {
  AccountName from;
  AccountName to;
  uint64_t    amount = 0;
};

With this definition in hand we can convert the binary blob to something similar to:

{ "from" : "sam",  "to": "alice",  "amount": 100 }

Scheduling

Now that we understand the structure of an EOS.IO transaction, we can look at the structure of an EOS.IO block. Each block is divided into cycles which are executed sequentially. Within each cycle there are any number of threads that execute in parallel. The trick is to ensure that no two threads contain transactions with intersecting scopes. A block can be declared invalid without reference to any external data if there is any scope overlap among the threads within a single cycle.

Conclusion

The single biggest challenge to parallel execution is ensuring the same data is not being accessed by two threads at the same time. Unlike traditional parallel programming, it is not possible to use locks around memory access because the the resulting execution would necessarily be non-deterministic and potentially break consensus. Even if locks were possible, they would be undesirable because heavy use of locks can degrade performance below that of single threaded execution.

The alternative to locking at the time data is accessed, is to lock at the time execution is scheduled. From this perspective, the scope field defines the accounts a transaction wishes to acquire a lock on. The scheduler (aka block producer) ensures no two threads attempt to grab the same lock at the same time. With this transaction structure and scheduling it is possible to dramatically deconflict memory access and increase opportunities for parallel execution.

Unlike other platforms the separation of code (currency contract) from data (account storage) enables a separation of locking requirements. If the currency contract and its data were bundled then every transfer would have to lock the currency contract and all transfers would be limited by single threaded throughput. But because the transfer message only has to lock on the sender and receiver's data the currency contract is no longer the bottleneck.

Removing the currency contract from being a bottleneck is incredibly important when you consider an exchange contract. Every time there is a deposit or withdraw from the exchange the currency contract and the exchange contract get forced into the same thread. If both of these contracts are heavily used then it will degrade the performance of all currency and all exchange users.

Under the EOS.IO model two users can transfer without worrying about the sequential (single threaded) throughput of any other accounts/contracts than those involved in the transfer.

Sort:  

Dan, thank you, you brought me in programming world. I am value so much that I was early involved with BitShares, because I was introduced to your talks and writings about what technology can do. It was like some catalyst for me, I started to learn programming, two years latter I left my sales job and became full time web developer. It was great gift, no appreciation of token will compare with acquired ability to earn money with your own brain. In the end, stake in crypto not so important, price could decline for years, but skills is much more tengible. Human mind creating these systems to serve humanity, we should not be slaves to them by depending on price of the token. I hope someday I could contribute code to projects like STEEM, BitShares or even EOS.

Really a great work i have seen so far.keep continue ur gd work


@dan This architecture is same as AWS LAMBDA. Pay how much you use.I found serverless computating architecture may helps devs save some time to reinvent wheels.

What happens if sam is evil and implements a sam::apply_currency_transfer(data) that never returns?

That depends on how much EOS sam buys. If he bought the entire supply of EOS, he could freeze the network until all the witnesses abandoned him. The previous holders of EOS could all thank him for the massive piles of cash he gave them, and then fork the network with sam excluded.

If he only has a small stake in EOS, his code should execute briefly (proportional to his stake) on the witness nodes and then be abandoned with minimal inconvenience to anyone.

I was thinking more along the lines of that he could block any transfers involving him, but I reread that part and it seems the sender could just exclude him from the recipients list and still have the transfer go through?

No, currency contract requires both sender and receiver to be in recipients list. Sender or recipient could implement apply_currency_transfer to auto reject every transfer.

An exchange would only block transfers that exceed user's balance with exchange contract, meanwhile other contracts may block all incoming transfers that they do not expect.

Ok I think I got it, it would be up to the contract to make sure any required recipients hook cannot be abused by a third party and in the currency case it's not a problem if either of the parties would like to stop the transaction from happening.

That's really powerful! I can't wait to try this stuff out on the testnet :)

Very nice answer !

An transaction that cannot execute in a couple of milliseconds is abandoned and never included in a block.

I have no doubt that EOS should take centre stage in cryptocurrency discussion in a few years to come for obvious reasons.

Unlike bitcoin, EOS is a product of years of experience. @dan has successfully pulled off a number of project that are individually a success.

Steem and bitshare are obvious success story of a man' s vission.

I can bet that EOS will be a standard for anything crypto in the next 5 years.

That said, I want to bring @dan attention to the current EOS Token ICO. Many of us who are not vast in blockchain cannot dare to participate in the sale for obvious reasons.

The procedure for participation is so complex for an average investor.

I really feel something needs to be done because alot of people have been waiting for this ICO, so so seeing it passing by like this just for not having enough technical knowledge to buy is not good.

This should be the people's coin so make it accessible to the people. Thank you.

For this exact reason i only buy on exchanges, bought a few days ago at $3, ouch hahaha :-) Thanks for the comment.

Which exchange did you buy from

EOS is an upgraded like an ugrade to blockchain tech . Lets be clear here: Bitcoin, was built on the ingenious blockchain tech. Bitcoin is what led the way for more innovation and every other variation of the blockchain. To imply that bitcoin is not a product of years of experience is misleading.

Ps you can buy tokens after ICO. If EOS delivers in its promises and based on activity and consistent update report from @dan , it is most probable that it will. If it does, the future value of EOS tokens can increase significantly. In that case; purchasing it on exchanges early may prove to be just as profitable in the nxt 3 to 4 years.

I made the right choice investing on EOS,this is what brought me on Steem.

Which do you believe in more?

I have faith in EOS and I have just discovered Steemit so it's really hard to decide at this point. But steemit is freaking awesome!

Which do you think is the better option in your opinion?

Steemit, no doubt.

Dan you are great.

I feel I haz the dumbz every time I read one of your posts LOL.

Have a good week my man.

Ironically I wrote a post today again about Poloniex and their nonsense, I think it is my best one yet with balance and fun and facts....

I honestly thought about you the whole time I was writing that post, which took me about 2 hrs.

Good post thanks for sharing

This comment has received a 0.14 % upvote from @booster thanks to: @hamzaoui.

Way to go @Dan. We wil be cheering you!

Parallel Execution seems very intricate. Good luck @dan keep up the good work!