Two puzzles with the condenser_api.get_account_history call

in #steemtalk15 days ago

Can anyone explain this?


Image by Bing Creator

  1. Why does the condenser_api.get_account_history call from api.steemit.com change the transaction numbers that it returns based upon the "limit" parameter?
  2. Why is the last transaction number different, depending on which API server I connect to?

api.steemit.com

Last transaction number for steemcurator01 is 1024795

$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", -1, 0], "id":1}' https://api.steemit.com | jq -S '.result[] | .[0],.[1].op[0]'
1024795
"comment"

It looks the same here, when I change the "start" parameter from -1 to the actual transaction number (1024795).

$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", 1024795, 0], "id":1}' https://api.steemit.com | jq -S '.result[] | .[0],.[1].op[0]'
1024795
"comment"

But when I query transaction 1024795 with a limit of 1 (or higher), it jumps back to transaction numbers 1024726 and 1024727

$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", 1024795, 1], "id":1}' https://api.steemit.com | jq -S '.result[] | .[0],.[1].op[0]'
1024726
"claim_account"
1024727
"comment"

api.steemitdev.com

here is the same sequence on the steemitdev API, where everything looks as expected (except that the last transaction number is different)

$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", -1, 0], "id":1}' https://api.steemitdev.com | jq -S '.result[] | .[0],.[1].op[0]'
1024774
"claim_account"
$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", 1024774, 0], "id":1}' https://api.steemitdev.com | jq -S '.result[] | .[0],.[1].op[0]'
1024774
"claim_account"
$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", 1024774, 1], "id":1}' https://api.steemitdev.com | jq -S '.result[] | .[0],.[1].op[0]'
1024773
"create_claimed_account"
1024774
"claim_account"

api.moecki.online

And here is the same sequence on api.moecki.online. Again, it looks as expected, except for another different last transaction number.

$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", -1, 0], "id":1}' https://api.moecki.online | jq -S '.result[] | .[0],.[1].op[0]'
1024286
"claim_account"
$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", 1024286, 0], "id":1}' https://api.moecki.online | jq -S '.result[] | .[0],.[1].op[0]'
1024286
"claim_account"
$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", 1024286, 1], "id":1}' https://api.moecki.online | jq -S '.result[] | .[0],.[1].op[0]'
1024285
"create_claimed_account"
1024286
"claim_account"

Perhaps it has something to do with the fact that api.steemit.com is ten minutes behind the others? But that doesn't explain the changing transaction numbers for the (apparently) matching transaction.

$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", -1, 0], "id":1}' https://api.steemitdev.com | jq -S '.result[] | .[0],.[1].op[0],.[1].timestamp'
1024792
"create_claimed_account"
"2024-06-19T18:17:03"
$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", -1, 0], "id":1}' https://api.moecki.online | jq -S '.result[] | .[0],.[1].op[0],.[1].timestamp'
1024304
"create_claimed_account"
"2024-06-19T18:17:03"
$ curl -s --data '{"jsonrpc":"2.0", "method":"condenser_api.get_account_history", "params":["steemcurator01", -1, 0], "id":1}' https://api.steemit.com | jq -S '.result[] | .[0],.[1].op[0],.[1].timestamp'
1024809
"claim_account"
"2024-06-19T18:07:30"

FWIW, the account_history.get_account_history call doesn't have the renumbering behavior.

$ curl -s --data '{"jsonrpc":"2.0", "method":"account_history_api.get_account_history", "params":{"account":"steemcurator01", "start":1024824, "limit":0}, "id":1}' https://api.steemit.com | jq -s '.[].result.history[][0]'
1024824
$ curl -s --data '{"jsonrpc":"2.0", "method":"account_history_api.get_account_history", "params":{"account":"steemcurator01", "start":1024824, "limit":1}, "id":1}' https://api.steemit.com | jq -s '.[].result.history[][0]'
1024823
1024824

But the steemit API is still 10 minutes behind the others, and the transaction numbers still don't match:

for S in https://api.steemit.com https://api.steemitdev.com https://api.moecki.online;
do
echo $S
curl -s --data '{"jsonrpc":"2.0", "method":"account_history_api.get_account_history", "params":{"account":"steemcurator01", "start":-1, "limit":0}, "id":1}' $S | jq -s '.[].result.history[][0],.[].result.history[][1].op.type,.[].result.history[][1].timestamp'
done

https://api.steemit.com
1024823
"claim_account_operation"
"2024-06-19T18:30:33"
https://api.steemitdev.com
1024803
"claim_account_operation"
"2024-06-19T18:40:09"
https://api.moecki.online
1024315
"claim_account_operation"
"2024-06-19T18:40:09"

Color me confused...

Sort:  

Some time ago I had a similar observation.
I later saw an explanation from Steemchiller.
That should explain question 2.

Indirectly, this also explains question 1:
We learnt from the last error with Hivemind that the request to api.steemit.com is not only forwarded to a single server. Obviously there are several servers operating an RPC node. As Steemchiller has written, the id can differ depending on the node. The response therefore depends on the server/node to which the request was forwarded.

Thanks, I remember when you posted that, and now that I (re?)read this comment from @steemchiller, the different ID number at varying endpoints makes sense:

It's not really a bug, because the field just contains an index for the data and the nodes can be configured with different plugins and settings. When a node does not hold all types of operations for example, the id will of course not be the same as for other nodes.

If steemit's endpoint relies on backend devices with different plugins and settings, that really cripples the account_history call, though. AFAIK, we should be able to iterate through that list with unique transactions from 0 through N (for whatever N is with the plugins and settings in question), right? I think that's the whole point of the paginated API methods. Yesterday, that was only possible by querying transactions one at a time. Roughly 50 transactions in that account's history were not addressable at all if the limit parameter was greater than 0. (Today, the issue seems to have cleared, at least for now.)

Maybe I should just switch to SDS for this purpose, but I was hoping to avoid reliance on a single point of failure. The idea was to let the user switch between API nodes if one goes offline. I guess the only two options for reliable data are to switch to SDS or else query the account history one transaction at a time.

Maybe I should just switch to SDS for this purpose, but I was hoping to avoid reliance on a single point of failure.

I'm planning to provide a downloadable SDS version in the near future. I currently still have a few other important things on my list, but I guess we will see this being realized within the next three months.

I have noticed that on some the tran# is one less than the desired return tran# requested. A falled return to take into account the count 1 to n versus the computer count 0 to n-1?

Upvoted. Thank You for sending some of your rewards to @null. It will make Steem stronger.