Programming Diary #27: Browser extension updates, new visuals, and a new Steem-based game idea

in Steem Dev23 days ago (edited)

Summary

After 26 previous programming diary entries, today's post finally provides an overview of most/all of the various programming activities that I have worked on. These include browser extensions, python scripts, spreadsheets, and PowerBI visuals.

Looking back at the last 3 weeks, the post also tracks progress against previously stated goals as well as activity that had not been anticipated before. Successes include the growth of "Developer Delegation Day (DDD)" during October and also enhancements to all of my Open Source browser extensions.

The post also provides some specific and general goals for future programming activity as well as general reflections about the Steem ecosystem.

Background

Overview

It occurs to me that I've never really provided an overview of the different things that I have worked on, so it might be useful to start off with that information for the current post. Here is the list:

  • The Steem Conversation Accelerator - Provides browser notifications when conversations of interest happen on the Steem blockchain, and - when clicked - loads a browser tab with a list of the conversation threads.
  • The Steem Follower Checker - Gives a score to a Steem account based on the account age as well as the number and reputation of the account's followers. The intent here is to help identify the authors who put in the effort to build a follower network and help their followers to succeed.
  • The Steem Curation Extension - Provides visual highlighting for posts that have beneficiary settings to burn Steem and posts that get promoted by sending SBDs to @null.
  • mod-bot (private repo): This was written by @cmp2020, but I have contributed some bug fixes, and I've been keeping the tool running for several years now. This is a Steem automaton that pins and unpins posts in a community if the posts have @null beneficiary settings or if @null receives post promotion payments.
  • autovoter (private repo): I run an autovoter that supports authors with automated votes from my accounts and accounts that are owned by some other people that I know. This incorporates the algorithm from The Steem Follower Checker as well as some other factors to try to focus on organic content whereas curation rewards are a secondary consideration. (if you ever see this voting on inorganic posts, please let me know so I can make adjustments).
  • I also maintain a variety of private/ad-hoc/throwaway shell & python scripts, spreadsheets, and PowerBI reports for gathering and reporting blockchain metrics.
  • @penny4thoughts (private repo): This was a bot that was intended to incentivize engagement by distributing beneficiary rewards to commenters in a thread based upon the author's upvotes. I disabled that because I don't want to be in the middle of the transfers when network connectivity issues arise. I have some ideas to eventually bring it back in a different form, but it's been shelved for a year or more now.
  • New
    : During this interval, I introduced a proof of concept Word Search game that will make a word search out of interesting Steem posts and engage with players as they guess the words and try to identify the source posts for the puzzles.
  • Update
    Steemometer (private repo): How did I forget to include the Java Steemometer toy program that I spent so much time on? This reports activity metrics about the Steem blockchain in near-real time and also displays promoted posts and messages in a "visibility as a service" section of the tool. I'm probably done working on this, but eventually I'll make it open source.

My programming activities generally focus in one of those areas. Clearly, this is far too much for one person in their spare time, but I do my best and keep plodding along😉.

  • (Not to mention, I also have a couple other ideas that I think would be useful, but I haven't started on because there already too many irons in the fire.)

So, there's an overview that (I think) covers most of my programming activities. Now, let's revisit the goals from Programming Diary #26.

Previous Goals

Here are my previously stated goals:

  • Developer Delegation Day for October is coming up in a week, on October 5. As with September, I'll be deciding who to support with 5 SP delegations.
  • I'm going to finish up with the SCA changes for displaying replies to the observer account, and hopefully start on adding notifications about conversations that the observer has participated in.
  • Hopefully, I'll be able to merge the existing changes into the Steem Curation Extension and I'll start thinking about ways that mouseover and/or clicks can be used in the browser extension to complement the Steem curation experience with supplemental information for the curator. This exercise with voting power spurred my thinking about other things that might be possible.
    • (For example, the browser operator could click on a certain location in the post container and get a lightweight popup message to display info like word count, follower network strength, and the number of votes from delegation bots)

  • Also, not related to programming, but it's nearly the end of the quarter, so I'll probably be posting the Q3/2024 inflation post tomorrow.

I definitely got a little bit sidetracked during the last three weeks, but here is a summary of progress towards those goals:

  • DDD: Delegations made, and we had at least 3 new participants in the initiative! Thank you very much to @moecki, @steemchiller, and @sbamsoneu for joining me in making DDD delegations in October!
  • SCA
    • Display of replies to the SCA observer account: This was completed.
    • Progress towards notification of activity in conversations where the observer has participated - none.
  • Steem Curation Extension
    • Merge voting power display changes: Done
    • Progress towards display of other author/post metrics: None
  • Quarterly Steem inflation summary - Done

Now, let's look at some more details from my programming activity since Programming Diary #26

Activity Descriptions

Steem Conversation Accelerator (SCA)

As previously noted, I was able to get replies to the observer account incorporated into the activity list. I decided against notifying or displaying posts and comments, because the observer would presumably know about those.

In addition, I fixed a bug where replies from a muted account were appearing in the activity list. Muted accounts are now excluded. This is a little bit ambiguous if an account is both muted and followed, but as it stands now, I think the "Mute" setting wins.

Finally, I spent most of the day yesterday tweaking some timing settings and fixing some bugs in an effort to balance API load with completeness. Here are the current constraints as I recall them:

  • If an account has 1000 operations (these could be things like votes, block producer rewards, transfers, curation rewards, etc.) after their most recent post/comment/reply, the post/comment/reply will be missed.
  • If the browser is idle and comes back to life, it will bring in up to 2 hours of previous activity.
  • If an API call fails 10 times, activity may be missed.

Excel and PowerBI reporting

As previously mentioned, I posted the Q3 inflation summary.

I also took a look at the distribution of account sizes by staked VESTS in these two posts:

  • In the top row, we see numbers of accounts, by class (and additional sub-divisions by stake for nanoplankton). The "redfish" category is repeated on the left and right side, showing the degree of the "zoom" between the two visuals.
  • In the bottom row, it's possible to visually compare STEEM, SBD, and VESTS on an equal basis. All values are converted to their STEEM/SP equivalents.
  • Summary: Most accounts are nanoplankton, but within that classification, many of the accounts are in the top-2 subgroupings. Most SP is held by Orcas and above, most STEEM and SBDs are held by exchanges, but the SPS also has a significant share of SBDs.

The biggest surprise to me was that we're now seeing 4 "blue whales". Four years ago, there were only 2 (I mis-remembered it as 1, so I'm glad I dug up that comment).

Steem Follower Checker

The scoring algorithm was adjusted to reduce the follower weight according to the age of the account in an effort to begin accounting for accounts that become inactive as time passes. This update has also been incorporated into my autovoter, and seems to be delivering reasonable results, so I'm planning to merge it into the main branch at the end of the month.

Steem Word Search

I created a couple python scripts that can download a post's body from the Steem API, turn it into a word search puzzle, and then post the puzzle. The idea is to eventually automate the posting process and the interaction with players in the comments as they solve the word search puzzle and also identify the post that generated the puzzle. I'm thinking that the bot might give out hints about the source post as a reward for identifying each of the words in the word search.

Looking ahead

SCA

I think I'm finally happy with the current iteration of the SCA as it pertains to handling followed accounts, so now I really am ready to move on to the next step. I'm torn about which that will be, though.

The whole reason I started the SCA was to give notifications for replies to other people in a conversation where the observer has participated, so now that the first major feature is basically in place, I had planned to move to that as a second feature. But, recently, I had the idea to incorporate tag following. So, to recap the major features that are intended, they are:

[x] - notifications and activity list for activities by followed accounts.
[ ] - notifications and activity list for activity in threads where the observer has participated.
[ ] - notifications and activity list for followed tags
[ ] - notifications and activity list for subscribed communities
[ ] - visibility for burn/promotion activity

And I need to pick one of those for my upcoming focus, but I'm not sure which one I want to tackle first. Please let me know if you have any feedback on the priorities and preferences for those enhancements.

Whichever one I pick, I expect that each one will take several months or more.

Steem Follower Checker

I'll be thinking about how to improve the accounting for inactive accounts, probably following-up on past suggestions by @moecki. I'm not going to be making any changes in the next few weeks, though.

Steem Curation Extension

Next up is to add metrics like word counts, delegation bot usage, and Steem follower network strength on a per-post basis. I might be able to make good progress on this in the next few weeks.

mod-bot

I've asked @cmp2020 for some code changes to this. No details for now, but I think I have an interesting idea. We'll see.

Goals

For the next iteration, I guess I'll state two specific goals:

  1. Pick the next major direction for the SCA and start working on it.
  2. Make some progress on delivering per-post metrics in the Steem Curation Extension

Reflections

I'm already at 1600 words, and I have yet to write the summary and conclusion, so let's keep this short. I don't really have much on my mind anyway. It's hard to zoom out after spending yesterday zoomed in on the SCA.

SP/STEEM/SBD distribution

Following up on the Steem/SP/SBD distribution, one thing that jumped out at me when reviewing that data was the large amount of liquid STEEM and SBD that are held by the Danumu/Upbit exchange (~190 million STEEM out of 474 million, i.e. 40% - and this is just one or two of their multiple wallets).

If this visualization suggests any single action for consideration, it might suggest the potential for Steem's Tier-1 stakeholders to negotiate with that exchange to establish a staking service for their clients. In '80s jargon, it looks like a potential "win/win scenario" for the exchange and the blockchain.

JavaScript browser extensions

I realized that yesterday was my first day of JavaScript programming where I hardly used any of the AI assistants at all. I think I only asked Leo one question. Maybe this indicates that I'm starting to pick up the language, a little.

SteemDB

I'm glad to see that SteemDB is updating at a speedy pace again for the first time in quite a while. I've noticed that curation and beneficiary rewards have been catching up at a rate of multiple weeks per day. Thank you to @ety001 for the recent changes there.

I guess that's enough for today, even if the reflections section is a little light this time. On to the conclusion and summary😉.

Conclusion

I like to do these diaries each fortnight, but it's been three weeks between posts this time. I was surprised by how much there was to write. Before I sat down to draft this post, it didn't feel like I had accomplished much, but now it seems that I did more than I had realized.

Sadly, the "to-do list" is much bigger than the "can-do list" and it feels like the gap is always growing bigger, but at least a look at the completed activities makes me aware that there has been some good progress.

Overall, I made progress in all three of my open source browser extensions, implemented an updated scoring algorithm in my auto-voter, visualized some new stakedholder metrics, and introduced the concept for a new Steem-based social game. Not a bad list of accomplishments for part time activities over the course of three weeks.


Thank you for your time and attention.

As a general rule, I up-vote comments that demonstrate "proof of reading".




Steve Palmer is an IT professional with three decades of professional experience in data communications and information systems. He holds a bachelor's degree in mathematics, a master's degree in computer science, and a master's degree in information systems and technology management. He has been awarded 3 US patents.


image.png

Pixabay license, source

Reminder


Visit the /promoted page and #burnsteem25 to support the inflation-fighters who are helping to enable decentralized regulation of Steem token supply growth.

Sort:  

From the Steem conversation to the updated scoring algorithm from the steem conversation to steem curator extension…. This are really nice projects, you have got here. To be honest, you have really achieved a lot in a space of 3 weeks. It’s not easy..

I also do love the initiative behind the DDD and I appreciate your shoutouts. Is not all the time you get lucky to be mentioned by some big names in steemit and to top it all, I was mentioned alongside big steemians like moecki and steemchiller… thank you. I’m really glad to be part of this growing initiative

 18 days ago 

Thanks for the reply! I appreciate the feedback. And thanks again for participating in DDD!

Hello, maybe you will find one more of my feedback useful. We are talking about The Steem Conversation Accelerator. I installed this extension on my home computer. But I had a few busy days at work and didn't turn on my computer for probably about 10 days. After turning on the computer, as usual, I received a notification about possible interesting interactions. When I clicked on this notification, the activity page took several minutes to generate as the extension tried to give me all the information for 10 days. I think this is not very good for your server.

 13 days ago 

Thank you for the feedback! I might have fixed that in the 20241020 version (assuming you haven't updated to that?).

I had fixed a problem with the time compare logic and also updated it to collect the last two hours after some idle time. From daily use, even that time frame seems to be too much with my follower list, so I think I'll cut it down to 30 minutes whenever I update again, but for now at least 2 hours is better than 10 days. ;-)

Please let me know whether you're still seeing any downloads beyond two hours with the newest version. Hopefully not.

Yes, you're right, I haven't updated versions in a while. 🙂

What to say? If only 284M Steem is available to trade, why burn?

The problem has to be identified correctly: Inability to purchase Steem?
Inability to receive, buy, or sell Tron?
I have a lot of problems with the Microsoft Edge Browser Working.
Yet, this is a browser that allows for better, easier writing when the browser works.

It is easy to make HTML posts or markdown.

Perhaps, people want more media types available?

They say there's a movement to upgrade jpg although so far it has failed in adoption, for example.

Perhaps, it is the amount of hack attempts that is causing the problem: Steem has gone down numerous over the last week or so and you steemdb is functioning again. Which is good.

I can't hardly get online unless I have a VPN. Is it possible to:

  • create a browser solely for Steemit. Years ago I made an IE 4 browser considered to be the most secure.

In order to do that, I used deep nesting of private classes in c/c++ something like nested namespaces.

Meanwhile, I will be working on cryptocurrency upgrade from SHA 256 here shortly. Whether I start in a month or sooner, I lean towards that and doing it in Windows.

 19 days ago 

I was having stability problems with Steemit for a few days, too. Seems like things have stabilized more recently.

I have a lot of problems with the Microsoft Edge Browser Working.
Yet, this is a browser that allows for better, easier writing when the browser works.

I'm not a fan of Edge. It always feels bloated and slow on my computer, though I have to use it at work. Same with Chrome. At home, I use Brave almost exclusively. It seems a good bit zippier to me.

What to say? If only 284M Steem is available to trade, why burn?

This is a really good point. I'm not opposed to burning, obviously, but we have to be realistic about what it can accomplish. What we really need is to move lots of STEEM from the exchanges (redfish & plankton) into SP that's held by (new) minnows and dolphins. Burning and club#### participation don't help much with that. As long as ~79% of all SP is held by 323 Orcas and above, the "crab bucket" concept from the Steem whitepaper is fundamentally impossible.

create a browser solely for Steemit. Years ago I made an IE 4 browser considered to be the most secure.

Certainly possible. It doesn't even really have to be a browser. It would just need to be a desktop application that interfaces with the Steem APIs. I think it would be a full-time job, though. Personally, I wouldn't be able to accomplish it as a "side project".

Loading...

Hello @remlaps,

I have read your post and seen the Excel and Power BI dashboard. Does Power BI provide the ability to use the SDS API to visualize Steem data summaries?

I have worked with OBIEE and OAS tools, which are Oracle products, but they do not support SDS API integration.

Please cast your witness Vote to bountyking5

 13 days ago 

Does Power BI provide the ability to use the SDS API to visualize Steem data summaries?

I hadn't tried it before, but apparently, it does (at least to some degree). I generated this by retrieving data straight from getVestingStats.

It would be more complicated to add parameters to the API call, but I'm fairly sure that it is possible.

Ok brother, I will try.

Please cast your witness Vote to bountyking5

Upvoted. Thank You for sending some of your rewards to @null. It will make Steem stronger.