Programming Diary #28: Thoughts on the problem of overvaluation
Summary
This post describes my programming activities from October 20 until today. During this time, I implemented changes to the Steem Curation Extension, finalized the latest version of the Steem Follower Checker, and resurrected one of the old visualizations that had become inoperable until SteemDB was recently brought up to date again.
The changes to the Steem Curation Extension provide a new overlay for curators to get some supplementary information on a per-post basis, prior to deciding whether or not to click into the post. A new scoring algorithm was implemented for the Steem Follower Checker.
Finally, testing of the Steem Curation Extension led me into a few days of reflections on the existence of paid voting and overvalued content on the Steem blockchain. Some of these thoughts are posted here.
Background
In Programming Diary #27, I listed (more or less) all of the major programming/development initiatives that I have worked on for the Steem ecosystem. This includes browser extensions, python scripting, LibreOffice Calc spreadsheets, and PowerBI visualizations. With all of these items to pick from and with the limited time that I have available, it is inevitable, unfortunately, that when I work on one thing, all of the other things will sit idle.
So, the big decision for every interval between diary posts is, "What will I work on"?
In the previous post, I set myself the following primary goals:
For the next iteration, I guess I'll state two specific goals:
- Pick the next major direction for the SCA and start working on it.
- Make some progress on delivering per-post metrics in the Steem Curation Extension
I accomplished the second, but didn't achieve the first. However, I also worked on some other stuff that I hadn't planned. So, let's look at the details.
Activity Descriptions
PowerBI
I also mentioned in the previous post that SteemDB had started updating more quickly. As a result, I was able to resurrect some visualizations that had stopped working in Q2/2023. One of the interesting pages is the 90-day summary of burned tokens and social rewards (author, curation, beneficiary). Here it is, updated through November 1:
To me, the most interesting thing here is to glance at the difference between today's graph and the last one I was able to post on September 2, 2023. The main two points that jump out at me are that:
- The average total of social rewards per day has dropped by almost 550 STEEM/SP, from 55,892 to 55,351. This is not a surprise, but it's interesting to see the observation match the theory over the course of 14 months; and
- Meanwhile, average burned beneficiary rewards per day has dropped by about 350, from 1,161 to 809. (note that this number is adjusted for liquid rewards, but the scaling doesn't handle 100% powerup posts right, so it's an upper boundary estimate.)
I'm not posting the other graphs today, but we see this same decline in @null beneficiaries over most of the last two quarters.
Browser Extensions
Steem Curation Extension
I added a new overlay with mouseover action for curators. Now, if you mouse over the "CURATION INFO" label in the upper right-hand quarter of a post summary, it will give you word count and information about votes from bid/delegation bots. Here's what it looks like:
| Dark Mode | Light Mode |
|---|---|
 |  |
So far, I haven't spent any time cleaning the bot account list. I just slapped together a minimal list and was mainly focused on getting the functionality working. Here are descriptions of the fields:
- Word count: Strips out HTML & markdown fields, and then counts the strings that are left behind. I'd expect this to be almost useless for posts in languages like Korean or Bangla, but hopefully it's not a bad estimate for posts in many languages.
- Reading time: Number of words * 200 words per minute.
- Bot count: Number of voters that are in the paid bot list. This runs into some challenges because some of the bid/delegation bot accounts are also voting organically, but for now they are being counted as paid votes. It's important to note that I don't intend to count curation trails or unpaid auto-voters as paid voting.
- Paid percentage: Percentage of total rshares that are allocated to a post by paid voting accounts.
- Organic value: The unpaid percentage times the total value.
- Total value is calculated as
total_pending_payout_value + 2 ( curator_payout_value )in order to handle values before and after payout time and to also include the post value that went to beneficiaries at payout time.
- Total value is calculated as
I merged the changes into the main branch on October 31, so feel free to download it or clone the repo and give it a try.
I also posted a thread, here looking for peoples' thoughts on what metrics to include. Feel free to weigh in. I plan to continue updating this.
Steem Follower Checker
As previously planned, I merged the latest changes into the master branch on October 31.
Looking Ahead
From now until 2025, I think my focus will be on 3 areas: The Steem Curation Extension, the Steem Conversation Accelerator (SCA), and the word search game that I discussed a few weeks ago. Here's the intended direction for each:
The Steem Curation Extension
So far, I have enabled the curator info overlay functionality and added post and voter information to it. I intend to follow-up with additional information about the author and the reach/strength of the author's follower network. Some of the ideas that I'm considering for inclusion can be found, here.
At this point, the main challenge seems to be the need to find a reasonable balance among usefulness, network consumption, and the size of the overlay. I'm not sure whether it makes sense to include all the information in a single overlay or to separate the four categories out into more than one overlay.
I'm also thinking about adding a checkbox to toggle the account feed between showing and hiding resteemed posts.
The Steem conversation accelerator
I'll just repeat this section from the previous post, since nothing has changed:
[x] - notifications and activity list for activities by followed accounts.
[ ] - notifications and activity list for activity in threads where the observer has participated.
[ ] - notifications and activity list for followed tags
[ ] - notifications and activity list for subscribed communities
[ ] - visibility for burn/promotion activity
Word Search Game
Basically, most this still needs to be built from the ground up. I already wrote some code that downloads a post, picks out some words, creates a word search, and then turns it into a new post. Next, I need to create a way to interact with the players to tell them when they guess the right words, and to give hints about the source post.
Goals
I can already see that my time is limited for the next two weeks, so I'm only going to set myself one specific goal for this interval, and that is to start displaying information about the author and their reach in the Steem Curation Extension. We'll see how much progress I can make.
Reflections
One of the things I noticed while testing the new overlay to the Steem Curation Extension is that I have apparently become totally desensitized to the state of the /trending page. This testing forced me to actually look at some of the posts that are sitting out there.
For example, I came across a post like this one:
console
take off
get down
get on
landing
tail guided
pitch
roll
vase
bass
garden
garment
hose
husk
That's it. That's the whole post (minus a footer). And it's valued at $84. Here's what it looks like in the overlay.
Can you imagine how a first-time visitor to the website reacts to seeing this on the trending page? I had a whole post written to whine and complain about it, but I self-censored in order to try for a more productive approach. I'll excerpt from that post in my reflections here, though.
- If I am an investor, this sort of overvaluation is devaluing my stake.
- If I am an author who is struggling to earn rewards by producing relevant, organic content, this is demoralizing me. If this sort of content is what dominates, eventually, I will probably respond in one of two ways:
- By leaving; or
- By setting up my own daily digital pollution factory.
- If I am a random visitor who happens across the trending page, this is discrediting the web site and chasing me away as quick as a flash.
- If I am a curator, this is reducing my inclination to find deserving content when I can just front-run the votes for the known polluters.
My first inclination is to wonder why nobody is downvoting this content? Were we so hamstrung by the Hive fork 4 1/2 years ago that we have to stay willfully blind to it, even now?
Well, it's easy to see why most of our accounts aren't downvoting. There are 324 accounts that can make any impact at all on these numbers, since those 324 accounts hold almost 80% of the SP in existence.
When a fraction of a percentage of all accounts controls 80% of the influence, the idea of crowd-enforcement to control valuation is simply not possible.
This is not a complaint or a criticism. Just a description of reality.
And of those 324 accounts, how many are players in the overvalued posting game? I don't have that metric yet, but easily 10-20%, I would guess. Did you know that they even have downvote insurance now?
So, what do we do about this? Unfortunately, for most of us, the answer is that we simply can't do anything. "Suck it up, buttercup", as they say😉. But that's not a satisfactory answer, and persuasion is still possible, so let me approach this as a short term and a long term problem.
Short term
- Web site operators and investors should treat this as an urgent problem. This sort of activity is harming the web site credibility and it is devaluing the investors' stake. To me, addressing both of those impacts should be an urgent matter.
- Web site operators should find ways to lower the prominence of this sort of content.
- Investors should look at levying a "tax" to disincentivize this sort of behavior. I'm not talking about reincarnating the downvote wars that we've seen in the past, but rather just adding some amount of friction. This might be done by downvoting the posts by a certain percentage, even as low as 1, 5, or 10% as a starting point. Given the distribution of SP in the ecosystem, Orcas, and even Dolphins can pitch in on this effort, but it probably needs to be spearheaded by whales and blue whales.
Long term
- I remain convinced that downvoting is not the long term solution to this problem. We have to stop the bleeding, but we also need to rearrange the incentives.
- I also remain convinced that the solution to this problem harnesses the simple idea that
(rewards + audience) > rewards alone. - From these beliefs, I conclude that the most important long term need for the ecosystem is an aggressive focus on expanding the products, tools, and services that deliver attention to the creative content producers in the ecosystem.
- This means, at a minimum, improving SEO, providing visibility as a service, and building bridges between Steem and the rest of the attention economy.
- Fundamentally, Steem is a play at the intersection of the crypto-economy and the attention-economy. If we want a piece of content to be valued on a basis other than the size of a stakeholder's crypto investment, we have to make it possible and rational for authors to focus on audience along with rewards.
Conclusion
All three of my open source browser extensions have been updated in the last couple of weeks, so I invite you to try out the latest changes.
The Steem Follower Checker has a new scoring algorithm; the SCA has a bug-fix with its timing logic; and the Steem Curation Extension has a new overlay that provides some supplemental information about posts that appear in the various feeds.
Also, I continue to invite contributions from anyone else who wants to help improve these tools.
Finally, this post raised the topic of overvaluation of posts in the reflection section here. This is a thorny and long-standing problem in the Steem ecosystem, but ignoring it won't make it go away. Do you have any productive ideas on how to respond to the problem? Or do you disagree that it is a problem at all? Please add your thoughts.
Thank you for your time and attention.
As a general rule, I up-vote comments that demonstrate "proof of reading".
Steve Palmer is an IT professional with three decades of professional experience in data communications and information systems. He holds a bachelor's degree in mathematics, a master's degree in computer science, and a master's degree in information systems and technology management. He has been awarded 3 US patents.
Pixabay license, source
Reminder
Visit the /promoted page and #burnsteem25 to support the inflation-fighters who are helping to enable decentralized regulation of Steem token supply growth.
There's no incentive to look for it, most people don't have a big enough stake to have an impact anyway, and voters that can overvalue garbage posts also have the power to downvote a small account (there's little upside in making a powerful enemy).
I disagree. I think systematically hiding the problem will perpetuate it. Garbage is trending because that's what the chain (currently) values. The sites should be transparent about that, not obfuscate it.
I am skeptical it would do much good, but I think we should be doing more to discuss the philosophy of how things are valued, e.g. seeing "Garbage post is worth X Steem" is a signal that 1 Steem is worth what that post is really worth divided by X. That DeFi that provides bank-like "interest" returns without doing bank-like economic activity on the other side (e.g. making money from loaning out the deposits) is a signal that however much Steem is earned in interest is worth 0 real value. Maybe explore what it means to "invest" in a cryptocurrency, and consider whether passive-income-extracting speculators create a barrier to entry to genuine investors by not only risking the long term value but creating a short-term bubble that limits the possible return on a real investment. I also think it would be beneficial for the community to have statistics about how much of the reward pool is flowing through particular categories, e.g. self-votes, paid-for votes, etc.
True. I even thought carefully about whether to post this, relatively gentle, critique. If there's going to be any downvoting, it almost definitely has to start at the top of the investment chain. Though, downvote insurance might be an interesting concept for players on the other side of the debate, too😉. Still, I'd prefer to avoid that whole cycle of retaliation. That drives people away, too.
Two points here. This general desire for awareness was part of my motivation for putting this sort of information into the overlay in the first place. It's also part of the reason why I posted about it here. Maybe talking about it won't fix the problem, but it's got a better chance than ignoring it.
The second point is interesting, too, that STEEM's value is pegged to the quality of the content that gets rewarded. It was a buried assumption in my claim that this content is devaluing the investor's stake, but you're right that it should be stated explicitly and understood better.
Edited to add:
I forgot to respond to this point:
This might be true, but maybe the web site operator could devise a way to hide it from the casual visitor while also providing a mechanism for visibility to investor-class stakeholders who could decide how to respond to it.
This literally frames the EXACT thought of the struggling new author on this site. I know because I'm one of them. Add in uneven rewarding from the SCx accounts and you're going to have issues keeping good creators here.
I think the uneven rewarding from SC* accounts is sort of unavoidable, but even that might be mitigated if the digital pollution weren't sucking so much air out of the rewards pool.
True. One other thing I've learned is that those high prize pools favor the curators and, in effect, the users with high SP, and in the process, give the small share to the actual creator of the content. Which is off, in my opinion. For example, I had no clue that a $150.00 post would break down to approximately $42.00 for the author. I thought it would be 50-50 between author/curator...it is not. Kind of demeaning, to be honest. Curating for a couple of months opened my eyes to quite a few things. I plan on making a post about all of that as soon as I have some free time.
As I understand, it actually should be 50% to the author (and beneficiaries) and 50% to the curators, but you have to be careful with how you understand the total value. Up until recently, the total value displayed included TRX distribution that wasn't happening any more, and with Tron's relatively high price that was inflating the reported totals by a lot. That was fixed now, so the discrepancy should be smaller now. But, I believe that the reported total is based on the median price of STEEM over 3 1/2 days, so if the STEEM price is volatile, the value at payout time can be very different from the price that's reported on the exchanges. (this can move the actual payout higher or lower, but the discrepancies should balance out over time)
Update:
I guess this is the one you're talking about?
What happened here is what you expected. The post paid out at $166.40. Of that, $83.20/50% went to curators, and $84.68/50% went to the author and beneficiary. Of that 50%, $42.34/25% in beneficiary rewards disappeared from the post's value at payout time, but it was distributed as beneficiary rewards. The other $42.34/25% went to you and stays in the reported value of the post. There are some rounding discrepancies in there - probably due to voters who didn't contribute enough to get rewarded, I guess, but for the most part it was a 50/50 split where half went to author + beneficiary and half went to curators.
For whatever reason, the web site and the post's reported value just don't remember the payouts to beneficiaries after a post pays out.
That's exactly how it is. I've been working on this recently because I wanted to check the pending payout amounts again by resetting the TRX/VESTS ratio. Unfortunately, they're still not correct because there's still an error in the Condenser (which only becomes apparent when the haircut rule takes effect, which probably didn't happen when the Condenser was being developed). I will suggest a fix for this.
To the display of the payout amounts:
It does not actually take the beneficiaries into account. The reason for this is relatively simple. The blockchain does not return this data. It would have to be calculated separately. And this is not entirely trivial because the curator amounts are not always fully 50 %. As you know, for votes that are made within the first few minutes, the curator contributions are reduced. I have already thought about this. In this context, I suspect an inaccuracy in Steemworld, which I had already reported to Steemchiller. Once this has been clarified, I can suggest changes.
cc @philhughes
Ah... I knew there was a reason why I didn't like pegging the so-called "median" to track the haircut price. I thought it was harmless, but it always seemed like a kludge. But yeah, of course any value depending on that would be wrong. By a lot, right now.
I guess the argument was that it's not wrong in terms of the internal SBD price, but it's labeled with a "$" (at least in the US), and of course the external SBD price isn't anywhere near the peg, so reporting in terms of SBDs is confusing, at best.
I compared this with one of your posts:
This was the display according to the current code:
... and so it should actually be correct:
The total amount in $ does not change.
The only difference is that 31 STEEM are not paid out. However, the amount for SP is correct if the curator amounts are also included.
I'll have to take a closer look to see where the code needs to be changed and which amounts should be included.
I'm confused about where the second screen shot comes from, but I think you're showing basically the same as what I was seeing.
Here's a an example after payout time with no beneficiary rewards in play.
At the haircut price, $6.20 is about 24 STEEM. At actual dollar value, it's about 36. The actual rewards seem to match the haircut price.
Which means the actual dollar amount of the distributed STEEM and SP would be closer to 2 * $4.05 = $8.10. Similarly, it looks to me like your second screenshot probably matches the STEEM/SP distribution, but not the dollar value. The dollar value of 42 STEEM is in the neighborhood of $7.14, not $10.93.
No idea how I have never noticed this before. It's a substantial difference. Or maybe I just forgot, but it seems like something I would have remembered. Or maybe I had overlooked it due to the previous inclusion of TRX values.
Thank you for explaining that. I'm still a novice on how this all works, which leads to some confusion on my end. I feel sort of silly that I didn't realize this on my own, but glad I asked nonetheless.
Oh - I'm excited! Since you've really been looking at the whole theatre quite recently, almost "from the outside", there could be some remarkable insights...
As I understand, it actually should be 50% to the author (and beneficiaries) and 50% to the curators, but you have to be careful with how you understand the total value. Up until recently, the total value displayed included TRX distribution that wasn't happening any more, and with Tron's relatively high price that was inflating the reported totals by a lot. That was fixed now, so the discrepancy should be smaller now. But, I believe that the reported total is based on the median price of STEEM over 3 1/2 days, so if the STEEM price is volatile, the value at payout time can be very different from the price that's reported on the exchanges. (this can move the actual payout higher or lower, but the discrepancies should balance out over time)
This is from a payout four days ago. In this example, curators are getting almost double. It's the same for every payout among the curator report posts I've looked at. Is this some sort of issue left over from the TRX removal? It seems odd to me that the curator pool would be that large versus the author reward.
ok. Our replies crossed in the mail😉. See my other reply. The web site just doesn't report the beneficiary numbers after payout time, but it did split 50/50.
That was what was confusing me. I'm glad I asked now; thank you for explaining. I was looking at it wrong. But see, it's things like this that go unanswered that further hurt this site. Without a way to ask questions, it gets left up to the imagination, which = people leaving in the process once frustration fully sets in.
Yes, after that I also moaned a lot ;-)) By the way, a good place to ask questions and get well-founded answers is the Talk Channel in the WOX... Mostly no one uses it.
Yeah, we can definitely use better documentation and places to ask questions. These are the sorts of things that we can do in a decentralized way if we grow the community enough, but it's sort of a catch-22. In the meantime, I'm happy to answer questions like these whenever I am able.
Coincidentally, this reporting with beneficiary rewards is exactly why I calculated the total the way I did in the browser extension. Here's what the curator's overlay shows for that post:
As described above:
I've thought a lot about overrated content and I've come to the conclusion that it can't be fought. The only way out is to remove the need to publish such nonsense. A fundamental change is needed for investors to receive their returns without having to post. This can be implemented in different ways. For example, there could be an account to which investors would delegate their SP and receive their percentage without publishing posts. You can even set the minimum delegation limit to, say, 100,000 SP.
I agree that it can't be eliminated, but I think it can be fought, and the problem can be reduced. This is why I talked about "adding friction". We can't eliminate it, but we can shift the balance.
This makes sense in concept, but I'm not sure if it's really practical to implement. If investors can receive these rates of returns without posting, nobody would curate. If you lower the rates, then the pollution returns. Definitely something to think about, but there are a lot of implementation details to work out.
Yeah, but delegation is useful for other things, too (like maintaining a cold-ish wallet). Also, it's always possible to just pay for votes, directly. I mean, the bots could even price their votes in BTC or TRX or some external token if they wanted to.
Your comments are very valid and emphasize once again that the problem is so complex that it is not so easy to come up with a solution. I think it is necessary to make curation so attractive that there is no need to publish garbage content. For this, you can come up with different solutions. Perhaps it is worth increasing the percentage of payments to curators, perhaps these payments could also be liquid. If we had a consensus that we want to change something and can do it, then this discussion would be much more interesting 🙂
I still like the idea of separating investors and bloggers. So - bloggers create the content that makes up Steem as a platform. Investors curate these blogs (whether manually or as a trail...) and so authors are rewarded based on quality and likes. Investors, for their part, can NOT blog with their accounts, so they cannot vote themselves or buy such votes. In return, they receive a variable (price-dependent) return...
As a concept, I like this idea, but I have trouble thinking how it could be implemented on a decentralized blockchain when an investor can always set up multiple anonymous accounts. Two of the core ideas that Steem was pushing when it launched were censorship resistance and anonymity. Both of those principles that were designed into the chain make it very hard to limit someone's capabilities based on their identity.