Scraping and Analyzing the Steemit Trending Page – Blockchain Business Intelligence

in #utopian-io7 years ago

Reaching the Steemit trending page is an awesome achievement. Posts tend to make nice sums of money, however previous analysis shows that to get on trending you must make a nice sum in the first hour anyway on the post. Trending then accelerates the posts exposure.

On the 2nd Aug I scraped the Steemit trending webpage. I did the same again on the 16th October and I did it again on the 21st Dec. Included in this analysis and report is the top 50 posts on trending on each date.

The reason I used web scraping and not the SteemSQL as my main data source is because trending posts are front end and the data is not stored in the blockchain. There is now way to know what post were trending and when from the blockchain. It is also worth noting that to be able to carry out analysis and comparisons on the data, scraping the data at one point in time would give nothing to compare too. So for this analysis I have been collecting data since August.

High Level Overview

3.png

150 posts were pulled over the 3 scrapings. Over the three scrapings there were 107 unique authors. In August there were 39 unique authors. October there were 43 unique authors and in December there were 40 unique authors.

The average votes per post on the trending page was 392 in August. In October it was 210 and in December the average number of posts on the top 50 posts in trending was 510.

In August the average post payout value was $251, this reduced to $165 in October and December is coming in at $369

We can see from the table that some authors on Steemit have multiple posts on trending, in multiple months. 30 of the 107 Authors have had more than 1 post in trending. That’s 28% of authors appear more than once. 72% of Authors on trending only trended once.

Of the 150 posts, 73 of these have come from the 28% of authors that appear more than once. That equates to 48% of the posts from 28% of the authors.

Detailed Analysis

4.png

The table above shows the number of votes by authors and by scraping date. It is clear to see that posts from August and December received a higher number of votes that October. We can see @buildteam have the highest number of votes. This relates to just 1 post that @buildteam got on trending in August. @sweetsssj tops the votes in December, however this vote count is based on two posts

Below the first table shows SBD payout value and number of Posts per author first for Aug, the Oct and finally December. @dan and ned toped the charts in Aug and October with @adsactly topping the chart in December. In December @adsactly appears for the first time in trending with 4 out of the 40 posts.

5.png

6.png

7.png

Below is a breakdown of the Categories in which the posts belong. It is really good to see such a wide spread of categories reaching trending.

9.png

The authors reps for the 150 posts scraped over the 3 scrapings can be show below

10.png

With the exception of one account with a rep of -18, all other authors on the trending page have a rep of 49 or higher. This can be see each month in the chart below, and then below that details the average rep of Authors in trending for each scraping

11.png

12.png

We can see in Aug 8 posts have authors with a rep of 73, in Oct, 11 posts have authors with a rep of 68 and in Dec 7 posts have authors with a rep of 70.

When scraping the trending page, it is possible to get the age of the post. We can see from the table below that the % of posts that were trending on the same day as publishing was 60% in Aug, 78% in October and 46% in December. 84% of posts on trending were published either that day, or the day before.

13.png

Just out of interest here is a word cloud based on the 'clip' of the post shown on the trending page
14.png

Conclusion

It is very difficult to get a post on trending, we don’t need data to tell us that. But what is interesting is that more than 60% of posts trending trend within 24 hours of posting. The faster a post receives high value votes pushing the payout value up, the easier it is to get a post on trending. This is one of the reason people use voting bots and bid bots.

A trending page works well to showcase posts for people that have not yet signed up to Steemit. However when you log in, I believe this page should change and should be more based on the individual users preferences. That way, everyone’s trending page would be different.

As certain data is not included in the blockchain, such a post view count and trending details, I would really like to get my hands on some data from Steemit Inc. Or maybe Steemit Inc could do a data post. I would like to see information such as view count, vote to view count ratio and other view count ratios, geography information, gender information and general traffic information.

What post on Steemit has received the most external traffic? What author brings the most external traffic. There is so much more we could learn from Steemit Inc if we had the data. So anyone in steemit inc reading this, I would be happy to work with you on the data and present some information to steemit users that is not in the blockchain. If you would like to support this idea, please do resteem this post for extra visibility.

What do you think of the data above? What do you think about the steemit trending page? Please do leave your comments and feeback below.

This post has been set to 50%/50% payment. Any SBD received from this post will be used to purchase STEEM at a lower price.

I am part of a Blockchain Business Intelligence community. We all post under the tag #BlockchainBI. If you have an analysis you would like carried out on Steemit or Blockchain data, please do contact me or any of the#BlockchainBI team and we will do our best to help you...

You can find #BlockchainBI on discord https://discordapp.com/invite/JN7Yv7j



Posted on Utopian.io - Rewarding Open Source Contributors

Sort:  
Loading...

Thanks for the very detailed information
You are doing a great job with your articles

thank you very much @greatvideos

You are welcome 😉