Steemit is bad at connecting users to relevant content - We need Smart FeedssteemCreated with Sketch.

in #steemit7 years ago (edited)

It's a known fact to all Steemers that here we are having a hard time to find the content we like. We have some basic tools like tags and following to keep track of the content we are interested in but we are still missing tones of good articles and see a lot of crappy ones.

Facing the problem

The reason is that Steemers rarely keep posting about only one category of content with their account. Someone may write an article about the benefits of a multi-sig wallet in Bitcoin and the next day send a cute photo of his dog. Reach to non relevant content kills attraction and weaken the chances of users coming back into platform.

Also if we rely on following we are missing tones of articles that are written by new authors that we still do not follow. So times to times we should go to latest articles section and after ignoring dozens of article we may find a new author we might like (or do we?).

Tags are also killing us in another way. We like them because they categorize content but there is no sorting based on the quality or relevance of article. Hot and Trending section are mostly showing us authors with more popularity or friends in Steem platform. We already know that best articles in Steemit are not necessarily the trending articles. That's fine but we still need to reach to good articles to upvote them right? How to make good articles trending while we cannot reach to good articles at all?

All of this aside, recent growth is making it look like an emergency as new users are having a hard time to reach to their audience since they are simply ignored in the new tab due to the high volume of posts in the platform.

Please not like Facebook

We dislike how corporate tech companies like Facebook or Twitter make us to only see what we like to see. I agree, that's BS. We don't want to get under controlled media again but we want to have better reachability to content we are looking for. That's why we need a different model of personalized feed which probably is not implemented yet. We need Optional Personalized Feed.

How it should look like?

We can have two optional button on top of homepage so users can have the freedom to use the chronological timeline or smart timeline based on their interests. The settings of Smart timeline can appear the first time users click on it and can be accessible on settings menu.

steemit-wireframe.jpg

How it could be?

1. Should be discretionary (optional)

Users should not be obliged to use this feature at all or in any way this feature should not replace the chronological feed for users.

2. Should be simply configurable

Users should be able to configure it in a clear and concise manner. A JSON configurations string (which can be generated for less advanced users by any UI) seems to be a good option.

3. The algorithm should be open source

The algorithm to generate the feed based on the configuration should be open source and well documented so it can be reviewed by anyone. Developers of other apps (like Busy or eSteem) could have the freedom to fork the algorithm and implement their own smart feed based on the same pre-defined configuration standard (in case of implementing it off-chain).

4. The dataset should be protected by encryption and be stored on user's device

The dataset of user's previous engagement and interactions, likes, dislikes, reading time etc should be stored and used by machine learning algorithms to make a real personalized experience for users. In companies like Facebook this dataset is usually stored at centralized databases and used for other commercial purposes like target marketing. It's crucial that users should be able to protect their privacy and reading habits by encrypting this data using their posting WIF and store it on their own devices.
Storing datasets on personal devices has many interoperability and synchronization issues that could be solved by providing centralized tools to access this data. For example UI platforms like Steemit, eSteem or Busy can provide cloud features to store the encrypted data for users in a centralized manner (without having access to keys to decrypt them) but it should be optional for users.

How it should be implemented? (technical)

There are two stages of making a smart and personalized feed. The first stage is to ask user what they want to see? For example users can indicated they want to see posts from a small list of users at first. They want to see posts related to books and politics. These are relatively simple but very important to implement. But the second stage is to gather information about what users actually like by observing their interactions on user interface. If they vote, read or ignore a post, the features of the post should be stored and flagged with the behavior so it can later be used in a classification machine learning algorithm to prioritize the feed for users.

Here I am suggesting some solutions which I think all have their own pros and cons. It would be very good if you can also share your own thoughts about this as I believe non of my ideas are perfect at all.

Off chain, solution 1:

We can keep the blockchain neutral and have user interfaces implement their own version of algorithm. The only problem is that it needs many centralized indexing applications to provide the feed off-chain and the centralized apps should receive the decrypted info of users's engagement data-sets to generate the data so you should trust the centralized app is not storing and using your private data later.

Off-chain, solution 2:

Machine learning algorithms can be implemented on front-end also. Your browser can handle the whole process of storing the engagement data and use it to show you relevant content. However it cannot index data and fetch content in the way you need it because it doesn't have access to the whole database of content to pick the articles you really like. Even so, it can change the sorting and emphasize the content you are interested in and ignore those you probably are not in the normal chronological feed.

This wireframe can be a good way to show how different it is:

steemit-smartfeed-2.jpg

On-chain

First of all, I'm not the right person to propose chain features since I'm honestly bad in it. However, I assume more knowledgable people can come up with a way to have a neutral smart-feed configuration on chain per users. Obviously we cannot store the whole user engagement data on-chain because it's heavy and expensive. However, users can address their datastore along with a hash to their smart feed config json to make it safely accessible to steem's blockchain to use it.

Having an embedded on-chain feature for smart-feeds has a huge advantage since there is no need for any centralized server to index data, and in the same time the chain has full access to chain and can provide feed APIs for all the apps. It omits the flexibility of other Steem UI apps (Busy, eSteem etc) to make their own algorithm of smart feeds, though make it very easy to implement on every platform for users.

Conclusion

Smart and personalized feeds are proved to increase the engagement for users and save their time significantly on internet. For example Google is a simple example of smart and personalized feed. Even so, they are not much popular these days because the algorithms are kept as secret and the data are used for other purposes outside the user's control. I believe we need it for Steemit as soon as possible to move the platform to the next level and I hope we can figure out how to implement this feature with a high respect to privacy and use it as a powerful tool to leverage the connectivity of platform for new users.

If you agree with this feature, please re-esteem so it would be noticed more. Thank you.

Sort:  

These are great ideas @p0o. I guess Steemit being in beta there are a lot of features that could be added like searching for users, keywords, and even a search within a certain profile so you don't have to scroll through thousands of posts to find the one relevant to what you want to read. UpVoted.

hey @p0o!!! This post is dead on. Some of us are okay with wading through all kinds of posts to find the gems, but for most users, there needs to be better ways to access content.

I've been thinking about this a lot lately, in addition to other UI changes.

Last night, I finally got started coding a chrome plugin that begins to address this with resteems on Profile pages. In the process, I've had some thoughts about how one might go about building more robust infrastructure on TOP of Steemit.com within the browser.

Take a look, I outlined my thoughts here. And I think there are some really creative ways we could work on this, without needing permission from anyone.

That's a very nice addition and you are right, we can start and do it small. By the way, in busy we don't show re-esteems in the blog page and have a different page for it. Take a look: https://busy.org/@itchykitten

oh nice!!! I didn't realize we were on the same chain! Who started busy? Thats super cool

@itchykitten - There are several apps being built ontop of the STEEM blockchain. Steemit.com is only one "Window" into the STEEM blockchain itself. Busy.org is also a tool you can use to look/interact with the STEEM blockchain. You should also check out SteepShot if you have an android or apple phone - search the store for "SteepShot". Its an Instagram like app built on the STEEM blockchain currently in alpha testing.

SteemQ is also on the way - it is a decentralized video hosting site like YouTube built on the STEEM blockchain.

I heard SteemQ was cancelled or something.

No, it is still in development.

Oh! Someone made a post about it saying it was dead. If it is still in development that is so great!!

mmmhmmm....!! curious! That's super cool, I knew there were some different 'portals' in, like chainbb, but I didn't know know the others. That's so wild, that we can have a big blockchain fungal mass buried in our internet, and as many different ways of accessing it as people are willing to build.

SteemQ sounds like it could be really useful. Do you know what they're planning on using for storage?

They are keeping it under wraps, will know more soon. IPFS is also going to be integrated into STEEM.

Who is doing SteemQ and how can I follow the progress?

@furion has been working on SteemQ. He is a well respected fellow from our community.

I'm looking forward to see the result ;)

@fabien and @ekitcho started busy months ago and yes it's a really cool project on the same chain as steemit.

please follow me and reward i'll do the same ...thanks!!!!

intermediate steps i see before we could implement your very flexible and smart idea , is something i've been thinking for busy.org, and very easy to do:

  1. tag Filters inside your Feed
    With that, you can navigate on your feed (posts from ppl you follow) on a specific tag like #photography (with one or more tags)
  2. Custom Feed: Create a feed, give it a name (like a Youtube Playlist name), add filters: tags/author/maybe keywords/.
    With that feature, you could create a custom Feed called "MyBestFriend", with friend1, friend2, friend3, to only display their posts, and another custom Feed named "Photography", where you would add triggers for all posts in multiple tags related to photography #steemitphotochallenge #picture #travel

Awesome. I believe permanent tag filter in busy timeline can solve a lot of problems here however the process gets in the way since users should bother with filtering every time their interests change. Also tag filters in the main timeline doesn't show new posts from users we don't follow so both of you solutions are needed to be implemented to fix it. I don't think really doing it with smart feeds would take much more time since the ML algorithms are already available.

@ekitcho, I like how you're thinking! I don't wonder if it might be confusing to to be able to access a raw tag versus filtering your feed through a tag. But you're right, we need to get specific. Maybe it's a silly one but what about..

  1. Being able to view other people's feeds?
    There are lots of friends whom I really like on steemit...and I can view their following list, but wouldn't it be sweet to drop in on someone elses feed for a moment?

  2. average length of comments
    I don't know what level you could do this on, it might have to be onchain, or pre-processed by a server running steemd...but that would be pretty cool to filter a feed by posts that have longer average comments

  3. multiple $$$ levels of trending?
    what if you isolated a few levels of trending, like you could click on $5, $25, $125, $500+, and it would show you all the 'trending' posts that are around that reward level. part of the problem now it seems is that you can get whatever support you garner from your followers, but after that, your sources of fresh eyes are either resteems, or trending. Maybe this would help some of those smaller and inbetween posts get some fresh eyes, and it would be a great place for curators to scout too

  4. sort by upvote and comment percentages?
    so posts that have a really high percentage of upvotes/comments per views might show up higher in this sort

just a few thoughts

I like your ideas most.

thankyou swiss! I thought they were pretty good myself =)

Great post @p0o!

I too, feel like missing on a lot of good content because let's face it, a ton of good content gets burried pretty quick if you're following a lot of people (like myself).. so your suggestion is a thing that could be experimented with and put to the test, I believe it should help a lot of people including myself.

On the other hand, I think with the introduction of 'communities' and the ability to have sort of 'subreddits', it will help a lot in having things organized. I cant wait till we have that feature to be honest, I think it will change a lot of our experience with steemit. So lets wait and see!

Yeah, I think it will be huge. Moderated communities of some sort or another would go a long ways.

Definitely! I honestly can't wait till the race of setting up communities begin. I have a lot of interest in many very unrelated topics, so let's just try to be patient and wait for it :)

I've been using this site for one day and I feel like so much of it is fake. There's really good content here but it feels too commercial most of the time.
It would be nice to see more community here and less self promotion.

have you seen MY FEED?!?!?! :P /s

but actually...it's very true...there are some amazing people here, but they're hard to find

You're onto something here, we need better features to encourage content consumption, right now it feels like everything is designed for creators - which is great, BUT, at some point we need to get more people coming here as consumers!

One devil's advocate sort of note - isn't this how Facebook feed started? First it was an optional, experimental feature, next thing you know it was permanent and could not be taken away. Your ideas about keeping it off-chain would help avoid this, although it doesnt stop the main portal (right now steemit.com) from eventually adopting a "permanent" smart feed that warps who gets seen.

I don't think it would be like Facebook anytime. Facebook has a private centralized and closed-source backend and it's pretty quick and easy to change things. For us, if it's on-chain everyone can see the algorithm and community should approve to adapt. If it's off-chain still the algorithm is visible (since condenser is open source) and if someone doesn't like it they can use another solution like Busy or eSteem.

Yeah, good point.

I think this is a huge reason for building some of these tools into the front end via browser extensions + a steem-rpc call to the blockchain. It would also allow for endless and more personal customization.

yeah i posted something similar about this about profile pictures some time back, i think those kind of curated or smart feeds based on reputation and other factors might end up being outside of the main site which is a shame as i'd also like them here.

i agree humble, but the cool thing is I bet this could be done in a javascript browser extension TO the site. Have you seen the reddit enhancement suite?? super inspiring!!! why should we wait for the devs to make changes, we can do all this on the front end querying the blockchain through steem-rpc.

sweet, if that can be done or discussed then it should be!

I agree, I'm going to be slowly aspiring my way up to something like that. My javascript and software engineering abilities are somewhat fragile :P

Totally agree about smart feed though. How about doing something similar like the search in Quora? I mean there has to be a better way to classify content. I was thinking more in the line of favorite topics on the individual basis.

Now, since everyone of us have different interests, we can simply list the ones we most interested in. That way, there's no need for complex algorithm that "feeds" us content.

What do you guys think about that?

i personally think that some kind of viewer app, a bit like flipboard would better as a third party app, i'm always seeing the main website steemit getting 503 loading errors these days, not sure it could handle doing smart feeds unless it was all client side, the servers need some serious load balancing work moving forward if we want a million users here..

I don't think there's a main server for Steemit since its running on blockchain. There's definitely work to be done. Besides, Steemit don't seem to run well with third party app like Grammarly. I'm quite sure the developers are working hard to get the coding done right.

of course there is a server with steemit.com that does'nt live on the blockchain, it calls the data from the blockchain, but i've seen 503 errors which is normally load issues on the webserver end, imagine hundreds of thousands of people loading the webpage all at once. why would steemit run well with grammarly. it will not be designed for the way that text is displayed on steemit probably so the problem is not with steem it's with the plugin from grammerly. you could always just write in grammerly and come back to steemit to post.

hmm interesting. I didn't know that. Well, I'm not a programmer or coder but I'm interested in learning more about how webpages and the Internet works. Definitely learn something new here.

Having a list of most interested topics with personalized list is totally different. Yes with most popular lists you can attract the crowd but does not necessarily bring a great experience for all of your users. And as I said, algorithms with their data should be controlled by community so instead of it "feeding us" it will "curate for us"

Really good idea. Cleverly thought out! I also feel like I'm missing great content, and people that I want to connect with. I'm spending a lot of time looking for interesting people, likeminded people and content providers. Upvoted and followed!

Thank you very much :-)

@p0o I agree with you here. There should be a filter of some kind my news feed is almost feels like adds now. Thanks for this post.... Have you looked into ONG.social? It's starting to pick up and it might be something big soon. Would like to know your thoughts.

Very interesting dashboard. I like the concept to be implemented for Steem but making good dashboards take time :-)

Excellent post.

Well written and an important subject i wholeheartedly agree with. I love Steemit but it does need to add some more functions to get really competitive in the social media niche, to have a smart feed function as you suggest and give clear examples off is great to see.

Just a clear simple to use filter when searching for posts would be nice as a start, would be awesome to be able to choose different categories/users and make a more advanced search.

Thanks for bringing up the subject and also giving ideas on how to fix it.

Upvoted and Resteemed, ChooChoo!!!:)

Thank you and ChooChoo :-D