Calibrae Day 2 Progress Report
There
is not so much to report today, apart from things not going so well, and the initial offer of places in the Calibrae database, to copy user accounts and balances into it.
As you can see, I have created a basic initial logo with text, over there on the top right, using one of my favourite fonts, the Ubuntu font.
Ubuntu is a concept that I want to implement with the Calibrae project - it basically means humans caring for humans. So for now, it will provide the font for this project's logo.
SS Steemit has hit the HF19 Iceberg, and is listing badly
I also learned how incredibly large the Steem blockchain has grown, so I decided to call a stop to any further accumulation of it, and put out the call for interest in having accounts replicated into the new chain, based on the snapshot of the Steem blockchain.
To be clear, HF19 simply made exploiting the chain more easy and profitable for everyone, instead of just the mostly premined whales, who have been milking every investor and sweat equity contributor for the last year.
Progress with the forking
I had to strip back the Git repository I made, because I need to have a legacy copy of Steemd in here, and to do that, required trickery with Git that I don't know how to do just yet, so, I have to start again, slowly.
I have all the assets saved that I was working on, and they will be re-integrated into the changed repository structure.
First thing I need to do, is get a fully functioning steemd that is happy to not update (I think, just remove its list of seed-nodes) while answering queries based on the data in the snapshot.
The users who are declaring interest, I will be adding them to a script that queries the frozen Steem blockchain, in order to generate the VESTS both liquid and vested, in their accounts, as well as grabbing their reputation scores, json_data, incept date, and all the data included in the user account entries in the blockchain.
I should have that happening tomorrow, and I will steadily add the list of users, and a list of new accounts related to these user accounts, as several interested users have said they want to have friends or family have new accounts within the new chain.
Calibrae needs workers
The most important thing that needs to happen now, is I need people to work with me on the fork. We need C++/Boost people, we need node.js/react people. They need to be willing to volunteer their time on this, unless someone is going to step up and donate funds to a pool. If that happens, we will also need a fund manager.
There is a growing momentum building with this project, and it is only a matter of time before we find the people, but I will continue to report on what progress I am making. I think if it was all down to me, it would take 2-3 months. If we have 5 more people, two C++ people, 2 javascript people, and a funds/sales person, we could shrink that to 1 month.
I have sufficient funding and motivation to bear with the 2-3 month timeline, but I'm sure that once more progress is made, people will be onboard SS Calibrae, in the engineering room and bridge, and we will sail this thing into the sunset.
Thanks to all the good people who are giving their moral support in this enterprise, and stay tuned for daily after-action reports from me as to what has been happening.
Here's something scaleable for you:
Waaah! Beautiful!
I'll drop that here:
I like that though I think the green is a little unsaturated. I haven't made a place for these things yet. I will make a repo now.
https://github.com/calibrae-project/assets
If you want to do more of this kind of work, feel free to upload it to the repository. I created a quick-and-dirty textual brand version you can see on these posts, using the Ubuntu font, but maybe you have some ideas for a more appropriate design.
I'll put those versions up directly into the repo.
By the way, when I read the preview, I thought you were posting something about the blockchain scalability problem :)
Have you looked at the golos code?
They don't have the premine.
I know about that already, someone already mentioned it. I'm pretty sure that overall their code lags behind Steem's code, and I want to keep, for instance, the HF19 changes, which I doubt that Golos has made.
Yeah, they seem to still be pre hf17 for most of it.
I've heard about how crappy the Golos Dev team is... They make Stinc's dev team look like a bunch of beavers making dams in fall.
Lol, well, there is that.
I haven't gotten an English source for info, yet.
While on one hand, HF19 was a necessary change, it made the bad incentives 1000x more problematic. They would be wise to make changes I will be making, that will mitigate this. Specifically, using reputation as a vote power coefficient. I am uncertain as yet whether the normalised form of reputation, or its native logarithmic pattern should be used, I'm inclined to think the native value would be better. It would effectively give a limitation effect like the old, geometric curve used in vote weighting before, except using a social regulation scheme. If zero reputation also caused accounts to have severely limited bandwidth, it would allow humans to flag bots down to zero, and basically solve the bot problem.
With the explosion of blockchain size in the last 6 weeks, this change is now past being necessary, and going towards 'too late'.
This sounds awesome, but how can you flag bots down to zero if they don't post anything thought?
True enough, a vote-only bot can't be flagged down. But they also can't be flagged up and boost their reputation above 25, so such a bot is a mosquito, like the one time famous @ asshole bot. That was run by nextgencrypto aka berniesanders. I left out their @ symbols because I don't need that cunt weighing in here.
Furthermore, if the botfarmer paid for the account, that's another $5 worth of market cap, or if they use the 'freemium' free account type that will be added in Calibrae, the bot will be heavily restricted, unable to have Stake delegated, and besides that, with starting rep score, its actual influence level will be even smaller than a 25 rep new bot account on Steem.
The power limiter effect of reputation is calculated by the highest reputation account in Calibrae, which sets the 100% coefficient level. Probably that will be around what is 70 now, which is about 40 billion rep points. This would put the coefficient of a non-upvoted account at around 1% or even less, so loading it up with a million bux worth of Stake would only make it equal to a $10,000 account on Steem. If they want to put that much into the market cap, and by this only gain that much power, well, that's still good for the platform, and the effect of their votes will still be no more than a 20k account at 50% of the top reputation level.
By making reputation a coefficient limiter on vote power, this kind of abuse is made very expensive. The moment the bot makes a squeak, also, the users can pile on high rep downvotes and once it drops under zero, I am thinking the logical limit drops to 1 post a day, or vote, and that's gonna burn the botfarmer-troll...
Steem definitely needs course corrections more than new users at this point.
If you ask me, with 50% growth in the last 6 weeks, in blockchain size, the ship has taken on too much water, the course is only part of the fix, they need to patch up the hull, bail out the water, and replace most of the engine. Or to use more literal terms, they have to stop bots, yesterday, they have to stop the preminer bullies from playing social manipulation games, yesterday, and ya know what, they are doing nothing, and if history is any record, Ned is the captain of the titanic, but not even rearranging the deckchairs.
Definitely worth an upvote and a resteem :)
I'm good with money, but have NO coding skills. I'd be willing to manage the funds as long as it doesn't turn into a full-time job. I have too many other things on the go to take on a full-time job.
Sure, I will keep that in mind. Nobody is offering to donate funds or make an agreement to pay for some specified thing yet, so don't worry, you won't have any big job just yet :)
I've been poking around condenser learning how it works, making small changes. I can do some work on condenser clone for calibrae, hopefully rewarded in Steem or Juice.
Is my understanding correct, that anyone who doesn't show interest won't have their stake nor account information will be carried over into the new database?
That's right. The purpose is to both make this process voluntary, and to thin out the amount of data I have to trawl to build the genesis blocks. I am excluding preminers, so the decent folk have a chance to get to know the platform before the gates open via exchanges, and 'freemium' signup account type, I have to fully specify that yet, it needs to be quite restrictive to prevent botfarmers.
I'm interested as a user.
Sorry, I don't have any programming skills that would be of benefit to you.
We need users!
Also, I intend to make running a witness for Calibrae pretty much point-and-click. There will be binaries, and scripts to make things easier. I had fun, with the endless puzzle of making use of
steemd, but it is such a high barrier of entry that I don't think is reasonable.I also intend to eventually alter the witness scheduling system so that there is no flat payout cycle for top 19s, but a steadily declining rate of block allocations so that the competition for votes is more fierce, and the churn on positions is higher as new users join the election.
Looks like you are really putting some thought into this. I've brought your project to the attention of a few others.
What OSs will you support with your binaries? I'm a Linux man.
Well, linux is easy. I am still tinkering with how to persuade it to build static binaries, I can see easy enough that it probably will just run without integrating the shared libraries into it, on most distros, probably, but I'd like to just have the option of download-and-run, portable, and all (It's written that way, pretty much, if you launched it next to the binary via
./steemd)I'm working on that today, actually... I have the snapshot here on my pc now, it took some time to get 23gb downloaded, so I can finish the changes to the cmake files to make it do static binaries.
As for windows and mac, well, I'll leave the mac binaries to mac people, I haven't the faintest idea how to do that. Hopefully we can get windows binaries going too. There is absolutely no reason why they can't be made. Of course Docker could be used but I'd rather not force people to use that, handy as it can be, it's just another layer of complexity.
Have you heard of the AppImage project? Many software projects are now offering these stand alone packages for Linux now.
http://appimage.org/
Thanks for the tip... if that works like it looks, this is gonna solve a lotta problems.
So, from what I am reading so far about it, this does require that I will have an ubuntu 14.04 environment, which I know it will build for. This is not a difficult constraint to satisfy, sounds like I should be able to just build an ubuntu 14 chroot environment (this may be a little tricky, but doable, in an automated way, like using debootstrap), then write the scripts that do the build inside that, and then voila. Binaries for everyone! Well, everylinux anyway :)
How does this differ from just using docker, which I'm fairly familiar with? You can do that with docker right?
I've been using https://github.com/phusion/baseimage-docker and love it. You can run one or multiple processes, and scheduled stuff. It's a PC in a docker container, and very widely supported by VPS providers, for really simple deployment.
I'm running the same docker container on my laptop, and when I make change, just replace the docker image on the remote machine, it saves state, it's great!
Am I missing some point?
https://www.reddit.com/r/docker/comments/5eptm9/are_docker_containers_cross_platform_or_no/
Docker also lets you run on any x86-64 system as well. I will be for sure updating and uploading a docker to the docker hub also, so it can be as easy as docker run calibrae-project/calibrd. The docker can be thus built on ubuntu 14.04, same as the appimage, and run on any linux, or within a docker.
There has recently been a privilege escalation exploit that can let the code escape from the container though - https://threatpost.com/docker-patches-container-escape-vulnerability/123161/ but since this project won't involve running arbitrary code, it should not be a problem. However, the appimage version will not have this problem run outside a container.
So it will be entirely arbitray, up to the operator how to deal with this.
Excellent! =)
Even popular software such as Gimp is offering .appimage packages along with all of their other standard ones these day. Even Linus Torvolds thinks its a good idea, and he's hard to please.
There's quite a few bits of software that I run this way now, especially if I want to give new development versions a spin, since they may not have landed in the repositories yet. I'm an openSuse user.
I'm looking forward to getting involved when you have something up and running.
BTW, will you establish your own coin for this, or will you use Steem/SBD ?
We will be removing all the abstractions and SBD from the ledger - everything will be a renamed version of VESTS. The ratio of Vests per SBD and STEEM can be found by querying the chain at any given moment, and it changes according to the median of the witness price feeds. The snapshot has a specific state on this set up, and every balance will be queried by its VESTS base, and liquid VESTS will become JUICE and Steem Power (Vested VESTS) will become Stake.
We should have a testnet up and running within a couple of weeks I think.
What does the blockchain JSON compress to? I'd like to do some analysis, including what takes up space, so we can model how much proposed changes would affect blockchain size/performance.
I dunno. It should be protobuf, for space reasons, but I have no idea, to be honest. Let me less the block_log.
It's probably some kind of native Boost binary serialization format. Probably, knowing boost, this format changes after some version and before some version... IDK. but I would use protocol buffers, personally. Likely it's not a lot different.
Ok. My home PC doesn't have enough RAM really, so I may just have a go at running a witness node on your network on an VPN hourly basis for a little while, once you've simplified it a bit, and get it that way.
To run a witness, especially, on a brand new chain, would only require about 2gb of memory, even with a slow spinning disk, at least for backup. But if it was getting a block every minute you might find you want it to have an SSD, and the connection has to be super reliable.
I take a bit of a different angle on reliabilities of witness servers - so long as, I figure, there is 22 witnesses, it doesn't matter if half of them miss blocks a few times a day. I may look into it down the track to have the scheduling system account for this and deprioritise failing nodes automatically and slowly re-add their slots in the schedule.
There is no real issue if the witnesses are recalculating the schedule every few minutes, in reality, it's not that complex a process, I don't believe it impacts on block production. The way it works here, and with the culture, it makes the job very rigid. If 8 out of 10 blocks make it on, then the total capacity of the network is only reduced by 20%, and until it reaches over 3000tx per block do you start to see a ceiling on this. or so. I forget how fast it goes, but the network has never been saturated yet, at 95% or so uptime on nodes.
As a rule, nodes only have something like 6 seconds to prepare to make a block. The schedule is not mapped out very far ahead, for obvious reasons that this would make attacking by schedule, a specific node difficult.
I personally don't think the scheduling system is the optimal solution, at all, but it seems to work so far.
By the way, running a witness, you would not want to rent by the hour, as they have to be running every 3 seconds to grab new blocks. I can show you VPS services that will cost you under 10 euros a month and be ridiculously ample.
If there is anything non-technical you need, I'm here :)
"Woow man love this!