How to setup and run the EOSIO Genesis Snapshot Generator
These instructions and acompanying video provide detailed steps for creating a genesis snapshot used to bootstrap an EOS.IO blockchain that will reflect the contributions of stakeholder in the EOS tokensale.
Update 15th Feb 2018
We've just heard from the EOSIO development team that there may not be a 'genesis block' as such with the new EOS BIOS process so don't invest too much time with the process we documented here until we all get the update. The GitHub repository is also being updated with new code so the procedure in this post uses now used the0.2.1Testnet Release of the snapshot generator.
We are using the software provided by the EOS.IO development team at https://github.com/EOSIO/genesis and expanding on the high-level installation guide, so any developer or enthusiast can create a snapshot, genesis file and so validate the tools and process.
As we get closer to a community blockchain launch, it’s important for as many people as possible to validate this code, it’s output and ideally develop alternate implementations so we have certainty in the starting stake of the EOS community in any EOS.IO based blockchain.
At a very high level, the snapshot generator queries the Ethereum EOS tokensale contracts to determine all contributions and balances and then outputs this to a comma seprated file snapshot.csv along with a snapshot meta-data file snapshot.json which contains some high-level stats and checksums.
These snapshot files, can then be used to create an EOS.IO compatible genesis file using the 'Genesis Block Generator' at https://eosio.github.io/genesis/
The EOS.IO development team have told us to expect a significant update to the Genesis Snapshot Generator and accompanying documentation at some point. We'll update the post and video when it's available.
Let’s get started…
How to setup and run the EOSIO Genesis Snapshot Generator
Prepare a fresh VM, physical or cloud server with the following specifications:
Ubuntu 16.04 (LTS) Server, 4 CPU cores , 8GB memory and at least 120Gb of SSD storage.
Update unbutu with the latest repository package lists
sudo apt-get update
Install screen
Let's install screen which allows us to keep our processes running with output even if you get a dropped connection.
sudo apt-get install screen
screen –S eos
If you've never used screen before, check out this quick tutorial https://www.mattcutts.com/blog/a-quick-tutorial-on-screen/
Exit the eos screen by typing Ctrl-a then d.
Install parity.io and full sync the ethereum blockchain
bash <(curl https://get.parity.io -Lk) -r stable
Run parity in it’s own ‘screen’ so we can monitor it easily
screen –S parityio
parity --no-warp
It’s very important to use the --no-warp option with parity to be able to create accurate snapshot data
warping will skip over historical data that the snapshot requires, tricking web3 into thinking the chain is synced when it is not.
The current ethereum full blockchain is over 50GB in size and will likely take a few days to sync!
Again, exit the screen by typing Ctrl-a then d, leaving parity to sync.
EOS.IO Genesis Snapshot Build Process
Clone the EOS.IO genesis GitHub respository
git clone https://github.com/EOSIO/genesis
cd genesis
git checkout 0.2.1
Install MySQL
cd
sudo apt-get install mysql-server
When prompted be sure to enter a root password for the mysql root account
Test that the mysql service is running...
mysqladmin -u root -p version
Create the mysql eos_snapshot database and load the schema
Log into mysql command line
mysql -u root -p
Create the eos_snapshot database
CREATE DATABASE eos_snapshot;
exit
Create the eos_snapshot database schema by importing the EOS.IO developed schema definitions
mysql -u root -p eos_snapshot < ./genesis/bin/schema.sql
Check that the mysql schema import was successful
Login.
mysql -u root -p eos_snapshot
and
SHOW TABLES;
You should see the following output:
+------------------------+
| Tables_in_eos_snapshot |
+------------------------+
| buys |
| claims |
| keys |
| reclaimables |
| registrations |
| snapshot |
| state |
| transfers |
| wallets |
+------------------------+
9 rows in set (0.00 sec)
exit the mysql client.
Install Node.js and NPM
Ensure your ubuntu installation can access the correct Node version repositories, this will enable us to install Node.js v6.x
curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
Install nodejs (JavaScript Runtime)
sudo apt-get install -y nodejs
Change to the EOS.IO genesis root directory
cd genesis
Install the 'build-essentials' so we can compile the scrypt 'cryptographic hash' npm dependancy
sudo apt-get install build-essential
Now install/compile the required npm dependencies for the genesis snapshot software that are referenced in package.json (npm is a package manager for JavaScript)
sudo npm install
Set your snapshot generator defaults by creating a config.js
cp config.default.js config.js
edit the config file using your favourite editor
nano config.js
Remember to change the eth_node_path to your home folder / parity defaults.
module.exports = {
period: 1,
include_b1: true,
cache: false,
fallback: false,
overwrite_snapshot: false,
recalculate_wallets: false,
author: "[email protected]",
//ETH node
eth_node_type: 'ipc',
eth_node_path: '/home/rdold/.local/share/io.parity.ethereum/jsonrpc.ipc',
//redis
redis_host: "localhost",
redis_port: 6379,
mysql_db: "eos_snapshot",
mysql_user: "root",
mysql_pass: "eos",
mysql_host: "localhost",
mysql_port: 3306
}
This snapshot config file will create a snapshot for Period's 0 and 1, without the experimental fallback support.
Run the snapshot
From the genesis directory run
nodejs snapshot.js
:~/genesis$ nodejs snapshot.js
prompt: Config file detected, load it?: (true)
.--------------------------------------------------------------------------------.
| Settings |
|--------------------------------------------------------------------------------|
| period | 1 |
| include_b1 | true |
| cache | false |
| fallback | false |
| overwrite_snapshot | false |
| recalculate_wallets | false |
| author | [email protected] |
| eth_node_type | ipc |
| eth_node_path | /home/rdold/.local/share/io.parity.ethereum/jsonrpc.ipc |
| redis_host | localhost |
| redis_port | 6379 |
| mysql_db | eos_snapshot |
| mysql_user | root |
| mysql_pass | 42 |
| mysql_host | localhost |
| mysql_port | 3306 |
'--------------------------------------------------------------------------------'
Starting in 5 seconds.
sequelize deprecated String based operators are now deprecated. Please use Symbol based operators for better security, read more at http://docs.sequelizejs.com/manual/tutorial/querying.html#operators node_modules/sequelize/lib/sequelize.js:242:13
MySQL: Connected
Web3: Connected
Web3 is Still Syncing (At Block #5041978). trying again in 30 seconds
Web3: Synced
Truncating tables, starting clean
---------------------
INFO: Period Map needs to be Updated - not synced, syncing to Period (Today is Period 230)
Generating new row(s) for Period Block Map (Up to Period 1)
This will take a little while
---------------------
Added period 0 to the Period Map :)
{ begin: 3904416, end: 3957867 }
Added period 1 to the Period Map :)
{ begin: 3957868, end: 3962664 }
Syncing Contract State between block #3904416 & 3962664
Syncing Contracts between block #3904416 and #3962664, this may take a while.
.-----------------------.
| 3904416 ~> 3958215 |
|-----------------------|
| Transfers | 3648 |
| Buys | 19967 |
| Claims | 2917 |
| Registrations | 6537 |
| Reclaimables | 0 |
'-----------------------'
Started: 29.86 seconds ago, Average: 55.50 ms/pass
finished
.-----------------------.
| Complete: 3904416 ~> 3962664 |
|-----------------------|
| Transfers | 14127 |
| Buys | 22442 |
| Claims | 8044 |
| Registrations | 7981 |
| Reclaimables | 5 |
'-----------------------'
Started: 40.30 seconds ago, Average: 69.12 ms/pass
Syncing Wallets
...
...
...
....
{ period_map:
[ { begin: 3904416, end: 3957867 },
{ begin: 3957868, end: 3962664 } ],
block_begin: 3904416,
block_end: 3962664,
sync_contracts:
{ buys: 22442,
claims: 8044,
registrations: 7981,
transfers: 14127,
reclaimables: 5 },
total: 15992,
tests:
{ total_supply: true,
daily_buys: true,
negative_balances: true,
validation_balance: true } }
.------------------------.
| Tests Passed |
|------------------------|
| total supply | ✓ |
| daily buys | ✓ |
| negative balances | ✓ |
| validation balance | ✓ |
'------------------------'
Fallback: Skipping Fallback
Snapshot Table Synced
5229 Records Saved to CSV
Generating snapshot.json
Generating DB Table Checksums
Snapshot meta written to snapshot.json
Snapshot for Period #1 Completed.
Genesis
If all has run well, you'll have a 'snapshot.csv' that identifies all the EOS ERC-20 stakeholder addresses along with their registered EOS public key and snapshot EOS balance.
You can find these files in $HOME/genesis/data/ with a folder for each period snapshot you constructed. To see the Period 1 snapshot files we just created :
ls $HOME/genesis/data/1
We'll leave actually creating a genesis block file for another tutorial, but if your keen head over to https://eosio.github.io/genesis/ and use the files you just created as input.
As always, leave any feedback or suggestions/requests for other walkthroughs in the comments.
EOSphere are passionate group of EOS enthusiasts and data centre professionals setup to help in the global community launch of an EOS.IO blockchain later this year. Based in Australia, EOSphere are initially focused on bootstrapping the Australian EOS community by hosting a series of meetups in each state capital.
Follow us on steemit @eosphere and on twitter @ https://twitter.com/eosphere_io
Thanks !!!
Brilliant..... I love it always anytime I hear EOS. Thanks for sharing this info.
Its a great development!!
You guys rock. Love your posts.
Amazing... Nice post..
I would like to learn about this :) but i have no knowledge about coding :|
Can't wait to for the launch.
Thenk you guys for providing the steps here. Think it's brilliant. Will definitely get around to trying it!
Cheers.
good tutorial to getting started eos development? keep up
amazingly good writing, just I do not know where to start?
EOS it is gonna revolutionalize everything....this is really a great concept....although i am not getting it as whole by now...but my interest is developing....