Gridcoin 5.4.0.0-mandatory "Kermit's Mom" release

[5.4.0.0] 2022-08-01, mandatory, "Kermit's Mom"

https://github.com/gridcoin-community/Gridcoin-Research/releases/tag/5.4.0.0

The Gridcoin core developers are proud to release 5.4.0.0 which represents the
Kermit's Mom milestone. This is a mandatory release and includes the long-
awaited implementation of Manual Reward Claims (MRC) and significant
improvments to poll and project structures among many other enhancements and
fixes.

Highlights

Manual Reward Claims (MRC)

The importance of this release to the community cannot be overstated. One of the
most significant challenges to new "crunchers" joining the Gridcoin community
is the payment of the 2nd layer crunching rewards to the crunchers for work they
are doing for network approved BOINC projects. The implementation of constant
block rewards (CBR) a number of years back improved the participation of wallets
in the staking process dramatically, raising the average difficulty on the
network from about 3 to greater than 18 today, representing more than
180000000 GRC continuously online and staking. This has drastically improved
security of Gridcoin, making it much more resistant to 51% attacks, but has had
the downside of increasing the effective minimum GRC required to solo crunch
to 5000 GRC or more. Without MRC, which is in this release, solo crunchers are
required to stake to get paid their rewards.

Most people regard having enough GRC to stake on average once a month is the
practical minimum for solo crunching, so 5000 GRC is effectively the minimum
bar for entry for solo crunching. Needless to say this is a giant challenge
for many newcomers and greatly harms the growth of the network. The pools help
with the problem by providing a way to earn GRC and eventually collect enough
to solo crunch, but they have a drawback in that the project research when pool
crunching is done in the name of the pool, not the cruncher. MRC sweeps that
giant hurdle away. With MRC you can solo crunch with only about 1 GRC to start.
You simply need enough to send a beacon advertisement (which is the new solo
cruncher's way of letting the network know that they want to get rewarded for
crunching). MRC will now allow a solo cruncher with an active beacon to request
payment at intervals as small as 14 days from the last time they were paid.
There is no minimum payment threshold, only the minimum time threshold. This
request for payment is a "manual reward claim" request, which is where the name
MRC came from originally. Note that what is going on under the hood is that the
requester is sending out a "MRC" request contract that is attached to a
transaction in the Gridcoin blockchain, which is picked up by the next staker,
and causes the staker to stake the reward for the requester as a delegate.
Essentially MRC is a form of delegated staking. Note that for this privilege
the requester will pay a portion of their reward as a fee to the staker and the
foundation. The total fee starts at 40% of the rewards at 14 days and then
declines as 14/(number of days) as the interval between payments is increased.
The foundation gets 80% of the fees and the staker 20%. These ratios were set
and approved by official poll
(id 651a3d7cbb797ee06bd8c2b17c415223d77bb296434866ddf437a42b6d1e9d89,
https://github.com/gridcoin-community/Gridcoin-Tasks/issues/250).

To give a specific example, a new cruncher registers their beacon and starts
crunching. They have a pretty good computer and their magnitude has been
building to the point where at 14 days they have about 35 GRC pending as
rewards. At 14 days after the activation of their beacon, the MRC payment
button on the main screen will allow them to request payment of their rewards.
If they press the button at 14 days, then the requester will get 35 * (1 -
0.40) or 21 GRC. The foundation will get 0.80 * 0.40 * 35 or 11.2 GRC and the
staker will get 0.20 * 0.40 * 35 = 2.8 GRC. If the cruncher waits until 28 days
to request the reward, lets assume his or her magnitude is constant, so the
rewards have doubled, so now the requester has 70 GRC pending. If they press
the button at 28 days, then the requester will get 70 * (1 - 0.20) or 56 GRC.
The foundation will get 0.80 * 0.20 * 70 or 11.2 GRC and the staker will get
0.20 * 0.20 * 70 or 2.8 GRC. Notice that the effective fee to the foundation
and the staker at 28 days is the same as at 14 days. This is by design. The
foundation and staker fees decline as a percentage from the point of view of
the requester's total rewards as the time between payment requests increase
beyond 14 days, but from the point of view of the foundation and staker, the
fees essentially are computed at the 14 day point and remain capped as absolute
numbers.

The MRC request button will appear on the wallet main screen for crunchers
with active beacons and will allow MRC requests once the block v12
functionality block height is reached. This is the post Kermit's Mom release
hardfork point, and it will be at block 2671700, which will occur on or about
August 31, 2022.

Some resources to review for MRC and related concepts:

• Solo crunching guide: https://gridcoin.us/guides/boinc-install.htm
• Staking: https://gridcoin.us/wiki/staking.html
• Beacons: https://gridcoin.us/wiki/beacons.html
• MRC core functionality:
  https://github.com/gridcoin-community/Gridcoin-Research/pull/2425
• MRC GUI functionality:
  https://github.com/gridcoin-community/Gridcoin-Research/pull/2513

Polls

Kermit's Mom has significantly improved both the rigor and functionality of the
polling system. Poll categories were included in the GUI rewrite a while back,
but those categories were GUI only and were not enforced by protocol at the
blockchain level. At the Kermit's Mom hardfork block (2671700), polls will now
use the poll payload v3 contract and associated protocol code, which will
enforce and validate poll categories and the validation rules associated with
them. Minimum poll duration and minimum vote weight percent of active vote
weight, along with poll type requirements are now encoded and/or enforced by
protocol. The GUI will display a "validated" tag for polls that have met
validation requirements for their given poll type. Polls that fail validation
requirements by the end of the poll will show "invalid". Polls with invalid
combinations of poll type, duration, and poll weight type cannot be created and
will also be validated by all nodes and rejected if an invalid combination is
received.

Polls also support the inclusion of additional fields, and the PROJECT poll
type now requires the additional fields of project_name and project_url in
the poll. This is in anticipation of driving automated whitelisting
functionality in a future release.

For more information, please see:

• Voting: https://gridcoin.us/wiki/voting.html
• Whitelist: https://gridcoin.us/wiki/whitelist-process.html
• Poll validation:
  https://github.com/gridcoin-community/Gridcoin-Research/pull/2522
• Poll GUI additional fields components:
  https://github.com/gridcoin-community/Gridcoin-Research/pull/2525

Projects

Kermit's Mom has implmented the inclusion of a GPPR controls required flag in
the project structure. After the Kermit's Mom hardfork point (2671700), all
project entries for the whitelist will require the GDPR protection flag to be
set to either true or false to indicate whether the project requires the user
to give permission for project statistics for that user to be exported. This is
used to drive a new column in the projects tab table in the researcher
(cruncher) wizard, and indicate a possible situation where a cruncher may not
have granted the necessary permission.

For more information, please see:

• Whitelist: https://gridcoin.us/guides/whitelist.html
• GDPR Protection Display:
  https://github.com/gridcoin-community/Gridcoin-Research/pull/2527

libsecp256k1 port from Bitcoin Core

This one is not visible on the outside to anyone, but is a major improvement to
Gridcoin's plumbing. @div72 did a fabulous job porting over Bitcoin Core's
libsecp256k1 to Gridcoin. This replaces OpenSSL for all elliptic curve
cryptography on the blockchain itself with Bitcoin's more efficient (and more
secure) version. OpenSSL is now only used by the CURL library for https
communication with external websites in the scraper project download. This will
remain as CURL is well encapsulated and well maintained.

There were many other improvements and fixes that are in the Kermit's Mom
release. Please refer to the change log below for details. This release was a
group effort. Thanks to everybody who submitted a PR merged into the release. A
special thanks to @barton2526 for doing many backports/fixes to the build
system, Qt version bumps, and other utility backports. Thanks to @Pythonix for
submitting backports and fixes to the network code. And thanks to @iFoggz for
submitting enhancements to the scanforunspent rpc. A special thanks to m3prx for
submitting the winning icon for MRC submission used in the GUI.

A teaser of the major feature to look forward to for the next milestone,
LaVerne: HD wallet support! This is another long sought piece of
functionality that will allow folks to store one keyphrase to regenerate all
keys if necessary and provide the foundation for proper hardware based wallet
functionality.

Jim Owens

[5.4.0.0] 2022-08-01, mandatory, "Kermit's Mom"

Added

• test: Add TrimString(...) tests #2447 (@barton2526)
• test: Add dead code detection #2449 (@barton2526)
• test: Add explicit references to related CVE's in comments #2467 (@barton2526)
• test: Add testing of ParseInt/ParseUInt edge cases with leading +/-/0:s #2470 (@barton2526)
• consensus, contract, mining, researcher, rpc, staking, gui: Implementation of MRC - baseline functionality #2425 (@jamescowens)
• consensus: MRC mandatory implementation code #2471 (@jamescowens)
• test: Add upstream sync_tests.cpp #2481 (@barton2526)
• net: Countermeasures against eclipse attacks #2454 (@Pythonix)
• lint: add script to check for https violations #2491 (@div72)
• util: Add flatpath BOINC data directory path resolution for Linux #2499 (@jamescowens)
• gui: Add beaconExpired() to researchermodel #2498 (@jamescowens)
• consensus: Add missing block nVersion check for v12 blocks in AcceptBlock #2502 (@jamescowens)
• gui, util: Add AccrualChangedFromStakeOrMRC core signal #2503 (@jamescowens)
• util: Change default -dbcache to 100 MB and also implement -txindexdbcache #2507 (@jamescowens)
• rpc, util, consensus: Implement exception handling framework for MRC and fix ValidateMRC to deal with testnet consensus issue #2508 (@jamescowens)
• gui: Initial implementation of GUI MRC submission form #2513 (@jamescowens)
• build: Port over Bitcoin's translation docs #2439 (@jamescowens)
• [2/3] build: integrate libsecp256k1 #2492 (@div72)
• gui: New MRC request icon #2526 (@jamescowens)
• mandatory, voting: Implement poll type validation in protocol #2522 (@jamescowens)
• gui, voting: Implement poll additional fields gui components #2525 (@jamescowens)
• gui, researcher: Add GDPR protection display #2527 (@jamescowens)
• consensus, rpc: Kermit's mom hardfork (2671700) #2551 (@jamescowens)

Changed

• net: Hard Coded Seed Node Cleanup #2427 (@barton2526)
• script: Add More Generated Files to Gitignore #2435 (@RoboticMind)
• gui: Update copyright year to 2022 for Gridcoin About dialog box #2443 (@jamescowens)
• rpc: Change type field in ListTransactions to lower case #2441 (@jamescowens)
• refactor: Replace memset calls with array initialization #2452 (@barton2526)
• refactor: Changed some parameters from pass by value to pass by reference #2455 (@Pythonix)
• ci, cd: improve caching #2461 (@div72)
• contrib: port recent macdeployqtplus changes #2465 (@div72)
• test: Test for expected return values when calling functions returning a success code #2464 (@barton2526)
• build: Improve error message when pkg-config is not installed #2460 (@barton2526)
• test: Bump shellcheck, mypy versions #2463 (@barton2526)
• build: Update depends packages (expat, fontconfig, freetype, libXau, libxcb, xcb_proto, xproto) #2466 (@barton2526)
• lint: run mypy over contrib/devtools #2475 (@barton2526)
• build, lint: Remove x-prefix's from comparisons, Fix some shell script issues the linter complains about, Re-enable boost include checks #2478 (@barton2526)
• test: Avoid copies of CTransaction #2479 (@barton2526)
• ci: change windows CI to Focal, modify wrap_wine to use wine64 for 64bit binaries #2484 (@barton2526)
• build: Qt 5.15.2 #2486 (@barton2526)
• net: No longer send local address in addrMe #2459 (@Pythonix)
• voting, gui, rpc: Enhance PollResult and AVW calculation to improve pool handling #2489 (@jamescowens)
• [1/3] refactor: port some misc changes from upstream #2485 (@div72)
• build: Try posix-specific CXX first for mingw32 host, Fix Windows cross-compiling with Qt 5.15 #2494 (@barton2526)
• Improve upon scanforunspent rpc #2468 (@iFoggz)
• rpc: Change tail_fee and head_fee to display in GRC rather than Halfords in createmrcrequest #2501 (@jamescowens)
• scripted-diff: change http to https in copyright text #2504 (@div72)
• qt, refactor: Use enum type as switch argument in *TableModel #2496 (@barton2526)
• build, qt: bump Qt5 version to 5.15.3 #2510 (@barton2526)
• utils: run commands using utf-8 string on Windows #2514 (@barton2526)
• prevector: enforce is_trivially_copyable_v #2516 (@div72)
• crypto: Unroll the ChaCha20 inner loop for performance #2515 (@div72)
• gui: Modify VerifyTCPPort to use the status of CheckOutboundConnectionCount #2506 (@jamescowens)
• gui: Fix transaction history table column size behavior #2520 (@jamescowens)
• log: Use consistent wording in random.cpp log #2538 (@div72)
• lint: Use newer versions of our lint packages, remove yq #2541 (@barton2526)
• [1/2] validation: move CBlock validation methods to validation.cpp #2539 (@div72)
• gui: Implement proportional column resizing for Addressbook with memory #2543 (@jamescowens)
• build: remove redundant warning flags #2546 (@barton2526)
• qt: Prefix makefile variables with QT_ #2547 (@barton2526)
• build: remove build stubs for external leveldb #2550 (@barton2526)
• build, refactor: Improve package version usage #2549 (@barton2526)
• build: minor boost tidyups #2548 (@barton2526)

Removed

• rpc, util: Remove caching from BlockFinder #2490 (@jamescowens)
• test: remove obsolete check sig test #2552 (@div72)

Fixed

• build: fix unoptimized libraries in depends #2428 (@barton2526)
• build: don't use deprecated brew package names #2429 (@barton2526)
• qt: fix shutdown on MacOS #2440 (@div72)
• net: Do not add random inbound peers to addrman #2451 (@barton2526)
• util: skip trying to set the locale on NetBSD #2448 (@barton2526)
• build: change bundle id #2462 (@div72)
• net: Do not propagate obviously poor addresses onto the network #2453 (@Pythonix)
• ci: Fix CI build title to reflect that we are building for bionic, not xenial #2469 (@barton2526)
• lint: Fix misc typos #2472 (@barton2526)
• util: Fix crash when parsing command line with -noincludeconf=0, Properly handle -noincludeconf on command line #2473 (@barton2526)
• build: Fix several minor linter errors #2476 (@jamescowens)
• tests: Don't access out of bounds array index: array[sizeof(array)] #2480 (@barton2526)
• script: Fix and Minify Icon SVG #2488 (@RoboticMind)
• gui: Add missing null pointer check for m_beacon #2500 (@jamescowens)
• util: Fix BN_zero macro in key.cpp for OpenSSL 3.0 #2497 (@jamescowens)
• rpc, contract: Adjust ValidateMRC, CreateMRC, and createmrcrequest to correct provided fee handling #2505 (@jamescowens)
• consensus: Move DoS into contract validators to allow variability of DoS based on context and further fixes to ValidateMRC #2512 (@jamescowens)
• lockedpool: When possible, use madvise to avoid including sensitive information in core dumps #2509 (@barton2526)
• refactor: Fix some minor linter complaints #2517 (@jamescowens)
• miner: Miner Logger bug fix #2518 (@iFoggz)
• key: properly parse short DER private keys #2519 (@div72)
• researcher: Fix ReadClientStateXml() crash on wrong BOINC directory permissions #2524 (@jamescowens)
• init: fix daemon forking #2521 (@div72)
• scraper: Change open mode from append to truncate for auth file #2528 (@jamescowens)
• gui: New mrc contract icon try #2 #2529 (@jamescowens)
• gui: Remove white outlines on MRC icon #2530 (@a123b)
• voting: Change m_additional_fields serialization #2531 (@jamescowens)
• build, qt: Fix QMAKE_CXXFLAGS expression for mingw32 host #2537 (@div72)
• logging: fix logging empty thread name #2535 (@div72)
• trivial: fix comment in account header guard #2542 (@div72)
• build: Restrict check for CRC32C intrinsic to aarch64 #2544 (@barton2526)
• build: force CRCCheck in Windows installer #2545 (@barton2526)