The Perl script I wrote to help me tabulate the votes on the contest I just ran

in #life7 years ago

I was considering sharing the Perl script I had written, but didn't want to clutter up my post about the winners. Someone asked for it in a comment on that post, so I'm sharing it.

I generally write code with lots of comments, so that a year later when the "maintenance coder" (which might be me!) comes to fix something, there'll be a mostly-accurate guide. Now that I've got these concussions, verbose comments are that much more useful -- because now merely "a day later" I've forgotten a lot of what I was doing.

Also, I generally split code into sections, a habit I started way back working in C, and have dragged it forward to Perl, Python, etc. And, another throwback to C, my scripting code generally just has a line like "exit main();" and then all processing happens inside the main() function/subroutine, and it calls out to others. I've also written my own logging module which I have reproduced several times (since the previous one was locked up in the previous job, and I don't steal code like some people steal pets[1]). (Logging module isn't included here, or used -- the script is self-contained.)

Thus, I won't annotate the code any further; I'll just post it below. Oh also, I'll note that I did this in an Ubuntu 16.04.3 VM, running on this Windows 10 laptop. When I went to copy/paste, it "helpfully" added a CR (carriage return; or, an extra blank line) to every line ending. Rather than going through the below and hitting "down, del" 178 times until each line ending was corrected, I just did an "svn up" on the laptop, as I had checked the code in to my personal Subversion repository. Subversion handles the line endings of text files correctly, so once it appeared on the laptop I was able to easily copy/paste, with no line-ending difficulty.

Also, wanted to note that there are many "print" statements throughout the code, so that as I was developing the solution I could see whether things were happening correctly. I left those in, so the actual output of the votees and number of votes, appears after that -- with a line of text indicating the above are the top 50, and the below are the rest.

I use ActiveState's Komodo IDE to edit and run the script while developing it. Excellent Integrated Development Environment! It's commercial, but they also have a free version Komodo Edit, which just edits, doesn't run the script (and I haven't used it, as I bought a personal license for the IDE years ago and have been upgrading every few years -- still on version 9 right now; I think, like Spinal Tap, they've turned it up to 11 :) ).

If you have any questions on anything in the below, post a comment and I'll do my best to clear up any confusion!

#!/usr/bin/perl
=head1 tabulate_votes2.pl

Second script to tabulate votes, to help me with Steemit.  This is for the contest: https://steemit.com/life/@libertyteeth/libertyteeth-community-minnow-support-vote-for-up-to-five-of-your-favorite-minnows-50-will-get-a-5-slot-in-my-steemvoter

1. I do a voting contest, people nominate others, and the contest ends.
2. Gather the votes in a text document, with the voter on the first line, then their votes separated by spaces, commas, and "extra text", finally a blank line.
3. Run this script on it.

From my work log, a full description which I'll massage:

---
The script will need several aspects to it:

1. Detect self-votes and eliminate all votes from that account. Also keep a list of these, so it can present it.
2. Verify that the voter, and each voted account, is a valid account and alert me so I can try various changes to see if I can find the account that was intended.
3. If a voter has more than five votes, then replace the earlier ones with the later ones, so there is a maximum of five votes from any voter.
4. Then sort based on number of votes (likely I'll use Perl, and a hash, with the key being the account name, and the value being an integer, starting at 0 and
   increment for each vote; then sort on the values in descending order and choose the top 50).
5. Verify each voted account qualifies as a minnow, and set aside any account that's "close" like between 100% and 110%, but no more.
6. Then spit out the list of the top 50, and those who were "close", and I'll finish manually. Also, those who self-voted, because it was the first freaking rule! :)

By "finish manually" I mean that there might have been an account which, when voted on, qualified. I might be able to determine that via steemd.com; or, perhaps, using
steemworld.com or steemnow.com. Alternately, I might just say "congratulations on escaping minnow status during this contest!" We'll see.
---

This script will produce the following output, separated into sections, with one line for each unique account within each section:

Actual votes:
account,number of votes

"Close" accounts:
account,number of votes

Self-voters:
account

Actually, this script now ONLY produces the "actual votes" -- this round, there weren't any self-voters, so don't need to account for that; and, to get the "close"
accounts, I need Python in order to access the STEEM blockchain.

=cut


=head2 ALWAYS USE STRICT AND WARNINGS
=cut
use strict;
use warnings;


=head2 MODULES USED FROM CPAN
=cut
use Getopt::Long;


=head2 MODULES WRITTEN IN-HOUSE
=cut


=head2 VARIABLE DECLARATIONS
=cut
my $vote_file; # file with votes, from the post -- format described above


=head2 SUBROUTINE DECLARATIONS
=cut
sub main();


=head2 MAIN FUNCTION
=cut
exit main();


=head2 SUBROUTINE DEFINITIONS
=cut
=head2 main

Main routine, which tabulates the votes and writes output.

=cut
sub main() {
    GetOptions( "vote_file=s" => \$vote_file,
              ) or die "Error in command line arguments.\n";
    die "ERROR: no vote_file specified!\n" if !defined $vote_file;
    open FH, "<$vote_file" or die "ERROR: can't open file [$vote_file]!\n";
    my @lines = <FH>;
    close FH;
    
    my (%voters, %votes, %close, %self);
    # The %voters hash: key will be the voter; value will be an array with up to five elements; remove first if adding sixth.
    # %votes: key=votee, value=number of votes.
    # %close: key=votee, value=number of votes. NOTE: NOT IMPLEMENTING, as I need Python to talk to the STEEM blockchain.
    # %self: key=voter, value=array with up to five elements; remove first if adding sixth (these are who they WOULD have voted for, if one of them wasn't a self-vote).
    my $voter;
    LINE: foreach my $line ( @lines ) {
        chomp $line; # remove trailing '\n'
        if ( $line eq "" ) {
            if ( !defined $voter ) {
                # Could be two blank lines between voters' blocks; or, could be a blank line at the beginning of the file.
                print "Skipping blank line\n";
                next;
            }
            print "END current voter [$voter]\n\n";
            $voter = undef; # saw a blank line while processing; so, we're done processing the current voter.
            next;
        }
        # Okay, we checked for a blank line.  Next, what to do when the line's not blank.  First, if we're NOT processing a voter,
        # then this line is the voter's name, so set it to start processing.
        if ( !defined $voter ) {
            $voter = $line;
            print "BEGIN processing voter [$voter]\n";
            next;
        }
        # If we're here, the line was not empty, and we are processing a voter's votes.  So, extract any "@word" from this text, and add them as votees for that voter.
        my @votees = split /@/, $line;
        shift @votees; # remove the first element, which is "everything before the first '@'"
        foreach my $votee ( @votees ) {
            $votee =~ s/ .*$//; # remove any space to EOL
            $votee =~ s/,$//;   # remove any comma at the end of the votee's name
            $votee =~ s/\.$//;  # remove any period at the end of the votee's name
            print "  voted for [$votee]\n";
            if ( $voter eq $votee ) {
                print "  SELF-VOTE DETECTED!  All votes by [$voter] will be ignored.\n";
                delete $voters{$voter};
                $self{$voter} = 1;
                next LINE; # Stop processing this voter's votes, they voted for themselves.
            }
            
            if ( grep( /^$votee$/, @{$voters{$voter}} ) ) {
                print "  NOTE: already voted for [$votee] so not adding to the array.\n";
                next;
            }
            push @{$voters{$voter}}, $votee;
            # At this point, if the voter has voted for more than five votees, remove one (from the front) so that there are only five.
            if ( scalar @{$voters{$voter}} > 5 ) {
                my $v = shift @{$voters{$voter}};
                print "  MORE THAN FIVE VOTES, so dropping [$v]\n";
            }
        }

        # FYI, there weren't any self-votes, so that's good!  Looks like it processes the vote file (votes.txt) just fine.
        # Next step: output the top 50 that were voted for, as well as the rest -- I will look up whether they are "minnow status"
        # either manually, or with a Python script -- as it requires Python in order to interact with the STEEM blockchain.
    }
    
    # Okay, now we have the %voters hash completely filled out.  So, let's process it into the %votes hash.
    foreach my $k ( sort keys %voters ) {
        foreach my $v ( sort @{$voters{$k}} ) {
            $votes{$v}++;
        }
    }
    
    # Now, do the reverse: make a hash where the key is the number of votes, and the value is an array containing those voted for.
    my %votes_by_num;
    foreach my $k ( sort keys %votes ) {
        push @{$votes_by_num{$votes{$k}}}, $k;
    }
    
    # Now we can output it.  Walk through %votes_by_num, then walk through the array that is each key's value, printing out the account and
    # the number of votes it obtained, as well as a leading count.  If the count reaches 50, say "above are the winners".
    my $i = 1; # counter, so can print another header once it hits 50
    print "Winners:\n";
    foreach my $k ( reverse sort { $a <=> $b } keys %votes_by_num ) { # The "{ $a <=> $b }" does a numerical sort; default is alphabetical, which put "51" after "5"...
        foreach my $a ( @{$votes_by_num{$k}} ) {
            print "$i: $k $a\n";
            $i++;
            if ( $i eq 51 ) {
                print "\n\nBELOW ARE THE REST OF THE VOTING RESULTS:\n";
            }
        }
    }
    print "breakpoint here!\n";
    
    exit 0;
}

 
Enjoy!


[1] -- I haven't followed her TV career, but a very good friend many years ago shared the "Fuck Me, Ray Bradbury" video of hers (also hilarious!), and I looked for others back then and found this one which is ... odd. :)

(And, has a sad explanation of the behavior, towards the end.)




Sort:  

Furthermore to see this work in coding ,, I am thinkg on other point that on the off chance that you are doing coding work well and in the event that you are a programers you have to attempt some learen progressively in the event that you didnot still and make an application on steemit base like regard yet better from regard which give us to warnings on new feedings and posts as message with a notice tune like different applications or cell phone msgs @liberteyteeth

Thank you! Your English is painful to parse but I got the gist of it -- I am working on a script to email me (which could also be an SMS) for new posts, which could also be massaged to recognize and react to comments from others.

I just sent @armandocat 5 SBD -- okay, I need to stop for a moment. As I've got the Steemit More Info plugin for Chrome running, and while typing the beginning of this paragraph, inadvertently found an Easter Egg in it! :)

Check this out:

Was very weird seeing that animation appear at the top of the screen!!!

Okay, back to the comment: I sent him 5 SBD earlier today to ask if he could split the "Replies" tab into two: "Replies on my posts" and "Replies on others' posts", because I get so many of the former that some of the latter slip through the cracks. It's simple to find the former, as I can click on my posts on my "Blog" feed, which I usually do so I can process many comments at once.

Thus, having "replies that weren't on my posts" is something I need which I am willing to pay for, so I gave him a tip and asked if he could add that feature.

With your comment, I'm realizing that I am pretty close to having written that functionality for myself.

Time just gets away from us... (That's the last line of dialogue in both the book and movie, "True Grit", which was such a good book that I started it over as soon as I had finished it! Of course, a review in the book indicated they had done that, so I might have been prompted, but still -- a masterpiece! :) )

Oh, and for your English -- it might be more readable if you wrote in your first language, and then used an online translation tool to convert to English? (My apologies if that's what you're already doing!)

Let us know when you are done with Post notification script . Email or sms any notification would be cool . I wonder why steemit does not have notifications yet .

In your intoduceyourself post , you have said that you are from Canada !! Why is your english a bit different from Canadians ? Doesn't sound canadian atall , Have you recently shifted to Canada ?

i think your this inicitive have potential to solve a very big problem i want to give thank you for sharing your great work on steemit

Wow, I actually hadn't thought of that! I am considering making this into a service. Thanks!

well though, even in that way it gona helping them to make all that lenthy process to a simple end

While most of the codes in this post isn't all clear to me, (because i've got very limited knowledge on programming), I acknowledge your efforts to add value to this platform in your own unique way.

So i took the "liberty" of making a gif footer image for you... just my little way of saying "Hey, keep it up sir..."

download(8).gif


@pangoli, one of the winners of the latest episode of Community Minnow Support you've got started...

Thank you.

Neat, thanks! If I may ask for a slight change -- pan a little lower on the image (i.e., so one of the "orbs" is fully visible, possibly two or one and a half?), and then move the addition up if necessary so it's over the background?

Sure, i'll get to work....
Okay, done...
download(9).gif

Nice work dude . Thanks for sharing your codes with us . Have you worked on any other projects on steemit ?

Not a whole lot. There's the link in my profile, to http://libertyteeth.xyz/show-steemit-edits from which you can see all edits to a post or comment.

I also published a "Binance Trading Spreadsheet" there: http://libertyteeth.xyz/uncategorized/binance-trading-spreadsheet/ (had an article here about it as well).

I've dabbled with the blockchain using Python but keep having difficulties connecting to nodes, so stopped. I was writing a script to send me an email when someone makes a new post; I switched to web-scraping when the nodes kept going up and down. It's on hold right now because I'm in the middle of adding the feature "if not emailing to a phone, include the entire web page" and the formatting gets thrown off. I asked (in a post here a few weeks ago I think) if anybody knew how to email a web page using Python, but got no solid answers.

Plus BTC went bearish, so @haejin's posts (with excellent analysis) weren't "jump on this coin now!", so I didn't "need" it as much and haven't touched it in a while.

This one I started in Perl, which is my strongest language. Python is pretty neat but I prefer Perl; I get the solution developed faster with it. I'm sure, with enough experience with Python, that'd change, but that's where it is right now. So I'm very glad to have found @hoffmann's posts with Perl Steemit integration! Had some issues with it yesterday but I think I'll look into it more today, he replied to my comments on his blogs and I think I'm unblocked now.

From feedback on this post, I've decided that I will write a service to help "Steemit users tabulate votes for a contest."

I'll need to make it far more robust than it is now; I'll want it to be able to take the URL to the contest, and then either web scrape or STEEM blockchain scrape, rather than (like I did for this one) have the contest runner copy/paste the page and then "fix it up" like I did. So I'll mostly be starting from scratch, but will be able to reuse a lot of the above.

I'll need database integration as well, and a way for people to provide inputs -- i.e., some contests don't allow self-voting but some do, so that should be a variable; some contests only allow winners to be of a certain size, and those sizes can be calculated differently, so I'll need to provide multiple "size calculations" and then multiple variables within each; and probably more than I can think of, which means I'll need to make a "what would you like to see?" blog post.

In fact, I'll start on that right now. Thanks!

How many years how long have you been coding ?

Thanks for sharing this with us.

Someone asked for it in a comment on that post, so I'm sharing it.

Many people will not even bother to read through comments talkless of reacting to them.
You are a role model and you are leading by example

Thank you for your kind words! As the Beatles sang, "We're all doing what we can." :)

well its extremely a decent work a significant number of challenge proprietor was battling with such issue however now with the arrangement given by you it would turn out to be simple for them to choose the champ.

Thanks! I'm making this into a service, you're now the fourth comment here that said I should do so! :)

@magoo-1 always on duty.

Hlw author this guy owner multi ID and always comment by multi ID.This person always try to chest and get your reward.

You can check transations

Oh my God!!! There actually are coders that practice proper documentation!! You should be considered a national resource.

Oh, and now I have to go grab all of her music, because that was fucking hilarious.

You made me laugh out loud with your last sentence! Here's the other one I referenced -- I haven't heard them all, as I mentioned I didn't follow her TV career, so haven't seen any of her output for years.

Thanks for the coding praise as well!

well its really a good work many of contest owner was struggling with such problem but now with the solution given by you it would become easy for them to select the winner.

You and @googlee had the same idea, thanks so much! I am now considering making this into a service.

Thing is, it'll take time (a whole lot of precious time), it's gonna take plenty of time, uh oh:

Haha, neat! The first object to "dance" is the clock -- and it does so aligned with the vocal "time"!!!

Ever since I (today) installed the plugin "YouTube ad blocker" -- YouTube has been glitchy, pausing each video about 10 times. It's doing it right now, as I type, and the song is half over! It should have buffered the whole thing by now. Guess they want us to use competitors! http://d.tube

your work is good and i appriciate your this work to help some ones who need it
and also to see this work in coding ,, i am thinkg on other point that if you are doing coding job well and if you are a programers you should make a app on steemit base like esteem but better from esteem which provide us better features and also quick notifications on new feedings and posts as message with a notification tune like other apps or mobile phone msgs
@libertyteeth maybe my question is out of your post but i just ask you to see your coding,, hope if you will do it sure this work will help every one much more

Thanks! I don't know that I'll do a full interface like eSteem or busy.org; however, from two other comments here, I have decided to make this into a service to help others who run contests on Steemit.

Wow i can see that a lot of work was put in place as coding arent easy at all. Hmm @libertyteeth you are really for the minnows

you are a legend in coding we all knew it ;)