The problem would be how to eliminate the automated comments.

Yeah, things just become increasingly complex when we try to consider everything. The bots that just do [almost] the same comment over and over again can probably be eliminated without too much difficulty. As can really short comments like 'Thanks' or 'Good job'. Or there could be a human element where moderators look for people with over a certain amount of comments and just manually reduce the rank.