Yes, some times it can be that way, depend on the voting strength that your voter applied during voting, there is a level that one will reach in steemit and he/she will have a 'voting slide' which he can slide up to 100% or slide down to 1% during any up vote, so something like that can affect your rewards respectively, sometimes just two up votes from whales with their power slide up to 100 percent during voting can be bigger that 200 votes from minnows put together.