At the end of the day the biggest influencers for college athletics aren't the fans but the big money boosters