Mutualistic species vary in their level of partner specificity; yet, the evolutionary mechanisms which underpin partner specificity and generalism are not yet fully understood. One factor which may underpin variation in specificity is the degree of antagonism/cooperation in the relationship between hosts and symbionts. It is possible that mutualist hosts cooperatively specialize, maximizing mutual symbiotic benefit with a preferred symbiont, or antagonistically specialize, maximizing resource extraction from a preferred symbiont. Specialization in a preferred symbiont reduces the benefit of association with non-preferred symbionts, while generalists receive similar benefit from all symbionts. Here, we employ evolutionary game dynamics and adaptive dynamics in order to assess the evolutionary stability of cooperative specialization, antagonistic specialization, and generalism. When hosts specialize cooperatively, our system is bistable, favoring one of the specialist hosts and its preferred symbiont. When hosts specialized antagonistically, host and symbiont frequencies cycle continuously when average specialist payoff is greater than average symbiont payoffs. Higher average generalist payoff causes generalism to be an evolutionary stable strategy. Cooperative specialization unilaterally favors greater cooperation between specialist hosts and preferred symbionts, while antagonistic specialization leads to an evolutionary arms race in which symbionts attempt to escape host exploitation. We conclude that the cooperation-antagonism continuum which exists in mutualisms may play a key role in determining the pattern of partner specificity which develops within mutualistic relationships.