Why does the speed of the propellant limit the speed of a space ship in open space?

Isn't speed a relative thing in space? If so, why would the speed of a propellant matter? Why can't a space ship accelerate infinitely?