Where does the energy of a magnetic field come from?

If I were to hold two magnets apart, north facing south, close enough that they were drawn together, I would need to do work to them from falling together. I would need to expend a certain amount of energy to keep them apart for a certain amount of time. The magnets oppose the force I apply to them. How is it that the magnets can continue to oppose my force indefinitely? How is it that they can 'do work' forever? There may be a simple answer, but I'm baffled.