Why does a gas inside a closed container experience atmosperic pressure when the gas itself is not in contact with the atmosphere?

I know this is a very basic (and probably stupid) question, but I have been confused about this for a long time.

Answer

Both Ignacio and Anna's answers are correct, but let me see if I can expand on them a little.

You ask:

Why does a gas inside a closed container experience atmosperic pressure when the gas itself is not in contact with the atmosphere?

but actually a gas inside a closed container doesn't necessarily experience atmospheric pressure. If you've ever shaken a bottle of pop then opened it you'll know that the gas pressure inside is a lot higher than atmospheric pressure. This is an example where the pressure inside the closed container is higher than atmospheric pressure, but it can be lower as well.

What happens to the gas in a container depends on how flexible the container is. If I take a glass bottle and pump half the air out, the gas inside will stay at half atmospheric pressure. That's because the glass walls of the bottle are rigid and they take some of the pressure.

On the other hand if I fill a plastic bag with gas then pump some out, the pressure inside the bag stays at atmospheric pressure. That's because the plastic bag is flexible. The atmosphere presses on the bag and the bag transmits the pressure to the gas inside it.