Q. Why can’t Mormons swim on Sunday?
Swimming is something we do mainly for recreation, like sailing, golfing, four-wheeling, and going to the movies. The Lord has asked us to spend one day of our week in worship. Recreation often distracts from this goal. We prefer to find activities that focus our thoughts on the Savior and bring us together as families.
Is there something inherently evil about swimming? No, but we can lose our spiritual balance when we overfill our time with fun-seeking. 
The Lord instructed Joseph Smith, “That thou mayest more fully keep thyself unspotted from the world, thou shalt go to the house of prayer and offer up thy sacraments upon my holy day; for verily this is a day appointed unto you to rest from your labors, and to pay thy devotions unto the Most High” (D&C 59:9–10). Notice that there is a blessing associated with Sabbath observance, and you can choose to claim it or not. Freedom from the ‘spots’ or moral pollutions of the world is more important to me than swimming.
The blessings of closer communion with God easily outweigh the pleasure I might get playing Marco! Polo! in the community swimming pool (called the ‘Municipool’ where I live), especially when I can enjoy a swim on six other days of the week.
The blessings are not reserved only for Mormons, either. Give it a try and see how you feel. See David’s excellent article on Sabbath Day worship to learn how.