Do I need my ex’s permission to enroll my child in swimming lessons during my parenting time?

No. You do not need your co-parent’s permission to enroll your child in extracurricular activities during your parenting time. The decision to permit participation is a parenting decision.  So long as a parent has sole or joint legal custody, they may enroll a child.  If there is a dispute about whether the enrollment unreasonably infringes on the other parent's custodial time or if the child's participation is in the child's best interest are issues that a court can resolve if the parents disagree.

The requirement for an agreement between the parents is usually tied to the expense of the activity, so that one parent cannot unilaterally commit the finances of the other parent.  However, since extracurricular activities are a discretionary add-on, there is no absolute right to require the other parent to share the cost (although, in practice, this is usually what courts order).