Preheat the oven to 350°F (175°C). Grease and flour a 9x13 inch baking pan.
In a large mixing bowl, combine the mashed sweet potatoes, sugar, vegetable oil, and eggs. Mix until well combined.
In another bowl, sift together the flour, baking powder, baking soda, cinnamon, and salt. Gradually add this dry mixture to the sweet potato mixture, alternating with the milk. Mix until just combined. Do not overmix.
In a small bowl, mix together the brown sugar and ground cinnamon for the swirl.
Pour half of the cake batter into the prepared baking pan. Sprinkle half of the cinnamon-sugar mixture on top, and if desired, add half of the chopped pecans. Pour the remaining batter over the top, then sprinkle the rest of the cinnamon-sugar mixture and pecans, if using.
Using a knife, swirl the batter gently to create a marbled effect.
Bake in the preheated oven for 40-45 minutes, or until a toothpick inserted in the center comes out clean.
Allow the cake to cool in the pan on a cooling rack for at least 15 minutes.
While the cake is cooling, prepare the glaze by whisking together the powdered sugar, milk, and vanilla extract in a small bowl until smooth.
Once the cake has cooled, drizzle the glaze over the top and let it set before serving.