Preheat the oven to 350°F (175°C) and line a muffin tin with paper liners.
In a large bowl, whisk together the flour, granulated sugar, baking powder, salt, and ground cinnamon.
In another bowl, combine the melted butter, milk, eggs, and vanilla extract. Mix well.
Pour the wet ingredients into the dry ingredients and stir until just combined. Do not overmix.
In a small bowl, mix together the brown sugar, ground cinnamon, and softened butter to create the filling.
Spoon half of the muffin batter into the prepared muffin cups, then add a teaspoon of the cinnamon filling on top of each.
Top with the remaining muffin batter, covering the filling completely.
Bake for 18-20 minutes or until a toothpick inserted into the center comes out clean.
While the muffins are baking, prepare the glaze by whisking together the powdered sugar, milk, and vanilla extract until smooth.
Once the muffins are done, allow them to cool for 5 minutes in the pan before transferring them to a wire rack. Drizzle the glaze over the warm muffins.