Preheat your oven to 350°F (175°C). Grease a 9x5-inch loaf pan and set aside.

In a medium bowl, whisk together the flour, baking powder, baking soda, salt, cinnamon, and nutmeg.

In a large mixing bowl, cream the softened butter and granulated sugar together until light and fluffy, about 3-4 minutes.

Add the eggs one at a time, mixing well after each addition. Stir in the vanilla extract.

Gradually add the dry ingredients to the wet mixture, alternating with the buttermilk, starting and ending with the dry ingredients. Mix until just combined.

Fold in the diced apples and nuts (if using) until evenly distributed.

Pour the batter into the prepared loaf pan. In a small bowl, mix together the brown sugar and cinnamon for the topping, then sprinkle it evenly over the batter.

Bake for 55-65 minutes, or until a toothpick inserted into the center comes out clean.

Allow the bread to cool in the pan for about 10 minutes before transferring it to a wire rack to cool completely.