Preheat oven to 350 degrees Fahrenheit. Grease a 9x13 baking pan and set aside.
In a large mixing bowl, combine the flour, sugars, and butter with a pastry mixer, or your hands, until you have coarse crumbs. Reserve 1/2 cup of the crumbs.
To the rest, add the baking powder, baking soda, cinnamon, and nutmeg, mixing thoroughly.
In a separate small bowl, lightly beat the eggs, then add to them the buttermilk and mix.
Fold the buttermilk/egg mix into the other ingredients, mixing until just wet.
Spread batter into the prepared 9x13 baking pan.
Add the nuts to the reserved crumb mix and stir. Sprinkle over the top of the batter.
Bake for 40 minutes, or until a toothpick inserted into the center comes out clean.
Cool slightly, then serve warm.