Preheat oven to 350 degrees F.

Use a paring knife to cut the plums in half through the center, twist and separate. Remove the pits. If the pits don't come out with ease, use your paring knife to cut them out. Set aside.

Sprinkle the brown sugar to cover the bottom of a 9-inch cake pan. Dot 2 tablespoons of the butter over the brown sugar. Set the pan directly on a burner of your stove over low heat. Allow the butter and sugar to melt together, swirling occasionally so nothing burns. You want the butter/sugar mixture distributed evenly over the bottom of the pan. Use a spatula to help spread the mixture, if needed.

Place the plums cut-side-down into the cake pan so the fruits are touching. Start with the outside perimeter of that pan and work your way to the center. Set aside.

Put the whole-wheat pastry flour, almond flour, baking powder, and salt in a medium bowl. Stir with a fork and set aside.

Use an electric mixer to cream the butter and sugar together in a large bowl until smooth. Add the eggs and mix until thoroughly blended. Add the buttermilk and vanilla and mix again. If the batter is a little curdled looking, don't worry.

Add the flour mixture and to the butter/sugar/egg mixture and mix together into a creamy, smooth batter.

Pour the batter over the plums. Set in the oven with a baking sheet beneath to catch any plum juices that may spill over.

Bake until a toothpick inserted into the cake comes clean, about 45 to 50 minutes.

Remove from oven and cool completely in the pan, at least an hour.

To remove cake, set a plate large enough to cover the pan on top. Flip the plate over so the cake inverts onto it. Lift off the cake pan.

Serve with lightly sweetened whipped cream or Greek yogurt.