Preheat oven to 350 degrees Fahrenheit. Grease a 9x13 baking dish and lay out a layer of French toast in the bottom; you will probably have to cut some pieces in half to cover the entire surface without overlapping (8-9 slices).
Drizzle half the pancake syrup over the French toast.
Spread half the eggs and ham over the French toast, then sprinkle half the cheese over that.
Repeat with the remaining French toast, pancake syrup, eggs, and ham.
Press the hash browns evenly over the ham and eggs, then drizzle with the olive oil. Sprinkle the salt and pepper over that.
Bake for 30 minutes uncovered.
Sprinkle the remaining cheese over the top and return to the oven for 10 minutes.
Turn the broiler on high and bake for an additional 2-3 minutes, or until the cheese is caramelized.
Let sit for 5 minutes before serving. Drizzle with additional pancake syrup, if desired.