Mahogany would be a poor choice for the top on a Classical guitar. Nylon strings are different from steel, and that difference drives most of the differences in the structure between the two types of guitar. I have to warn everybody that it's going to get technical in a hurry here.
Nylon strings have much higher 'damping' than steel: they dissipate energy faster as they vibrate. There are two reasons for this. One is that nylon, as a material, has higher damping than steel. If you tap on a plate of steel it clangs, while a similar plate of nylon just goes 'thud'. The energy of the tap is wasted very quickly, so the pate won't ring on. Also, nylon strings, because of their lower density, have to be fatter than steel to have similar mass and tension. As they vibrate they have to push more air aside. This doesn't generate sound; it's more like trying to run in knee deep water.
It's in the nature of damping that it tends to 'eat' high frequencies faster than lows. If you actually look at the vibrations of steel and nylon strings you can see this. If you pluck a steel sting and a nylon string in the same place and the same way, the initial wave forms will look exactly the same. They will have the same amount of energy in each partial. A second or so later the nylon string will have practically no energy in the higher frequency range, above, say, 4000 Hz or so, while the steel string will still have plenty of energy all the way out to 8000 Hz or higher.
From the maker's standpoint, this means that nylon and steel string guitars present quite different problems. With steel strings the problem is to get enough low end sound to balance out all of the high frequency in the strings. There are several ways to do this, but the two most basic ones are to make the box bigger, and make the bridge and top heavier. With nylon strings you have the opposite problem: getting the most out of the small amount of treble energy the strings give you. For this you want a smaller box, and, most especially, a lighter top and bridge. Just to put a couple of numbers on this: the ebony 'belly' bridges that Martin uses on their Dreadnought guitars weigh 32 grams or a bit more, and to that you need to add another 3-10 grams for the bridge pins; call it about 40 grams with the saddle. A Classical bridge that weighs half that is bordering on heavy, in many maker's judgment.
The top weighs more than the bridge, of course. In making the top for a guitar the thing that limits how light and thin you can make it is the stiffness, particularly along the grain. Tops fold up over time under the load of the strings, but you will seldom, or never, see one that has simply broken under the string load alone. 'Young's modulus' is a measure of how much force it takes to stretch or compress a piece of material by a given amount, and it's a good predictor of how stiff a top will be at a given thickness. The mahogany samples that I have tested have all had Young's modulus values along the grain that were similar to that of a medium density spruce, but they've been as much as twice as dense. To have the same stiffness the mahogany top would need to be nearly twice as heavy.
There's not much horsepower in a plucked string, and high frequency response is related to acceleration in a car: if you have a small engine, and want good acceleration, you need to keep the car light. Getting good, strong, full sounding trebles out of a Classical guitar begin with making light, stiff top and a light bridge. There's more to it than that, of course, but you have to start there.
The long and short of it, then, is that it's possible to get away with a mahogany top on a steel string guitar. If somebody were to insist on a mahogany top on a Classical guitar, I'd strongly suggest making it even smaller than the usual Classical size, which is about an inch narrower in the lower bout than the Martin 000 shape. As you make the outline smaller the balance of treble to bass shifts more toward the treble, and you can get away with a thinner top and lighter bracing. Something on the order of a Martin size 1 might work reasonably well with a mahogany top. It would be even easier to get a good sound out of it by substituting in cedro for the top, which can look similar to mahogany, and have reasonably stiffness, but much lower density. Martin's 1 was the 'normal' size for a guitar around 1850 or so.
Sorry to get so technical. I hope it helped.