If that is the case, it would just bring the price/rent attached to the land down to the point were it is feasible.

I accept agriculture is more complex because of the existence of subsidies, speculation and other non-market interventions/decisions in relation to that land, but any exception can be managed as such. No need to throw the principle out completely