"There are... some irresponsible 4x4 vehicle and track owners, whose activities continue to damage sensitive sections of our inland environment," he said in the national council of provinces on Thursday, opening debate on his department's budget vote.
"When we banned them from the beaches, some of them simply moved inland, and they are destroying sensitive mountainous environments."
A recent announcement by the industry that it planned to hold a national off-road workshop in October this year was pleasing.
"An independent grading system for 4x4 tracks, including environmental grading, and the creation of a representative industry body would be positive steps.
"However, regulations still will be required, and if there are not concrete proposals forthcoming from the industry within the next six months, the government will have to issue these unilaterally - although this would not be our first choice," he said.