A BJP spokesperson, however, rubbished the minister's claim as an attempt to "garner cheap publicity" adding that if Mr Rajbhar had any grievances, he should air them at an appropriate forum.
"There is more corruption prevailing under the BJP regime as compared to the previous SP and the BSP regimes. Though I am a part of this government, this is not my government...We have an alliance with the BJP," Mr Rajbhar told Press Trust of India over the phone.
Mr Rajbhar, who leads BJP ally Suheldev Bhartiya Samaj Party (SBSP), also said that his party was not getting the due respect under the present government.
"I have talked to the chief minister on these issues," he said.
Asked whether his party would contest the 2019 Lok Sabha polls with the BJP, Mr Rajbhar said it depended on the BJP.
"It depends on the BJP whether it wants an alliance with us or not. Like local body polls, if the BJP does not forge an alliance with us, we will contest the Lok Sabha polls alone," he said.
On the violence in Kasganj, Mr Rajbhar said the incident was unfortunate and claimed some officers were defaming the government.
Reacting to the charges about corruption levelled by Mr Rajbhar, state BJP spokesperson Rakesh Tripathi said that it was not the tradition of his party to break pre-poll alliances.
"The statement by Om Prakash Rajbhar, who is a Cabinet minister in the Yogi Adityanath government, is simply to garner some cheap publicity," Mr Tripathi said.
Mr Rajbhar is a responsible minister and if has some grievances with the government, then he should air them at an appropriate forum, and not publicly declare them, he said.
"It seems that he is behaving as an Opposition leader. And, if he has some serious problems with the BJP, then he can leave the government," Mr Tripathi said.
The SBSP, which has four MLAs in the 403-member UP Assembly, had contested the Uttar Pradesh civic polls separately. It had even contested a few seats in the Gujarat Assembly elections on its own, asserting that its alliance with the BJP was not at the national level.