Addressing a press conference in Amin Gaon in Guwahati, the prime minister also said he would prefer to put an end to the slanging match with Bharatiya Janata Party's (BJP) prime ministerial candidate L.K. Advani.
"We can't be blamed," the prime minister said replying to a question on his reaction to Lalu Prasad's remark on the Babri razing. He said that the then BJP chief minister of Uttar Pradesh Kalyan Singh "did what he did" despite his affidavit to the apex court that he will protect the 16th century disputed structure.
"It is known to everybody that then BJP chief minister Kalyan Singh gave a solid assurance to the Supreme Court and after that he did what he did. The Congress party certainly cannot be blamed because it believed that Kalyan Singh would honour the affidavit given to the Supreme Court?.
"But he didn't honour it and that is the only fault you can find with the Congress party with regard to the demolition of the Babri Masjid," the prime minister said.
Lalu Prasad, the railways minister, had said in Bihar that the Congress was also to blame for the demolition as it did nothing to prevent it, despite being in power at the centre then.
Signalling an end to his slugfest with Advani, who has repeatedly called him a "weak prime minister", Manmohan Singh, replying to a journalist's query, said: "I don't want to prolong the match and the dialogue? whatever I had to say I have said already".
Speaking on terrorism, he said: "Communalism, terrorism, and Naxalism (Maoism), is today a big threat to the country's unity and integrity, but we are committed to fighting all these evils with a firm hand. To defeat communalism, we all have to vote for the Congress party as we are the only party that stands for secularism."
On the global financial crisis, the prime minister said the meltdown was a result of "financial mismanagement" by developed countries and it would blow over "partially" by September.
"This crisis has arisen due to mishandling of the financial system by the major developed countries," he said.
"We expect the crisis to be partially over by September when we can go back to our growth rate of 8 to 9 percent."
He also said the Congress if voted to power would make a renewed effort to curb militancy and terror.
"I am confident the Congress party and its alliance partners would be able to form the next government at the centre and we shall work for development and fight terrorism with a renewed effort," the prime minister said.
The prime minister, who is a Rajya Sabha member from Assam, was in Guwahati to address an election meeting at Amin Gaon on the outskirts of Assam's main city of Guwahati.