First i want to correct you about some thing that you wrote cynically - " go ahead build your temple " .
Allow me to just share , since i have seen others misunderstand this, that this is not a possible. Let me point to you a passage in the bible that mention how the fist temple was built .
King david , who on all accounts in the bible was God's favourite king wanted very much to built a permanent temple for the lord. But God did not allow it. As it is written :
" But God said unto me, Thou shalt not build a house for my name, because thou hast been a man of war, and hast shed blood. "
1 Chronicles 28:3
So king david was not allowed to build the temple because he was man of war and the only his son Solomon was allowed to build the temple. The temple can only be built in a time of peace.
Also, the two temples where build at a time when prophets walked the land and where the living connection between the nation and God . With out them no temple can be built. Sure , someone can build some structure there , but it would only be a house of brick and stones with out any soul or spiritual value. And of those we have plenty off , we sure don't need another one.
The temple can only be built by divine intervention. It is not up to man to decide. So let God judge between us.
And in the meantime , let the Muslims pray in their Mosques , let the Jews pray in the Western wall and let there be peace in the land.
With out it all those prayers aren't worth much anyway.