In terms of ethnic/cultural diversity the subcontinent is the size of western and central europe ,and even greater in population.How many empires united europe except roman and charlemagne?
To control subcontinent you need to have sufficient economic base.You need to control gangetic plains to be master of northern india,this is not very difficult for a strong emergent power(especially strong in elephants or cavalry) as terrain obstacles are absent.But to be a pan-indian power controlling gangetic plains is not enough.You have to fulfill 2 more conditions -
You have secure resources of the 2nd great fertile belt -the indus river valley,and push west enough to create a buffer against invasions through the kandahar-kabul natural mountain gateway.
Once western frontiers are secure you have to secure the flanks and the richest ports in India -bengal-odisha controlling trade with S-E asia and china and Gujarat-sindh controlling trade with middle east and europe.
If an empire succeeds in accomplishing the above then they can afford to cross the natural vindhya terrain barrier and enter south india.Only 2 empires achieved total conquest - British and the mauryans.4 others came close - Mughals,Guptas,Marathas and Delhi sultanate at its peak under khiljis.
The british expansion followed a different pathway because they were a maritime external power,but the other 4 followed the above model.
Mauryans under chandragupta secured power in Magadha which was the dominant power of the gangetic valley first.Then expanded west and took over the indus valley area.Defeated the greeks and pushed the frontier upto the hindukush in afghanistan,thus securing step 2.Then he took gujarat and then entered south.Bindusara continued expansion in south.Ashoka finished off the last remaining anomaly - kalinga(odisha area).One reason mauryans succeeded was because during that time regional languages and identities had not developed so much.So you see maurya edicts in pali and prakrit all over india.
Mughals fought for Indus and gangetic valleys simultaneously.They took control the indus valley quickly,but had to fight the rajputs and afghans continously for victory in the gangetic plains.After akbar finally achieved this,he next turned to Gujarat and Rajputana had to be controlled because it formed the link between gujarat ports and delhi core area.Then orissa.Akbar controlled both kandahar and kabul -the 2 western frontier entry points.Jahangir lost kandahar,and shahjahan tried desperately to retake it but failed.But the iranians (safavids) were themselves threatened by uzbeks and ottomans on 2 sides so never attacked until much later(nadir).This allowed shahjahan to enter south india.Aurangzeb continued the policy but didn't suceed,overextended,made enemies everywhere .
Look at the guptas -same model.Chandragupta I begins the empire by becoming the main power in eastern gangetic plains.Samudragupta conquers all of it and expands till the indus.He also takes orissa and penetrates deccan on the eastern side.Chandragupta II conquers the sakas,gujarat-sindh and th indus valley.He secures the western frontier to an extent with vassal buffer states in the indus region.And this holds until the hepthalite white huns tear down the system.In the south Guptas followed slightly different policy.In the eastern part of deccan they ruled via vassals.In the western part of deccan and south india they followed matrimonial alliance policy. The vakataka kingdom was the main power and allied with guptas and its queen was generation after generation a gupta princess.
Delhi sultanate under alauddin same model.Already controlled gangetic plains and indus belt.Secured western frontier by defeating ilkhanate mongols.Then took gujarat.Then penetrated deccan.Ultimately didn't work long term and never reached the depths of south india like the mughals attempted to.
Marathas failed because -1.They had no central imperial administration,but a loose confederate structure.2.Their sardars were mainly raiders by nature focusing on loot to be acquired by force,rather than streamlined bureaucracy to extract revenue efficiently over long periods,this also alienated friendly forces in north india.3.They lost 10 precious years due to panipat debacle in 1761.It took them 10 years to recover and return to north india and delhi.But by this time English position had become impregnable,they had taken over bengal,orissa,bihar,eastern U.P (awadh was vassal)and were now the dominant power of gangetic plains.They were commercially and technologically far superior.Even then as long as confederacy fought united they had a chnace (stalemate in 1st anglo-maratha war).Once confederacy unity was broken,they fell easily.
Apart from marathas,the 3 south Indian empires that might have had a chance at pan-indian domination were satavahans, cholas and rashtrakutas.Its difficult for a south indian power to as easily snowball because land revenue potential is not as great as in gangetic and indus belts combined.2ndly local dynasties are very resilient and we see perseverance of pandyas, cheras, cholas, chalukyas,pallavas for hundreds of years -even if temporarily defeated cadet branches persist and revive later.
Satavahans were almost a pan-indian empire,but most of their energies which could have have been expended at expansion into north india were spent in the 200 year rivalry against the sakas.Still satavahans are an important empire as they were the greatest indian power when after the fall of the sungas,north india had a political power vaccuum and no other power could resist the sakas until the rise of the guptas.
Rashtrakutas were the strongest power of their day,stronger than their northern rivals -the gurjara pratiharas and pala .But they always had to fight too many enemies,due to continous attempts by local dynasties to undermine their supremacy in the south -they were never able to establish a permanent presence in the north,even though whenever they entered north in force they usually won most of the battles.But inevitably they had to leave or weaken their presence and oppurtunistic palas and gurajars waiting on 2 flanks pounced and retook kannnauj.
Cholas made a strategic choice.Unlike the rastrakutas who actually made a bid for pan-indian domination.Cholas instead focused their military might on a maritime empire - in lanka,-SE asia.In the north they only penetrated bengal and orissa,that too to complete their complete mastery of the -SE asian trade route with china by taking all the eastern ports.Bay of bengal was a chola lake.