Well if reaching W Pak was difficult, E PAK was virtually next door . Those migrating to / from E Pak did not face pogroms as those in the West did.
The language of the East Pakistanis (Bengali) was completely different from Biharis. My grandparents speak Bhojpuri, the dialect of Bihari in Western Bihar. Urdu is much closer to Bhojpuri than Bengali is, Bengali has almost no connection with Bihari or Urdu. Which is why many Biharis tried to go to West Pakistan. And as you know, a lot of Biharis did go to East Pakistan as well, and when 1971 happened; were caught in no man's land and were essentially nationless.
Btw, my paternal grandfather did work in the railways that operated from Dhaka to Lahore for a few years. My father did some of his schooling between the end 1950's till the early 1960's, and he did pick up Bengali in school, but my grandparents never did. It's easier for kids to pick up a language in school, but not adults.