There are several reason why Chittagong Hill Tract was included in East Pakistan.
First of all, Bengal's premier port Calcutta went to India on the ground of being Hindu majority(Capital city and principal port needed a special consideration than just being which religious group reside in it, because capital represent the entire nation), so it was only logical that smaller Chittagong port will go to East Pakistan(Chittagong district which supported the port is Muslim majority unlike CHT). Otherwise partition can not be presented as fair or equitable. It was a big blow for East Bengal to loose Calcutta, despite being 2/3rd of the undivided province, it was stripped off from owning the capital and largest city,Calcutta as well as it's principle port. Moreover East Bengal was deprived of large swath of Muslim majority territory in western sides(Murshidabad, Malda, half of Dinajpur districts) on the ground of maintaining Indian land connection with North East and navigability of Calcutta port.
On the other hand CHT was the hinterland of Chittagong district and it give some buffer territory to the Chittagong port. Moreover, Chittagong's principal river, Karnaphuli flows through the CHT. So it was absolutely vital for East Bengal to retain CHT. Another important consideration was, CHT was only accessible through Chittagong district. So, communication point of view, it was only logical for CHT to go with East Bengal. So, you can say, Chittagong Hill Tract(CHT) was a small compensation for East Pakistan for it's much, much larger sacrifice in other sectors.
Sardar Patel obviously was a self-serving man who wanted to maximize the gain without regarding the sense of justice or fair play or reciprocity. He completely forgot or did not want to remember, how many favors India got at the expense of East and West Pakistan in the Radcliffe award. He just wanted to take...take and take.