The old relics in different areas of the Union Council indicate that the area remained under the influence of Buddhists which flourished in the whole Swat district around the 2nd century BC. The Elum Mountain situated in the area is considered to be sacred place for Buddhists. The Hindu Shahi dynasty succeeded the Buddhism around the 5th century. By the time of the Muslim invasion (1000 AD), the population was by and large Hindu. The Muslim era in Swat began with the Mahmud of Ghazni from Afghanistan in 1001 AD. The Yousafzai Pukhtun tribe succeeded Swati Pakhtuns in the sixteenth century.
The grandson of Shekh Saadi –Turk by origin-, Akhud Darveza Baba a towering spiritual personality got the land of the valley. The land was distributed between Main Noor Baba and Mian Dawlat Baba, the grandsons of Akhund Darveza Baba, where their progeny still inhabited in the area.
Islampur has remained of significant importance in the Swat state era formed in 1926 due to its proximity with the capital and the influence of its people in the state. The First ruler of Swat State, Mian Gul Abdul Wadood (Bacha Sahib) had built White Palace in 1940 as summer residence in Marghazar village of the Islampur Union council. The architecture of it reflects the grandeur and greatness of the State which was turned into hotel later on.