It is said that Milarepa also visited the place. So did Thangtong Gyelpo who discovered a terma (sacred treasure) at the site and Zhabdrung Ngawang Namgyal in 1646.The Paro Penlop, Gyalse Tenzin Rabgye, built the first monastery in 1692 around the holy cave in which the Guru meditated.
However in 1998, a fire destroyed the main structure and all its contents. Previously, it had suffered a fire in 1951. It was reconstructed in April 2000.
According to legend, Guru Rinpoche flew to the site as the fiery Dorje Drolo, one of his eight manifestations, on the back of a tigress, a form assumed by his consort Yeshe Tsogyal to subdue the local demon, Singye Samdrup. After that, he meditated in a cave for three months. It is said that when the monastery was first built, it was anchored to the cliff by the hairs of khandroma (female celestial beings).