The Udaigiri Caves are cut into a sandstone hill and feature 20 Gupta cave shrines. Two are Jain and the rest are Hindu. Buddhist monks also inhabited these caves in 2 BC. The Rani Gufa or "Queen's Cave", is tucked away around the corner. The best sculptures are found on the pillars, arches and to the rear of the courtyard on the lower level. On the back wall of the upper story, a long frieze shows rampaging elephants, panicking monkeys, sword fights and the abduction of women. The other two caves are double-storied caves that contain sculptures of a lion holding its prey, elephants with snakes wrapped around them and pillars topped with peculiar winged animals.

For years Buddhist and Hindu dynasties ruled Madhya Pradesh. Ashoka, the emperor of the Mauryan Empire, ruled Madhya Pradesh, then known as Malwa. He built many of the Buddhist monuments at Sanchi. The area was then ruled by the Sungas, and then the Guptas. The Parmaras ruled what is now southwest Madhya Pradesh about 1,000 years ago. Bhopal was named after one of their rulers, Raja Bhoj. The Chandela dynasty, who ruled the northern part of Madhya Pradesh, built the impressive temples in Khajuraho between 950 and 1050. This culture and tradition is a hallmark engraved on the creations of Udaigiri caves.