The name 'Cave Hill' appears to be a translation of the Irish Binn Uamha. The hill was originally known in Irish as Beann Mhadagáin (meaning "Madagán's peak"), after a king of Ulaid (Ulster), Matudán mac Muiredaig, who died in 856 AD. It could also be named after a later king, Matudán mac Áeda (reigned 937–950). The later king's grandson, Eochaid mac Ardgail, was killed at the battle of Crew Hill in 1004, in which the men of Ulster were defeated by their old enemies, the Cenél nEógain. It is from him that McArt's Fort derived its name.

The residential neighbourhood at the foot of Cave Hill's entrance is derivatively known as Ben Madigan, with street names to match, and is a wealthy semi-outer city, semi-suburban area.[citation needed] The name 'Ben Madigan' can also be found attached to buildings and schools close to the area, e.g. the Belfast Royal Academy has the Ben Madigan Preparatory School on the Antrim Road.

The crowning stone Giant's Chair of the O'Neill clan was apparently sited on Cave Hill summit until 1896 and gave its name to the nearby Throne Hospital.

During World War II, a bomb dropped prematurely during a German bombing raid on Belfast exploded, causing a large crater near the grounds of Belfast Castle. It is understood that RAF Bomber Command was situated on Cave Hill in the early years of World War II before relocating to Castle Archdale in County Fermanagh. Hence the German bomb may have been intentional.

On 1 June 1944, an American Air Force B-17 bomber crashed into Cave Hill during heavy fog, killing all ten crew instantly. The incident inspired Richard Attenborough's film, Closing the Ring. Some scenes of the film were shot on Cave Hill. The site of the crash is accessed via Carr's Glen Country Park where the field, known locally as 'The Bomb hole Field' remains open to public.
McArt's Fort