The Castle of Good Hope is the oldest surviving colonial building in South Africa, a five-pointed star of yellow stone that the Dutch East India Company finished in 1679 to keep the English out of Table Bay. The English came anyway, in 1795, and walked in. What the walls have kept in, according to three centuries of soldiers, jailers and night staff, is a small and rather persistent population of the dead.

The oldest story is about a governor. In April 1729 Pieter Gijsbert van Noodt had seven soldiers condemned to hang for desertion. Some tellings say the sentence was meant to be lighter and he hardened it. On the gallows, one of the seven turned and called on God to judge the governor before the day was out. Van Noodt, the story goes, did not attend the hanging. He was found that afternoon in his chair, dead, with an expression on his face that the people who found him would not describe. That last detail is the kind that stories add, and no surviving document confirms it. What the archives do record is that van Noodt died in office in April 1729, which is enough to keep the tale alive. His heavy footsteps, they say, still go along the upper passages.

Then there is the bell. It hangs in the tower over the main gate and was rung to mark the hours and to warn of danger. At some point in the 1700s a soldier is said to have hanged himself with the bell rope, and the tower was bricked up. People who work in the castle report that the bell still rings, on its own, at no fixed time, with nobody near the rope.

The most watched of the regulars is a tall man. In 1915 sentries on the walls reported a tall, faintly glowing figure walking the battlements between the bastions, then stepping off the edge and vanishing. He appeared for a stretch of nights and stopped. In 1947 the same figure was reported again, for about a fortnight, and then not. The 1947 sightings happened to coincide with the royal tour, and a separate story from that visit has a weeping woman in grey seen running across a courtyard with her hands to her face, with members of the royal party among the witnesses. That one is thinly sourced and much repeated.

Lady Anne Barnard, who kept house in the castle from 1797 for the first British governor, is the gentlest of the set. She is seen in a ballgown of her period at functions, and near the Dolphin Pool that she wrote about and that was rebuilt from her sketches. There is also a black dog, seen in the courtyards, which is said to leap at people and pass straight through them. Down in the Donker Gat, the windowless cell that floods when the tide is high, guides report cold spots and voices.

Some years ago the night guards went to the press. They said they heard men arguing in the guardroom when it was empty, that footsteps followed them along the walls, and that some of them would no longer do the rounds alone. The castle’s management said, in effect, that old buildings have their moods.

Visitors still ask about the bell. The guides will tell you it has not rung this week. Then they will pause.