Harold Holt was more a victim of his own self-belief than the vicious ocean tide at Cheviot Beach, according to police.

He played up to his image as the outdoors prime minister and was photographed in the British press in his scuba mask surrounded by three women in bikinis in an 007 pose.

On the day he disappeared, Harold Holt, 59, should have known not to go swimming. If his boast "I know this beach like the back of my hand" was true, he should have stayed on the sand.

He was with four others - one was Marjorie Gillespie, his secret lover. According to Holt's wife, Dame Zara, she wasn't the only one.

"She was one of the queue formed on the right. It went on all the time," Dame Zara would say years later. She said he had lovers in Melbourne, Sydney, Canberra and Hong Kong as well as Portsea.

But as with US president John Kennedy, nothing was ever said. It was a different time.

Despite the growing controversy of the Vietnam War, Holt did not feel the need to surround himself with security guards.

On Friday, December 15, he flew to Melbourne from Canberra in an RAAF VIP plane and later drove his own maroon Pontiac Parisienne to Portsea. He had an early night and played tennis with friends the next day.

On Sunday morning, he dropped three crayfish at the Gillespie holiday house next door.

Before lunch, he took Gillespie to Point Nepean to see round-the-world solo sailor Alec Rose sail into Port Phillip Bay.

They were followed in a second car by young businessman Alan Stewart with Gillespie's daughter Vyner, 20, and her friend, medical student Robert Simpson, 19.

Despite the blustery, muggy conditions Holt wanted to swim at the private and exposed ocean at Cheviot Beach, rather than at one of the more protected beaches in the bay.

The nearby patrolled beach at Portsea was closed because of the weather, but the prime minister did not hesitate when he arrived at Cheviot. He remarked the tide seemed to be high but headed straight in.

Stewart said: "If Mr Holt can take it I had better go in, too." But because of the undertow he did not go out of his depth.

Holt swam on, entering an area that quickly became turbulent, and disappeared.

Gillespie said: "It was like a leaf being taken out. It was so quick and so final."

Friends said that although Holt was an excellent snorkeller, with strong endurance, he was not a powerful swimmer.

He used to practise holding his breath for up to two minutes during boring times in Parliament, or in the bath, so he could dive deeper during his frequent snorkelling trips.

He once told his press secretary, Tony Eggleton: "Look, Tony, what are the odds of a PM being drowned or taken by a shark?"

But he should have known the dangers. He had to be assisted from the water at Cheviot Beach eight months earlier when he was swept out after his snorkel sprang a leak.

He suffered a lifelong complaint from a shoulder injury he received playing football at university and he found it difficult to swim freestyle.

Despite a massive air, sea and land search, the body of Australia's 18th prime minister was never found. Portsea Surf Life Saving Club president Milton Napthine, a friend of Mr Holt, said: "He knew damn well that the surf was too high. God only knows why he went in for a swim."

One suggestion was that he could have been struck by a large piece of timber discarded from a cargo ship. There were many theories about what happened to Holt, from the reasonable to the ridiculous.

But the man who probably knew best was Laurie Newell, the police inspector in charge of the search who rose to the rank of deputy commissioner and later became the head of the Country Fire Authority.

"I think he fell for his own publicity," he said.

"He believed he couldn't drown. Remember, he wasn't a young man any more."