British judges have ruled that Bermuda’s ban on same-sex marriage is permitted under its constitution, in a setback for gay rights in the British overseas territory.

The UK’s judicial committee of the privy council (JCPC) – the ultimate court of appeal for Bermuda and dozens of other British overseas territories, dependencies and Commonwealth states – on Monday overturned a decision by Bermuda’s highest court, which ruled the ban to be unconstitutional.

The JCPC also ruled separately that there was no right to same-sex marriage under the constitution of the Cayman Islands.

The attorney general of Bermuda appealed to the JCPC after the country’s court of appeal and supreme court held that section 53 of the island’s Domestic Partnership Act (DPA) 2018, which confines marriage to a union between a man and a woman, was invalid.

The case attempting to establish the right to gay marriage had been brought by six people including Roderick Ferguson, a gay Bermudian; Maryellen Jackson, a lesbian Bermudian and the LGBTQ charity OUTBermuda.

The judges sitting in London allowed the attorney general’s appeal by a four-to-one majority.

In the written judgment by Lord Hodge and Lady Arden, they said: “No restriction is placed on the worship, teaching, practice and observance of the respondents which manifests their belief in the validity of same-sex marriage. The protection of a ‘practice’ does not extend to a requirement that the state give legal recognition to a marriage celebrated in accordance with that practice.”

They also said the exclusion of same-sex couples from the institution of marriage was not attributable – as the respondents had argued – to their creed but because they were of the same sex.

Responding to the decision, Ferguson said: “The Bermuda government’s crusade against same-sex marriage was waged to convince you that there’s something shameful about your sexuality.

“Don’t believe that tired old lie … Our work as a society is not done until everyone’s humanity is recognised both in law and in life.”

In his dissenting judgment, Lord Sales said the attorney general’s arguments failed to take seriously the text of the constitution as a legal instrument “which contains a general protection for freedom of conscience and the right to manifest one’s conscientious beliefs by living in accordance with those beliefs (subject to a power on the part of the state to interfere with that right where that is justified for social reasons, which justification is absent in this case)”.

OUTBermuda said future generations and future courts would be inspired by the dissenting judgment. But the charity also raised concerns about people who had already entered into gay marriages in Bermuda, including when the DPA was suspended.

The case in the Cayman Islands concerned Chantelle Day and Vickie Bodden Bush, a lesbian couple who were denied a marriage licence in 2018 on the basis that marriage was defined in the territory’s bill of rights as “the union between a man and a woman as husband and wife”. They won an initial legal challenge but lost at the court of appeal and took the case to the JCPC.

The same five judges as heard the Bermuda case unanimously rejected the appeal. In the written judgment, Sales said there was nothing in the constitution to stop legislation permitting same-sex marriage but the bill of rights had been “obviously” written “to emphasise the limited ambit of the right [to marry] and to ensure that it could not be read as capable of covering same-sex marriage”.