Who are the new warlords undermining society in Latin America and the Caribbean? Ioan Grillo's new book investigates

Gangster Warlords: Drug Dollars, Killing Fields and The New Politics and Latin America, by Ioan Grillo (Bloomsbury)

Mass beheadings, widespread torture and rape, murderous child soldiers and terrorised civilian populations. It all sounds familiar – but this time it has nothing to do with Islamic State. The setting is Latin America and the Caribbean, where armed conflict between drug gangs and nation states has been gaining momentum since the turn of this century.

The loaded question is whether the region is experiencing full-blown wars. Within the countries concerned it is a divisive issue. While security forces have no doubts they are engaged in full-scale combat, politicians fear – in fairness, correctly – that labelling their states as war zones would be disastrous for foreign investment and tourism.

The death toll is staggering. In the seven years between 2007 and 2014, 83,000 people were killed in Mexico alone. A civil war is generally accepted to involve more than 1,000 casualties each year. To put this into more perspective, during the Troubles in Northern Ireland, 3,500 lives were lost over 30 years.

Gangster Warlords follows on from Grillo's book El Narco, which investigated the rise of Mexican drug cartels. This time he has carried out painstaking – and frequently hazardous – research over several years to cover all of the countries riven by drug warfare in the region. He takes in Mexico, Brazil and Jamaica, as well as the "Northern Triangle" of Nicaragua, Honduras and El Salvador. The resulting picture is grim, but also fascinating.

The conflicts have diverse origins. In Brazil, Grillo interviewed a founder of the notorious Red Commando, now living in retirement in Rio, who revealed a strange history. During the 1970s criminals mixed with revolutionaries in prison and cross-fertilisation occurred. Politicised gangs started by robbing banks, but soon seized upon the drugs trade as a burgeoning revenue stream.

The Brazilian army is in the midst of battles in urban slums to curb the gangs' activities before this year's Olympics, but its successes are likely to be temporary. The gangs are the de facto government in many favelas, providing their own rough justice with the official state sidelined.

Over in Jamaica the no-go areas are the projects (social housing estates) in Kingston, with the innocuous-sounding Tivoli Gardens the home of the fearsome Shower Posse – so called for their practice of showering their opponents with bullets. Grillo interviewed several gangsters and their families, covering some of the same bizarre ground as Marlon James's Booker Prize winning novel A Brief History of Seven Killings.

The ruler of Tivoli Gardens, one (perhaps fittingly named) Christopher Coke, even ran his own prison. The grisly nadir of Grillo's book comes when he reproduces testimony from a defendant who witnessed Coke hacking apart a crack dealer in the prison, complete with imitations of chainsaw noises, screams, severed body parts and faecal smells.

But Coke also played the role of benign monarch, providing uniforms for schoolchildren and dispensing food to hungry residents. His tentacles reached far into the state – hence his nickname President – but eventually the army duly laid siege to Tivoli Gardens, with Coke's gunmen holding out for as long as they could in a surreal reprise of the Alamo.

But the weirdest gangster warlord of all was Mexico's Nazario "The Maddest One" Moreno. In between suborning whole strata of government, he declared himself the head of his own religious cult, the Knights Templar. Shrines to Moreno were dotted around the countryside. If this sounds chillingly reminiscent of the terrifying Kurtz in Joseph Conrad's novella Heart of Darkness, it is a reasonable comparison.

Eventually Moreno was killed when ordinary Mexicans formed vigilante groups to combat him. Unfortunately members of other drugs gangs joined their ranks, which means the bloodshed making much of Mexico resemble a failed state is far from over. Meanwhile, over in the Northern Triangle, gangs recruit homeless children and put them through brutal initiations. Often the price of admission to a gang is to kill an enemy on their behalf.

Grillo looks into possible solutions. Scrapping existing police forces and recruiting and training new officers untainted by corruption has taken place in Mexico's Nuevo Leon state with considerable success. Another partial solution is to legalise marijuana throughout the region, which appears feasible given the repeal of prohibition in several US states.

Grillo also points out that initiatives to assist and rehabilitate addicts of harder drugs – specifically heroin, cocaine and crystal meth (or methamphetamine) – could also help to reduce the gangs' revenues. Some countries outside the region, notably Portugal and Uruguay, do not prosecute users of hard drugs, with benefits for their social and medical welfare.

But full legalisation of hard drugs is most unlikely, in the Americas or elsewhere, because of the potentially highly harmful nature of the drugs themselves. Tragically, a phenomenally lucrative black market will remain in place for exploitation by the gang warlords, so any ending to the carnage remains elusive. In the meantime Grillo delivers a consistently impressive in-depth account of an appalling situation.