Gunmen in more than 10 vehicles launched the attack on Ghodwa early in the morning, firing “indiscriminately at public buildings, including the police station”, a security source told AFP.
Two civilians were killed and four abducted, said the source in the town about 700 kilometres south of Tripoli.
The assailants also set fire to the homes of security forces in the attack, which lasted about half an hour, the source added.
IS claimed responsibility for the attack via its propaganda arm Amaq, saying its “fighters… attacked Haftar militia houses in the locality of Ghodwa in southern Libya”.
In a statement on Thursday, Libya’s Tripoli-based unity government condemned “cowardly acts” in which two people were killed.
Libya has been mired in chaos since the 2011 ouster of Muammar Gaddafi.
Haftar backs an administration in the east that rivals the internationally recognised Government of National Accord.
Ghodwa is controlled by Haftar’s self-styled Libyan National Army which launched a military operation in January to “purge the south of terrorist and criminal groups”.
After winning the backing of tribes, the LNA seized several towns without a fight, including the region’s main city Sebha, 70 kilometres north of Ghodwa.
Haftar then launched a military offensive on April 4 to capture Tripoli.
Jihadist groups have capitalised on Libya’s instability to establish a presence in the country.
IS had its main stronghold in Kadhafi’s home town of Sirte, around 450 kilometres east of the capital, until it was driven from the city by pro-GNA forces in December 2016.
Despite being weakened, the jihadists regrouped in the desert and still pose a threat in the North African country.