We need to play hard ball. I think going behind the Elders was a mistake... But too late.
We basically need to create a class system. We take care of those loyal to us, and however we can very, *VERY* discreetly sabotage the others, we must. We should separate the resources of the loyalists and the separatists. Then we must make the loyalists dependent on us, which shouldn't be hard. Once we have our base, we poach the others. Certainly some will be enticed by rewards of cloth, ale, and food. And if, say, the granary conveniently burnt down, and all the stockpile of separatist wheat was destroyed, and we had all that was left.... That might convince a few more.