Moduli spaces of connections with toral singularities