The plants are continually too wet. Avoid it by providing good soil drainage and not over watering. Leaf spot can also affect pansies and shows up, unsurprisingly, as spots on leaves, which can widen and spread to flowers as well.
