Need more details to determine necessity of said washer. A plastic washer is usually to absorb some vibration or reduce conductivity of metal on the pcb. A metal washer is to spread the force of the screw head to prevent damage.
That said, it's unlikely any washer would have an impact on performance. It will just be less "safe" maybe.
Generally you can achieve more mounting pressure if you use the washers. But sometimes they are just spacers depending on the application. Generally best to follow the directions included.