Without brass washers, you X20 will never be as accurate as it could be.
They may wear the same, who knows? The real issue here isn't wear. It's the way that plastic can mash and flex, and in that way, effect accuracy