It is naturally regrettable that the counting rules do indeed produce different results, that is, elect different candidates. This is to be expected, especially when comparing the Meek algorithm with the hand counting rules. Approximations must be made to provide a feasible manual process, so if it is required that a witnessed count be undertaken (and hence the moving of ballot papers between piles for each candidate) then a manual counting rule is required.
Unfortunately, real election data is hard to collect due to the confidentiality that usually applies to such data. However, a computer program has been written to produce such data anonymously by a random process which would not invalidate statistical tests on the anonymous data. This has resulted in a few more data sets from which a comparison can be made. The two counting algorithms being compared here are Meek[2] and ERS 97[3].
In the table, the last entry records the number of seats whose occupancy changed and, in brackets, the number of votes less than the quota which the Meek algorithm recorded against the candidate which ERS97 elected (expressed as a percentage of the total number of votes). Hence for M005, the last remaining candidate which the Meek algorithm did not eliminate was the one elected by ERS97 and had 6358.85 votes against a quota of 6517.76 (6517.76-6358.85=158.91 votes = 0.57% of 27,757). The star indicates that the remaining candidate in the Meek count was not the one elected by ERS97 and hence the two counts diverged at an earlier point - not just the last stage. Of course, in the one case in which two seats differed, it is not possible to provide a simple numerical difference.
It can be seen from the table that the differences are significant and large in some cases. In five cases (M070, R004, R005, R046 and R048) the differences are small and perhaps could be regarded as acceptable. The total number of seats in these 19 elections is 106 with 20 differences and hence a discrepancy in those elected of 18.8%, or 2.1% difference if all the elections are considered.
The difference in the handling of non-transferables between the two algorithms is a matter of controversy. To indicate whether the number of non-transferables is a factor, the difference that the two algorithms give in the number of non-transferables is expressed as a percentage of the total votes. In the case of R046, ERS97 has a very much lower number of non-transferables which surely has a key effect on the result. However, in general, the pattern is not so clear.
It could be that the method of constructing the Mddd data (first class above) produces results which would not be typical of real elections. However, the table clearly shows that the Rddd (real elections, second class) examples show similar differences.
Any of the data upon which this paper is based can be provided to interested parties.