Try before you buy

I’ve commited the first sets of results to the repository in the aptly named results/ directory.

To refresh your memory, the question I intended to answer was:

does the number of queue directories (on a single disk) make a significant impact on the time taken to create new entries in the queue?

They’re quite surprising.

First, let me point you to the disclaimer.

With that out of the way… I’ve run a total of four tests. Each test consisted of five runs.

The first test was run with one queue directory (/var/spool/mqueue/qdir00). The second with five (qdir00 through qdir04), the third with ten (… through qdir09), and the last with 20 (… through qdir19).

After each test run Sendmail was stopped, the queue directories removed and recreated, and Sendmail was restarted. Nothing else changed between tests except the number of queue directories.

Test 1 - 1 queue directory

View raw data

The first thing I did was run the data through ministat to see if there were any glaring discrepencies. To get it in a format suitable for ministat I knocked together “to_ministat.pl, which extracts the second column from each results file. This is the column that contains the numbers I’m interested in. This converted test1/results.1 to test1/m.1, results.2 to m.2, and so on.

The results are (omitting the plot, as it doesn’t add much in this case, and converting to HTML):

So I can be 99.5% confident that the results I got from the five test runs are consistent. There’s an outlier in the second test, but that can be put down to a random factor in the environment — it didn’t seriously perturb the test results.

Given that, here’s the results after plotting the figures from result.1 and adding a trendline and confidence band.

I’m quite surprised by that. I’d been expecting a much larger upward slope to the trend line, on the assumption that as the number of entries in the directory increased the time for creating a new entry would also increase. That’s clearly not the case on this system, at least up to 60,000 entries in the directory.

Why 60,000 entries, when the testing generated 30,000 messages? Remember that each message in the queue is represented by at least two files, a qf* file and a df* file.

This does show that there’s a penalty paid as the number of entries in the queue gets larger (every additional entry increases the amount of time required by ~ 1.94 uSec) but it’s a much smaller penalty than I had anticipated.

Given this result, I don’t expect the results from the 5, 10, and 20 queue directory tests to show a marked improvement.

Lets look at them.

Test 2 - 5 queue directories

View raw data

Here’s the ministat comparison of the raw data:

Again, no significant differences between the datasets. So again, I’ve selected dataset 1 as the representative sample.

This time I can compare this against the first results and see if there is any significant difference.

x test1/results.m.1
+ test2/results.m.2
: = Mean
M = Median
+----------------------------------------------------------+
|                                  x   x +                 |
|                                  x  ++ +x  x             |
|                              +   x +++ ++ xxx            |
|                          +   +  x+++++ ++ +xx            |
|                          +++++++++++++x++x+xx+  +        |
|+                        ++++++++++++++x++++x++ x+     x +|
|                                 |____:_____|             |
|                            |_______:_______|             |
+----------------------------------------------------------+

As you can see, no difference worth writing home about between 1 and 5 queue directories at the 99.5% confidence level. If you’re prepared to let things slide to 95% confidence then there is a difference.

Difference at 95.0% confidence
        -38.9836 +/- 38.0923
        -3.40834% +/- 3.33041%

but, as the figures show, it’s very small. That difference is also visible in the slope of the trend line, which is increasing at approximately 25% of the rate of the “single queue directory” case.

Test 3 - 10 queue directories

View raw data

Again, here’s the ministat comparison of the data from this run.

As with the results from test 2, there’s no difference at the 99.5% level between this test and the results from the first test. However, in addition, there’s no difference at the 95% level either.

As you can see, the graph shows that queuing time does increase at a slower rate than with one queue directory it’s actually slower than with 5. However, these differences are so small as to be insignificant.

Finally, the fourth set of data.

Test 4 - 20 queue directories

View raw data

Again, no significant difference between the individual result sets. And also again, there’s no significant difference between these results and the results in the “1 queue directory” test.

This is borne out by the chart. Yet again the trendline has a shallower slope than the single queue directory case, but the difference is minimal.

Conclusion

I had expected that multiple queue directories would have made a much larger difference than they have when Sendmail is queuing mail. What this demonstrates is that (at least in the situation where there is only process queuing messages) that it doesn’t appear to make much difference whatsoever.

The next set of tests will experiment with varying the number of concurrent connections that are used, to see if that workload makes any difference.

Note, also, that these results have no bearing on the effectiveness of multiple queue directories when it comes to trying to deliver mail that has been queued. That’s going to be yet another set of tests.

Share This