The practical implication of this more-refined analysis of averaging bounds is more support for the practice of averaging. Sufficient averaging over the hypothesis space can reduce the “complexity” allowing tight estimates on the true error rate—possibly even tighter than could be guaranteed with a single hypothesis.
The improved averaging bound is not yet the tightest possible and it appears there are several possible theoretical improvements.
Open Problem: In practice, is the averaging bound ( 7.3.15) ever better than the PAC-Bayes bound 6.2.1? The extra triangle inequality applications in the averaging bound may make it worse than the PAC-Bayes bound in practice.