Results reported in Figure 5 further reinforce the observation that qNEHVI produces a large pool of non-optimal solutions for all benchmarks problems, where many points exist away from the PF. Additionally, the darker coloration for qNEHVI in Figure 5 b), d), f) and h) indicates a much lower probability of occurrence, which reinforces our hypothesis, that HV improvement can be partially attributed to the stochastic nature of QMC sampling. Additionally, Figure 5 b) and d) for ZDT1 and ZDT2 respectively also show that there were many solutions being proposed at the extrema of f1=x1.
This is the same behaviour as that observed for a single run in Figure 4 b) and d), and we further elaborate upon it in SI 1. In contrast, the heuristic nature of U-NSGA-III provides more consistency between optimisation runs, which is shown by the brighter regions of points near the PF in Figure 5a), c), e) and g). indicating a higher probability density. Notably, the bright regions are not spread across objective space evenly. There is a preference for the lower range of f1=x1 since it is easily tunable, i.e. it is simple to derive improvement by simply decreasing x1. This is in line with our previous discussions based on results reported in Figure 4, where U-NSGA-III prefers solutions with immediate improvement. Furthermore, we observe that the bright regions are concentrated near the PF, which indicates that U-NSGA-III was able to consistently approach the PF and maintain a larger pool of near-Pareto solutions over the optimisation runs, despite the limited evaluation budget.
In contrast, qNEHVI had relatively few points, although they are lying directly on the PF, which is then shown as a higher mean HV compared to U-NSGA-III. In a real-world context, the larger pool of near-Pareto solutions could have scientific value, especially for users looking to build a materials library and further understand the PF. However, this is not reflected by the HV performance indicator.
Optimisation trajectory in objective space for a single optimisation run of 100 iterations x 8 points per batch. a-b) ZDT1. c-d) ZDT2. e-f) ZDT3. g-h) MW7. The red line represents the true PF, while MW7 being a constrained problem has an additional blue line to show the unconstrained PF. The colour of each experiment refers to the number of iterations. All problems clearly show a more gradual evolution of results as the number of iterations progress in U-NSGA-III whereas qNEHVI rapidly approaches PF and then fails to converge further.