In charts that use layers, Size is set for each Scatter layer, and each size field has its own legend in the Styles tab.
Configuration
The only configuration available is showing or hiding the size legend, which will hide the circles that correspond to size at the right. Note that you may also want to hide the color by legend, which is done under color configuration.