Periphery is probably a part of the answer.
But I also think that one sees less brightness with their weaker eye simply because the picture gotten from it is more grey, hence with less contrast. Add grey to any colour and you'll see the result. Why is grey? You see worse centrally with your weaker eye and your mind which is responsible for gathering thousands images in every single second from the retina of your both eyes cannot make a very clear whole with the data received from your weaker eye. What it makes is a greyish picture with dim parts in dark areas, not vivid enough in light areas and with smudged borders between objects.
Also the brain tends to surpress the data from your weaker eye since it's less efficient but then that creates some imbalance which is problematic because it puts too much pressure to your stronger eye.