lens - Why does it seem like large sensors are necessary for good low-light performance?

Phone manufacturers have recently started advertising the size of the photosites on smartphone camera sensors. They argue that larger photosites lead to better low-light performance. I think a good analogy would be car manufacturers claiming that larger wheels lead to faster cars. True, given the same f-number, a larger photosite captures more light and given the same axle RPM, larger wheels make a car go faster. However, larger wheels means that less force is applied to the road given the same engine torque, cancelling the effect.

I thought it would be the same for cameras. A larger sensor requires a larger focal length to deliver the same field of view. Given the same entrance pupil size, this increases the f-number, cancelling the effect of capturing more light. I was under the impression that the f-number of phone camera lenses was limited by the largest entrance pupil size that can fit into a smartphone. Surprisingly, phone manufacturers have been able to enlarge the entrance pupil enough to keep the f-number constant while increasing the sensor size. In fact, they were even able to lower the f-number significantly in some cases.

A similar phenomenon can be observed with regard to full-frame vs. crop cameras. The f-number of a crop lens is usually at least as high as that of its full-frame counterpart and there are hardly any fast prime lenses for crop cameras at all. It seems like it is much easier to get large entrance pupils on lenses built for large sensors. Take a 35mm f/1.4 full-frame lens with an entrance pupil diameter of 25mm. Is it not possible to build a similar lens that concentrates the light captured by this entrance pupil to a smaller image circle fit for a crop sensor, yielding a 22mm f/0.88 lens? Why does it seem like large sensors are necessary for good low-light performance?

Note: I know that sensor size also influences electrical characteristics, but I am only interested in optical considerations here. Let us pretend that sensors are ideal photon detectors with infinite dynamic range, leaving only shot noise to determine low-light performance. Let us also assume that all these sensors have the same number of photosites.

The question Why are larger sensors better at low light? does not answer my question as all the answers assume constant f-number, which is exactly the proposition I would like to challenge.

EDIT: I should mention that my question is not about what happens when you mount full-frame lenses on crop bodies. Obviously, this wastes a lot of the light captured by the lens so it's an apples to oranges comparison in my book. Instead, I'm wondering why it seems to be so difficult to concentrate all of this light to the crop sensor image circle in order to get a lens that makes the same total amount of light available to the smaller sensor.

Answer

One thing to consider is that the size of the image that the lens projects is completely independent of its f-number and focal length, but a factor of the lens design. F-number is a product of the width of the aperture, and the focal length. The focal length is the distance between the point of convergence, and the sensor, completely ignoring factors beyond that further into the lens.

You can produce an image of the exact same quality as an FF sensor, on a phone sensor as long as you design an 'equivalent' lens. But to create an equivalent field of view, as you alluded, the focal length needs to be reduced, as it needs to bend the light more to compress it into that image circle. With the size of phone sensors, these figures are increasingly small, often less than 5mm.

When you reduce both the sensor size and the focal length, the f-number needs to be increased to compensate. While decreasing the focal length will collect more light, you're only using the light in the same field of view. The lens design can take this into account, and simply not collect the wasted light. The actual aperture width (or entrance pupil) will need to stay the same, which increases the f-number.

When the focal length is so small, and the aperture so comparatively large, this represents an increasingly difficult technical challenge, as glass takes up space, and as the aperture increases, so will the size of these elements, but with the focal length so small, they still need to be either absurdly close together where they're occupying the same space, or so absurdly large and heavy that you'd need impractical and massive contraptions on the back of the device.

It's also worth noting the effect of chromatic abberation, caused by the fact that different wavelengths of light passing through a lens will bend differently, slightly more or less, depending on the wavelength. Lens makers have gotten pretty good at correcting for this effect to some degree, but it becomes increasingly difficult when taken to extremes.

Total light gathered is the important factor in image quality, sensor efficiency is basically the same across all modern sensors. Larger sensors are not necessarily better, but they reduce the challenge of lens design significantly.