You need to strip the binaries. Use the strip command.
But I still don’t think it would make any sense to mix them into the boot image or initrd. As long as it can mount the rootfs, it can get those modules from the rootfs.
Modules that go in the boot image or initrd should only be those modules that are required in order to get to the point where it can obtain the rest of the modules from the rootfs. That means things like filesystem modules, and storage controllers. If the GPU drivers are modularized (which I hope they aren’t, since the kernel is build specifically for that GPU), then early initialization of graphics could require THOSE modules (and their firmware, and firmware loading modules) to also be included in the initrd.
Further, you will need to duplicate the modules between the initrd and the rootfs, because once the rootfs is mounted at /, it will obscure the initrd.
Now from an ANDROID point of view, (be aware that things are setup a bit different), NO modules are actually required to boot. Once it is up, some things obviously don’t work, wifi being the big one.
Could you post a list of the modules that you have?