it should be fairly straight forward to run a headless system (no display). It's just a few Kconfig that needs to set properly. You can check the file distro.config that we use in the regular builds and use it.
peripherals that work with Android should work with Linux, with the following exceptions:
* display and GPU
* h/w accelerated video
Of course some of the Android features (camera, GPS, ..) require some user space (and often proprietary bits), in which case getting them to work without Android might require additional work.