How much neural rendering is too much neural rendering?