Yes you can. The problem is when you go the other way ie replacing 8 with 4 as they may be over driven by the output stage of your amplifier. You will get a lower "watt" output as the output stage is driving a higher impedence, but it is safe to check your 8`s as you will not do any harm
Cheers,
bryce