Reusing previous object files after modifying src.conf

Does anyone know if it is safe to reuse previous object files when
building FreeBSD from source and then modifying src.conf? The man page
is not very clear on this behavior.

For example, if I build release with the default src.conf settings and
then later add a bunch of WITHOUT_XXX=yes to src.conf and build another
release, reusing previous object files. Is this supported, or could
this confuse the build system and still package the components disabled
with WITHOUT_XXX=yes since the binaries are already present in the
object directory?

Thanks.