By recursing using FileSystemEnumerable we know the file type and
can omit the stat calls made by DirectoryInfo.Exists.

For the top level path, we can omit the call also and handle
non-directories when rmdir errno is ENOTDIR.

For the recursive case we can avoid recursion when the top level path rmdir
succeeds immediately.

FileSystemEntry is updated so IsSymbolicLink remembers the file is symbolic link
and does not make a syscall for it.