By default, a Microsoft CA certificate has no EKU or application policies. If there are no specifically defined policies, the certificate is considered valid for "All Application Policies." This means that the CA is technically able to issue certificates that have any defined application policy or EKU. This includes OCSP. It is not necessary for the CA certificate to explicitly contain the OCSP signing EKU. If it did contain the OCSP signing EKU, it would also have to explicitly contain the EKU values for any other kinds of certificate that it issues. This is because if any EKU or policies are explicitly defined, the certificate is valid only for the EKU or policies that are included, and the CA would be unable to issue certificates that have other policies.
For a CA that chains to a specific certificate vendor (or other third-party root) to be able to issue OCSP certificates, the root certificate also has to either contain the OCSP EKU explicitly or have no OCSP EKU defined at all. Typically, root certificates are not constrained at all. However, we apply constraints to the third-party roots that we include in Windows when they are included in the trusted root list, even though the certificate itself is actually unconstrained.
Functionally, when the CA issues an OCSP certificate, it performs a standard chain validation on the OCSP certificate. This fails because the OCSP certificate has the OCSP EKU whereas the rest of the issuing CAs are constrained but do not have the OCSP EKU.