To recap, here is where we left off:

    if (d.try_as<::INoMarshal>()) {
        in_context_deleter del;
        void* p;
        if constexpr (std::is_reference_v<Delegate>) {
            p = winrt::detach_abi(d);
        } else {
            winrt::copy_to_abi(d, p);
        }
        return
            [p = std::unique_ptr<void, in_context_deleter>(p, std::move(del)),
            token = get_context_token()](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    }

Precreating the deleter means that an exception in its construction happens before we do any funny business with the raw pointer. That way, we close the gap between creating the raw pointer (with its reference obligation) and putting it into a unique_ptr.

Or did we?

In C++, the order of construction of the captures of a lambda is unspecified.

[expr.prim.lambda.capture]

(10.2) ⟦ … ⟧ For each entity captured by copy, an unnamed non-static data member is declared in the closure type. The declaration order of these members is unspecified.

Since the order of construction is the order of declaration, the fact that the declaration order is unspecified implies that the order of construction is unspecified. And just to make sure you get the point, this is reiterated in paragraph 15 where it discusses the initialization of captures:

(15) ⟦ … ⟧ These initializations are performed when the lambda-expression is evaluated and in the (unspecified) order in which the non-static data members are declared.

Therefore, it’s possible that the get_context_token() happens before the creation of the std::unique_ptr, and if get_context_token() fails, then the reference held in the raw pointer is leaked because it never got put into a unique_ptr.

One solution is to put it into a unique_ptr before we create the lambda.

    if (d.try_as<::INoMarshal>()) {
        in_context_deleter del;
        void* p;
        if constexpr (std::is_reference_v<Delegate>) {
            p = winrt::detach_abi(d);
        } else {
            winrt::copy_to_abi(d, p);
        }
        std::unique_ptr<void, in_context_deleter> up(p, std::move(del));
        return
            [p = std::move(up),
            token = get_context_token()](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    }

By creating the unique_ptr immediately, we remove any opportunity for an exception to sneak in between the time we create an obligation in the raw pointer and the time we assign that obligation to the unique_ptr.

One thing that bugs me about this is that we introduce another unique_ptr, which means that its destructor will have to check something for null, when it’s almost always null.

We can avoid this temporary unique_ptr by using copy elision directly into the capture.

    if (d.try_as<::INoMarshal>()) {
        auto make = [](auto&& d) {
            in_context_deleter del;
            void* p;
            if constexpr (std::is_reference_v<Delegate>) {
                p = winrt::detach_abi(d);
            } else {
                winrt::copy_to_abi(d, p);
            }
            return std::unique_ptr<void,                      
                       in_context_deleter>(p, std::move(del));
        };                                                    
        return
            [p = make(std::forward<Delegate>(d)),
            token = get_context_token()](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    }

Bonus reading: Previously, in copy elision.

But an easier solution is to create the token early, just like we did with the deleter.

    if (d.try_as<::INoMarshal>()) {
        in_context_deleter del;
        auto token = get_context_token();
        void* p;
        if constexpr (std::is_reference_v<Delegate>) {
            p = winrt::detach_abi(d);
        } else {
            winrt::copy_to_abi(d, p);
        }
        return
            [p = std::unique_ptr<void, in_context_deleter>(p, std::move(del)),
             token](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    }

Okay, are we done?

Maybe.

But maybe this all wasn’t worth it.

We’ll talk about that next time.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 8 appeared first on The Old New Thing.

Let’s take another look at what we have:

    if (d.try_as<::INoMarshal>()) {
        void* p;
        if constexpr (std::is_reference_v<Delegate>) {
            p = winrt::detach_abi(d);
        } else {
            winrt::copy_to_abi(d, p);
        }
        return
            [p = std::unique_ptr<void, in_context_deleter>(p, {}),
            token = get_context_token()](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    }

We had originally broken the rule that the unique_ptr(p) constructor requires that the deleter’s default constructor not throw an exception. We fixed it by constructing the deleter explicitly as a parameter, so that the unique_ptr constructor can move it into the stored deleter without an exception.

But wait, if an exception occurs in construction of the in_context_deleter, the raw pointer we created in the previous block will be leaked. It owns a reference count but doesn’t clean up in the case of an exception.

We can fix this by creating the deleter first.

    if (d.try_as<::INoMarshal>()) {
        in_context_deleter del;
        void* p;
        if constexpr (std::is_reference_v<Delegate>) {
            p = winrt::detach_abi(d);
        } else {
            winrt::copy_to_abi(d, p);
        }
        return
            [p = std::unique_ptr<void, in_context_deleter>(p, std::move(del)),
            token = get_context_token()](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    }

If there is an exception constructing the custom deleter, it happens before we initialze the raw pointer, so there is no leak of the reference owned by that raw pointer.

Okay, so are we done now?

Nope.

More next time.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 7 appeared first on The Old New Thing.

But we missed something.

Again.

    if (d.try_as<::INoMarshal>()) {
        void* p;
        if constexpr (std::is_reference_v<Delegate>) {
            p = winrt::detach_abi(d);
        } else {
            winrt::copy_to_abi(d, p);
        }
        return
            [p = std::unique_ptr<void, in_context_deleter>(p),
            token = get_context_token()](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    }

The first part gets a raw ABI pointer, either by moving it out of the inbound delegate if we can, else by copying it from the inbound delegate. The reference count is owned by the raw pointer.

The second part wraps the raw ABI pointer inside a std::unique_ptr with our custom deleter. The unique pointer now owns the reference count, and the custom deleter will release it.

The problem is that one of the requirements for a custom deleter is that if you use the unique_ptr(p) constructor, the custom deleter must not throw an exception at construction.

[unique.ptr.single.ctor]

constexpr explicit unique_ptr(type_identity_t<pointer> p) noexcept;

Constraints: is_pointer_v<deleter_type> is false and is_default_constructible_v<deleter_type> is true.

Preconditions: D meets the Cpp17DefaultConstructible requirements, and that construction does not throw an exception.

But our custom deleter could throw an exception if Co­Get­Object­Context fails. So it doesn’t meet the preconditions.

We can fix that by using the constructor that takes an explicit deleter from which the stored deleter can be move-constructed. If an exception occurs, it happens during the creation of the parameter and not inside the unique_ptr constructor.

    if (d.try_as<::INoMarshal>()) {
        void* p;
        if constexpr (std::is_reference_v<Delegate>) {
            p = winrt::detach_abi(d);
        } else {
            winrt::copy_to_abi(d, p);
        }
        return
            [p = std::unique_ptr<void, in_context_deleter>(p, {}),
            token = get_context_token()](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    }

Okay, so now we’re done?

Nope, still broken.

More next time.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 6 appeared first on The Old New Thing.

But we missed something.

Let’s look at it again.

    if (d.try_as<::INoMarshal>()) {
        return [d, token = get_context_token(),
                context = winrt::capture<IContextCallback>(CoGetObjectContext)](auto&&...args) {
            if (token == get_context_token()) {
                d(std::forward<decltype(args)>(args)...);
            } else {
                throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
            }
        };
    }

While we copy and invoke the delegate only from its original context, we still destruct it from a possibly-wrong context.

This is a serious problem not just because the non-agile delegate might not be using thread-safe atomic instructions to manage its reference count, but even worse, if the release drops the reference count to zero, the delegate will destruct on the wrong thread, and that will probably create lots of problems.

We have to destruct the non-agile delegate in its original context. We can do this with a std::unique_ptr and a custom deleter. The std::unique_ptr handles all the move operations and the deleter cleans up the last pointer.

struct in_context_deleter
{
    winrt::com_ptr<IContextCallback> context =
                winrt::capture<IContextCallback>(CoGetObjectContext);

    void operator()(void* p)
    {
        if (p) {
            ComCallData data{};
            data.pUserDefined = p;
            context->ContextCallback([](ComCallData* data) {
                winrt::IUnknown{ data->pUserDefined, winrt::take_ownership_from_abi };
                return S_OK;
            }, &data, __uuidof(IContextCallback), 5, nullptr);
        }
    }
};

This stateful deleter remembers the context to use for final destruction. When it’s time to do the destruction, we switch into the target context via IContext­Callback::Context­Callback and take ownership of the raw pointer into a winrt::IUnknown. The destructor of the winrt::IUnknown will perform the release.

We can use this stateful deleter around the raw delegate pointer.

// Don't use this yet - read to the end of the series

template<typename Delegate>
std::remove_reference_t<Delegate> make_agile_delegate(Delegate&& d)
{
    if (d.try_as<::IAgileObject>()) {
        return d;
    }
    if (d.try_as<::INoMarshal>()) {
        void* p;                                      
        if constexpr (std::is_reference_v<Delegate>) {
            p = winrt::detach_abi(d);                 
        } else {                                      
            winrt::copy_to_abi(d, p);                 
        }                                             
        return
            [p = std::unique_ptr<void, in_context_deleter>(p),
             /* context = winrt::capture<IContextCallback>(CoGetObjectContext), */
            token = get_context_token()](auto&&...args) {
                if (token == get_context_token()) {
                    std::remove_reference_t<Delegate> d;
                    winrt::copy_from_abi(d, p.get());
                    d(std::forward<decltype(args)>(args)...);
                } else {
                    throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
                }
            };
    } else {
        return [agile = winrt::agile_ref(d)](auto&&...args) {
            return agile.get()(std::forward<decltype(args)>(args)...);
        };
    }
}

The first block gets a raw ABI pointer, either by moving it out of the inbound delegate if we can, else by copying it from the inbound delegate.

The second part wraps the raw ABI pointer inside a std::unique_ptr with our custom deleter, and the custom deleter makes sure that the Release of the original delegate happens in the correct context.

Are we done?

No!

More next time.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 5 appeared first on The Old New Thing.

    if (d.try_as<::INoMarshal>()) {
        return [d = std::forward<Delegate>(d),
                context = winrt::capture<IContextCallback>(CoGetObjectContext)](auto&&...args) {
            if (context == winrt::capture<IContextCallback>(CoGetObjectContext)) {
                d(std::forward<decltype(args)>(args)...);
            } else {
                throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
            }
        };
    }

We did this by comparing context objects.

This obtains the current object context in order to compare it with the original one, and that means an internal Add­Ref, and then we have to explicitly Release it.

But there’s a way to do this without having to obtain any objects.

The Co­Get­Context­Token function gives you an integer that uniquely identifies a live context object. You can then compare integers instead of having to compare COM objects.

Note that the context must be live. Once you allow the context to destruct, the value might be reused. (You’re already used to this. Process and thread IDs work the same way: They remain unique as long as they are running or you still have a reference to them by a HANDLE.)

Since we are keeping the context alive by the IContext­Callback returned by Co­Get­Object­Context, we can pair that with a context token to make for faster checks in the future.

ULONG_PTR get_context_token()                       
{                                                   
    ULONG_PTR token;                                
    winrt::check_hresult(CoGetContextToken(&token));
    return token;                                   
}                                                   

    if (d.try_as<::INoMarshal>()) {
        return [d = std::forward<Delegate>(d),
                context = winrt::capture<IContextCallback>(CoGetObjectContext),
                token = get_context_token()](auto&&...args) {
            if (token == get_context_token()) {
                d(std::forward<decltype(args)>(args)...);
            } else {
                throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
            }
        };
    }

Are we done?

Of course not!

There’s a flaw in the above code. More next time.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 4 appeared first on The Old New Thing.

That case is an object that implements the INo­Marshal interface, which means “Do not marshal this object.” For these objects, the Ro­Get­Agile­Reference function fails to create an agile reference and returns CO_E_NOT_SUPPORTED. This function is what powers the agile_ref class, so if you ask for an agile_ref to an object that refuses to be marshaled, you get an CO_E_NOT_SUPPORTED exception.

The catch is that this error is produced at the creation of the agile reference. If in practice all your uses of the agile reference are from the original context, you never actually needed to marshal the object, but too bad, you get the error anyway.

So let’s teach our agile delegate wrapper about delegates that deny marshalability: If the wrapper is invoked on the same context that the original delegate belongs to, then everything is fine. But if you try to invoke the wrapper from another context, then you get the CO_E_NOT_SUPPORTED exception.

Here’s our first try. (Foreshadowing: Since I call this a “first try”, that suggests we’re going to have a second try.)

// Don't use this yet - read to the end of the series

template<typename Delegate>
Delegate make_agile_delegate(Delegate const& d)
{
    if (d.try_as<::IAgileObject>()) {
        return d;
    }

    if (d.try_as<::INoMarshal>()) {                                                                
        return [d, context = winrt::capture<IContextCallback>(CoGetObjectContext)](auto&&...args) {
            if (context == winrt::capture<IContextCallback>(CoGetObjectContext)) {                 
                d(std::forward<decltype(args)>(args)...);                                          
            } else {                                                                               
                throw winrt::hresult_error(CO_E_NOT_SUPPORTED);                                    
            }                                                                                      
        };                                                                                         
    }                                                                                              

    return [agile = winrt::agile_ref(d)](auto&&...args) {
        return agile.get()(std::forward<decltype(args)>(args)...);
    };
}

If the object has the INo­Marshal marker interface, then we capture the original context into our wrapper delegate. At invoke time, the wrapper checks whether the invoke context equals the captured context. If so, then all is good, and we call the original delegate. Otherwise, we throw the exception that Ro­Get­Agile­Reference uses to say “Sorry, I can’t marshal this object.”

If we take a universal reference to the Delegate, we gain the ability to std::move() out of the inbound delegate if it is an rvalue reference.

template<typename Delegate>
std::remove_reference_t<Delegate> make_agile_delegate(Delegate&& d)
{
    if (d.try_as<::IAgileObject>()) {
        return d;
    }

    if (d.try_as<::INoMarshal>()) {
        return [d = std::forward<Delegate>(d),
                context = winrt::capture<IContextCallback>(CoGetObjectContext)](auto&&...args) {
            if (context == winrt::capture<IContextCallback>(CoGetObjectContext)) {
                d(std::forward<decltype(args)>(args)...);
            } else {
                throw winrt::hresult_error(CO_E_NOT_SUPPORTED);
            }
        };
    }

    return [agile = winrt::agile_ref(d)](auto&&...args) {
        return agile.get()(std::forward<decltype(args)>(args)...);
    };
}

Next time, we’ll look at a small optimization we can make to this implementation to reduce the amount of work needed to check the context at invoke time.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 3 appeared first on The Old New Thing.

In many cases, the delegate is already agile, so there’s no need to make an agile wrapper for something that is already agile. We can detect this case by looking for the marker interface IAgile­Object, which all agile objects possess.

template<typename Delegate>
Delegate make_agile_delegate(Delegate const& d)
{
    if (d.try_as<::IAgileObject>()) {
        return d;                    
    }                                

    return [agile = winrt::agile_ref(d)](auto&&...args) {
        return agile.get()(std::forward<decltype(args)>(args)...);
    };
}

If the delegate declares itself as agile, then the delegate can be its own agile wrapper.

Wait, there another case that we missed. We’ll look at that next time.

Bonus chatter: You might think that we could try to get copy elision in the case of an agile delegate:

template<typename Delegate>
Delegate make_agile_delegate(Delegate d)
{
    if (d.try_as<::IAgileObject>()) {
        return d; // copy elision?
    }

    return [agile = winrt::agile_ref(d)](auto&&...args) {
        return agile.get()(std::forward<decltype(args)>(args)...);
    };
}

Unfortunately, this doesn’t work because function parameters are not eligible for copy elision.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 2 appeared first on The Old New Thing.

The easy way is to wrap the delegate in an agile_ref, and then resolve the agile_ref back to a delegate when you want to invoke it.

template<typename Delegate>
Delegate make_agile_delegate(Delegate const& d)
{
    return [agile = winrt::agile_ref(d)](auto&&...args) {
        return agile.get()(std::forward<decltype(args)>(args)...);
    };
}

But if it were that easy, why would we call this article “part 1”?

More in part 2.

The post Making an agile version of a Windows Runtime delegate in C++/WinRT, part 1 appeared first on The Old New Thing.

When we checked the driver version numbers on all the crashing systems, they were something like “build 314”, when the current driver build number is something like “build 2718”. These users are running drivers that are ridiculously old! The vendor fixed that bug ages ago, but the user hasn’t gotten the fix. What’s going on?

My theory was that these users have turned off Windows Update or are otherwise declining to upgrade their video drivers. But I learned that my theory was probably wrong.

The deal here is that these are video drivers, which are a category of drivers where computer manufacturers have a lot of control. The manufacturer certifies the drivers for use on their PCs after performing their own acceptance testing on their specific hardware configurations. (Which are probably not hardware configurations that the video card vendors themselves are aware of.)

This responsibility carries forward post-sale. The computer manufacturer remains responsible for certifying driver updates, presumably by testing them against reference PCs that they maintain in their labs. Sometimes, manufacturers get customized versions of the video cards (all the better to differentiate your product with, my dear), which is why the video card vendor “driver downloads” sites often warn you to check with your computer manufacturer before installing a driver.

In practice, computer manufacturers are diligent about certifying drivers for a year, year and a half, two years tops.¹ After that, it’s not uncommon for them to abandon that model and not bother certifying drivers for it any more. All customers with that model of PC are just stuck with whatever video drivers were current as of the time the manufacturer stopped certifying drivers.

Microsoft maintains generic drivers for many classes of hardware, but intentionally sets them as low priority so that the PC manufacturer-provided drivers take precedence. The video drivers received directly from video card manufacturers are similarly deprioritized by the video card vendors. The computer manufacturer-certified drivers take precedence, even if that certification is horribly out of date.

¹ I wouldn’t be surprised if the length of time they certify drivers is somehow correlated with the length of the computer warranty.

The post Why has the display control panel pointer truncation bug gone unfixed for so long? appeared first on The Old New Thing.

How could something like this happen?

My guess is that this code started out as perfectly good 32-bit code:

HWND hwndButton = GetDlgItem(hdlg, ID_BUTTON);
SetWindowLong(hwndButton, GWL_WNDPROC, (LONG)g_originalWndProc);

And then they recompiled it as 64-bit code and got an error.

error C2065: 'GWL_WNDPROC': undeclared identifier

They then went back to the documentation and saw that for 64-bit Windows, GWL_WNDPROC was renamed to GWLP_WNDPROC.

So they fixed it by changing GWL_WNDPROC to GWLP_WNDPROC.

HWND hwndButton = GetDlgItem(hdlg, ID_BUTTON);
SetWindowLong(hwndButton, GWL_WNDPROC, (LONG)g_originalWndProc);

However, the point of renaming the value was not to annoy you. The point of renaming the value was to call your attention to places where pointer truncation is likely to occur. In this case, it’s the final parameter, the original 64-bit window procedure. The build break is telling you that you are probably passing a 32-bit value as something that should be 64-bit. In this case, because it was being cast to (LONG). You are expected to upgrade the GWL_WNDPROC to GWLP_WNDPROC and at the same time upgrade the cast from (LONG) to (LONG_PTR).

HWND hwndButton = GetDlgItem(hdlg, ID_BUTTON);
SetWindowLong(hwndButton, GWL_WNDPROC, (LONG_PTR)g_originalWndProc);

Now, this was likely an oversight rather than a systemic failure, because they did manage to subclass the window properly:

WNDPROC g_originalWndProc;

HWND hwndButton = GetDlgItem(hdlg, ID_BUTTON);
g_originalWndProc = (WNDPROC)SetWindowLong(hwndButton, GWLP_WNDPROC,
    (LONG_PTR)subclassWndProc);

They merely missed a spot. Perhaps the developer got distracted after fixing the symbol name and forgot to come back and fix the pointer.

Next time, we’ll look at why this bug has remained unfixed for so long.

The post Speculating on how the buggy control panel extension truncated a value that it had right in front of it appeared first on The Old New Thing.