WebView2 based app and Steam Overlay

[Elanis]

Hello !

I'm currently building an application using Tauri which uses WebView2 under the hood .... and I'm stuck on a pretty huge issue with Steam Overlay. It looks like it doesn't hook up to my app. Tauri team agree that doesn't look like an error on my side or their.

Here's what Steamworks Documentation says about Steam Overlay:

Your game does not need to do anything special for the overlay to work, it automatically hooks into any game launched from Steam! While in development and running your game in a debugger, the overlay is loaded when you call SteamAPI_Init. As such you'll need to make sure to call SteamAPI_Init prior to initializing the OpenGL/D3D device, otherwise it won't be able to hook the device creation.

The overlay supports games that use DirectX 7 - 12, OpenGL, Metal, and Vulkan. The overlay will not be active in software-rasterized games.

Does anyone here have knowledge about internals and rendering of Webview2 and can tell me why this doesn't work ? (Or at least, if you have a small idea, it would help me a lot 😄 )
It works with a chromium based app (Electron, nw.js), but not with Tauri (WebView2 based), from my own experience on this.

Thanks.

PS: see this Tauri issue if you want more informations: tauri-apps/tauri#6196

[novac42]

Thanks for your bug report. I've assigned this to a dev that can best try to replicate the issue and follow up on this.

[bradp0721]

novac42, so I'm not familiar with Tauri or the Steam overlay, so I have a few questions. Do we know how Tauri is using WebView2? Does it use one of our WebView2 framework elements (WPF, WinForms, WinUI)? Is it using the CoreWebView2 API surface directly? If yes, is Tauri using CreateCoreWebView2Controller or CreateCoreWebView2CompositionController?

Depending on how Tauri creates the WebView2, it is either being hosted in the application through a cross-proc child HWND or getting rendered into a DComp or WinComp visual that is connected to the app's visual tree.

Do we know how this works with Electron? How is an Electron app able to get the D3D device to call SteamAPI_Init?

[joshf26]

Thanks for your response Brad Peters (@bradp0721). I'm not very familiar with this but I'm also interested in using the Steam overlay with Tauri, so I looked into the Tauri source code and can answer this question:

is Tauri using CreateCoreWebView2Controller or CreateCoreWebView2CompositionController?

It appears to be using CreateCoreWebView2Controller: https://github.com/tauri-apps/wry/blob/6ef820b97dd505bacdc7d3f906112ffe0a6a1e60/src/webview/webview2/mod.rs#L187

[bradp0721]

Since Tauri is using CreateCoreWebView2Controller, the WebView2 is creating a child HWND in the application process. That HWND has a cross-proc child HWND in the msedgewebview2.exe process. WebView2 does not create a D3D Device in the application process or use the application's D3D Device. I'm not sure whether the issue is child HWNDs, something in WebView2, or something in Tauri.

Joshua Franklin (@joshf26), to try to narrow this down, could you try the following and see which ones do and don't work with the Steam overlay?

• Create a Win32 button in an application.
• Use the WebView2APISample app
• Use the WebView2 WinComp Sample

[Elanis]

Thanks to both of you for your answers ! I haven't been able to look into this earlier, but I'm glad you guys look into this issue.

I'm not used to c++ usage so it's kinda difficult for me to do these tests.
I succeed to start WebView2APISample app with Steam but without Steam overlay working but I'm not sure if this come really from the bug or from my lack of knowledge in this field.

I'm pretty sure starting SteamAPI_Init at the begin of WinMain() is a right way to do it as per steamworks documentation that state it should be done prior to device initialization

While in development and running your game in a debugger, the overlay is loaded when you call SteamAPI_Init. As such you'll need to make sure to call SteamAPI_Init prior to initializing the OpenGL/D3D device, otherwise it won't be able to hook the device creation.

But I'd prefer one of you checking to be sure 😅

If I have enough time before you're able to test, I will try to create different cases to help debug (e.g. a simple opengl context with overlay) to be sure of my C++ code, then adapting for your 3 examples.

[skymen]

Interesting article made by the lead developer of Construct about that issue of rendering the Steam Overlay over WebView2
https://www.construct.net/en/blogs/ashleys-blog-2/trying-show-steam-overlay-1861

A possible solution that could be done would be to make the --in-process-gpu merge all 3 threads into one so the rendering can happen on the main thread.

[AshleyScirra]

I wrote that blog, and while I think this is something Valve ought to fix, it might be possible for the WebView2 team to work around it. I suspect the problem is that WebView2 uses a multi-process architecture, and Steam only supports adding the overlay for rendering done in the main process. In other tools like Electron and NW.js the --in-process-gpu flag fixes this, as the GPU code then runs in the main process.

However with WebView2 it seems even with --in-process-gpu, there is still a separate app process and WebView2 process, so the GPU code still runs in a different process and so Steam doesn't show its overlay. I went as far as trying to work around this with DirectComposition, but Steam renders its overlay with unpremultiplied alpha (!), and DirectComposition uses premultiplied alpha, so it looks wrong.

If it was technically feasible for WebView2 to support a mode where the main app process can host the GPU process, or even WebView2 host itself in the main app process instead of its own process, then I think that will work around this as it would be possible for Steam to show its overlay on rendering done in the main process.

[Elanis]

Thanks for both your answers !

That could indeed be cool if we were able to have an integrated GPU process.
I actually didn't think about reaching Steam support, but that might be a solution as well.

[Gurigraphics]

Fake Steam Overlay

1. Capture the "Shift+Tab" event from the system (not from window) with SetWindowsHookEx
2. Use to creates a new window with directx11 on top of Webview2
3. Focus on the window: SetFocus(hWndNewWindow)
4. Simulate "Shift+Tab" to activate the overlay

---

However, I was unable to perform step 4 simulate keyboard event in new window with

void SimulateShiftTab(HWND hWnd) {
    SendMessage(hWnd, WM_KEYDOWN, VK_SHIFT, 0);
    Sleep(1000);
    SendMessage(hWnd, WM_KEYDOWN, VK_TAB, 0);
    SendMessage(hWnd, WM_KEYUP, VK_TAB, 0);
    Sleep(1000); 
    SendMessage(hWnd, WM_KEYUP, VK_SHIFT, 0);
}

Not with:

void SimulateShiftTab(HWND hWnd) {
    SetFocus(hWnd); 
    keybd_event(VK_SHIFT, 0, KEYEVENTF_EXTENDEDKEY, 0);
    Sleep(1000);
    keybd_event(VK_TAB, 0, KEYEVENTF_EXTENDEDKEY, 0);
    Sleep(1000);
    keybd_event(VK_TAB, 0, KEYEVENTF_KEYUP, 0);
    Sleep(1000);
    keybd_event(VK_SHIFT, 0, KEYEVENTF_KEYUP, 0); 
}

Not with:

void SimulateShiftTab(HWND hWnd) {
    SetFocus(hWndNewWindow); 
   
    INPUT inputs[4] = {};
    ZeroMemory(inputs, sizeof(inputs));

    inputs[0].type = INPUT_KEYBOARD;
    inputs[0].ki.wVk = VK_SHIFT;
   
    inputs[1].type = INPUT_KEYBOARD;
    inputs[1].ki.wVk = VK_TAB;

    inputs[2].type = INPUT_KEYBOARD;
    inputs[2].ki.wVk = VK_TAB; 
    inputs[2].ki.dwFlags = KEYEVENTF_KEYUP;

    inputs[3].type = INPUT_KEYBOARD;
    inputs[3].ki.wVk = VK_SHIFT;
    inputs[3].ki.dwFlags = KEYEVENTF_KEYUP;

    UINT uSent = SendInput(ARRAYSIZE(inputs), inputs, sizeof(INPUT));
    if (uSent != ARRAYSIZE(inputs))
    {
        
    } 
}

Keyboard events only worked with pyautogui

import pyautogui
import pygetwindow as gw

window_name = 'Steam Overlay'
window = gw.getWindowsWithTitle(window_name)

if window:
    window[0].activate()   
    pyautogui.hotkey('shift', 'tab')
else:
    print(f'window with name "{window_name}" not found.')

In this case the solution would be to render an image with the message:

Press Shift+Tab again to open the Steam Overlay

Anyway, when exiting the steam overlay, you need this other message

Press Esc to return to the game

And destroy the window with Esc