Audio output settings - Xbox app for PC sends game chat audio to wrong device

It's only with the game audio? Have you check on the settings of the app or even the audio settings of your windows PC, try to change your output device, put your headset on it.

Are you playing on a Xbox console?

If so, go to the audio settings in the console settings and set up the console audio to output game audio via headset.

In the headphone settings you can also set up spatial 3D audio, by turning the audio output for headphones to Windows Sonic, or if you have purchased the Atmos App (the free app only works 14 days with headphones), you can also set Dolby Atmos for headphones.

If you are on the PC, you need to make the headphone your device for audio output (not only voice communications).

Click on the speaker icon in the lower right, and choose the output channel for the headphone. Depending on your soundcard or USB headset, you need to select the proper audio output.

If you are using a capture device and capture the gameplay on a PC, make sure that you have set the chat output to external speakers as well in the audio settings.

Or are you using the Twitch app on your console?

If using headset for game audio and chat, mute the TV or any device your game and chat audio is coming through and make sure 'chat' is set to output through headset only. If you have your tv volume up in the background while you're on the headset there
will probably be echoing especially if you have your party audio set to output through your TV.

Of course, mate.

Ok:

Open the Xbox Game Bar and join/initiate a party chat (don’t leave the party till the completion of the solution).

Now open the Xbox Game Bar Settings by clicking on the Gear icon and in the menu shown, click on Party Chat (in the left half of the window).

Then set your Audio Output to the headset and if it is already set to the headset change it to another device (if you only have one single device, then try to use a dummy device from applications like Virtual Audio Cable).

Repeat the same for Audio Input and then close the Settings Windows.

Then click on the Widget Menu and choose Audio.

Now make sure the output (in the Mix tab) and input (in the Voice tab) is set to your headset. And make sure the system sounds are on max.

Then repeat steps 2 to 4 but make sure your headset is set as audio output and input device.

Now close the Settings window and check if the party chat and the game's volume have started to work fine.

If not, then check if disabling Push to Talk in the Party Chat settings of the Game Bar solves the issue