Switch b/w Edge open windows only

If you want to go forward, left to right, then press CTRL + SHIFT.
If you want to go backward, right to left, then press CTRL + SHIFT + TAB.
If you want to go to a specific tab, you can press CTRL + N, where N is a number between 1 and 8. Unfortunately, you can't go past 8, so if you have more than eight tabs, you'll have to use a different keyboard shortcut or just click on it.

these are obviously for tabs, if you really meant windows, you can use virtual desktops and put multiple each windows on each one, then use CTRL + WIN key + Right/Left arrows to switch between each virtual desktop.

these are very convenient to be honest, just needs a little time to get used to the shortcuts then your hands will automatically move to the right keys 😉