Welcome to the 1980's!That would be cool to have windows (and session) manager that can manage windows from several X servers on one X terminal.
That is exactly how X was designed to work. A dumb "terminal" with graphics that displayed on behalf of programs running else where.
You have things backwards though. The terminal/display runs the X server, the remote programs that want to display are the X clients. The programs connect to the display, not the other way around.
Last time I checked this still all worked except a lot of modern programs require direct access to hardware for rendering so they don't work over the net.