Sometimes for obscure reasons I ssh into a device and want to start a graphical
app on the device. Maybe I wanna reload new groups on my user without
restarting my compositor or maybe I just wanna troll somebody.

You essentially just need to set a few environment variables to access the
graphical session and be the same user.

Wayland
XDG_RUNTIME_DIR=/run/user/1000
WAYLAND_DISPLAY=wayland-1

X11
DISPLAY=0

You might have to adjust the userid or display number based on your setup.

This only deals with the most basic display things. If there are further
libraries and/or utilities like a polkit, you'll additionally have to link
that.