Can you write something like, window that displays a blue square on a black background, and this square moves when you press arrow keys?

Something with minimal use of libraries, and that even uses syscalls directly instead of using libc. Studying musl internals may help here.

I'm not sure what api you should use for this exactly, maybe x window system? Or maybe something like gtk/qt? Or framebuffer? What is most widespread, what is most lowlevel/fast?

I'm trying to do this for windows right now, would be nice if somebody would study this for linux in parallel. It's quite a fruitless job, the only advantage is being able to say that you know every byte in your program.

Here is a good starting point: https://www.reddit.com/r/ProgrammingLanguages/comments/7il6f1/a_quick_beginners_guide_to_handcrafting/

Huh, actually this guy is pretty close right now: https://www.reddit.com/r/ProgrammingLanguages/comments/88stz8/monthly_what_are_you_working_on_how_is_it_coming/dwsv3nz/?context=0

A long time ago, SDL was pretty common. It was cross-platform and I was able to get it to work pretty easily. It was also supposedly powerful, making it a candidate for 3D apps. I just looked at their website and I think they said Valve used it.

A more minimal approach may be with syscalls, curious what you end up picking. I looked up what NVidia's OpenCL samples used (my Linux setup is not very functional at the moment, so I can't really install them) and I think they mention:

in the Linux variants. Vulkan's samples seem to use:

with their windowing also. Good luck...