Creating a window and Vulkan context and receiving input events (e.g. what game-like applications need) is fairly trivial via Xlib but far worse in Wayland. The Xlib API also sucks compared to Win32 or Cocoa, but writing a Wayland client for something as trivial as creating and managing a window is much worse than on any other platform (see: https://www.p4m.dev/posts/29/index.html)
Basic window system functionality really needs to be provided by the OS, not by 3rd-party libraries. And there's no reason why simple things like creating a window should be drastically more code in an OS API than in a wrapper library like SDL.