With the current setup, it does work, however it requires you entering both the username and pat in the token field. It would be nicer if you could have a username field to keep them seperate.
git ls-remote https://<your_github_username>:<your_pat>@github.com/<your_username>/<your_repo>.git