Currently, if you select 1 or more email, to deselect it without pressing on each of them you need to first press the "select all emails" button, and then press it again, once it turns into "deselect all emails" to remove that selection.

This feels kinda clunky so I would like an alternative, using a key modifier (Shift + Click on the "select all emails" button) or a keybind (like Ctrl + Shift + A or something like that, so it's out of the way and harder to do accidentally) or an extra button on the UI or any combination of the previous suggestions.