Add Delete Button in Message View

I found an alternative - add Delete to the Quick Access Bar - right at the top. Then Delete is there for All views. Select e.g. Message view, click on a blank ribbon space, select Customise the ribbon. On the left select Quick Access Bar and select Delete and Add.

There's no way to add it there, but it will still be available within the Ribbon on top, or when you hover over the message in the message list above (additional buttons will appear in the right-most column). You can also make the habit of hitting the Backspace key on your keyboard as a quick "delete" option.