How do I convert video to mp3 audio on my pc?

Hi folks, I got a bunch of videos (mostly MP4s and MKVs) and I just want to rip the audio out as MP3s, ideally without killing the quality. I'd rather not use riksy online video to mp3 converters be...