This is expected behaviour as TOC is a field, not a text.
There are many ways of doing what you want. One of them is to style the text you don't want to have included in your TOC as not heading (apply similarly looking style with 'Outline level' set to Body text). Another is to (unless it is level 1 heading) limit the TOC to higher levels only. And many others, all are, however, workarounds.