creating a column from comparing two columns

If extract from B into separate list values which are not in A that could be done with

=IFERROR(INDEX($B:$B,AGGREGATE(15,6,1/ISNA(MATCH($B:$B,$A:$A,0))*(ROW($B:$B)-ROW($B$1)+1)/($B:$B<>""),1+ROW($C12)-ROW($C$1))),"")

for data structured like

if drag it down till first empty cell appears (first file attached).

But on four hundred thousand rows it could take years to recalculate.

With Power Query (aka Get&Transform) it will much more effective, If convert range with data in A and B into the table, just Ctrl+T (here left table with 400000 rows is generate by helper query)

and apply query to it like

let
    Source = Excel.CurrentWorkbook(){[Name="Source"]}[Content],
    MergeWithItself = Table.NestedJoin(Source,{"B"},Source,{"A"},"Source",JoinKind.LeftAnti),
    RemoveUnused = Table.SelectColumns(MergeWithItself,{"B"}),
    RenameHeader = Table.RenameColumns(RemoveUnused,{{"B", "B not in A"}})
in
    RenameHeader

result will be the right table (another file attached).

Query loads the table from Excel sheet and left anti joins (merge) it with itself on column B against column A, rest is cosmetic.