Apply where clauses after union in Eloquent ORM

I'm working on a Laravel project and I need to search through 2 models as the same time.

I decided to use a union for this query, as I would do it in SQL. One of my models is converted into the other, and then the two queries are unioned. I was expecting to be able to filter as well after that, but any where clause I'm using ends up applying only on the first part of the union, and not the second.

Here's the code

EntityMember::union(
    User::select(*convert User into something similar to EntityMember here*))
)
->whereNull('user_id');

and I end up with the following query generated :

(select * from "entity_members" where "user_id" is null)
union
(select *conversion to entity_member here* from "users")

When what I would like to get is:

(
  (select * from "entity_members")
  union
  (select *conversion to entity_member here* from "users")
) where "user_id" is null

What is a way to achieve this kind of union? Which replacement method would work? Or am I doomed to do this whole query inside a selectRaw?