Is giving private data to big models a privacy breach if it can't be searched?

I came across an interesting question on social media by chance, what is your opinion?