You might be interested in "Text Embeddings Reveal (Almost) As Much As Text":
> We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes.
> We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes.
https://arxiv.org/pdf/2310.06816.pdf
There's certainly information loss, but there is also a lot of information still present.