Wait, what?I think that WAS Frigga. She has the illusion power too, and it's implied in a later scene she taught it to Loki.
Loki definitely learned his illusions from Frigga, but the scene of
her in his cell happened so soon after Odin stated that he would never see her again, I thought it was clear that it wasn't her. This was reinforced when Thor demanded no more illusions, and we see how broken Loki is (and, really, how few books were actually in his cell, compared to the massive stack he supposedly was receiving from Frigga).
My wife also saw the scene this way, so I know I'm at least not the only one who did. I can understand seeing it the other way, but I think it makes the scene far more poignant if it's
Loki having to create a false conversation. It also explains why she disappears when he reaches out to her. Having fooled himself with his own illusion, it falls apart when he's unable to touch her. If it were her illusion, the image would remain until the conversation was over (which it wasn't).