When Connor first came back Angel could still nurture the hope that underneath all the Quortoth upbringing the boy he'd dreamed about was still there. It often takes people a long time to accept a diagnosis. After Deep Down Angel never spent very much time with Connor. Admittedly, I don't remember S4 that well, it was one of those season that hurtled along but gathered no moss until Home. What I do remember is Angel only being able to love Connor at a distance. I remember him having the intention to help but but never by being close, always watching from rooftops or round corners, nothing face to face. Not that Connor's own behaviour helped but the net effect was that Angel never got to know Connor well enough for his reality to drive out the dream.

Love is complicated. I don't think Angel only loved his dream of Connor, you always end up overstating things in these debates, but in the end he let that desire for the dream win out over his love for the living boy. Lilah tempted him and he gave in. However, I don't feel that the story validated that position in Origins. In my mind, although he never intended it so and in fact worked hard to prevent it happening, Connor's memories being restored acts as a metaphor for Angel finally being able to accept his son for all he was. It's more by luck than judgement but then sometimes people do get lucky.