I recently watched a video where they tested if the replies were generic/context-generic or fed by their own comments.

They replied to comments on their video with a fake ad. And sure enough, after a while, the AI suggested such replies.

Here’s the video

Is this summary helpful?

When the summary is presented there, like that, of course, it seems helpful.

I’m certain the majority of people will respond with “yes”, without checking the user’s post history to verify the summary. The responses will be biased, making the whole thing worse, and giving opportunity to disingenuous marketing and justification.

We can only hope they interpret and use it for that: indicating usefulness, not correctness. Who knows what kind of metrics they use for correctness, if at all, and what type of faults they deem acceptable.