Yes, so far I've encountered something like 10-20 (didn't count exactly) cases such as this.
This is quite embarassing : since they can't be parsed with a compliant XML parser, they're missed by my programs, meanings I have holes in my data.
And since the data is not encoded, there may be other characters that are bad, such as '<'.