The above posts may give you comfort, but face it, you will probably always feel inferior when you are around other Chinese people who can speak the language. I suggest you pick one of the many Chinese languages and learn it.
My wife is Chinese, and we recently had a child. I am 42 and started learning Mandaran a year and a half ago in night classes. I can carry on a pretty good general conversation and will soon start learning the writing. The reason I am learning the language is to support my child’s learning so my child doesn’t have the same feelings you do. It’s not that difficult to learn, especially if you apply yourself. It will reward you greatly if you do, so just go for it.