Interpersonal sensitivity to ambiguous facial expressions of emotions was examined using individual growth modeling. Generalized socially phobic (n = 54), depressed (n = 44), comorbidly socially phobic and depressed individuals (n = 39), and non-clinical controls (n = 65) were assessed on an emotion labeling task. Patients were as good as controls in labeling happy expressions as positive, and better than controls in labeling faint angry and sad expressions as negative. Depression and social anxiety (SA) were positively correlated with a response bias, and negatively correlated with sensitivity, to angry expressions. Depression was also correlated with higher sensitivity, and SA with higher response bias, to sad expressions. The effects of depression and SA on the identification of negative expressions were sub-additive. Finally, depression, but not SA, was associated with longer examination of happy, but not angry emotional expressions. Implications of these findings for the understanding of interpersonal impairments in social phobia and depression are discussed.