A  number  of  facial  expression  recognition  algorithms  based  on  hand-crafted  features  and  deep  neutral  networks have been developed. Motivated by the similarity between the hand-crafted features and features learned by deep network, a  new  feature  loss  is  proposed  to  embed  the  information  of hand-crafted  features  into  the  training  process  of  network, which tries to reduce the difference between the two features. Based on the feature loss, a general framework for embedding the  traditional  feature  information  was  developed  and  tested using CK+, JAFFE and FER2013 datasets. Experimental results show that the proposed network achieves much better accuracy than the original hand-crafted feature and the network without using our feature loss. When compared with other algorithms in literature, our network also achieved the best performance on CK+ dataset, i.e. 97.35% accuracy has been achieved.