Human Epithelial type 2 (HEp-2) cells are of great importance in the diagnosis of autoimmune disorder. Traditional approach requires specialists to manually observe the cells and make decisions, which is laborious, time-consuming and easily influenced by subjective experiences. Therefore, in this paper, we proposed a general framework based on Gabor Ternary Pattern (GTP) and joint sparse representation to automatically classify cell images. The method firstly searches the affine invariant key points in cell images by a multiscale Canny detector, and then extracts GTP features from the local region around the points. The sparse representation classifier is applied to determine the labels of the cell images. We conduct experiments on the publicly available ICPR cell image database and get a promising result. The experiments show that the approach based on GTP outperforms the SIFT-based approach and the sparse representation classifier provides excellent performances.