With the expansion of the application of classification analysis, the classification of multi-source heterogeneous data has recently received a significant amount of attention in the fields of data mining and machine learning. However, due to the lack of prior knowledge, it is still challenging to effectively integrate the complementarity and correlation among multi-view features to classification analysis. Motivated by this scientific problem, there are three main themes within the proposed research based on ensemble learning theory: (1) Build the support vector-tensor machine ensemble models for supervised classification problems via joint feature selection and classifier design; (2) Build the semi-supervised support vector-tensor machine ensemble models for semi-supervised classification problems via joint feature selection and classifier design; (3) Design the nonlinear multi-kernel based on vector-tensor compound pattern and joint feature selection and (semi-supervised) support vector machine for nonlinear classification problems. What is of significance in this proposal will be not only building some support vector-tensor machine models and designing more algorithms for various applications, but also making the research contents of data mining and machine learning richer and promoting research and development of machine learning and mathematical theory. Expected outcomes of the proposed research will provide solutions for the critical problems of the classification of multi-source heterogeneous data, and lead to novel techniques and fundamental theoretical basis for their applications in various fields, as well as provide a new theoretical perspective for ensemble learning based algorithms.