Abstract: We design a pertinence graphical model, combined with domain-specific heuristics among the components of human hand, to describe the hand's 3D structure, kinematics, dynamics and self-occlusion. The modular structure facilitates tracking each hand component (sixteen variables of six degrees of freedom) separately instead of tracking hand configuration of 27 degrees of freedom as a while to reduce the computational complexity. Then, a more efficient belief propagation method embedding continuously adaptive mean shift (CAMSHIFT) algorithm to obtain configuration space (C-space) is proposed. Belief propagation is processed in the feasible C-space to increase the tracking efficiency. The experimental results show that our proposed method can track articulated hand robustly and efficiently under self-occlusion.