Abstract: In order to make up for the insufficiency of WIMP-based user interface model for vision interaction,a new user interface model based on vision is proposed.First,we analyze the process of vision interaction.Then,based on the activity theory,a vision-based model is proposed,which is user centered,task oriented and event driven.Then the structure and relationship of the model is detailed.Finally,we report an interactive prototype system based on vision gesture.The prototype system is able to support interactive design and satisfy the demand of usability.