This work aims at dynamically understanding the properties of a scene from the analysis of moving object trajectories. Two different applications are proposed: the first one is devoted to identify abnormal behaviors, while the latter allows to extract the k most similar trajectories to the one handdrawn by an human operator. A set of normal trajectoriesâ models is extracted by means of a novel unsupervised learning technique: the scene is adaptively partitioned into zones by using the distribution of the training set and each trajectory is represented as a sequence of symbols by taking into account positional information (the zones crossed in the scene), speed and shape. The main novelties are the following: first, the use of a kernel based approach for evaluating the similarity between trajectories. Furthermore, we define a novel and efficient kernelbased clustering algorithm, aimed at obtaining groups of normal trajectories. Experimentations, conducted over three standard datasets, confirm the effectiveness of the proposed approach.