The aim of this study was to determine whether motor repertoire can influence kinematic observation (i.e., observing action from only the movement of an actor's main joints) in improving episodic memory performance and whether there are differences between young and older adults. To investigate this, 45 young and 45 older participants performed a free recall task after an encoding phase consisting of the visual presentation and naming of point-light displays (PLDs) representing actions performed by either a young or an older actor. The results show that matching the age of the actor represented in the PLDs to the observers' age improves recall performance for both young and older participants, and that the age effect in episodic performance is reduced when PLD represented an older actor. This study then shows that the effect of kinematic observation on memory is modulated by the observer's own motor repertoire, suggesting that memory is directly based on sensorimotor simulation.