Gulliver (1977)

A mature recluse takes refuge in a town inhabited by a troupe of dwarfs and discovers that the chief keeps a beautiful woman hidden in a tower. He ends up becoming its leader.