They lived with Humans originally but were constantly ill-treated by them and eventually turned against them. It is thought they became enemies after their leader
Red Nose mysteriously gained power. However, it is said that some Imps have begun admiring human society lately.