What made Joseph decide to take his family to Egypt?

An angel told to him in a dream.

The angel of the Lord appeareth to Joseph in a dream, saying, Arise, and take the young child and his mother, and flee into Egypt, and be thou there until I bring thee word: for Herod will seek the young child to destroy him. Matthew 2:13