According to the gospel of Matthew, when did Joseph "know" Mary?

Sometime after Jesus was born

Now the birth of Jesus Christ was on this wise: When as his mother Mary was espoused to Joseph, before they came together, she was found with child of the Holy Ghost. Matthew 1:18

Then Joseph being raised from sleep did as the angel of the Lord had bidden him, and took unto him his wife:

And knew her not till she had brought forth her firstborn son: and he called his name JESUS. Matthew 1:24-25