"Talks about Sea Raids"
The reason why there were all those castles was because of a constant series of internal wars in Europe between various lordlings over petty bullshit.

1st Crusade: Pope has to deal with an anti-pope and needs to establish his authority. A Holy War should help. Never actually end up helping the Byzantine Empire because you never give them the territory you conquered, even though the Byzantines did a lot of the heavy lifting.

4th Crusade: The Pope wanted to unite western and eastern Christians under his authority. He diverted
this Crusade, with the help of Venice, and captured Constantinople in 1204. Christians fought Christians. Constantinople never recovered and that weakened state ultimately led to their conquest by the Ottomans.

Like looking historically at Crusades from Islamic perspective, the Crusades to them were like the Viking raids to us: unwashed, uneducated barely literate barbarians coming to rape and pillage. And that point of view wasn't entirely wrong during that time. Literacy rates in Europe were lower than in the Islamic world, India, China, Japan etc.

Also, Al-Andalus, Islamic Spain, was one of the most developed places on EARTH during that time, with an emerging financial banking system and advanced healthcare. Part of why pilgrimages to Spain were so popular was so that nobles would have an excuse to go to Cordoba and visit the banks for a loan and maybe also get a medical checkup without raising suspicion.