Jami' at-Tirmidhi
Jami` at-Tirmidhi 822
حَدَّثَنَا أَبُو مُوسَى، مُحَمَّدُ بْنُ الْمُثَنَّى حَدَّثَنَا عَبْدُ اللَّهِ بْنُ إِدْرِيسَ، عَنْ لَيْثٍ، عَنْ طَاوُسٍ، عَنِ ابْنِ عَبَّاسٍ، قَالَ تَمَتَّعَ رَسُولُ اللَّهِ صلى الله عليه وسلم وَأَبُو بَكْرٍ وَعُمَرُ وَعُثْمَانُ وَأَوَّلُ مَنْ نَهَى عَنْهَا مُعَاوِيَةُ
Unreviewed translation. All hadiths in this collection are available in five languages. Titles and introductions may remain in Arabic.
Abu Musa Muhammad ibn al-Muthanna narrated to us: Abd Allah ibn Idris narrated to us, from Layth, from Tawus, from Ibn Abbas, who said: The Messenger of Allah, Abu Bakr, Umar, and Uthman performed tamattu'. The first to prohibit it was Mu'awiyah.
Notes and open questions — unreviewed
This brief report states that the Prophet and the first caliphs combined Umrah with Hajj, and attributes the first prohibition to Mu'awiyah.