The Church has traditionally used the color red for Pentecost to symbolize the flames of the Holy Spirit. At first, only members of the clergy wore red vestments, but now it is common to see members of the congregation wearing something red on Pentecost Sunday.
Red is not the only color associated with Pentecost. Red symbolizes the divided flame-like tongues but the Holy Spirit is symbolized by a white dove. White is also worn by people who are being baptized. Baptisms and confirmations are done throughout the year, but having them on Pentecost Sunday is in keeping with the day described in the book of Acts where 3,000 people were baptized.