The Dove is the symbol of the Holy Spirit, part of the Godhead/Trinity. One of the scriptures where the Holy Spirit as Dove appears is Matthew 3:16:
As soon as Jesus was baptized, he went up out of the water. At that moment heaven was opened, and he saw the Spirit of God descending like a dove and alighting on him.
Rev. George T. Freeman 1873 – 1910
Pastor of this Church 1905 – 1907
Context in the window: