For excellent comments about the Trinity and its relationship to this hymn I would recommend the book "Amazing Grace - 366 Inspiring Hymn Stories for Daily Devotionals" by Kenneth W. Osbeck, published by Kregel Publications. See his devotional for May 25 printed on Page 160.
This beautiful hymn praises the loving kindness of the Father, the compassion of the Savior, and the comfort of the Spirit. What an awesome God we serve! Take time this week to praise the three persons of the Trinity as they are active in our world and in our lives. Praise ye the triune God!
1 Praise ye the Father for His loving-kindness,
tenderly caring for His erring children;
praise Him, ye angels; praise Him in the heavens;
praise to the Savior!
2 Praise ye the Savior for His deep compassion,
graciously caring for His chosen people;
young men and maidens, ye old men and children,
praise to the Savior!
3 Praise ye the Spirit, Comforter of Israel,
sent from the Father and the Son to bless us;
praise to the Father, Son, and Holy Spirit!
Praise to the triune God!
Listen to it here. PRAISE