Holy Trinity Church, Ystalyfera, was constructed in 1844. At the time its location was within the Parish of Llanguick, which was then in the Diocese of Saint David's. When built, it was capable of 'only' seating a maximum of 350 persons. The £500 necessary to build the chapel was raised by public subscription, among those who donated being the Queen Dowager, and Mrs Budd of Ystalyfera Iron Works.

Holy Trinity was expanded in 1862-4, designed and carried out by Mr Newton, the works manager of the Yniscedwyn Iron Company. Funds were raised by a Fancy Fair held at Ynisydaren, the home of J. Palmer Budd, in August 1862.

The interior of Holy Trinity Church included lovely stained glass windows, some of which were dedicated to the life of James Palmer Budd and his wife, one panel of which is preserved in Ystradgynlais Hospital.

James Palmer Budd, of Ystalyfera Iron Works, and his wife Emily (née Rawson) were prime movers amongst those working intitially to see Holy Trinity Church built, and then twenty years later seeing it expanded and rebuilt. Over the 150 years of its existence, other prominent people included the Rev. Secundus Jones, and dedicated organist Mr D.R. Jones. Also included is a list of Vicars of Ystalyfera, by incumbency date.

Holy Trinity Church, Ystalyfera, celebrated its 125th anniversary in 1970, and also successfully raised money to restore the Walker organ, installed when the church was expanded in 1864.

The graveyard of Holy Trinity Church, Ystalyfera covers a large area and is still in use today. It is currrently accessible only from the A4067, due to the landslip at Pantteg blocking the upper road.

Holy Trinity Church, Ystalyfera, has a number of wargraves and remembrance graves. The fallen remembered on these are listed on several local war memorials, including Ystradgynlais and Ystalyfera.

Holy Trinity Church was demolished in 1988, after being condemned as structurally unsound in mid 1986. The final service was held on 25th January 1987.

After it was demolished in 1988, the site of Holy Trinity Church was replaced by a memorial garden. This was completed in the shape of the church, covering the same plot of ground, and was opened in a ceremony in 2004 officiated over by the Bishop of Swansea and Brecon.