Built in the 15th century, the chapel belonged to the Goštautas (Gasztold) noble family, who are buried in the crypt below.

Two valuable 16th century tombstones have survived in the east and west walls of the chapel. The first is dedicated to Albertus Goštautas, the chancellor of the Grand Duchy of Lithuania (by Bernardo Zanobi da Gianotti). This is the oldest memorial sculpture in Lithuania. The influence of both Gothic and Renaissance art forms are obvious. The second tombstone is dedicated to Bishop Paulius (Povilas) Alšėniškis (Pawel Holszański) (by Giovanni Maria Padovano).