The history and owners of Sarvilahti Manor are known since the 15th century. The castle was built in the 1670s, and it is one of the two palatial manor castles in the Late Renaissance in Finland. However, some Baroque elements can be seen in the castle as compared with Louhisaari Manor Castle. It has been severely damaged during the wars in the 18th century, and by fire in 1880. Nowadays, it is owned by Swedish culture foundation.