Lee's ancestors probably arrived in Taiwan during the period of Emperor Yon-chien and then moved to Yeh-mei of Da-shi. Bing-shen Lee, father of Teng-fan, was in the business of tea and rice. Bing-shn Lee was very successful and became a rich man in Da-shi. Teng-fan Lee passed the government exam in the 6th year of Emperor Sien-feng (1856) and was then recommended by local government to the Emperor in the 9th year of Emperor Sien-feng (1859). In the 4th year of Emperor Tung-chi (1865), Teng-fan passed the government exam held in Fu-kien and was then recommended to serve in the Emperor's cabinet in 1868. He was considered a very important person at that time. Later on, the original name of Da-shi was changed from "Da-du-kan" to Da-ke-kang", then changed again to "Da-keh-kang".and then to Da-shi.

Lee's residence was built in the 9th year of Emperor Sien-feng (1859) after Teng-fang was recommended to the Emperor by the local government. The compound was huge. There are two main buildings and four side buildings. The living and reception rooms are located on the center line of the layout. The side buildings on both sides guard the courtyard. The buildings on the center are most important. The farther from the center the less important. The Lee family worships their ancestors in the center of the main hall. The roof of this portion of building is taller than other portions to signify its supreme position. The reception area is the major entrance to the building.
The main gate is placed slightly behind its ordinary position to make this space the focus of sight. The side buildings are living quarters. The rooms are assigned to family members according to their seniority.

The woodcarvings are the most interesting part of this building. There are many carvings. The craftsmanship is superb. You can see a number of fine carvings on the main gate. All carvings of flowers, birds, and animals look very delicate. The appearance is so tender and masculine. Teng-fang Lee was a scholar and was good at writing and painting. There are many pieces of calligraphy and paintings in this building. If you visit this building, you must not miss them.