He said three staff have retired and around eight staff have left in the past 18 months.
OPW Minister Kevin Boxer Moran has claimed that OPW staff are being "bullied" at Castletown House.
He said he examined a letter given to him by staff members.
He said the letter handed to him outlined how staff feel and he "can't believe" some of the aspects staff highlighted.
He said staff said they are being filmed going to work or if they "go for a drink at night" they are being bullied or contacted and that they don't "feel safe".
He said three staff have retired and around eight staff have left in the past 18 months.
He said he has received a letter from the Irish Wheelchair Association outlining concerns about accessibilty on Lime Avenue.
He said "all OPW sites" are shared entry and "work extremely well".
Speaking on national radio, he said putting traffic control on the avenue at a cost of €10,000 per week is not "practical".

Aerosol Misuse In Kildare Raises Fears Of Sudden Death Risk Among Teens
New Housing Plan Could See Kildare County Council Bidding Against First-Time Buyers
Remembrance Service For JoJo Dullard: Parish Marks Three Decades Since Disappearance
Naas Special School Delay - 12 Children Promised A Place Still Not In Class
Local Property Tax Deadline Extended To Wednesday
Lawless Rejects Claim He Let Apprenticeship Exam Leak Scandal "Fester"
Warnings Over Speeding In Kilcock School Zone But No Funding Available To Tackle The Issue Until 2026
Three Teenagers Arrested In Connection With Rioting At Citywest Hotel Last Month