File robots.txt now read the each domain.