James located Caroline County, Virginia where he lived with his wife and children until his death.

Note: There seems to be a difference of opinion as to whether James Taylor II was son of James Taylor I and first wife, Frances, or by the second wife, Mary Gregory. In some places James Taylor is mentioned as born 1673 and others 1674.