Place the sweet potatoes into a large pot and cover with water. Bring to a boil over high heat, then reduce heat to medium-low, cover, and simmer until tender, about 20 minutes. Drain and allow to steam dry for a minute or two. Mash the sweet potatoes in a large bowl, then mix in the white sugar, brown sugar, butter, eggs, brandy, and vanilla extract. Spread potatoes in a 9-inch square baking dish.