Place the potatoes into a 6-quart saucepot and add water to cover. Heat over medium-high heat to a boil. Reduce the heat to low. Cook for 10 minutes or until the potatoes are tender. Drain the potatoes well in a colander.
Stir the soup, mustard, lemon juice, vinegar and horseradish in a large bowl. Add the potatoes, celery and onion and toss to coat. Season with the black pepper. Cover and refrigerate for 2 hours.