Whisk the sugar into the milk in a mixing bowl until dissolved. Sprinkle the gelatin over top, and let stand for 1 minute to soften. Whisk the boiling water and almond extract into the gelatin mixture until the gelatin has dissolved. Pour into a 7-inch square pan. Refrigerate until firm, about 4 hours.
Once firm, cut the gelatin into 1-inch squares, and place into a mixing bowl. Gently stir in the mandarin oranges and their juice to serve.