1) Preheat the oven to 180 degrees C/Gas 4. Generously butter a 32-by-22 cm baking dish.
2) In a medium saucepan, combine the milk, sugars, sweet potato puree, vanilla, cinnamon and nutmeg. Cook over medium-high heat, just until the mixture begins to boil. Remove from the heat and set aside.
3) In a medium bowl, whisk the egg yolks until smooth and combined. Using a ladle, pour 115 ml of the hot milk mixture in a slow, steady stream into the yolks, whisking constantly. Add another 115 ml of the hot milk mixture and whisk to combine. Pour the egg mixture into the milk mixture. Cook over medium-high heat, stirring constantly, until it's thick enough to coat the back of a wooden spoon, about 3 to 5 minutes. Strain the custard and reserve.
4) Cover the bottom of the baking dish with one layer of bread slices. Pour about a quarter of the custard mixture over the bread. Repeat with remaining bread and custard, ending with custard. Cover and refrigerate for 8 hours or overnight.
5) Sprinkle the bread pudding with the pecan crumble. Bake until the crumble mixture is golden and the custard is bubbly, about 45 minutes. Serve warm.
6) For the pecan crumble: in a medium bowl, combine of all the ingredients and mix with a hand mixer on low speed, until well combined.