1. Mix the single and double creams in a small saucepan and stir in the sugar.
2. Cook over a low heat, until the cream begins to bubble and the sugar has melted. Stir the grated lemon rind into the cream mixture and allow it to cool in the saucepan.
3. Break the cake into fine crumbs, and divide these equally between 4 individual serving dishes.
4. Stir the brandy into the cream mixture together with the lemon juice.
5. Pour the cream mixture over the cake crumbs and refrigerate for 30 minutes, or until the cream has thickened.
6. Sprinkle the toasted almonds over the top of the cream before serving.